Pattern-Guided Adaptive Prior for Structure Learning
Authors: Lyuzhou Chen
Yijia Sun
Yanze Gao
Xiangyu Wang
Derui Lyu
Taiyu Ban
Xin Wang
Xiren Zhou
Huanhuan Chen
Venue: NeurIPS 2025 poster ()
Keywords: Machine Learning
Causal Discovery
Prior Knowledge
Adaptive
Primary area: general_machine_learning
TL;DR: Common prior knowledge can distort edge weights in DAG structure learning, creating characteristic graph patterns. Our framework detects these patterns as signals to adaptively adjust learning process, improving prior integration and DAG accuracy.

Abstract: Learning the causality between variables, known as DAG structure learning, is critical yet challenging due to issues such as insufficient data and noise. While prior knowledge can improve the learning process and refine the DAG structure, incorporating prior knowledge is not without pitfalls. In particular, we find that the gap between the imprecise prior knowledge and the exact weights modeled by existing methods may result in deviation in edge weights. Such deviation can subsequently cause significant inaccuracies when learning the DAG structure. 
This paper addresses this challenge by providing a theoretical analysis of the impact of deviation in edge weights during the optimization process of structure learning. We identify two special graph patterns that arise due to the deviation and show that their occurrence increases as the degree of deviation grows. Building on this analysis, we propose the Pattern-Guided Adaptive Prior (PGAP) framework. PGAP detects these patterns as structural signals during optimization and adaptively adjusts the structure learning process to counteract the identified weight deviation, thereby improving the integration of prior knowledge. Experiments verify the effectiveness and robustness of the proposed method.

Pattern-Guided Adaptive Prior for Structure Learning
Lyuzhou Chen, Yijia Sun, Yanze Gao, Xiangyu Wang",
Derui Lyu, Taiyu Ban, Xin Wang, Xiren Zhou, Huanhuan Chen'
School of Computer Science and Technology, University of Science and Technology of China Jinzhai Rd, Hefei, China, 230026 fc1231415emaill sunyijia2022mail. gaoyzemail., sa312e,} {drlvemail., bantyEmail., w25200mail., zhou06120, hchen@Justc, edu. cn
Abstract
Learning the causality between variables, known as DAG structure learning, is critical yet challenging due issues such insufficient data and noise. While prior knowledge can improve the learning process and refine the DAG structure, incorporating prior knowledge is without pitfalls. In particular, the gap between the imprecise prior knowledge and the exact weights modeled by existing methods may result in deviation in edge weights. Such deviation can subsequently cause significant inaccuracies when learning the DAG structure. This paper addresses this challenge by providing theoretical analysis of the impact of deviation in edge weights during the optimization process of structure learning. We identify two special graph patterns that arise due the deviation and show that their occurrence increases as the degree of deviation grows. Building on this analysis, we propose the Pattern-Guided Adaptive Prior (PGAP) framework. PGAP detects these patterns as structural signals during optimization and adaptively adjusts the structure learning process to counteract the identified weight deviation, thereby improving the integration of prior knowledge. Experiments verify the effectiveness and robustness of the proposed method.
find that
Introduction
Discovering directed acyclic graph (DAG) structures of variable associations from observational data, also known as structure learning (Scanagatta et al. 2019 Heinze- et al. 2018 is gaining significant attention in many practical applications [Peters et al. |017] Gong et al. 2022 However, the broad application of structure learning is frequently hindered when relying data alone. Such difficulties arise primarily from challenges like insufficient data and unavoidable noise [Ramsey et al., 2018). To overcome these limitations and improve practical applicability, prior knowledge is often incorporated. This knowledge, typically derived from domain expertise about the existence of specific relationships [Constantinou et al. 2023], allows for the integration of established insights. By doing so, prior knowledge can guide the DAG's structure, reduce the search space, and thereby enhance the efficiency and reliability of the learning process |Eggeling al. 2019 Chen et al. 2023
In practical scenarios, the prior knowledge available from domain experts is often imprecise, since typically indicates the existence of dependencies between variables rather than specifying their exact magnitude or detailed functional forn Kitson et al.] 2023). This imprecision contrasts with the requirements of existing modeling approaches, which often describe the relationships within DAG structure using precise numerical weights for the connections between variables [Zheng et al. 2018 Indeed, most current methods widely incorporate such imprecise prior knowledge, yet the potential
'This paper refers to priors that correctly assert the existence of a directional relationship (e.g.. variable influences variable B) as accurate, but not precise because they do not specify an exact numerical strength.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

risks and problems arising from the inherent gap between the non-specific nature of these priors and the precise parameter demands of models have not been fully explored. This direct introduction of imprecise priors often means that existing methods face difficulties in the reconciliation process, potentially leading to suboptimal learning outcomes.
One critical consequence of this issue is what we term "deviation". Because imprecise prior knowl- edge typically lacks exact numerical specification for relationship strengths, forcibly applying con- straints based on it can compel learned edge weights to diverge from their true underlying values. Such deviation has two critical properties. Firstly, the misspecification of one edge's 's weight due to deviation can propagate and interfere with the accurate learning of other parameters within models. Secondly, the extent of this adverse impact can increase with the magnitude of the initial deviation.
Despite the potential for such deviations, existing prior-based structure learning methods primarily aim align the learned graph with given prior knowledge. These methods often overlook the risk that foreing adherence to imprecise priors can cause edge weight deviations that can distort the learned structure. Generally, approaches to integrate priors range from more rigid enforcement (e.g., compelling edge existence and specific weight ranges) to more flexible techniques (e.g., using penalty terms to encourage adherence Both types of methods can inadvertently lead to deviations. Rigid enforcement can result in weights unrepresentative of the true data-generating process if the constraints sharply conflict with data evidence or true relationship strengths. Similarly, flexible methods can also induce deviations by disproportionately emphasizing prior conformance over data fidelity, ultimately affecting the learned structure.
To address this challenge, this paper first theoretically explores the impact of edge weight deviation on structure learning within the widely-used linear Structural Equation Model (SEM) framework. We demonstrate that such deviation gives rise to two kinds of graph patterns. The prevalence of these patterns increases with the degree of deviation. To assess their association with the deviation, we quantify their expected frequency. This analysis shows they are likely markers of deviation- induced errors rather than naturally occurring structures. Therefore, leveraging these patterns as structural signals of edge weight deviation during the optimization process, this paper proposes the Pattern- Pattern-Guided Adaptive Prior (PGAP) framework for learning reasonable edge weights. Our main contributions are outlined below.
A comprehensive theoretical exploration of how imprecise priors cause edge weight deviation and impact continuous optimization in structure learning.
The formal identification and analysis of two graph patterns manifesting reliable structural signals of such edge weight deviation.
The development of the PGAP framework, which leverages these patterns as structural
signals to adaptively adjust the learning process, mitigating erroneous structures.
2
Related Work
Structure learning aims to discover variable associations from observational data. Traditional ap- proaches include constraint-based methods relying on conditional independence tests (Spirtes et al. 2001 Colombo et al., 20012) score- score-based methods evaluating graph fit [Chickering] 2002). and hybrid techniques [Tsamardinos et al. 2006]_ A significant recent advancement frames structure learning as a continuous optimization problem Zheng et al. 2018), enabling the use of peudient-baseed algorithms and spurring developments like NOTEARS, GOLEM [Ng et al. 2020). DAGMA (Bello et al.) 2022).
and various extensions addressing non-linear relationships or using different optimization strategies
WWei et al. 2020 Yu et al. 2019 Lachapelle et al. 2019 Deng al. 2023)
Prior knowledge is often used to improve learning accuracy, especially with limited data |Constantinou et al. 2023). In traditional combinatorial search, priors may be enforced via hard constraints (e.g.. requiring/Foriidding specific edges or ancestral relationships (De Campos and Castellano) 2007
De Campos et al. |2099] Chen et al. 2016)) used as soft constraints to guide the search process or scoring [Castelo and Sicbes] 2000 Boobouuakii and Tsamardinos/ 2014 Eggeling et al. 2019).
Similar strategies have been adapted for continuous optimization frameworks. Hard constraints can
"These two main strategies are often termed "hard" and "soft" prior integration methods, respectively. A detailed discussion of these approaches can be found in Section

---

restrict edge weight parameters to specific ranges [Sun et al. 2023 Chen and Ge 2022] while soft constraints typically add prior- based penalty terms the objective function Yang al. 2023 Wang
al. 2024 or directly manipulate gradients during optimization [Bello et al. 2022
However, existing methods for incorporating prior knowledge in continuous optimization often focus primarily on ensuring the resulting graph qualitatively satisfies the given constraints. They may overlook a crucial side effect: enforeing qualitative prior (e.g., edge existence) within a quantitative model can lead to deviations in the enforced edge's weight, which, as we show, can propagate and distort the learned weights of other edges. Addressing this challenge is the central motivation for our work. A more detailed discussion of related work can be found in Appendix A
3 Preliminaries
Notation
This paper employs standard matrix and vector notation. Subscripts typically indicate specific elements, column vectors, or selections based on index sets. For example, given Aij denotes the element in the i-th row and j-th column, and A, may denote the j-th column vector. For subset of indices S C (1,...,d) d), Xs denotes the subvector (Xt)
a matrix A,
Bayesian Network Consider a Bayesian network defined over set of random variables X Xz). (X1...., the set of nodes and E denotes the set of edges. The joint distribution of X, denoted as P(X), is
Its structure is represented by DAG G = (V, E), where V = {X,...., Xa) denotes
Markov with respect to DAG G, which means P(X) = I P(X; Xp::), where Pa, represents the set of parent nodes of node X; in graph G.
Structural Equation Model This paper considers the case where the random variables follow a linear structural equation model (SEM), described by:
X 8 +e
(I)
Here, B E Rddd represents the weighted adjacency matrix of graph G. with each non-zero el- ement Bi indicating an edge X,) in E and describing the linear dependency. The vector
= -e,...., +a)² denotes the noise, which is assumed to have zero mean and possess diagonal covariance matrix Ω = diag(of.......
Structure Learning The primary objective of structure learning is to estimate the DAG structure G (characterized by an estimated adjacency matrix W during estimation) by analyzing x data matrix X, where each row corresponds to an independent and identically distributed sample X from the joint distribution P(X). This task is often formulated constrained optimization problem:
min WER
Q(W:X) subject to G(W) E DAGs
(2)
Here, Q is scoring function that assesses the fit between the data matrix X and the graph G(W) corresponding to the candidate adjacency matrix W. Common scoring functions for linear SEMs include least squares [Loh and Bühlmann, 2014) and negative log-likelihood [Bühlmann et al. 2014):
Qw,(;
XW XI2
2n
d
(w; 2n
log XWW - 11
Continuous Optimization Recently, (Zheng et al. which employs a continuously differentiable function h(
2018 introduced the NOTEARS method,
characterize the acyclicity of the graph G(W). This transforms the combinatorial constraint G(W) DAGs into a smooth equality constraint, enabling continuous optimization techniques:
min WeRdaa
Q(W;X) subject to h(W) =
h(W) == tr(e tr(e -d
We do not distinguish between nodes and random variables, as this simplification does not lead to confusion.
3

---

Here, O denotes element-wiss multiplication and e eWow is the matrix exponential. It has been shown that G(W) is DAG if and only if h(W) O. Fulfilling this condition ensures that the solution the optimization problem corresponds to DAG structure. Following NOTEARS, subsequent research has introduced similar functions to encapsulate the acyclicity of a DAG and further incorporate properties that benefit the optimization process, such as the work of [Ng et al. 2020) and [Bello et al.; 2022].
Prior Knowledge In practical scenarios, structure learning can be enhanced by incorporating prior knowledge about the system being modeled, often available from domain experts or previous studies. This incorporation helps resolve ambiguities within the Markov equivalence class, reduces the search space, and improves learning accuracy, especially with limited or noisy data.
This paper defines prior knowledge as (X; + X)): which indicates the existence of directed edge from variable X; to X, in the true DAG G. To ensure prior knowledge benefits structure learning, most studies assume its accuracy (i.e., that the provided prior knowledge aligns with edges in the true DAG G). However, even accurate prior knowledge requires careful integration to avoid unintended negative consequences, particularly concerning learned edge weights.
4 Pattern-Guided Adaptive Prior Framework for Structural Learning
This section unfolds our approach. First, we theoretically analyze how the integration of imprecise priors can lead edge weight deviation and subsequently impact the structure learning process. Second, we explore how this deviation manifests as two specific structural patterns within the learned graph. Finally, we detail the PGAP framework, which leverages these patterns as structural signals to adaptively adjust the learning parameters in a two-stage process. Please refer to Appendix B for the proof of the mentioned lemma.
4.1 Impact of Edge Weight Deviation
In this section, we detail our principal findings concerning the impact of edge weight deviation on structure learning SEM continuous optimization, showing that enforcing prior knowledge introduces deviations that interfere with learning other weights.
Our theory is based on the interaction of deviations with other edge weights during optimization of the scoring function. Therefore, we will first analyze the properties of the scoring function. In fact, the optimization of the scoring function
W arg min Q(W; XX)
(5)
W
decompose into independent sub-problemn for each variable X,, resembling ordinary least squares (OLS) regression:
Wj = arg min (XW, X,)²
This conclusion is true for common scoring functions like least squares (Qis) and negative log- likelihood (Qnu). Since the variables in DAG have topological order, for each Xj, we only need to consider the edge pointing to it from its predecessors:
W ビッ = arg min (X,, WG,5 - -X,)
(7)
W y.s
Here, ty is the set of topological predecessor nodes to X, in the graph defined by W. In particular, the set of predecessors i, aligns with the predecessors in the true DAG, tj. (e.g., if the variables X1, X2,..., ...,,, Xa are ordered following the true DAG, then 17 = (1,2,..... 1}), the accurate edge weights can be determined directly from sufficient data, as stated in the Lemma
Lemma 1. Given node Xj, assume that tł represents the set of predecessors of X, consistent with an order of the true DAG G. The regression coefficient obtained by linearly regressing X, on tj corresponds directly to the relevant element in the adjacency matrix B of the true DAG, i.e.:
Bez. = arg min Wez0 (Xe5We3 - -x,)
(8)

---

Now, consider the influence of prior knowledge integration. Assume prior knowledge (Xg - X)) incorporated. Let W be the optimal weight for edge (X3,, (X,,X,) x)) obtained from the OLS regression of X3 onto its current predecessors E, (Eq. 6). If the prior- infuurnefeeeeed result (defined We,) differs from this data-optimal weight, we define the deviation as 7 Wa, W This deviation y represents the discrepancy introduced by imposing the prior on the quantitative SEM weight. This occurs, for example, when methods enforcing this prior either fix the weight Wa.5 (a hard constraint) or add penalties to strongly push Wo, towards nen-zero values (a soft constraint).
The crucial question is how this deviation "y on the prior edge (Xg + X3) affects the learning of other weights, specifically the weights Wa.5 for other incoming edges (Xk; where E i, - We analyze this by considering the optimization problem where 7 is fixed (abstracting the effect of prior The fix of y implies the fix of W R.j.
only need to optimize the weights W
W g- arg min
(Xe,-WWiy-a) + X,W,, -x,) -X,)
(9)
Let W B- denote the optimal weights for non-prior edges obtained from the original OLS problem (Eq. 6 involving all predecessors ty. The following lemma quantifies how the deviation 7 propagates to affect these other weights.
Lemma 2. Lel 7 = Wa.j 1e4j-Wa, Wä. represent the deviation of the prior-infleence weight W Wa. (derived
from 1 from the optimal OLS weight W obtained when regressing X, onto the set of predecessors t, (derived from ( The difference between the wrior-influencd weights for other incoming edges W e - and We,-88 8.. is a linear function of o expressed as:
- Wig-s.j 03.3
7 -B, 8,3
(10)
where 0 is the inverse of the covariance matrix of the variable set tj.
We further consider scenarios where the predecessors ty align with the true predecessors t7. this situation, 3 E {1,2,.... 1}. and - can be expressed as:
B EE=
B of Ba, Bui
1 <<D 3
Oug Ba.k
(1I)
Di
3 k
of
Equation (1] shows explicitly how the impact term Ok, depends on the true graph structure B (via direct edges like BR, or B3, and common children structures) and the noise characteristics Q (via o² terms). Combined with Lemma 2 this illustrates how deviation on edge (Xa, X,) propagates to affect edge (Xx, Xy) based the underlying true graph pathways connecting X₁. and X3--
Lemma and Lemma elabboraee on the dual role of prior knowledge. Firstly, the prior knowledge can qualitatively guide optimization to better structural approximations by helping identify true predecessors 17. However, if the prior knowledge introduces a quantitative deviation y WR, A.j" this deviation can propagate and affect the learned weights of other edges (X,,X,) for k t, - B, potentially leading to the generation of spurious edges or incorrect weights, even when the set of predecessors tú correctly matches the true predecessors tj. This underscores the importance of judiciously using prior knowledge, considering not just its qualitative correctness but also its quantitative impact.
Remark In the above analysis, we have not discussed the acyclicity constraint of the optimization problem 4 Although the acyclic constraint of DAG may cause differences between the real optimization and linear regression, the introduction of the constraint ((W) = imposes a topological order among the variables, ensuring that successor nodes do not point predecessor nodes. During the continuous optimization process, the constraint ((W) = must be met (or approached), meaning the learned weight matrix W always corresponds to a graph structure possessing an order. Consequently, even with the acyclicity constraint, the optimization problem to estimate the incoming edges for node X, still resembles linear regression.
*We use l - to represent the deletion of element / from set 4z. 'Note that although changes during the learning process, analyzing its impact for given value allows us understand the characteristics of the resulting DAG structure.
5

---

Tie edge
X
X
X,
X
X
Weight sparius odge
X
Wright priorinfhuorddddg
(8)
(b)
The of woiggtt chight
Figure 1: (a,b) If the edge (X,, Xs) (X,, Xx) exists in the true graph, then as y increases, the learned weight Wa, will gradually deviate from true weight Bx.5. (c) When there is a common child node X, between Xg and Xg. increases, the difference between WA, and Bx.j will increase.
4.2 Deviation-Inducee Graph Patterns Formation
In this section, we show that weight deviation introduces spurious edges, disrupting the learned DAG. This motivates the use of graph patterns deviation indicators.
Based on the findings from Lemma] 2 the impact the weight difference " on structure learning is linear. Setting aside the coefficient 0g for simplicity, concentrate on the coefficient Øx.p- Assuming the noise variance as a positive constant, we explore the conditions within the weighted adjacency matrix B where 0x.3 + 0:
1}. J. let 0 be the inverse of the submatrix of the covariance the variable set t.. Then the condition under which 0g.3 is non-zero given by: BR # 0 or X3 and Xx have a common child node X;, when ks < B; B,, # 0 or X and Xx. have a common child node X,, when B.
Lemma 3. Assuming tú =
of X corresponding
It should be noted that Lebesgue aerommeaurre set case is ignored in the Lemma which refers the case where Xg and Xx share a common child X,, and there exists specific non-zero value BR.s such that 0 = 0. Essentially, assuming this special case does not occur is equivalent to the faithfulness assumption [Koller 2009 Spirtes al., 2001].
When a node X₂ meets the conditions specified in Lemma 3 the weight deviation y will alter the solution to the optimization problem (9). In particular, if significant, might cause the edge weight Wa. obtained in the optimization problem (6) to deviate from zero in optimization problem (9). This means that even if B& 0 and all the true predecessor nodes of node ý are known, ? can still induce the formation of new edges. Figure [ visually illustrates how weight deviation can lead to the creation of extra edges. As can be seen in Figure [ the creation of extra edges gives rise to two kinds of graph patterns. In cases (a) and (b), a triangular pattern is formed within the DAG corresponding to the learned adjacency matrix W. This structure involves two nodes associated with the prior knowledge (X3 3 + X,) and third node X that meets the conditions in Lemma 3 In case (c), Xs, Xt. and X, form a collision structure, and simultaneously, X3. Xg. and X; constitute another collision structure. This case is referred to as a "double collision pattern"..
Our focus on these patterns is not arbitrary choice but theoretically grounded. Lemma precisely characterizes the necessary and sufficient conditions for deviation on a prior-enforced edge propagate and induce single spurious edge. The two conditions identified in the lemma, direct connection or a common child path between the two parent nodes, correspond exactly to the formation of the triangular and double collision patterns, respectively. This demonstrates that these patterns are the exhaustive structural manifestations of this fundamental eиror mechanism. Therefore, they are not arbitrary heuristies but are leveraged as the principled, preoretically-derivee indicators of prior-induced error in our framework.
As y increases, these extra edges and the associated patterns become more prevalent. To quantify the potential prevalence of these deviation-inducel patterns compared to naturally occurring ones, analyze their expected frequency using the standard Erdös-Rényi random graph model G(d, p). where d is the number of nodes and is the probability of an edge existing between any pair of nodes (Analysis of general models is possible but produces complex results that obscure intuition). This analysis helps justify using these patterns as indicators, but the underlying mechanism linking deviation to these patterns (Lemmas 2 and 3 based on general SEM properties and graph structures, not limited to ER graphs. Indeed, the conditions in Lemma 3 involve local structures (direct connections or common children paths), implying these patterns may appear in any topology (such as scale-free networks) meeting these local conditions.

---

Lemma 4. Assuming the true DAG G adheres to the Erdös-Rényi graph model G p). Assuming that the prior knowledge leads to the weight deviation 7 which sufficiently large, then the expected number of triangular patterns resulting from r approaches [ 2)p, and the number of double collision patterns approaches o (d 2) 3)p².
According to the lemma below. the number of both triangular and double collision patterns signifi- cantly surpasses their natural occurrence in the DAG: Lemma 5. In the ER graph G(d. p). the expected number of natural triangular patterns and double collision patlerns is tdldd 1)(d - 2)p² and d 1)(d 2)(d 3)p², respectively.
The expected number of the two patterns given Lemma 4 can also be approximately regarded as dp and _ (dp)². This result indicates that the frequency of two graph patterns can be characterized by dp, which is numerically close the average degree of the graph and characterizes the sparsity of the graph. Thus, we can further deduce that triangular patterns predominate when the average node degree low, while double collision patterns become more frequent as the average degree increases.
4.3 Pattern-Guided Adaptive Prior Framework
Thus far, we have theoretically established how deviation ? on prior-enforeed edge (Xg + Xj) can propagate (Lemma to induce spurious edges (X,, X,) when nodes X X, satisfy certain structural relationships (Lemma 3) leading to characteristic triangle or double collision patterns. We also showed that these patterns strongly indicate deviation-induced errors. (Lemmas
5 Leveraging these insights, this section introduces the Pattern-Guided Adaptive Prior framework, using these structural signals to mitigate adverse effects and foster more accurate weight learning.
The core idea of PGAP is to employ a two-stage process. First, we learn an initial DAG structure while incorporating prior knowledge. Second, we detect the number of deviatiioninducee patterns associated with each prior edge. If such patterns are found, we adaptively reduce the influence the corresponding prior and re-optimize the model to obtain more refined final graph. This procedure detailed in Algorithm for the common case where priors are integrated as soft constraints via penalty term in the loss function.
Algorithm 1 PGAP with Soft Prior Constraints
1: Input: Data matrix X, set of prior knowledge II, initial prior loss weights og (vector with elements 00,5 for each æ E II), pattern adjustment step size a.
2: Learn Wsngel using base structure learning algorithm, incorporating the prior loss term LP(W; II, an).
3: Initialize a1 f- ag.
4: for each prior π = (Xg + X,) E 11
5: Apaaterns # of triangle and double collision patterns in W sugee involving
6: 011, f max(0, 00 a X Mpattems)
7: end for
8: Learn Winal using the base structure learning algorithm, incorporating the prior loss term Lp(W:II.n)
9: Return Winal
In this procedure, the objective function of the base learning algorithm is augmented with a prior loss term LPWW; II, a.). Here, II denotes the set of provided priors, and is vector of weights governing the enforcement strength for each prior. The algorithm leverages the number of detected patterns, Mpaterss adjust these weights, producing an updated weight vector a for the second optimization stage. This is discussed in more detail in Appendix [ This adaptive adjustment allows the model to counteract the effects of weight deviation by down-weighting priors that cause structural distortions. In addition, a similar implementation for hard prior constraints, which adjusts local regularization parameters, is also provided Appendix C
5 Experiments
This section presents a subset of the primary experimental results and analysis. evaluate the performance of our proposed method across various synthetic data settings, including different graph
7

---

structures, numbers of nodes, dataset sizes, and noise types. We also assess its effectiveness when combined with different prior integration techniques and compare it against several baseline structure learning algorithms. Comprehensive details, extended analysis (including the impact of varying prior knowledge proportions and a detailed parameter analysis), supplementary experiments on real-world datasets, and explorations into non-linear data scenarios are available in Appendix D
5.1 Experimental Setup
Synthetic Data Random DAGs are generated using the Erdös-Rényi (ER) model or the Scale-Free
(SF) model, with the number of nodes (d) set to 20, 40, and 60. We use ERk and SFK to denote ER and SF graphs, respectively, where represents the edge-to-node ratio. The edge-to-node ratio (k) varies between 2 and 4, corresponding to average degrees of 4 and 8, respectively. For graphs with 60 nodes, an edge-to-node ratio of k = 8 is also tested. The weighted adjacency matrix B is created with uniform random weights drawn from U(-2, 0.55) U U0.55 2). and the data is generated as X B²x X + e. Noise is varied among Gaussian, exponential, and Gumbel distributions, referred to Gauss, Exp, and Gumbel, respectively. Variance settings include equal variance (-EV) and non-equal variance (-NV). Sample sizes are set to 2 and 20 times the number of nodes (2d, 20d), and data is normalized post-generation. For the NV settings, the individual noise variances of for each variable X₁ were drawn independently from a uniform distribution (0.5, 2). All reported results on synthetic data represent the mean values obtained from six independent trials for each experimental configuration. All used priors are randomly selected from a proportion of edges in the true DAG.
Methods We benchmark PGAP against several established structure learning methods, including VARSORT [Reisach et al., [021], GOLEM (Ng et al. 2020), DAGMA [Bello et al.) 2022). , and NT-LogLL [Deng et al. 2024]. To demonstrate its adaptability and effectiveness, PGAP is integrated with each of these baseline methods, and its performance is compared against the standalone baseline. The least squares scoring function is used when noise is EV, and NLL is used when noise is NV. We also evaluate its performance when combined with different prior integration methods NOTEARS. These include a hard prior integration method and two soft prior integration methods. The hard method specifies edge weights learned in the interval (-oo, -th) U (th, oc) where th is the threshold. The soft methods utilize two kinds of prior penalty functions:
Softmax: Lp(W;;) = Σ Wn - Softmar @ Wn))
(12)
(W; = Σ Relu(th x Wn - |W| o Wn)
In these equations, Wn represents the mask matrix derived from the prior knowledge, and Σ denotes the summation over all elements of the matrix.
Metrics Fl score, Structural Hamming Distance (SHD) or scaled SHD.
Setup The initial weight of the prior loss 1. The adaptive reduction parameter a, i.c., the amount of weight reduction for each graph pattern, is 0.1. Other settings follow the default configurations of the baseline methods. Experiments are conducted on a system equipped with a 4.5 GHz AMD Ryzen 9 7950X CPU, NVIDIA GeForce RTX 3090 GPU, and 32GB of memory.
5.2 Experimental Results and Analysis
Performance of PGAP in Various Graph Settings This experiment evaluates the performance of the PGAP by comparing it against the baseline NOTEARS-Sofee Soffmax across variety graph settings. Figure@lillustrates Figure 2 iillutraass the performance comparison, showcasing results for different numbers of nodes,
dataset Sizes under different noise. Note, "scaled SHD" is calculated number of nodes. Across these diverse conditions, the lines of proposed PGAP (the dark red and blue lines) are consistently positioned the bottom of the plots and demonstrate superior performance. Results for denser graphs like ER4 and SF4 are detailed in Appendix [.I] The supplementary results also show that the improvement offered by PGAP tends to increase as the proportion of prior knowledge grows, contrasting with the baseline performance degradation with higher proportions of prior knowledge. PGAP helps to alleviate this issue, underscoring the importance of its mechanism for the reasonable integration of priors.
the SHD divided by the
POTEARS-Softman which can occasionally exhibit
8

---

NOTEARS-2d
NOTEARS-PGAP-2d
NOTEARS-20d
NOTEARS-PGAP-20
Gauss-EV
Exp-EV
Gumbel-EV
Gauss-NV
Exp-NV
Gumbel-NV
1.3
1.0
Scaled SHID
1.0
ER2
Scaled SHD
0.7
ER2
0.7
0.4
0.9
0.9
SF2
0.7
0.7
0.5
0.5
20
40
60.0
40
6020
40
60
20
40
6020
40
6020
40
60
d (Number of Nodes)
(Number of Nodes)
Figure Performance of PGAP across Different Graph Settings (ER=2).
Table 1: Improvements of PGAP on different prior integration methods (Gauss-Ny NV case).
Data Size
2d
20d
ER
Metric
F1
SHD
SHD
Prior Proportion
0.2
0.4
0.2
0.4
0.2
0.4
0,2
0.4
NOTEARS-HoE
33.6+2.1
48.2+2.0
40.2+1.0
34.8
35.1+4.1
48.1
37.0#20
31.8=20
+PGAP
37.023.0
53,9#2,1
37.0#2.1
29.3+14
35.422,
52.1
35.3#1.7
28.3+23
NOTEARS-ReN
32.2+5.0
39.4+2.3
54.7+5.4
60.5+5.6
33.4+45
42.9+4.3
49.8
52.3#7.1
2
+PGAP
34.913.7
42.3+3.5
49.824.1
53.028.0
35.144.7
47.024.5
46.824.
42.029.4
SETEARRSSoftman
39.0+3.9
48.922.6
37.823.4
35.721.9
34.526.3
48.324.0
38.7245
34,7229
+PGAP
37.7#2.9
57.8+5.4
35.2+1.5
26.742.3
33.442.2
56.4+5.1
37.5#18
26.3+29
NOTEARS-Han
39.1
54.1
68.0+38
54.8+2.7
37.0#1.
55.0+1.2
70.3#2.7
54.0#2.1
+PGAP
39.8+3.7
55.5+2.0
66.224.3
52.243.5
37.622.9
55.5+2.4
69.3+3.3
53.2+27
NOTEARS-Rede
46.5t3.0
52.5+2.4
89.315.3
89.824.1
44.241.9
53.542.0
89.723.6
82.325.7
4
+PGAP
46.9t3.0
53.2+2.2
86.3+5.0
86.8+4.9
45.4+2.3
53.3+14
86,3+42
80.3+5.6
NOTEARS-Sofnne
38.8+3.5
55.9+1.8
72.8+199
60.342.7
38.542.7
57.5428
73.843.3
57,8+43
+PGAP
40.4#4.0
56.9
69.0229
57.0
39.0#1.6
60.4229
69.0s28
50.8
PGAP on Prior Integrating Methods In this experiment, we integrate PGAP into 3 baseline prior integration methods: NOTEARS-Hard -ReLU, and -Softmax, evaluate the impact on their performance. The specific implementation of PGAP can be found in Appendix C The results are detailed in Table Additional performance data under various noise types are available in Appendix D.2 The results indicate that PGAP generally enhances the performance of these methods, particularly terms of SHD, demonstrating its effectiveness across different prior integration methods.
Comparison of PGAP with Established Methods structure learning methods and compared these PGAP-enhanced versions against their original counterparts. The mainstream methods included in this comparison are NOTEARS, GOLEM, DAGMA, and NOTEARS-LogLL. For fair comparison, these baseline methods, both in their original
We integrated PGAP with several mainstream
Gauss-FV, FR4, DataSize=20d
Gauss-NV, FR4, DataSize=20d
VARSORT
VARSORT
60
80
100
120
70
80
90
100
110
NOTEARS
Sectne
NOTEARS
+PGAP
+PGAP
Fre-
GOLEM
Method
GOLEM
+PGAP
+PGAP
DAGMA
+PGAP
DAGMA
NI-LogLL
+PGAP
+PGAP
60
70
80
80 Structural Hamming Distance (SHID
60
Structural Hamming Distunce (SHI)
Figure 3: Improvements of PGAP on Mainstream Methods.

---

and PGAAPenhanced forms, utilized prior knowledge via a soft integration approach. Additionally. VARSORT also included, which is benchmark for structure learning. Comparative experiments were conducted across various prior proportions and noise types (see D.3 with the results presented structural learning methods
Figure 3 As shown in the figure, the PGAP-enhanced versions consistently outperform their respective original versions. This demonstrates PGAP's ability to improve upon existing methods by fostering a superior utilization of prior knowledge.
Limitation and Further Discussion
While theoretical and empirical results demonstrate the effectiveness of the PGAP framework, it important to acknowledge its limitations and the assumptions upon which it is built. A primary consideration is that formal theoretical analysis is grounded in the linear SEM setting under the faithfulness assumption. Although our exploratory experiments suggest that PGAP remains beneficial in some nonlinear scenarios, extending the formal characterization of deviation propagation general nonlinear models remains a challenging future direction.
Furthermore, the PGAP framework is designed theoretically motivated by the analysis of error propagation, the current work does not provide formal consistency guarantees for the resulting estimator. The framework's 's practical performance can also be context-dependent As our analysis suggests, PGAP demonstrates robust improvements in sparse graph settings. However, in denser graphs where the target patterns may occur more frequently by chance, the framework's is stability becomes a key consideration. We address this challenge and provide more extensive discussion it in Appendix [D.8 Furthermore, more extensive discussion of these aspects, alongside topics such as computational complexity and the handling of prior knowledge quality, is available in Appendix B
adaptive, two-stage procedure. While it is
7 Conclusion
This paper addresses the challenges of integrating prior knowledge into structure learning and offers practical solutions. Through theoretical analysis, we examine how edge weight deviation impacts the optimization process and specifically investigate two graph patterns whose prevalence increases with the magnitude of deviation. To counteract these issues, we introduce the Pattern-Guided Adaptive Prior (PGAP) Framework, designed to minimize the occurrence of these problematic patterns, thereby reducing the adverse effects of prior knowledge and improving its integration process. Our experimental results confirm the effectiveness and robustness of PGAP, demonstrating its superiority DAG structure learning. This study highlights overlooked aspects prior usage existing research and pioneers the exploration of applying prior knowledge judiciously in continuous optimization.
10

---

Acknowledgements
This research was supported in part by the National Key R&D Program of China (No. 00212D0111700) in part by the National Nature Science Foundation of China (No. 62137002, 62406302, 62176245),
part by the Natural Science Foundation of Anhui province (No. 2408085QF195), in part by the Fundamental Research Funds for the Central Universities under Grant WK21501 10035.
References
Mauro Scanagatta, Antonio Salmerón, and Fabio Stella. A survey on Bayesian network structure learning from data. Progress in Artificial Intelligence, 8(4):425 439, 2019.
Christina Heinze-Deml, Marloes H Maathuis, and Nicolai Meinshausen. Causal structure learning. Annual Review of Statistics and Its Application, 5:371-391, 2018.
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: Foundations and learning algorithms. The MIT Press, 2017.
Wenbo Gong, Digory Smith, Zichao Wang, Craig Barton, Simon Woodhead, Nick Pawlowski, Joel Jennings, and Cheng Zhang. Neurips competition instructions and guide: Causal insights for learning paths in education. arXiv preprint arXiv:2208. 12610, 2022.
Joseph Ramsey, Kun Zhang, Madelyn Glymour, Ruben Sanchez Romero, Biwei Huang, Imme Ebert-Uphoff, Savini Samarasinghe, Elizabeth A Barnes, and Clark Glymour. Tetrad-A toolbox for causal discovery. In Proceedings of the 8th International Workshop on Climate Informatics, pages 1-4, 2018.
Anthony C Constantinou, Zhigao Guo, and Neville K Kitson. The impact of prior knowledge on causal structure learning. Knowledge and Information Systems, (((8):338-34335 2023.
Ralf Eggeling, Jussi Viinikka, Aleksis Vuoksenmaa, and Mikko Koivisto. On structure priors for learning Bayesian networks. In Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, pages 1687-1695. PMLR, 2019.
Lyuzhou Chen, Taiyu Ban, Xiangyu Wang, Derui Lyu, and Huanhuan Chen. Mitigating prior errors in causal structure learning: Towards LLM driven prior knowledge. arXiv preprint arXiv:2306.07032, 2023.
Neville Kenneth Kitson, Anthony C Constantinou, Zhigao Guo, Yang Liu, and Kiattikun Chobtham. survey of Bayesian network structure learning. Artificial Intelligence Review, 56(8):8721-88) 2023.
Xun Zheng, Bryon Aragam, Pradeep K Ravikumar, and Eric Xing. Dags with tears: Continuous optimization for structure learning. Advances in Neural Information Processing Systems, 31, 2018.
Peter Spirtes, Clark Glymour, and Richard Scheines. Causation, prediction, and search. MIT press, 2001.
Diego Colombo, Marloes H Maathuis, Markus Kalisch, and Thomas S Richardson. Learning high- dimensional directed acyclic graphs with latent and selection variables. The Annals of Statistics, pages 294-321, 2012. 2012.
David Maxwell Chickering. Optimal structure identification with greedy search. Journal of Machine Learning Research, 3000555077555, 2002.
Ioannis Tsamardinos, Laura E. Brown, and Constantin F. Aliferis. The Max-Min Hill- -ilm-Ciiiing Bayesian network structure learning algorithm. Machine Learning, 65:31-78, 2006.
Ignavier Ng, AmirEmad Ghassami, and Kun Zhang. On the role of sparsity and DAG constraints for learning linear DAGs. Advances in Neural Information Processing Systems, 33:17443-17954 2020.
11

---

Kevin Bello, Bryon Aragam, and Pradeep Ravikumar. DAGMA: Learning DAGs via M-matrices and log-determinant acyclicity characterization. Advances in Neural Information Processing Systems, 35:8226-823) 8239,, 2022.
Dennis Wei, Tian Gao, and Yue Yu. DAGs with no fears: A closer look at continuous optimization for learning Bayesian networks. Advances in Neural Information Processing Systems, 33:3895-39066 2020.
Yue Yu, Jie Chen, Tian Gao, and Mo Yu. DAG-GNN: DAG structure learning with graph neural networks. In Proceedings of the 36th International Conference on Machine Learning, volume 97, pages 7154-7163, PMLR, 2019.
Sébastien Lachapelle, Philippe Brouillard, Tristan Deleu, and Simon Lacoste-Julien. Gradient-based neural DAG learning. arXiv preprint
2019.
Chang Deng, Kevin Bello, Bryon Aragam, and Pradeep Kumar Ravikumar. Optimizing NOTEARS objectíves via topological swaps. In Proceedings of the 40th International Conference on Machine Learning, volume 202, pages 7563-7595. PMLR, 2023.
Luis De Campos and Javier Castellano. Bayesian network learning algorithms using structural restrictions. International Journal of Approximate Reasoning, 45(2):233-2544 2007.
Cassio P De Campos, Zhi Zeng, and Qiang Ji. Structure learning of Bayesian networks using constraints. In Proceedings of the 26th Annual International Conference on Machine Learning. pages 113-120, 2009.
Eunice Yuh-Jie Chen, Yujia Shen, Arthur Choi, and Adnan Darwiche. Learning Bayesian networks with ancestral constraints. Advances Neural Information Processing Systems, 29, 2016.
Robert Castelo and Arno Siebes. Priors on network structures. Biasing the search for Bayesian networks. International Journal of Approximate Reasoning, 24(1):39-57 2000.
Giorgos Borboudakis and Ioannis Tsamardinos. Scoring and searching over Bayesian networks with causal and associative priors. arXiv preprint 1408 2057, 2014.
Xiangyu Sun, Oliver Schulte, Guiliang Liu, and Pascal Poupart. NTS-NOTEARS: Learning non- parametric DBNs with prior knowledge. In Proceedings of the 26th International Conference on Artificial Intelligence and Statistics, pages 1942-1964. PMLR, 2023.
Zhichao Chen and Zhiqiang Ge. Directed acyelic graphs with tears. IEEE Transactions on Artificial Intelligence, 4(4):972-983. 2022.
Haoyu Yang, Roshan Tourani, Jia Li, Pedro Caraballo, Michael Steinbach, Vipin Kumar, and Gyorgy Simon. Causal structure learning from imperfect longitudinal data in healthcare. Proceedings of the 11th International Conference on Healthcare Informatics, pages 1-11. IEEE, 2023.
Zidong Wang, Xiaoguang Gao, Xiaohan Liu, Xinxin Ru, and Qingfu Zhang. Incorporating structural constraints into continuous optimization for causal discovery. Neurocomputing, 127902, 2024.
Po-L.igg Loh and Peter Bühlmann. High-dimensional learning of linear causal networks via inverse covariance estimation. Journal of Machine Learning Research, 15(1):3065-110) 2014.
Peter Bühlmann, Jonas Peters, and Jan Ernest. CAM: Causal additive models, high-dimensional order search and penalized regression. The Annals of Statistics, 42(6):2526 - 2556, 2014. doi:
A0.1 URL https:::/// htops://ddar w/t.///oooooo//o /tt//i..og//////// 10. 1214/14-A011260
Daphane Koller. Probabilistic graphical models: Principles and techniques, 2009.
Alexander Reisach, Christof Seiler, and Sebastian Weichwald. Beware of the simulated DAG! Causal discovery benchmarks may be easy to game. Advances in Neural Information Processing Systems,
34:27772-27784, 2021.
Chang Deng, Kevin Bello, Pradeep Kumar Ravikumar, and Bryon Aragam. Markov equivalence and consistency in differentiable structure learning. In Advances Neural Information Processing Systems, volume 37, pages 11756997777, 2024.
12

---

Shohei Shimizu, Patrik O Hoyer, Aapo Hyvärinen, Antti Kerminen, and Michael Jordan. A linear non -an-Gausiian acyclic model for causal discovery. Journal of Machine Learning Research, 7(10), 2006.
Patrik O Hoyer, Shohei Shimizu, Antti J Kerminen, and Markus Palviainen. Estimation of causal effects using linear non-Gaussian causal models with hidden variables. International Journal of Approximate Reasoning, 49(2):36--378 2008.
Arthur Gretton, Peter Spirtes, and Robert Tillman. Nonlinear directed acyclic structure learning with weakly additive noise models. Advances Neural Information Processing Systems, 22:1847-1855, 2009.
Jonas Peters, Dominik Janzing, and Bernhard Scholkopf. Causal inference discrete data using additive noise models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(22): 2436-2450, 2011.
Asish Ghoshal and Jean Honorio. Learning linear structural equation models in polynomial time and sample complexity, In Proceedings of the 21st International Conference on Artificial Intelligence and Statistics, pages 1466-1475, PMLR, 2018.
Ilyes Khemakhem, Ricardo Monti. Robert Leech, and Aapo Hyvarinen. Causal autoregressive flows. In Proceedings of the 24th International Conference on Artificial Intelligence and Statistics, pages 3520-3528. PMLR, 2021.
Weilin Chen, Jie Qiao, Ruichu Cai, and Zhifeng Hao. On the role of entropy-based loss for learning causal structure with continuous optimization. IEEE Transactions Neural Networks and Learning Systems, 36(1):1594-1608, 2025.
Ignavier Ng. Shengyu Zhu, Zhitang Chen, and Zhuangyan Fang. A graph autoencoder approach to causal structure learning. arXiv preprint arXiv:1911.0744 2019.
Shengyu Zhu, Ignavier Ng. and Zhitang Chen. Causal discovery with reinforcement learning. In Proceedings of the International Conference on Learning Representations, 2020.
Xiaoqiang Wang, Yali Du, Shengyu Zhu, Liangjun Ke, Zhitang Chen, Jianye Hao, and Jun Wang. Ordering-based causal discovery with reinforcement learning. arXiv preprint arXiv:2105.06631, 2021a.
Olivier Goudet, Diviyan Kalainathan, Philippe Caillou, Isabelle Guyon, David Lopez-Paz, and Michele Sehag. Learning functional causal models with generative neural networks. Explainable and Interpretable Models in Computer Vision and Machine Learning, pages 39-80, 2018.
Diviyan Kalainathan, Olivier Goudet, Isabelle Guyon, David Lopez-Paz, and Michèle Sebag. Struc- tural agnostic modeling: Adversarial learning of causal graphs. Journal of Machine Learning Research, 23(219):1-62 2022.
Xun Zheng, Chen Dan, Bryon Aragam, Pradeep Ravikumar, and Eric Xing. Learning sparse nonparametric DAGs. In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics, pages 1414-34255 PMLR, 2020.
Yinwei Wei, Xiang Wang. Liqiang Nie, Shaoyu Li, Dingxian Wang, and Tat-Seng Chua. Causal inference for knowledge graph based recommendation. JEEE Transactions on Knowledge and Data Engineering, 35(1 11:1111111111114 2022.
Giorgos Borboudakis and Ioannis Tsamardinos. Incorporating causal prior knowledge as path- constraints in Bayesian networks and maximal ancestral graphs. arXiv preprint 12X66v1206.6331 2012.
Andrew Li and Peter Beek. Bayesian network structure learning with side constraints. In Proceedings of the International Conference on Probabilistic Graphical Models, pages 225-236 PMLR, 2018
Zidong Wang, Xiaoguang Gao, Yu Yang, Xiangyuan Tan, and Daqing Chen. Learning Bayesian networks based on order graph with ancestral constraints. Knowledge-Bond Based Systems, 206:10 2021b.
13

---

Jiaxuan Liang, Jun Wang, Guoxian Yu. Wei Guo, Carlotta Domeniconi, and Maozu Guo. Directed acyclic graph learning on attributed heterogeneous network. IEEE Transactions on Knowledge and Data Engineering, (8110):00455-08855 10856, 2023.
Karen Sachs, Omar Perez, Dana Pe' er, Douglas A Lauffenburger, and Garry P Nolan. Causal protein- signaling networks derived from multiparameter single-cell data. Science, 308(5721):553--222 2005.
Oliver J Rutz and Randolph E Bucklin. A model of individual keyword performance in paid search advertising. Available at SSRN 1024765, 2007.
Marco Scutari, Phil Howell, David J Balding, and Ian Mackay. Multiple quantitative trait analysis using Bayesian networks. Genetics, 198(1):12-1137, 2014.
Juliane Schäfer and Korbinian Strimmer. shrinkage approach large-scale covariance matrix estimation and implications for functional genomics. Statistical applications in genetics and molecular biology, 4(1), 2005.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The main contribution of this paper is propose a method counteract edge weight deviation in the structure learning optimization process. We confirm that this is stated in both the abstract and introduction.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of the proposed method in the Appendix
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sellsspeciiiationn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer: [Yes]
Justification: We confirm that complete proofs are provided for all theoretical results used, detailed in the Appendix B
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: The proposed method understandable and reproducible, and we will provide the relevant datasets and code.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
16

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
We provide access to the code and datasets used.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ praip..cc/pbblicguuu.ee CoodeSubeeieeeeee ssi iessonPolll mollicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide descriptions of the experimental settings and details in the Experi- mental Setup subsection under Experiment section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: For all major tables and figures in the paper, we have included error analysis error bars, with their specific interpretations explained in the corresponding sections.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
17

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: the Experimental Setup subsection of Experiment section, we provide the detailed computer configurations.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics fttishitttte///rrpps../p//////// wtc.c//ubbc//hhccsssiddel.....l.
Answer: [Yes]
Justification: We confirm the NeurIPS Code of Ethics and ensure that we do not violate them.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This paper focuses solely on fundamental methodological research in causal discovery, and its societal impact is expected to be minimal.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
18

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
The article does not pose such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The dataset and base model we use are both clearly publicly available. For the dataset, have explicitly provided the citation URL; for the base model, we have given clear citation details.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
19

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
20

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
The paper does not use LLM.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
21

---

Appendix
A
Extended Related Work
This section provides more detailed discussion of Section 2 in the main paper. Subsection A. discusses the evolution of structure learning, while Subsection A.2 explores various methods and classifications aimed at enhancing the quality of structure learning through the integration of prior knowledge.
A.1 The Development of Structure Learning
Structural learning seeks delineate and comprehend the causal relationships between variables to enhance accuracy and reliability in prediction, decision-making, and intervention strategies. Numerous methods have been devised to unearth the true DAG structure from observational data. Constraint- methods like PC [Spirtes et al. [001]] and FCI [Colombo et al. 2012] infer DAG structures through conditional independence tests between variables. Score-based methods evaluate the fit between candidate DAG and the data using scoring functions. The hybrid methods, such as MMHC (Tsamardinos et al.) [006). merge both conditional independence tests and score-based methods. Additionally, some studies leverage data attributes such as non Non--aussianity. nonlinearity, and discreteness to investigate the DAG structure, as noted in several studies [Shimizu et al. 20066
Hoyer et al. 2008 Gretton al. 2009
Peters et al. 2011 Ghoshal and Honorio 2018 Khemakhem
al.
2021
Recently, Zheng et al. 2018 introduced NOTEARS, transforming the combinatorial search into continuous optimization problem for the first time. This innovation streamlines the handling of acyclic constraints and the search of the vast DAG space, enabling the use of deep learning optimizers in the search process. Inspired by this breakthrough, subsequent research has expanded on continuous optimization methods for structure learning [Chen et al. [025]. For instance, (Wei et al. 2020] expanded the algebraic characterization of DAG acyclicity
broader class of matrix polynomials,
while [Ng et al. 2019) developed GOLEM, which employs soft acyclicity constraints to enhance optimization performance. [Bello et al. 2022 introduced DAGMA, noted for its efficacy in detecting longer cycles and providing better gradients and faster performance.
Extensions of continuous optimization have also been adapted diverse scenarios. For instance, (Zhu it al. 2020) and [Wang et al. 22021a] applied reinforcement learning to search for DAG structures, whereas Yu et al. [019)] employed graph-based deep generative models to capture the variable distributions using the variational inference mechanism's 's evidence lower bound as an alternative to
traditional scoring functions. [Lachapelle al., [019] utilized neural networks to model nonlinear
relationships between variables, among other innovative approaches [Goudet et al. 2018 Kalainathan
al. 2022 Zheng et al., 2020).
Additionally, some studies have focused on refining the optimization mechanisms NOTEARS. (Wei al. 2022) explored the KKT optimality condition to better understand the behavior of NOTEARS
and related algorithms. [Reisach et al., [2021] demonstrated that the performance of certain continuous
structure learning methods might be attributed to the consistency between the increase in marginal variance and causal ordering. Continuing this line of inquiry. [Deng et al. [023] introduced a likelihood-basse scoring function that incorporates quasi-MCP regularization, offering scale-invariant characteristics.
A.2 Structure Learning with Prior Knowledge
In practical applications, the scarcity of data and the presence of noise often hinder accurate results, prompting many studies to integrate human expertise or findings from previous research as prior knowledge into structure learning. There have been many works on structure learning under the combinatorial search framework.
Some studies enforce that the learned structure strictly adheres to prior knowledge, known as hard constraints. For instance, [De Campos and Castellano] 2007) adapted both the score-based hill climbing algorithm and the constraint-based PC algorithm to produce graphs consistent with hard constraints. Similarly, the Branch Bound algorithm (De Campos et al.) 22099] specifies mandatory and prohibited edges. More recently, some researchers have utilized ancestry as a hard constraint.
22

---

[Chen et al. 2016 combined the branch reduction capability offered by the prior with the A search method to explore the application of hard constraints precise searches. Related studies include [Borboudakis and Tsamardinos 2012 Li and Beek 2018 Wang et al., 2021b].
Alternatively, soft constraint methods use prior knowledge guide the structure learning process. [Castelo and Siebes] 2000] assessed candidate DAGs based on prior knowledge concerning the pres- ence or absence of edges between variables, as specified by experts. [Borboudakis and Tsamardinos) 2014) employed the joint prior probability distribution of ancestral relationships to score candi- date DAGs. (Eggeling et al., 2019] conducted a broader evaluation of prior knowledge on general graph characteristics, such as restrictions on maximum in-degree and balancing different in-degree probabilities.
In the realm of continuous optimization, efforts to integrate prior knowledge continue, which can also be categorized into soft and hard constraint methods. Hard constraint methods focus on learning edge weights within a specified feasible range. For example, (Sun et al. [023] utilized the L-BFGS-B optimization algorithm set boundaries on weighted adjacency matrix parameters by establishing prior upper and lower bounds. [Chen and Ge| [022) improved the NOTEARS framework using mixed integer linear models, incorporating prior knowledge through logical constraints and series of equality'nequality constraints. Similar approaches were also adopted by [Ramsey et al. 2018
and [Liang al. 2023
Soft constraint methods augment the original objective with a new loss function measuring DAG AAGor consistency and optimize continuously during the learning process. For instance, (Yang et al. 2023 constructed a loss function tied to a prior matrix that represents edge priors, ensuring that the wei weighter weighted adjacency matrix, when masked by the prior matrix, closely resembles the prior matrix include indicated edges. Similarly, [Wang et al. 20224) apply the ReLU function to the prior mask to optimize the degree which edge weights fall below fixed threshold, and this approach could be extended to softer activation functions. [Bello et al., 2022] manipulated the gradient of the objective function directly. They adjust the impact of individual elements on the sparse penalty term during updates based on prior knowledge to ensure specific edges are included in the result.
While these methods can generate DAGs qualitatively consistent with prior knowledge, they often overlook the nuanced learning of edge weights during optimization, which is a key distinction between continuous optimization and combinatorial search. Hard constraint methods exclude edge weights below a certain threshold to meet edge priors by setting feasible intervals, but they may fail to discern the positive or negative effects between variables, potentially leading inappropriate weight adjustments during optimization. The effectiveness of soft constraint methods relying on prior loss varies with the loss function's stringency. Mild loss functions may struggle shift the data's structural tendencies, making it challenging to satisfy prior knowledge. Conversely, when the loss function strongly emphasizes prior satisfaction, may excessively increase corresponding edge weights, diverging from the true weights.
In summary, while these methods offer avenues to integrate causal knowledge into continuous objective functions, the resulting edge weights can significantly deviate from the ground truth. This discrepancy may have severe implications. To address this, our paper proposes posterior strategy that examines the outcomes when the prior is either too strong or too weak, offering insights into how prior knowledge should guide learning.
B Proof of the Lemmas in the Main Text
The theoretical framework presented in this paper is grounded in the linear SEM, a widely adopted model for causal discovery. The validity and interpretation of our results rest on several key assump- tions, which we delineate here for clarity.
Assumptions for the Linear SEM (Eq. 1
Our model, defined as X = T + e, is based on two primary assumptions. First is "Linearity," which posits that the causal relationship between a variable X; and its parents Pa(X,) linear combination, weighted by the elements of the true adjacency matrix B. While a simplification of many real world processes, this assumption provides crucial tractability and interpretability. Second is "Acyclicity," which requires the underlying graph structure to be DAG. This is comnersoone
23

---

assumption in most graphical causal models, ensuring that variable cannot be its own ancestor and thus precluding feedback loops.
Assumptions for the Scoring Functions (Eq.
The optimization objective for learning candidate graph W relies scoring function, such as (W: XX) or (w; The choice between these scores implies assumptions about the noise distribution. The least squares score, Qia is statistically equivalent to the (negative) log- likelihood under the assumption of independent and identically distributed Gaussian noise terms, Le., c ~ (O,, o²) for all i. The more general negative ligeleilioood score, Qnit, relaxes this by allowing for differing variances across noise terms, i.e., G ~ (O, of).
Furthermore, we wish to emphasize that the core deviation propagation mechanism identified in our work appears robust even when these statistical assumptions are not perfectly met. As demonstrated in our experiments with non-Gaussian noise (e.g., Exponential, Gumbel), PGAP consistently pro- vides improvements. This suggests that the identified triangular and double collision patterns are fundamental indicators of prior-induced error, and their utility is not strictly confined to scenarios where the model perfectly specified. This robustness a key strength our proposed framework.
Additional Notations and Formulations
Before proving the lemmas, first introduce notations not used in the main text. Given that G is an acyclic graph, we can establish a topological order for the variables in X, ensuring no edges from lower to higher order variables. This ordering implies that B can be represented triangular matrix. According to Eq. (T). random variables X can be expressed as a linear combination of noises E:
strictly upper
x= (1- B²)-
(13)
The matrix 1 - B is invertible due to its upper triangular structure with non-zero diagonal elements. Let E represent the covariance matrix of X, Σ Cov(X () = E[XX"]. is related to Q as follows:
== (T B) TT( - B)-
(14)
These equations elucidate the relationship between the causal structure B, the noise characteristics Ω and the resulting covariance Σ. The inverse of the covariance matrix, 0 Σ E-', is known as the precision matrix. Submatrices of Σ and corresponding to specific subsets of variables play key role the derivation.
B.1 The Proof of Lemma
Lemma 1. Given node Xj, assume that 17 represents the of predecessors of X; consistent with an order of the true DAG G. The regression coefficient obtained by linearly regressing Xj on 13 corresponds directly to the relevant element in the adjacency matrix B of the true DAG, i.e.:
Be = arg min W WGs (X,:Wes - -X,)
(15)
Proof. To establish the claim, we need to demonstrate that the optimal weight vector in the minimiza- tion problem, which corresponds to the OLS solution, is equal to Bıj.y. The OLS solution is given by ( We need to show:
BBzs = (Zig.ez) Egzs
(16)
Given that tj is consistent with the order of the true DAG G. and utilizing the SEM property
Σ = (T B) -)-T B1- we have:
Eeg, (I --B1,,3) T (1 12z,;))
(17)
24

---

Additionally, the covariance between X,; and X, is given by Egys expressed as:
Xy It can be further
Ezjs = x
E (1 Bi Biy,1z)
exze ([-B))-o ej
(18)
(T- 1--Bi Bez,1z)
(og 0) B) ej
where ej the standard basis vector corresponding node Xy. Thus, the regression coefficients can be obtained as follows:
(2g,45) Egjst
-T (Sy; 0) (1-1B) B)= ej = Bij,ez 1r] (Rt; 0) (1- B)
-(1 Bij,3y 10 (1 - Bey,;)) (1 (1-Biaa3)
(19)
= - B1
- -
=B
Hence, we complete the proof of Lemma
B.2 The Proof of Lemma 2
Lemma 2. Let = W3.j Wi. represent the deviation of the prior-influenced weight Wa.s (derived from from the optimal OLS weight Wä,; obtained when regressing X, onto the set of predecessors t, (derived from (9). The difference between the prior-inftuence weights for other incoming edges W and W is a linear function of expressed as:
W W.5 Wi,- 03,9
02 8.8
(20)
where 0 is the inverse of the covariance matrix of the variable set lz.
Proof. According to the definitions of ordinary least squares (OLS) estimates in the main text, the optimal weights when regressing X, on X,,
W arg min (X,, W X,)² Ways
(21)
The weights for X1 when B.j is fixed to Was are denoted We, -B.j"
( + X - X,) 2
- arg min Wey-Bo
(22)
The OLS solutions of Eq.
21 and
can be expressed using covariances:
Wi (Et,,t)) E
(23)
We, 1 Bs (Ee,-A.ty =) Cov(Xe, -B - Xz))
(24)
where
Cov(X, -p X, Wa Wg.gXp3 En,- -i, Er,-888
(25)
So,
WG,00 = (2i, (Zi,-p.s W Wo.
(26)
Let - be the components of Wi,,; corresponding to b B. We consider the case when W is set to the optimal OLS weight W plus deviation ?
p.j
W W B
(27)
25

---

The difference we want to calculate is W - W 1,5
Wi--s- Wi, -
- -- -WWks + Wis-
(28)
-Et -1 (Z) - W ر En,- Wi - (Et,-Bat, Eng - -8,8
The term in the square brackets is zero because it represents the difference when 7 0, i.e., Wa, Wa. In this case, W should equal W 15- (This assumes that regressing X; - W X onto -ty-9 gives the same coefficients W 1.- as obtained from the joint regression on X,, This holds due to properties of OLS.) Therefore,
W - - -7 (Et, -
(29)
The Eq. (29 shows that the difference is linear in 7. We proceed by calculating this difference in more detail. Define the following matrices for convenience:
M Egy,ty+ A1 Ep,e, A Ep,t As D1y-0,3 A1 -
(30)
Thus, the difference can be rewritten as:
W W W = ((A,)
(31)
In the above equation, the most difficult thing to deal with is the inverse of A4. To this end, we first assume that and then introduce the Schur complement, which provides with:
S-1 A3S
A2 A2A
M M-1
(32)
AT
where S is the Schur complement of M with respect to As, defined as:
S= Ai A2A7 Aa
(33)
From the conclusions of Eq. 32
(M 12:j 12:j1112--11 A + A1i
(M 2:1-1,1 - A35
(34)
1,2--1 -5-1
Using these conclusions, we can get the expression of the inverse of A4:
AT 1 (M - 21 1M (AM
(35)
-M- = (M 1M - (M 1,, (M -M--,,,,MM--11------ (
When 3 * 1, the permutation matrix P can be used to transform M to the case where 8
M = PE pT
(36)
Therefore the inverse of M becomes:
M-1 = (PEN, PT)-1 P(
(37)
The second equality in the above equation is due to the fact that the permutation matrix P has the following property:
p P
(38)
Therefore obtain:
A (M (M- (1-1))2.-1122211 (M 12 1 "1M -,1)
(39)
= -8, -15-3,588.. -,6 (6
where
0 == (Ei
(40)
26

---

Therefore:
AT (0t,-,, 81, (0.-8,8081 - 11) Er, -998
(41)
Next, we calculate the components separately:
First Term T
(1,-5011,22 -,,,
= - (1 Be - eg
(42)
Second Term- - ' - -8,8
(43)
= - (1 -, -,0,0
- (05,3) 28
Consequently, by combining these two terms, obtain:
A. As 0,, -3,0 (0
(44)
Finally, the difference in the estimates is expressed as:
W -,j 7 A3 = 03.3
0, 0.8
(45)
This concludes the proof, showing that the difference between the adjusted and original weights is a linear function of the deviation y, scaled by the term derived above.
B.3 The Proof of Lemma 3
Lemma 3. Assume t; (1, (1.......j-.......... 1}. of X corresponding
and let 0 be the inverse of the submatrix of the covariance
the variable set 1j. Then the condition under which 05,9 is non-zero is given by: BR + 0 or Xạ and X₃. have a common child node X;, when 3; Ba k # 0 or X3 and X have a common child node X,, when h >>>>>>>>> =B.
Proof. Given that tú aligns with the true DAG order (say t, = {1, 21.2....., j - 1)), the inverse covariance matrix (Er;.y) can be expressed terms the true adjacency matrix and noise variances Ω = diag(o?. (iaago) oi):
0 = (I B1- ---1),b--11 (1 B --
(46)
This leads specific formulation for the components of 0:
BA. -111
Ba Br
1: < 3
11=
0..9
(47)
Bi E1
B < <<j
The analysis now splits based on the relative values and B. Case 1: k < B. From Eq. (47). 0g.9
will be non-zero if: (1) The direct edge weight Bg,s is non-zero. If Bg,s # O, the first term Be,
-3
contributes a non-zero value (assuming o3 > 0). (2) The common child term non-zero. If X,g and X₁ have a common child node X; such that E { + 1,..... - 1}. then B3 + 0 and Bu + In this case, the summation term 2i=
Ba. Ba. is non-zero. o?
Ox.s will be non-zero if: (1) The direct edge weight Bg is non-zero. contributes a non-zero value. (2) The common child term is non-zero. If X and X₃ have common child node X; such that i E {A + 1,..., 1}, then В3 # 0 and B&g # O. The summation term Ei Bs. will be non-zero.
Case 2: l: B. From Eq. 47
Ba, # O, the first term
B2 of
Combining these conditions yields the statement in the lemma. The exclusion of a Lebesgue zero- measure set corresponds the faithfulness assumption, ensuring that terms don't 'nn't accidentally cancel to zero.
27

---

B.4 The Proof of Lemma 4
Lemma 4. Assuming the tree DAG G adheres to the Erdös-Rényí graph model G p). Assuming that the prior knowledge leads to the weight deviation " which is sufficiently large, then the expected number of triangular patterns resulting from Y approaches 3(d 2)p, and the number of double collision patterns approaches & {d - - 3))p
Proof. Assume that the prior knowledge producing the weight deviation is (X,+ - Xj). To calculate the expected number of triangles generated by a given deviation y, we denote this expectation as Xz. 7). This expectation can be decomposed into expected sub-events:
E E #Tringle Xg
E Trianige
Xa Xj:7)
Σ E (Trianglegs) kAj
X5 Xj,7)
(48)
For cases where & < B:
E (Triangle pkj π X3 X,,3) P(Xx X3) > th) sufficiently large And for cases where 3:
p
(49)
sufficiently large
E' (Triangle
| π X3 + Xj+3) = P(X3 + X2) > th)
P
(50)
Thus, for the prior knowledge X, Xj. the expected number of triangles converges to:
Er# π X3 X,.7) - (j 2)p
(51)
To find the expectation over all possible prior knowledge π in the graph, compute the expectation:
d
j-1
Triange n. 7) = C2
E #(#TTrane Triangle π = X3 + X,, -// (d 2)p
(52)
j=2 1-11 8=1
Next, we address the expected number of double collision patterns, defined as:
El# Collision
X3 X).7)
(53)
ΣΣ>
Σ
E (Double Collisionaaai π X33 - x),7)
kcj kBBiiej
For cases where & < B:
Σ E (Double Collision x === Xg + +X,,7)
(54)
K
=(j - ( XX) X;)P(>> th)
7 sulliciently large
(j 13
And for cases where 3:
Σ E (Double Collisiongkij | Xg + +x,,7)
K
(55)
1 1 X,) + X,) th)
7 sufficiently large
(j 1- k)p²
Summing the contributions for all the expected number of double collisions for prior knowledge
Xg + X, is:
E B(#Doute Double Collision
= =x *,,7)
3-1
1-1
1
(56)
Σ 1 11-1 k-pp = 7 - 1 4)²²
1=1
k=9+1
Finally, calculate the total expectation:
Double Collision 7,7)
1
Σ Σ
E( # Double Collision π Xg + xx,7)
1
2)(d =3)p
(57)
C2 8=1
6
Hence, complete the proof of Lemma 4.
28

---

B.5 The Proof of Lemma
Lemma 5. In the ER graph G(d. p), the expected number of natural Iriangular patterns and double collision patlerns is gdlcd 1)(d 2)p² and 2z - 1)(d 2) - -3)p", respectively.
Proof. We consider the probability that any three nodes in DAG form triangle pattern. Given the acyclic nature of DAGs, we assume topological order for the nodes. Therefore, for three nodes b, arranged in order in the Erdös-Rényi graph model G(d. p). the probability that an edge exists between each pair of nodes is leading to combined probability of p³ for three interconnected nodes. Therefore, the expected number of triangle patterns in the DAG then calculated as:
E (riinngg) C. p
d(d 1)(d 2)p²
(58)
Similarly, for any four nodes a, b, c, and arranged in a DAG, the probability that nodes and simultaneously connect nodes and d in a structure resembling two collisions is p². Thus, the expected number of double collision patterns is:
1 Doublle Collision) C x p² d(d 1)(d 2)(d 3)p 3)p
(59)
24
C Algorithm for Pattern-Guided Adaptive Prior
This section details the implementation of the Pattern-Guided Adaptive Prior (PGAP) Framework. As general strategy, PGAP integrates three key steps: (1) learning DAG from data using a base structure learning algorithm, (2) detecting specific graph patterns that indicate potential prior-induced erros, and (3) adaptively adjusting parameters within the base learning algorithm mitigate these errors. The adaptation method varies depending on how prior knowledge is incorporated.
C.1 Implementation for Soft Prior Constraints
When prior knowledge is introduced as soft constraints via prior loss term in the objective function, the adaptive adjustment in PGAP involves modifying the weight of this prior loss term. Algorithm outlines the procedure.
Algorithm 1 PGAP with Soft Prior Constraints
1: Input: Data matrix X, set of prior knowledge II, initial prior loss weights og (vector with elements 00,z for each æ E IT), pattern adjustment step size a.
2: Learn Wstagel using base structure lcarning algorithm, incorporating the prior loss term LP(W; ap).
3: Initialize a1 + ag.
4: for each prior = (Xg + X,) E do
5: Appaterns t # of triangle and double collision patterns in W Wangel involving क
6: 011, f- max(0, Cg
Pppaerms)
7: end for
8: Learn Winal using the base structure learning algorithm, incorporating the prior loss term Lp(W;; II,
9: Return Wrnnal
In this procedure, (rWW; II, o;) represents the total prior loss, typically sum of individual loss components for each prior:
LP(: Σ ((;
(60)
where 0r is the weight for prior Its specific form is problem-dependent, but it should satisfy:
Lp(W; + 0 + prior edge exists in the learned G
(6
29

---

The "base structure learning algorithm" refers to chosen continuous optimization method (e.g.. based on NOTEARS) that minimizes scoring function Q(W :X) subject to acyclicity h(W) augmented with the prior loss Lp and a sparsity-inducing regularizer.
In this implementation, the weight of the prior loss term L. is adaptively reduced when deviation- indicating patterns are detected. This lessens the influence of the prior, allowing the base structure learning algorithm more freedom adjust edge weights based on the data.
C.2 Implementation for Hard Prior Constraints
When prior knowledge is enforced using hard constraints (e.g.. restricting edge weights to a spe- eific range), the adaptive adjustment in PGAP involves modifying the local regularization strength. Algorithm outlines this procedure.
Algorithm 2 PGAP with Hard Prior Constraints
1: Input: Data matrix X, set of prior knowledge II (used as hard constraints), initial regularization matrix Ao (with elements Aolk: j] e.g., 0.1), regularization adjustment step size w, maximum regularization strength Amax-
2: Learn W sugel using base structure learning algorithm with hard prior constraints and regular- ization matrix Ag-
3: Initialize A₁ f Ag-
4: for each prior
(Xp + X,) 1l do
5: TApatterns f- # of triangle and double collision patterns stagel involving the т
6: if "paaterns > 0 then
7:
for each k + do
8:
A1 A1[k.j] + min(Aolk:,)) + Pppatemss , Amax)
9: end for
10: end if
11:
end for
12:
Learn Wina using the base structure learning algorithm with hard prior constraints II and the adjusted regularization matrix A1.
Return W
Here, the regularization coefficient adaptively increased for edges suspected of being spurious (i.e., participating in the detected patterns). This encourages these edge weights towards zero, promoting a sparser and more accurate graph.
In the experiment, each element of the regularization matrix A starts at 0.1 with an upper limit of 0.2, and the regularization adjustment step size w is 0.02.
D Complete Experimental Results and Analysis
NOTEARS-2d
NOTEARS-PGAP-2d
NOTEARS-20M
NOTEARS-PGAP-ZN
Ga
Exp-EV
Gumbel-EV
Gauss-NV
Exp-NV
Gumbel-NV
1.3
Scaled SHD
1.0
ER2
Scaled SHD
0.7
ER2
0.7
0.4
0.9
0.9
SF2
0.7
0.7
0.5
0.5
20
40
20
40
6020
40
60
20
40
6020
40
6020
40
60
d (Number of Nodes)
d (Number of Nodes)
Figure 4: Performance of PGAP across Different Graph Settings (ER=2).
30

---

NOTEARS-2d
NOTEARS-PGAP-2d
NOTEARS-20d
NOTEARS-PGAP
Gauss-EV
Exp-EV
Gumbel-EV
Gauss-NV
Exp-NV
Gumbel-NV
1.1
1.2
BR4
ER4
Scaled SHDD
0.9
Scaled SHD
0.9
0.7
0.6
0.8
0.8
Fals
0.6
0.6
0.4
0.4
20
40
60.00
40
60
60
40
60
40
60
60
(Number of Nodes)
d (Number of Nodes)
Figure 5: Performance of PGAP across Different Graph Settings (ER=4).
NOTEARS
NOTEARS-PGAP
DataSize-2d
DaraSize-20d
ERI. DataSine=2d
DutaSize-0d
DataSiza-2d
140
Number Nodes d-20
Number of Nodes d 40
Number of d-60
3.5
70
110
Dat
DataSize-
DutaSize-0dd
DataSiam2d
180
250
75
1.50
210
0.1
0.2
0.3
0,40
0.1
0.2
0.3
0.4
0
0.1
0.2
0.3
0.40
0.1
0.2
0.3
0.4
0.1
0.2
0.3
0.40
0.1
0.2
0.3
0.4
Prior Proportion
Prior Proportion
Prior Proportion
Figure 6: Performance Comparison across Different Prior Proportion Values (Gauss-Nu case).
NOTEARS
DataSize-1d
DaraSize-10 104
DataSine-
DataNize-1Md
95
140
Number of Nodes 4-20
Number Nodes d-40 40
Number of Nodes 4-60
355
75
105
DataSize-1d
Dat
180
260
75
150
215
0.1
0.2
0,3
0,40
0.1
0.2
0.3
0.4
0
0.1
0.2
0.3
0.40
0.1
0.2
0.3
0.4
0.1
0.2
0.3
0.40
0.1
0.2
0.3
0.4
Prior Proportion
Prior Proportion
Prior Proportion
Figure 7: Performance Comparison across Different Prior Proportion Values (Exp-NV case).
NOTEARS
NOTEARS
DataSize-2d
DataSize-2M
ERI, DataSire-2d
DataNize-2dd
1335
Number Nodes 20-0
Number Nodes d-40
Number of Nodes d-60
35
65
105
50
DatsSize-2d
DataSize-2ti
130
260
75
150
215
0.1
0.2
0.3
0.40
0.1
0.2
0.3
0.4
0
0.1
0.2
0.3
0.40
0.1
0.2
0.3
0.4
0.1
0.2
03
0.40
0.1
0.2
0.3
0.4
Prior Proportion
Prior Proportion
Prior Proportion
Figure 8: Performance Comparison across Different Prior Proportion Values (Gumbel-NV case).
31

---

NOTEARS
+
NOTEARS-PGAL
Exp-EV, Gumbel-EV,
Gauss-NV, SF8
Exp-NV, SP8
Gumbe-NY, SES
500
420
SHD
450
360
400
300
0.1 0.2 0.3 0.40 0.1 0.2 0.3 0.4
0.1 0.2 0.3
0.4
0
0.1
0.2
0.3
0.40
0.1 0.2 0.3 0.40
0.1 0.2 0.3 0.4
Prior Proportion
Prior Proportion
Figure 9: Performance Comparison on Large-scale denser Data (Nodes=60, ER=8).
D.1 Performance of PGAP in Various Graph Settings
The main experiments (e.g., (igure 2 in the main text) have already established that PGAP demon- strates superior performance compared to the NOTEARS-Softmn haseline across a variety of settings for ER2 and SF2 graphs, including different numbers of nodes, dataset sizes, and multiple noise distributions (Gaussian-EV/NN, Exponential-EV/NY, Gumbel-EV/NV), using scaled SHD as the primary metric.
This appendix section provides further comprehensive details and extends this analysis to showcase PGAP's robustness and adaptability under additional conditions. The figures presented herein use scaled SHD, consistent with the main text. Specifically, this section expands on the findings by Presenting performance comparisons denser graphs (ER4 and SF4), detailed in Figure 5 (Figure 4 reiterates the result of FR2 and SF2 scenarios for completeness). We also investigate the impact of varying proportions of prior knowledge performance across different graph structures (ER graphs) and noise types (Ga Exp-NV, Gumbel-NV), as illustrated in Figures 6
7 and 8 Furthermore,
we also evaluate the performance PGAP on the denser graph with 60 nodes, as shown in Fig. 9
These extended results (Figures through 9 consistently show PGAP outperforming NOTEARS- Softmax. The enhancement in performance with increasing proportion of prior knowledge further suggests that PGAP not only effectively utilizes prior knowledge but also adapts more gracefully to its integration, alleviating the occasional performance degradation observed with the NOTEARS- Softmax baseline. This highlights the critical role of PGAP's mechanism for judiciously handling prior knowledge enhance rather than detract from the learning process.
D.2 PGAP on Prior Integrating Methods
The experiment in the main section demonstrated that PGAP enhances the performance NOTEARS- Hard, COTEARS-REL.U and NOTEARS-Softma. methods, with detailed results for the Gauss-NV noise case presented in Table D
This appendix further expands on these findings by providing performance data under additional non-equal variance (NV) noise conditions. reitea]esseeaatss the Gauss-NV results for completeness within this appendix. The truly extended results, showcasing PGAP's impact with Exponential-NV (Exp-NV) and Gumbel-NV (Gumbel-NV) noise, are presented in Table and Table respectively.
These comprehensive results (Tables 2 are important for assessing the robustness and adaptability of PGAP in handling different noise structures when applied to various prior integration strategies. The findings indicate that PGAP generally maintains or improves both Fl scores and SHD relative to the baseline methods across most configurations. This reflects PGAP's capacity to effectively adapt to different noise characteristics while beneficially integrating prior knowledge into the structure learning process.
D.3 Comparison of PGAP with Established Methods
The main text has discussed the comparison of the PGAP framework, when integrated with several established structural learning methods (NOTEARS, GOLEM, DAGMA, and NOTEARS-LogLA against their original counterparts. Figure in the main text provides a summary of these comparisons. This appendix section offers more comprehensive and detailed graphical breakdown of these performance comparisons across broader range of experimental conditions. The subsequent figures (Figures 10 through [5] illustrate these comparisons using SHD as the performance metric. These
32

---

Table 2: Improvements of PGAP on different prior integration methods (Gauss-NV case).
Data Size
2d
20d
ER
Metric
SHD
F1
SHD
Prior Proportion
0.2
0.4
0.2
0.4
0.2
0.4
0.2
0.4
NOTEARS-Han
33.6:2.1
48.2+2.0
40.2 40.2+1.0
34.8
35.1
48.
38.1#22 1:2.7
37.022.0
31,8+20
+PGAP
37.0#30
53.9+21
37.0+2.1
29.3+1.4
35,4:2.5
52.1+4.4
35.3
28.3+2.3
NOTEARS-ReLU
32.2=5.0
39.442.3
54.745.4
60.5+5.6
33.4
42.9+4.3
49.8
52.347.1
2
+PGAP
34.9=3.7
42.3+3.5
49.8+4.1
53.0+s.0
35.1+4.7
47.0#4.
46.8+4.9
42.0+9.4
POTEARS-Softman
39.023.9
48.942.6
37.843.4
35.72119
34.526.3
48.324.0
38.724.5
34.722.9
+PGAP
37.7
57.8
35.2*1.5
26.7+2.3
33,422.
56.4s5.1
37.5#18
26.3+2.9
NOTEARS-Hard
39.1
54.1+1.1
68.0+3.
54.8+2.7
37.0=1.9
55.0 1..0
70.3+2.7
54.0+2.1
+PGAP
39,8+3.7
55.5+2.0
66.244.3
52.2+3.5
37.6+2.9
55.5+2.4
69.3+1.
53.2+227
NOTEARS-ReLU
46.5#3.0
52.5+24
89.3+5.3
89.8+4.1
44.2+1.9
53.5+2.0
89.7#3.
82.3+5.7
+PGAP
46.9#3.0
53.2422
86.315.0
86.8+49
45,4
53.321.4
86.314.2
80.32:56
NOTEARS-Softna
38.8+3.5
55.9+1.8
72.84199
60.342.7
38.5+2.7
57.5+2.8
73.8+3.
57.8+443
+PGAP
40.424.0
56.9+1.5
69.0229
57.0+2.4
39.0#1.6
60.4+2.9
69.0#2.8
50.824.6
Table 3: Improvements of PGAP on different prior integration methods (Exp-NV case).
Data Size
20d
ER
Metric
F1
SHD
F1
SHD
Prior Proportion
0.2
0.4
0.2
0.4
0.2
0.4
0.2
0.4
NOTEARS-
40.627.6
49.5+1.6
35.345.3
32.241.5
35,8+2.4
51.744.0
37.7+12
31.0+18
+PGAP
39.724.3
52.4+2.1
34.244.1
29.3+1.4
35.912.7
53.023.
37.212.4
28.042.
NOTEARS- ReUL
32.623.0
41.624.0
54.7
57.8
33.8
43.0t3.1
51.5#22
54.3+t.1
+PGAP
30.4:4.5
44.5.t..
52.3*5.2
50.2*9.
35.2+2.
43.4
47.3#2.
50.7#26
NOTEARS-
33.227.0
47.216.5
41.715.6
36.7+5.7
33.3#36
48.4
40.8
35.8
+PGAP
37.1+53
54.3+6.0
37.743.9
27.543.5
35.8+4.1
61.1+5.2
37.3+2.6
23.843..
NOTEARS-Hard
39.5#1.
55.1
67.3+2.7
53.3+4.0
38.2+3.1
54.5#3.
69.8.1.3
55.242.6
+PGAP
39.5+3.5
57.3+22
67.542.5
51.0221
38.3#29
57.4
69.5a2.
51.214.5
NOTEARS- RRLU
46.723.0
54.7128
84.715.7
78.247.0
46.2
53.821.7
88.021.
83.325.1
+PGAP
47.923.1
55.542.7
81.724.5
75.726.3
47.4
54.4
84.526.
79.324.3
NOTEARS- -offaa
41.6#48
57.7*1.
69.7+6.9
58.2+2.6
37.7+18
57.0=1.0
74.8+3.3
59.2+3.
+PGAP
40.4#3.6
58.7+2.1
68.5+3.
52.7+2.
39.1+3.2
57.7
70.0
56.2425
Table 4: Improvements of PGAP on different prior integration methods (Gumbel-NV case).
Data Size
2d
20d
ER
Metric
F1
SHD
F1
SHD
Prior Proportion
0.2
0.4
0.4
0.2
0.4
NOTEARS-Hard
36,9#155
47.6#4.0
38.0+14
33.3+2.9
38.9#6.9
47.1
36.3#5.1
33.2+2.1
+PGAP
37.5+4.7
51.2448
35.823.6
30.0334
35.1+2.9
50.2+2.7
37.3
30.8
NOTEARS-ReLU
31.4+42
42,9+3.3
55.7+6.3
55.3+6.2
33.5+4.2
42,4+3.5
52.0=39
54.5+44
2
+PGAP
32.244.1
44.623.
52.326.2
49.8
33.924.3
43.243.2
49.822.9
50.324.5
NOTEARS-Softmax
32.6#416
48.7+3.1
41.842.5
37.343.3
34.2+4.9
49.747.0
40.5236
36.345.5
+PGAP
32.1#21
59.1
39.242.2
25.7+2.4
35.3+3.3
60.7+4.
38.2+2.0
24.52:3.
NOTEARS-Han
40.0#19
53,4+28
68.0+2.0
56.743,4
38.624.5
54.5#2.7
69.7.3.1
55,012.8
+PGAP
38.5#38
54.8
68.8t4.6
54.7#2,5
36.3218
55.7
70.2+29
52.2+2.
NOTEARS-ReLU
44.9+29
53.521.
89.843.8
86.7+8.1
43.9
53.2+2.2
92.7*6.0
85.2411
+PGAP
45.0=24
55.5+32
88.342.
81.747.1
45.3+2.3
54.1+2.3
88.026.5
82,2442
NOTEARS-Softmax
37.2+42
56.442.2
74.343.6
59.343.7
37.2+4
57.5+1.7
75.026.2
57.0427
+PGAP
37.323.
57.541.
70.243.0
55.5+1.7
38.8
58.3219
68.5458
53,842.2
33

---

figures cover various settings, including different noise distributions with Equal Variance (EV) and Non-Equal Variance.
These detailed results visually reinforce the findings from the main text, demonstrating that the PGAA-enhanced versions of the structural learning methods consistently tend to outperform their respective original versions across these diverse scenarios, highlighting PGAP's ability to foster superior utilization of prior knowledge.
Gauxy-EV, ER2, DataSize-204 20d
Gauss-EV, ER4, DataSize-20d
VARSORT
VARSORT
20
60
80
100
60
80
100
120
NOTEARS
Brales
NOTEARS
P0AM
+PGAP
Por
+PGAP
Method
GOLEM
GOLEM
PGAAP
+PGAP
DAGMA
DAGMA
PGAP
#PGAP
30
35
40
60
70
80
85
Structural Hamming Distance (SHD)
Structural Hamming Distance (SHD)
Figure 10: SHD Performance Comparison of PGAP with Mainstream Methods (Exp-EV case).
Exp-EV, ER2, 2ad
Exp-EV, ER4, DataSize-20d
VARSORT
VARSORT
50
60
70
80
90
100
70
80
90
100
110
120
NOTEARS
NOTEARS
+PGAP
+PGAP
Method
GOLEM
GOLEM
PPGAP
+PGAP
DAGMA
DAGMA
+PGAP
+PGAP
25
35
45
60
70
Structural Hamming Distance (SHD)
Structural Hamming Distance (SHD)
Figure 11: SHD Performance Comparison of PGAP with Mainstream Methods (Gauss-EV case).
Gumbel-EV, ERZ, DataSize-20d
Gumbel-EV, ER4, DataSize-2od
VARSORT
VARSORT
50
60
70
80
90
100
60
80
100
120
NOTEARS
KGs
NOTEARS
+PGAP
+PGAP
Method
GOLEM
GOLEM
+PGAP
+PGAP
DAGMA
DAGMA
+PGAP
+PGAP
30
35
40
45
50
55
60
65
70
75
80
Structural Hamming Distance (SHD)
Structural Hamming Distance (SHD)
Figure 12: SHD Performance Comparison of PGAP with Mainstream Methods (Gumbel-EE case).
D.4 Ablation Study on the Adaptive Reduction Parameter
The PGAP framework, when applied with soft prior constraints (as detailed in Algorithm 1), utilizes an "adaptive reduction parameter', denoted as a. This parameter determines the magnitude by which the weight of a specific prior in the loss function is reduced when deviation-indicating structural patterns
34

---

Gauss-NV, ERZ, DataSize=20d
Causs NV. DataSize-206
VARSORT
VARSORT
50
60
70
80
90
70
80
90
100
110
NOTEARS
Nexto
NOTEARS
+PGAP
+PGAP
GOLBM
GOLEM
Method
+PGAP
+PGAP
DAGMA
DAGMA
+PGAP
+PGAP
NT-LogLL
NT-LogLL
+PGAP
PGAA
20
30
40
50
60
70
80
90
Structural Hamming Distance (SHD)
Structural Hammiing Distance (SHD)
Figure 13: SHD Performance Comparison of PGAP with Mainstream Methods (Exp-NV case).
Exp-NV, ER2, DataSize=2d
Exp-NW, ER4, DataSize=
VARSORT
VARSORI
40
60
80
100
60
80
100
120
NOTEARS
NOTEARS
+PGAP
4PGAP
Fre 24
mer 14
GOLEM
GOLEM
Method
+PGAP
+PGAP
DAGMA
DAGMA
+GAAP
+PGAP
LT-LogLL
NT-LogLL
+PGAP
+PGAP
20
30
50
60
80
90
Structural Hamminn Distance (SHD)
Structural Hamming Distance (SHD)
Figure 14: SHD Performance Comparison of PGAP with Mainstream Methods (Gauss-NV case).
Gumbel-NV, ER2, 2ata
Gumbel-NV, ER4, DataSize-20d
VARSORT
VARSORT
60
100
50
60
80
90
100
110
NOTEARS
Bacii
NOTBARS
+PGAP
+PGAP
GOLEM
GOLEM
Method
+PGAP
+PGAP
DAGMA
DAGMA
+PGAP
+PGAP
NT-LoggL
LT
+PGAP
+PGAP
30
50
60
90
Structural Hamming Distence (SHD)
Structural Homming Distance (SHD)
Figure 15: SHD Performance Comparison of PGAP with Mainstream Methods (Gumbel-NV case).
ER2, Prior Proportion-0.2
ER2, Prior Proportion-0.4
Structural Hamming Distance (SHD)
55
NOTEARS
NGTEARS-FGAD
Structural Hamuming Distance (SHD)
MOTEARS
NOTEARS FCAP
40
40
30c
35-
30-
20
15
20
0.02
0.06
0.10
0.14
0.02
0.06
0.10
0.14
Adaptive Reduction Parameter
Adaptive Reduction Parameter
Figure 16: The Ablation Results for the Adaptive Reduction Parameter a (ER2).
35

---

ER4, Prior Proportion-0.2
ER4, Prior Proportion 0.4
Structural Hamming Distance (SHD)
NOTEARS
NOT NOTEARS PGAP
Structurai Hamming Dislance (SIID)
NOTEARS
NOTEARS-PGP
8,5
85
75p
75
651-
70
60
60
50
0.02
0.10
0.14
0.02
0.06
0.10
0.14
Adaptive Reduction Parameter
Adaptive Reduction Parameter
Figure 17: The Ablation Results for the Adaptive Reduction Parameter (ER4).
associated with that prior are detected during the initial learning stage (Wstagel). Understanding the sensitivity of PGAP's performance to this parameter is important for its practical application.
This ablation study investigates the impact of varying the adaptive reduction parameter a on the performance of PGAP, specifically when integrated with NOTEARS using a softmax- soft prior (referred to SETEARS-PGAR Figures 16 and T7 display the SHD for ER2 and ER4 graphs, respectively, under different prior proportions (0.2 and 0.4). In these figures, the numerous black scatter points represent the SHD values from individual runs of the NOTEARS with PGAP's adaptive mechanism.
The results show that NOTEARS-PGAN (solid red line) consistently achieves lower average SHD than NOTEARS results across a range of a values. For instance, an adaptive reduction parameter around 0.1 (the default value used in our other experiments) demonstrates robust performance, typically falling below the majority of the baseline scatter points. This indicates that PGAP's adaptive adjustment of prior loss weights provides a tangible benefit over a static prior weighting. While extreme values of might lead to diminishing returns or slight increases in SHD, the mechanism shows a favorable range of operation.
This study supports the conclusion that PGAP's adaptive approach to modulating prior influence based structural patterns is effective and not overly sensitive the precise setting of the adaptive reduction parameter within a reasonable range, generally outperforming the non-adaptive baseline.
D.5 Results on Real- World Data
To evaluate the practical applicability of our framework, we conducted experiments on four real- world datasets of varying scales and domains. The first the well-established Sachs dataset, which contains 11 nodes and 17 edges from measurements of protein and phospholipid expression levels [Sachs) jet al. 2005). We also include three additional datasets: the small-scale PaidSearch dataset (7 nodes, 6 edges) Rutz and Bucklin (2007). and two larger-scale datasets, Magic-niab (44 nodes, 66 edges) Scutari al. 2014) and Fcoli70 (46 nodes, 70 edges) Schäfer and Strimmer 2005).
Table[] the performance of three baseline methods (NOTEARS-Haree RReUU, and -Softmax) with and without PGAP enhancement on these four datasets, using Fl score and SHD as metrics with prior proportions of 0.2 and 0.4. The results demonstrate a consistent trend across all evaluated scenarios, Integrating PGAP generally leads improved outcomes, reflected in higher scores and lower SHD values. This robust performance across diverse real-world data underscores the practical value and generalizability of our proposed framework.
D.6 Experiments Nonlinear Situation
To further test the applicability of the proposed method in different scenarios, we also conducted experiments under nonlinear conditions, with the results presented in Table
36

---

Table Performance comparison on four real-world datasets: Sachs, PaidSearch, Magic-niab piggcc-nia,,o and
Ecoli70.
Dataset
Sachs
PaidSearch
Magic-niab
Ecoli70
Metric
SHD
Fl
SHD
SHD
Prior Proportion
0.2
0.4
0.2
0.4
0.2
0.2
0.4
0.2
0.4
0.4
0.2
0.4
0.2
0.4
NOTEARS- Hard
48.0
53.3
11
10
15,4
42.9
9
7
34,9
46.7
51
45
26.7
35.0
123
112
+PGAP
48.0
57.1
15.4
46.2
9
6
35.7
54.2
28.4
38.7
107
103
NOTEARS-ReLA.
48.0
55.2
10
13.3
11
35.3
42.7
23.0
29.5
161
176
+PGAP
48.0
57.1
11
16.7
42.9
8
7
37.2
53.2
41
27.9
35.7
117
99
NOTEARS-Sofhnn
37.0
48.5
13
13
30.8
40.0
8
8
35.3
44.9
51
46
20.6
29.0
155
176
+PGAP
40.0
519
33.3
46.2
7
6
37.2
54.7
25.2
36.0
113
108
Table 6: Improved Effect of PGAP on the Nonlinear Conditions (20 Nodes).
Nonlinear Function Type
MIM
MLP
Prior Proportion
0.2
0.2
0.4
Graph Type
Method
F1
SHD
F1
SHD
F1
SHD
F1
SHD
NOTEARS-ME
88.99
6,50
95.12
2.83
70.80
19.33
79.73
13.67
ER2
+PGAP
89.75
5,83
96.21
2.17
73.05
17.33
82.54
11.83
NOTEARS-MLP
68.45
16.33
76.10
12.50
53.19
25.50
66.62
19.50
SF2
+PGAP
67.80
16,50
80.11
11.33
53.26
24.33
71.09
16.67
NOTEARS-MEE
81.46
23.83
87.56
16.83
64.90
40.33
79.64
25.83
ER4
+PGAP
82.81
22.50
88.42
15.67
65.95
38.50
81.01
23,83
NOTEARS-HR
66.16
32,83
82.22
19.67
48.78
50.00
65.81
36.83
SF4
+PGAP
72.84
28.50
83.71
18.17
50.86
48.67
66.40
36.00
Compared to previous experiments, we employed nonlinear methods to generate data, including the MLP and MIM approaches. The MLP method applies the sigmoid function to introduce nonlinear transformations to the input X, while the MIM method uses trigonometric functions for nonlinear transformations. Specifically, the former can expressed as X + sigmoid( ) + z, and the latter X <- tanh(XX: Wi) + cos(XW2) sin(XW3)) generated weight matrices, and z is random noise.
where W1, W2, W3 are randomly
Table O compares the performance of the proposed method and NOTEARS-MLP under nonlinear conditions across different settings. The results show that even in non-linear cases, the proposed method still achieves effective performance improvements in most scenarios. However, we also observe that in some cases, the performance gains are not significant, which may be attributed the complexity of variable dependencies in nonlinear settings, limiting the capability of the proposed method.
D.7 Sample Analysis of Comparative Experiments
To illustrate the improvement offered by the PGAP method, we analyze DAG with 10 nodes, ER=2, and Gauss-NV noise. Figures 18 showcase the DAGs learned using SOOTEARS-Sfftane and PGAP, called DAG-Softmax and DAG-PGAP, The discrepancies between the learned DAGs and the true DAG indicated by different colored edges: black lines represent correctly inferred edges, red lines indicate extra edges not present in the true DAG. Dotted lines represent the incorporation of prior knowledge in the learning process.
DAG-Softmax results 6 extra edges, while DAG-PGAP has fewer, with extra edges, demonstrating PGAP's superior performance.
DAG-Softmax exhibits triangles that include both prior knowledge and extra edges that point to the same node, including 15,2,1), (5,2,6), and (6,8.5). In contrast, DAG-PGAP shows reduction in such triangle patterns, featuring such instances: (6..8)) The structure (5,2,6) is identified, leading to the extra edge from 6 to 2 being deleted. These 2 triangles in DAG-PGAP involve just extra edges. This reduction indicates that under the influence PGAP, even when the condition of Lemma is met, forming triangle patterns becomes more difficult.
37

---

Moreover, regarding the double collision pattern, DAG-Softmax displayed one instance: (6,8,3.9). In contrast, DAG-PGAP recorded no such instance. This comparison illustrates that PGAP effectively minimizes the number of special graph patterns, reinforcing its utility in refining the structure learning process by integrating prior knowledge more efficiently and reducing errors due to extra edges.
9
7
8
Figure 18: The structure learning results SOTTERSS-Sfftan (left) and DAG-PGAP (right) for DAG with 10 nodes, ER=2, and Gauss-NV noise. 40% priors are utilized.
D.8 Performance on Dense Graphs with a Data-driven Adjustment Mechanism
As discussed in Section the performance of the baseline PGAP adjustment can be sensitive in denser graphs. This is because the raw count of triangular and double collision patterns, used as the primary signal, may be as informative when such patterns occur more frequently by chance in highly connected structure. To address this challenge and enhance the robustness of PGAP, we introduce and validate a refined, data-driven adjustment mechanism that relies on non-oracle tuning procedure.
This refined procedure normalizes the pattern count for a prior-enforeed edge against a baseline derived empirically from the graph's own structure. The process is as follows. First, after learning an initial graph W Wagget. we iterate through every edge in the graph compute its associated pattern count, which yields empirical distribution of these counts for the specific graph. From this distribution, we establish baseline for the "normal" level of patterns, Teuselines using robust statistic such as the median or the 75th percentile. For each prior-enforced edge æ, we then calculate its excess pattern count: nexcess max (0. Ppattems.. Thaseline)) This excess count, representing an anomalous number of patterns, is then used to determine the adjustment to the prior's weight ax,, This non-oracle method ensures that the adjustment sensitive only to pattern counts that are unusually high for the given graph's 's density, thereby stabilizing performance.
To validate this data-driven procedure, we conducted extensive experiments on dense graphs. Table 7 shows the SHD results for graphs with 20 and 40 nodes with high average degrees. Table 8 presents results for even more challenging scenarios with 60 nodes and an average degree of 24. We compare the baseline NOTEARS- Softmax against PGAP enhanced with our data-driven adjustment, using both the median and the 75th percentile the baseline statistic.
The results consistently demonstrate that both data-driven variants of PGAP significantly outperform the baseline method across all tested dense graph scenarios and noise types. This confirms that the refined adjustment mechanism is not only theoretically sound but also a practical and effective solution for robustly applying PGAP in challenging, highly-connected environments.
D.9 Performance on Diverse Graph Topologies
To further assess the generalizability of PGAP and ensure its effectiveness is not limited to ER or SF graphs, we conducted additional experiments on three distinct random graph models that generate DAGs with diverse structural properties. The Watts-Strogatz (WS) model produces graphs with high
38

---

Table 7: Performance of the data-driven PGAP adjustment on dense graphs (d=20, 40). SHD is reported.
Node
d=20
d=40
Noise
Graph&Fddee
ER&41
SF&
FR&Sd
SF&
Prior Proportion
0.2
0.4
0.2
0.4
0.2
0.4
0.2
0.4
NOTEARS Softma
69.2.1844
64.0:6.3
54.517.77
49.2.16.9
296.011.88
218.31088
18000-1
Exp
+PGAP(median)
62.3+7.9
56.7+6.0
51.5+855
42.347.3
282.0+4.7
233.8+7.9
213.5+4.6
162.8-10
+PGAP(75th)
62.247.6
59.0:6.0
52.248.2
43.247.0
282.745.2
231.248.3
209.245.4
159,7=11
NOTEARS Softmax
67.816.2
60.814..5
53.51992
47.315.9
298.2.1.11
258.518.00
229.518.2
179.818.3
Gauss
+PGA PGGAP(median)
67.544.88
57.8+6.77
47.747.88
43.348..1
2848+44.5
238.74-6.8
218.54299
165.249.0
PGAP(75th)
66.714455
55.748.3
48.849.
43.247.0
286.8144.3
238.548.6
214.0#6.1
159.2=11
NOTEARS Softmax
718816.1
63.3:55.3
53.218.9
47.819.4
29415:16.3
256.811..6
222.015.9
181,5=10
Gumbel
+PGAP(median)
66.3116.7
59.016.7
280.516.5
164.2.9.2
PGGAP(75th)
64.5+555
57.744,7
48.7+-7.5
40,7+8.3
286.8+22.1
234.5+8.00
217.842.8
8++,5
Table Performance of the data-driven PGAP adjustment on highly dense graphs (d=60, average degree 24). SHD is reported.
Graph&Edge
ER & 12d
SF & 12d
Noise
Prior Proportion
0.2
0.4
0.2
0.4
NOTEARS-Softmin
648.818.9
601.2420
517.8
8.7
Exp
+PGAP(median)
617.817.77
541.7118
473.71120
351.5
+PGAP(75th) 620.317.5
548.0118
472.7+18
346.0+10
WOTEARS-Softmil
650.0
601.0+15
513.5+16
376.8=15
Gauss
+PGAP(median)
618.318.3
547.8114
485.3112
356.0110
+PGAP(75thh
627.5+7.44
549.7+13
478.8+14
353.0+:1
WITEARS-Softmar
647.717.2
594.3117
511.8114
385.8115
Gumbel
+PGAA(mmedian)
613.248.3
541.3116
480.7
345.3
+PGAP(75th)
619.316.9
548.7117
475.3113
344.0111
clustering coefficients, characteristic of social and biological networks, where triangular patterns may appear more frequently. The Stochastic Block Model (SBM) creates graphs with community structures by varying edge probabilities within and between node blocks, causing target patterns to be unevenly distributed. Finally, Geometric Random Graphs (GRG) connect nodes based on their proximity in metric space, resulting in spatially driven topologies.
The experimental settings for these models were varied to produce graphs with different densities and properties. For WS, we adjusted the lattice dimension; for SBM, the number of blocks; and for GRG, the connection radius. Tables 10 and present the performance of NOTEARS-Sofim (NT-Soft) with and without PGAP enhancement for the WS, SBM, and GRG models, respectively.
The results shown across these tables demonstrate that PGAP consistently improves both Fl and SHD metrics across all three graph models and their varying parameter settings. This robust performance particularly notable because these models generate topologies where the natural frequency and distribution of triangular and collision patterns can differ significantly from standard ER SF graphs. This provides strong evidence that the deviation mechanism identified our work is fundamental phenomenon and that PGAP's mnttern-guided approach is an effective and generalizable strategy for improving prior integration.
E
Limitation and Further Discussion
This section provides further discussion on key aspects of PGAP framework and evaluation. We elaborate on the interpretation of edge weight deviation (7), the applicability of our analysis to different scoring functions, the assumptions and generalizability of our theoretical findings, the robustness of the identified structural patterns, considerations regarding prior knowledge quality,
39

---

Table Performance of PGAP on WS graphs. (-) indicates the number of edges.
NT-Soft
+PGAP
Nodes
Parameter
F1
SHD
F1
SHD
Lattice=1 (60)
53.545.0
50.726.6
56.5+4.7
44.8+6.3
30
Lattice=2 (147)
50.6+3.6
137.7=9.7
54.5+4.3
107.0
Lattice=1 (120)
5..8+1.4
104.714.9
58.8+1.5
93.316.0
60
Lattice=2 (333)
46,9+2.0
345.7+11.2
50.5+2.5
190.
Table 10: Performance of PGAP on SBM graphs. () indicates the number of edges.
NT-Soft
+PGAP
Nodes
Parameter
F1
SHD
F1
SHD
Block=5 (71)
57.1+3.0
55.3+4.2
58.0+6.5
49.8+5.7
30
Block=10 (64)
58.9+5.4
44.5+3.0
58.3+8.4
44.8+9.3
Block=5 (118)
56.0=8.4
98.3+9.9
57.145.3
83.5+7.6
60
Block=10 (118)
56.9+3.5
90.3+9.3
62.542.5
73.8+3.2
Table 11: Performance of PGAP on GRG. (-) indicates the number of edges.
NT-Soft
+PGAP
Nodes
Parameter
Fl
SHD
F1
SHD
Radius=0.2 (43)
61.1+3.2
26.0+2.8
62.9+3.9
23.0+3.3
30
Radius=0.3 (101)
62.1
69.5+3.4
63.2+1.5
62.7+3.1
Radius=0.2 (195)
55.5+1.3
159.546.5 280.2+13.6
56.242.9
148.0+7.9
60
Radius=0.3 (368)
56.7#1.8
57.5+3.5
257.8
the positioning of PGAP within the landscape of prior integration methods, and its computational complexity.
On the Interpretation and Magnitude of Deviation Section 4.1 of the main text introduces the critical concept of edge weight deviation. This phenomenon arises when qualitative prior knowledge is integrated into the learning of quantitative SEMs, potentially causing learned edge weights to differ from those purely indicated by the data. Our analysis specifically examined how a localized deviation, defined as 7 W3,j W (representing the difference between prior-influence weight Ws.; and the OLS-optimal weight W can propagate and influence other weights. This appendix offers more detailed exposition of this deviation and its broader implications.
The theoretical framework presented (Lemmas 33) demonstrates that such a deviation y on a prior-enforced edge can linearly affect the weights of other potential incoming edges to the same target node. When this deviation 7 is substantial, can lead to the erroneous formation of spurious edges, where Wk.j becomes non-zero even if no true direct relationship X + Xj exists. As further elaborated in Section 4.2 these spurious edges, in combination with the prior-inftuenced edge and other true connections, result in the characteristic "triangle" and "double collision" graph patterns. Consequently, these patterns serve as observable structural indicators within the learned graph, signaling that significant weight deviations, driven by the enforcement of prior knowledge, may be actively distorting the local structure being learned around a particular node.
While the deviation 7 discussed in Section 4. is localized mechanism observed within optimiza- tion step, its repeated occurrence due strong or misaligned prior enforcement is what contributes to a more significant, global issue: the final learned edge weights in the SEM (W,,) can diverge considerably from the true, underlying weights of the data-generating process (Bt)). This ultimate discrepancy represents core challenge when attempting to bridge qualitative priors with the quan- titative nature of SEMs. The magnitude of this deviation from the true SEM weights is influenced
40

---

by several factors. A primary factor is the inherent inconsistency between the qualitative prior and the actual quantitative strength and nature of the relationship in the true data-generating system. Furthermore, the specific strategy used to integrate prior knowledge, be it hard constraints on weight values or soft constraints via penalty terms in the loss function (as discussed in Section 2) heavily dictates how strongly the learning process adheres to the prior versus fitting the observational data. The 'strength' or weighting assigned to these prior-related terms in the objective function acts as direct modulator of this critical balance. If prior enforcement is too rigid, it can overshadow the evidence present in the data, leading to learned weights that, while satisfying the prior, poorly reflect the true underlying causal mechanisms.
Although the precise threshold for the deviation y to induce the aforementioned spurious patterns is data-dependent (influenced by aspects such as noise levels and the magnitudes of other true edge weights), these patterns become increasingly probable as y grows substantial. This typically happens when the prior-enforcement mechanism compels an edge weight to value markedly different from what local data evidence among the current set of predecessors would otherwise support. often reflects situation where a qualitative statement (e.g.. "an edge exists") is translated into quantitative target that is excessively strong or misaligned with the underlying reality of the SEM. The PGAP framework is designed to leverage these observable structural patterns as signals of such problematic weight deviations. By detecting these patterns, PGAP adaptively adjusts the prior integration process (detailed in Appendix () to mitigate these deviations, aiming for a more accurate and robust estimation of both the graph structure and its associated edge weights.
Computational Complexity of PGAP The PGAP framework, as detailed in Appendix C (Algo- rithms] [ and [) involves a two-stage learning process. First, a base structure learning algorithm initial graph, W Sttagel. Second, after detecting specific structural patterns in W Wstaget and adaptively adjusting either prior loss weights (for soft constraints) or local regularization strengths (for hard constraints), the base algorithm is run again to yield the final graph, W final
is run to produce
The primary computational load thus consists of two executions of the chosen base continuous optimization learning algorithm. The intermediate pattern detection step, which involves checking local neighborhoods around each of the |TT] prior edges for specific triangle or double collision patterns, is efficient. For a graph with d nodes, the complexity of this pattern search is generally much lower than that of a full run of the base learning algorithm.
Consequently, if the base algorithm has a computational complexity of Thsse the overall complexity of PGAP is approximately 2 x Tinses plus the minor overhead of pattern detection. PGAP therefore maintains the same asymptotic complexity class as the underlying base structure learning method.
Applicability of PGAP to the Negative Log-Likeling Log-Likelihood Scoring Function The theoretical frame- work underpinning PGAP is developed based on the observation that optimizing common scoring functions like least squares (Qts) often decomposes into node-specific OLS problems. This appendix briefly discusses the direct applicability of this framework when using the Negative Log- Log-Likelihood scoring function, commonly employed for SEMs.
The NLL scoring function for a SEM, as presented the Preliminaries (Eq, Zusing Qnu), aims to find the weight matrix W that maximizes the likelihood of the observed data. During the continuous optimization process, an acyclicity constraint h(W) = 0 is enforced, which implies a topological ordering of the variables for any candidate W. Given this ordering. the estimation of the weights of incoming edges for any particular node X, from its set of candidate parent nodes ty effectively reduces a linear regression problem.
min Quu (W;X) e min W
log
X
XW
min W.
X - XW, i 11,1,,,
(62)
W
This is the same per-node optimization problem as considered for the Qta score in Section 4.1
Assumptions and Generalizability of Theoretical Findings Our theoretical framework, while utilizing assumptions like linear SEMs and faithfulness for analytical tractability (as discussed in Section 4.1 and Appendix [B). builds upon the identification of local graph patterns (triangle and double collision). As noted Section 4.2 the conditions in Lemma for pattern formation depend on local structures (direct connections common children paths). This suggests that the core mechanism
41

---

of deviation-induced pattern formation might still provide valuable insights even when some global model assumptions are relaxed. Future work could explore extending the theoretical analysis to non-linear models or other graph families, building on preliminary investigations such as those in Appendix D regarding non-linear scenarios.
Robustness and Specificity of Deviation-Indueee Patterns While our theoretical results (Lem- mas | and suggest that the identified triangle and double collision patterns are expected to be significantly more prevalent when substantial prior-induced deviation occurs compared to their natural occurrence typical sparse graphs, they may not be exclusively caused by this phenomenon. PGAP, therefore, uses their detection in conjunction with a known prior-enforced edge primarily a heuristic signal that such deviation is likely problematic and warrants adaptive adjustment. Investigating a broader range of structural signals more sophisticated detection mechanisms could be direction for future enhancements.
Handling of Prior Knowledge Quality If a significant portion of the prior knowledge incorporated into the learning process is fundamentally incorrect, PGAP's mechanism (reducing the influence of priors involved in the targeted deviation patterns) might inadvertently down- down-weight these incorrect priors if they lead to such structural distortions. In this sense, it could offer some resilience against grossly misspecified priors by reducing their impact. However, PGAP is not explicitly designed as a prior validation or correction tool. Its main goal is to rationalize the use of priors assumed to be qualitatively valuable but potentially quantitatively mis-specified, common assumption as noted in Section 3
Positioning Relative to Proactive Prior Integration Strategies PGAP is designed framework that can augment a wide range of existing structure learning algorithms and prior inte- gration techniques (both 'hard' and 'soft' constraint methods, as demonstrated in our experiments, e.g., in Section D.2 and related tables in Appendix Dj. Its strength lies in identifying and mitigating a specific, theoretically grounded adverse effect that can occur even with established prior enforcement methods. While developing more sophisticated upfront prior integration methods (e.g., learning prior strengths or incorporating prior uncertainty directly) is valuable and active area of research, PGAP offers complementary approach. It addresses an observable consequence of potential mismatches between qualitative priors and quantitative models, effectively acting as a refinement layer. The two-stage process allows for an initial assessment of prior impact, followed by targeted adjustment. Future work could explore synergies between PGAP's -uidded adaptation and more complex prior modeling techniques. For example, the structural signals detected by PGAP could potentially inform the learning of prior confidence scores or guide more nuanced regularization schemes in a unified framework.
adaptive
On the Linear Assumption and Generalization to Nonlinear Models Our formal theoretical analysis is developed within the linear SEM framework. This focus is deliberate. Linear SEMs are cornerstone of causal modeling, not only used extensively across scientific and industrial domains but also serving as critical object of study for developing more advanced methods. The continued prevalence of linear models is largely due to their interpretability. where linear coefficients provide a clear measure of causal influence, and their computational tractability. In many complex systems, particularly in sigh-dimessional, low-sample-size settings, linear models serve as powerful first-order approximations where fitting complex nonlinear models is often statistically infeasible and prone to overfitting. Addressing the unresolved problem of robustly integrating imprecise prior knowledge within this foundational domain is therefore significant contribution.
Although extending our formal analysis to general nonlincar models is a challenging task, the core principle of the PGAP framework is generalizable. Our path for such extension begins by generalizing our core concepts from the parametric to the nonparametric domain, leveraging the framework of Zheng et al. 2202).. The concept of an edge weight is replaced by functional dependency measure, and "edge weight deviation" becomes a more general "functional dependency deviation."
To understand how this functional deviation propagates, we then adopt a local perspective. When a prior introduces a functional deviation, the learned function fj and the data-optimal function f are no longer identical. In linear model, this difference inherently global. In the nonlinear case, however, the two functions fy and fÿ can be identical over large areas and only diverge in specific
42

---

local regions of the input space. It is within these regions of divergence that we can analyze the propagation mechanism. Here, the behavior of f, can be approximated by its first-order Taylor expansion, rendering it locally linear. The partial derivatives of f act as the coefficients of this local linear approximation. When pressure from the prior forces one of these "local coefficients" to be non-optimal, the optimization process is incentivized to adjust other local coefficients to minimize the loss. This local mechanism is analogous to the error propagation we formally analyzed in the linear setting. The adjustment of other local coefficients can cause the global functional dependency measure to become non-zero, signifying the creation spurious edge and leading to the formation of the same topological patterns we leverage in PGAP.
This local perspective also provides a crucial insight into why the effects of deviation might appear less pronounced in some nonlinear settings. If a local region where the prior causes significant deviation is sparsely sampled, its contribution to the global loss will be minimal, and the optimizer may find solution that largely ignores the prior's influence in that region. This does not mean the phenomenon negligible. On the contrary, points to an often overlooked aspect of causal discovery. While global dependency metrics are standard, they can average out important local effects that occur only in specific regions the input space. We believe that analyzing and accounting for such local prior-induced deviation represents key step forward for the field, especially for emerging applications such as discovering context-specific causal effects or understanding heterogeneous treatment effects in subgroups.
43


Reviews:
[{"review_id": "EvkHlkCnHj", "reviewer": "Reviewer_7Uc5", "summary": "This paper centers around Directed Acyclic Graph (DAG) structure learning and aims at addressing the pitfalls of integrating imprecise prior knowledge in the structure learning process, with a particular focus on Structural Equation Models (SEMs). The authors identify the edge weights deviation issue when specifying the edge prior and theoretically analyze how such deviations can propagate through the model, which adversely affects the estimation of other parameters. To mitigate this problem, the authors propose the Pattern-Guided Adaptive Prior (PGAP) framework. PGAP leverages structural signals in the form of two specific graph patterns as reliable indicators, specifically triangular and double collision. As such, PGAP utilizes such patterns during optimization and adaptively adjusts the prior to mitigate the weight deviation. As proof of concept, the authors conduct experiments on synthetic datasets confirming the effectiveness and robustness of PGAP.", "questions": "- It would be helpful if the authors could elaborate on the practical scope and applicability of linear SEMs. Specifically, in what types of real- world scenarios are linear SEMs commonly used, and how do the underlying assumptions hold in practice?\n- Can the theoretical foundation of the proposed method be extended to accommodate more general, nonlinear models? Given that different structural patterns or deviation mechanisms may emerge in nonlinear settings, it would be valuable to understand the potential for generalization and whether PGAP could be adapted accordingly.\n- As described in Algorithms 1 and 2, the prior is obtained through a single round of base structure learning. Could the authors clarify whether there is any guarantee or justification that this prior is sufficiently informative or reliable? Additionally, under what assumptions does this approach provide a good approximation, and how sensitive is the method to the quality of this initial prior?", "limitations": "The derivations and theoretical results in the paper are based entirely on linear Structural Equation Models (SEMs), which are relatively simplistic and may not adequately reflect the complexity of real-world data-generating processes. My primary concern is the practicality and robustness of the proposed algorithm in more general settings with more complex models such a generalized SEM or other nonlinear formulations that better capture real-world causal structures. It would be valuable for the authors to discuss the potential limitations of their method in such contexts, as well as the prospects for extending the framework beyond the linear case.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths: \n- The theoretical analysis is rigorous. The authors provide a clear and detailed mechanism for how imprecise priors can distort learned DAG structures, and justify that the error is influenced by the triangular and double collision pattern. The proposed algorithm is grounded in such an observation and has strong motivation.\n- The paper is well-written and easy to follow in general.\n- The experimental validation is thorough within the considered scenarios, lending credibility to the reported improvements.\n\nWeakness\n- My primary concern lies in the limited practical applicability of the theoretical framework and the proposed method. The theoretical analysis is confined to linear SEMs and relies on the faithfulness assumption—an assumption that may not hold in many real-world scenarios. For example, when analyzing the impact of edge weight deviation, the framework assumes optimization over linear SEMs, whereas in practice, structural relationships can be nonlinear or governed by more complex forms such as generalized SEMs. Similarly, the construction of the adaptive prior is based on the assumption that random DAGs follow Erdös-Rényi (ER) graphs, and key theoretical results (e.g., the lemma) depend on this assumption. These constraints narrow the scope of the method and limit the generalizability of its conclusions to broader or more realistic settings.\n- The experimental validation is currently restricted to synthetic datasets generated using ER and scale-free (SF) models. While the authors explore various configurations, including different edge-to-node ratios and noise types, the evaluation lacks diversity in underlying graph models. It would strengthen the paper to test PGAP on a broader set of random graph models, especially those where the frequency of triangular and double collision patterns differs substantially from ER/SF. More experiments on real-world datasets are also recommended. Moreover, results in Appendix C suggest that PGAP yields less significant improvements in nonlinear settings, raising questions about the method’s effectiveness beyond the linear domain and further indicating that the identified patterns and deviation mechanisms may not fully capture the complexity of nonlinear models.\n- I would also like to raise concerns about the method used to obtain the initial prior. According to Algorithms 1 and 2, the prior is constructed via a single round of base structure learning. It is unclear whether this process reliably yields a meaningful or accurate prior. Further clarification is needed on the conditions under which this approach can be expected to work well, and whether any theoretical guarantees or empirical justifications support it.", "was_revised": true, "final_justification": "I would like to thank the authors for addressing many of my concerns and acknowledge that the paper has clearly improved with the additional results provided during the rebuttal. I am therefore inclined to acceptance.", "rebuttal": "We sincerely thank you for your detailed and thoughtful feedback. We will address your questions in order below and would be happy to answer any further questions you might have.\n\n**Q1:**\nLinear SEMs are a foundational tool in causal discovery, not only being used extensively across many scientific and industrial domains [1-3] but also serving as a critical object of study for the development of more advanced methods [4-6]. Although nonlinear models can often better capture the complexity of real-world relationships, the continued prevalence of linear models is largely due to their interpretability and computational tractability. The linear coefficients provide a clear measure of causal influence. In many complex systems, linear models also serve as powerful first-order approximations of the underlying relationships. This is especially true in high-dimensional, low-sample-size settings, where fitting complex nonlinear models is often statistically infeasible and highly prone to overfitting. In such scenarios, the linear assumption becomes a pragmatic and often necessary choice for obtaining robust causal structures. Given that linear SEMs are a cornerstone of causal modeling, we believe that addressing the fundamental, unresolved problem of how to robustly integrate imprecise prior knowledge within this domain is a significant contribution.\n\n**Q2:**\nOur formal theoretical analysis is developed within the linear SEM framework to allow for a precise characterization of deviation propagation. Although extending this formal analysis to general nonlinear models is a challenging task, the core principle of our PGAP framework is generalizable. We outline our path for this extension as follows.\n\nFirst, we generalize our core concepts from the parametric to the nonparametric domain, leveraging the framework of Zheng et al. [7]. The concept of an edge weight is replaced by a functional dependency measure, $[W(f)]_{kj}:= ||\\partial _k f_j|| _{L^2}$, and \"edge weight deviation\" becomes a more general \"functional dependency deviation.\"\n\nTo understand how this functional deviation propagates, we next adopt a local perspective. When a prior introduces a functional deviation, the learned function $\\hat{f}_j$ and the data-optimal function $f_j^{\\star}$ are no longer identical. This is a critical distinction from the linear setting. In a linear model, this functional difference is inherently global; a change in a single coefficient means the two linear functions differ across the entire input space. In the nonlinear case, however, the two functions $\\hat{f}_j$ and $f_j^{\\star}$ can be identical over large areas and only diverge in specific local regions of the input space. It is within these regions of divergence that we can analyze the propagation mechanism.\n\nIn these regions, the behavior of $\\hat{f}_j$ can be approximated by its first-order Taylor expansion, rendering it locally linear. The partial derivatives of $\\nabla\\hat{f}_j$ act as the coefficients of this local linear approximation. The pressure from the prior forces the value of $\\partial _{\\beta}\\hat{f}_j$ in this region to be non-optimal. To minimize the loss within this local context, the optimization process is incentivized to adjust the other \"local coefficients,\" $\\partial _{k}\\hat{f}_j$ for $k \\ne \\beta$. This local mechanism is analogous to the error propagation we formally analyzed in the linear setting. In this case, the global functional dependency measure, $[W(f)] _{kj}$, will also become non-zero. This signifies the creation of a spurious edge $(X_k \\rightarrow X_j)$ and leads to the formation of the topological patterns we leverage in PGAP.\n\nThis local perspective also provides a crucial insight into why the effects of this deviation might appear less pronounced in some nonlinear settings. The global objective function is optimized over a finite set of sample points. If a local region where the prior causes significant deviation is sparsely sampled, its contribution to the global loss will be minimal. The optimizer, therefore, may find a solution that largely ignores the prior's influence in these data-scarce regions, resulting in a change to the global dependency measure $|| \\partial _k \\hat{f}_j || _{L^2}$ that is only minor.\n\nThis does not mean the deviation phenomenon is negligible in nonlinear models. On the contrary, it points to a crucial and often overlooked aspect of causal discovery. While global dependency metrics are standard, they can average out important local effects that occur only in specific regions of the input space. This local perspective is particularly valuable for many emerging applications, such as discovering context-specific causal effects or understanding heterogeneous treatment effects in subgroups. In these scenarios, analyzing and accounting for local prior-induced deviation represents a key step forward for the field, and we believe that as the field moves towards more fine-grained functional causal analysis, the issues of local deviation we have identified will become increasingly critical to address.\n\n**Q3:**\nWe thank the reviewer for pointing out this ambiguity. To be clear, the prior knowledge is an external input to our framework, not something generated by our algorithm. In line with standard practice in this research area, we assume this prior knowledge is qualitatively accurate. To strictly ensure this condition in our experiments, all priors were generated by randomly selecting true edges from the ground-truth graph. Our algorithms, therefore, describe a method for robustly using this externally-provided prior, not for generating it. We will revise our manuscript to make this distinction clear.\n\n**Regarding what is not mentioned in the question but mentioned in the weaknesses:**\n* Regarding the ER graph assumption, our core deviation mechanism (Lemmas 2-3) is entirely general. We used the ER model only for the analysis in Lemmas 4-5 to obtain a concise, closed-form expression for clarity, as a general model would yield a complex result obscuring the intuition. This simple expression allows us to clearly demonstrate our key justification: that deviation-induced patterns occur at a much higher frequency, making them a strong error signal. We will clarify this distinction in our revision.\n* In response to the reviewer's suggestion, we further validated PGAP's effectiveness on three real-world datasets of varying scales: the small-scale PaidSearch (7 nodes, 6 edges) [8], and two larger-scale datasets, Magic-niab (44 nodes, 66 edges) [9] and Ecoli70 (46 nodes, 70 edges) [10].\n\n**Table 1:** Improved effect of PGAP on the different datasets.\n| Dataset              |    PS    |          |         |         |    Mn    |          |         |         |   E70    |          |         |         |\n| :------------------- | :------: | :------: | :-----: | :-----: | :------: | :------: | :-----: | :-----: | :------: | :------: | :-----: | :-----: |\n| **Metric**           |  **F1**  |          | **SHD** |         |  **F1**  |          | **SHD** |         |  **F1**  |          | **SHD** |         |\n| **Prior Proportion** | **0.2**  | **0.4**  | **0.2** | **0.4** | **0.2**  | **0.4**  | **0.2** | **0.4** | **0.2**  | **0.4**  | **0.2** | **0.4** |\n| NOTEARS-Hard         |   15.4   |   42.9   |    9    |    7    |   34.9   |   46.7   |   51    |   45    |   26.7   |   35.0   |   123   |   112   |\n| +PGAP                |   15.4   | **46.2** |    9    |  **6**  | **35.7** | **54.2** | **50**  | **40**  | **28.4** | **38.7** | **107** | **103** |\n| NOTEARS-ReLU         |   13.3   |   30.8   |   11    |   10    |   35.3   |   42.7   |   51    |   47    |   23.0   |   29.5   |   161   |   176   |\n| +PGAP                | **16.7** | **42.9** |  **8**  |  **7**  | **37.2** | **53.2** | **49**  | **41**  | **27.9** | **35.7** | **117** | **99**  |\n| NOTEARS-Softmax      |   30.8   |   40.0   |    8    |    8    |   35.3   |   44.9   |   51    |   46    |   20.6   |   29.0   |   155   |   176   |\n| +PGAP                | **33.3** | **46.2** |  **7**  |  **6**  | **37.2** | **54.7** | **49**  | **40**  | **25.2** | **36.0** | **113** | **108** |\n\nAs shown in Table 1, the results demonstrate that PGAP provides consistent and significant improvements across all three datasets. When integrated with the baseline methods, PGAP leads to better outcomes, reflected in both higher F1 scores and lower SHD values. This robust performance on datasets of different sizes and domains underscores the practical applicability and generalizability of our proposed framework.\n\nRegarding the diversity of graph models, we also conducted additional experiments. Due to space constraints in this reply, we have included the detailed results in our response to **Reviewer xCbJ**, but they are intended to directly address the concerns raised here. We believe these new results, along with our experiments on additional real-world datasets, further strengthen the case for PGAP's practical applicability and generalizability.\n\n* We thank for raising this concern and have addressed this point in Q3.\n\n[1] Detecting abnormal connectivity in schizophrenia via a joint directed acyclic graph estimation model. \n\n[2] Benchmarking of Data-Driven Causality Discovery Approaches in the Interactions of Arctic Sea Ice and Atmosphere. \n\n[3] Scalable causal structure learning: Scoping review of traditional and deep learning algorithms and new opportunities in biomedicine.\n\n[4] DYNOTEARS: Structure Learning from Time-Series Data.\n\n[5] Federated Bayesian network learning from multi-site data.\n\n[6] A hybrid constrained continuous optimization approach for optimal causal discovery from biological data.\n\n[7] Learning sparse nonparametric dags.\n\n[8] A model of individual keyword performance in paid search advertising.\n\n[9] Multiple quantitative trait analysis using Bayesian networks.\n\n[10] A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics."}, {"review_id": "DOt0gmg77s", "reviewer": "Reviewer_ETuv", "summary": "Under the assumption of a linear model and causal sufficiency, this paper studies how integrating prior knowledge (edge existence) into continuous optimization-based causal discovery methods may lead to errors in the returned graph. They show that if an initial edge weight is estimated incorrectly, this error can propagate to other edge weights. They conclude that this error will show up in the form of spurious edges, and define two graphical structures (triangular collision and double collisions) that may occur artificially. Finally, they develop a sub-routine (PGAP) which can be added on top of any continuous optimization-based method, which adaptively lowers the weight placed on prior knowledge when the number of triangular/double collision patterns exceeds a threshold. They experimentally validate the efficacy their sub-routine in combination with a few standard continuous opt. methods in both synthetic and real data, showing that it almost uniformly increases performance, although by different margins in different settings.", "questions": "Q1 Can you provide a principled justification for focusing exclusively on triangular and double collision patterns, and characterize what proportion of weight deviation errors these patterns capture?\n\nQ2 How does PGAP perform in dense graphs without oracle hyperparameter selection, and can you provide a practical tuning procedure for α?\n\nQ3 Can you outline a concrete path for extending PGAP beyond linear models, or provide theoretical analysis showing why such extension is fundamentally difficult?\n\nI'm happy to increase my rating if the authors adequately address my concerns, both in this Questions section and as well in the Strengths and Weaknesses section.", "limitations": "See 'Clarity' and \"Quality\" section of the Strengths and Weaknesses section.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\n​Originality - Discovery methods tend to be sensitive to error propagation, and often return results that seem unlikely, prima facie. The subroutine PGAP suggested in this paper can be seen as a sort of \"self-correcting\" mechanism, which adjusts the optimisation function (in this case lowering the weight on the prior constraints) used to derive the causal structure whenever the graph structure suggested by the algorithm seems unlikely (# of triangle/double collisions is high). To my knowledge, explicit analysis and development of error-correcting sub-algorithms remains relatively unexplored in the discovery literature. Additionally, the way the authors reason from the occurence of a local error  (a \"weight deviation\") and consider the resulting cascade of global errors it might impart helps deepen our explicit understanding of error propagation in discovery methods.\n\nWeaknesses:\n\nClarity - Although the general organization of sections in the paper is fair, key information is often times not made explicit, or is deferred. For example, although this paper develops PGAP explicitly for causally sufficient linear models, leveraging the linearity assumption throughout the analysis, this functional assumption is not made clear to the reader until the middle of page 3 (Section 3). Ideally, an assumption this foundational to the methodology should be present in the title, abstract, and/or contribution part of the introduction. In Section 4.3, the authors defer the pseudocode description of their algorithm PGAP entirely to the Appendix, describing their proposed subroutine across only two sentences. This hinders the reader in understanding the author's proposed idea, and obscures the contribution of their work. Another example is in Section 6, where the authors defer all discussion of the limitations of their PGAP framework to the appendix, while the summary they supply does not provide any detail as to the shortcoming of their approach.\n\nQuality - Although the claims presented in the main text appear to be correct, and supported by the empirical evidence provided, it is not clear that the core idea by the authors is sufficiently developed to warrant a recommendation for publication. In Section 4.1, the authors characterize, in general, how weight deviations can propagate and affect learned weights of other edges (Lemma 2, Eq 11, etc,); however, in Section 4.2, they only explore one set of possible graphical structures that result from errors described in 4.1, namely the triangular and double collision patterns. Theres no explanation given why we should focus on these collision patterns, as opposed to any other set of structures that could arise. Although the initial analysis is interesting, it seems somewhat arbitrary that the authors chose to focus on only a pair of possible structures, without any compelling motivation.\n\nSignificance - Although the core idea of adaptive error-correcting subroutine (PGAP) is compelling, the much of this paper's analysis, both theoretical and empirical, seems to be of limited significance to the broader field. For example, they tackle error-correction only from the perspective of continuous-optimization based causal discovery methods, which have been uniquely criticized (among discovery methods) for being especially sensitive to unequal noise variances, performing poorly after standardization, and failing to generalize well to nonlinear and non-convex settings [1]. Additionally, the entirety of the analysis here remains limited to the linear setting, without discussion on how the error-correcting strategy could be generalized (although, it does not seem obvious to me that the approach here would generalize). Finally, even though the authors do show that PGAP does improve performance in many settings, it seems that both the theoretical and empirical results point to the utility of PGAP largely in sparse settings. Theoretically, in sparse settings relatively few collisions will occur, so adaptively penalizing the prior when collisions occur should be expected to improve performance. However, in dense settings, one would expect many more collisions, and thus penalizing might fail to improve performance. Figures 16 and 17 in the Appendix reflect this - in sparse graphs (Figure 16), NoTears+PGAP outperforms NoTears for almost all choices of the adaptive reduction parameter $\\alpha$. However, for dense graphs (Figure 17) NoTears+PGAP only outperforms NoTears for specific values of $\\alpha$, with the range being narrower when there is a lower amount of prior knowledge. Therefore, tuning of the hyperparameter $\\alpha$ seems somewhat critical to the performance of PGAP in dense settings with limited priors. Unfortunately, the authors do not suggest a tuning method, opting instead to use the optimal parameter here throughout the rest of the experiments, limiting the general applicability of PGAP to real world settings.", "was_revised": true, "final_justification": "The authors addressed my concerns, particularly by providing a data-driven way to set the threshold used to determine whether the graph has \"too many\" collisions. I recommend to accept.", "rebuttal": "We sincerely thank you for your thoughtful and constructive feedback. We have provided detailed answers to your comments and would be happy to answer any further questions you might have.\n\n**Weaknesses (Clarity):** To improve the clarity of the paper, we will revise the manuscript to be more self-contained by moving the discussion of the foundational linear model assumption, the PGAP algorithm details, and the core limitations from the appendix into the main text. \n\n**Weaknesses (Quality) & Q1:**\nOur focus on the triangular and double collision patterns is not an arbitrary choice, but stems directly from our theoretical analysis of the fundamental error propagation mechanism, as presented in Lemma 2 and 3. Lemma 2 establishes that a deviation on a prior-enforced edge can propagate to induce an erroneous weight on another potential edge, while Lemma 3 precisely characterizes the necessary and sufficient conditions for this propagation to occur. These conditions, which relate the two potential parent nodes $X_k$ and $X_\\beta$ that both point to $X_j$, are precisely: 1. A direct edge exists between $X_k$ and $X_\\beta$ in the true graph, or 2. $X_k$ and $X_\\beta$ share a common child node in the true graph.\n\nThese two conditions constitute the exhaustive set of elementary mechanisms by which a deviation on a single prior-enforced edge directly induces an error on one other edge. We term this fundamental, one-step process first-order error propagation. The triangular and double collision patterns are, in turn, the direct structural manifestations that arise when these pre-existing structure are combined with the spurious edge $(X_k \\rightarrow X_j)$ induced by the deviation. Therefore, we focus on them because they are the elementary building blocks of error arising from our theoretical model, not arbitrary heuristics. Furthermore, since any more complex error structure would be composed of these elementary patterns, analyzing them is the most logical and principled first step.\n\nRegarding the proportion of errors these fundamental patterns capture, they are exhaustive for the most elementary error type, capturing 100% of these cases. This is because within our theoretical analysis of a single deviation, Lemma 3 shows that these two patterns are the only possible structural outcomes.\nAs for more complex error structures that may arise from multiple interacting deviations, providing a single universal percentage of all errors captured is theoretically challenging. However, our work demonstrates this is not necessary because these two fundamental patterns already serve as a dominant and reliable signal of error. Firstly, our expectation analysis (Lemmas 4 and 5) shows their frequency when induced by deviation far surpasses their natural rate of occurrence, making them a strong theoretical indicator. Secondly, the consistent performance gains across our experiments validate their practical power. The fact that targeting these patterns is a highly effective strategy strongly suggests they are responsible for a sufficiently large and impactful proportion of the total errors.\n\n**Weaknesses (Significance) & Q2:**\nIn fact, our experiments also compare with the state-of-the-art method NT-LogLL [1], which addresses sensitivity to unequal noise variances. The results show PGAP consistently enhances even this modern and robust algorithm.\n\nRegarding the performance in dense graphs, our initial implementation reduces the prior's influence based on the raw count of detected patterns. As our experiments show, this is a highly effective strategy in the sparse settings common to many causal discovery problems. The core challenge in denser graphs is the higher baseline number of naturally occurring triangular and double collision patterns. A simple count-based adjustment therefore risks over-penalizing priors by reacting to structurally necessary patterns as if they were errors.\n\nFortunately, our theoretical framework allows for a straightforward and elegant refinement. For dense graphs where a high number of patterns occur naturally, the signal can be extended from the raw pattern count to the number of excess patterns that emerge above the graph's natural structural baseline. In the sparse settings, this natural baseline is negligible or near-zero. Consequently, the observed raw pattern count serves as a direct and effective proxy for the excess count, which explains why our initial implementation is so effective in those scenarios. For denser graphs where this simplification is no longer sufficient, our framework allows us to refine the procedure to explicitly isolate the true signal. This leads to the following practical and fully data-driven tuning procedure:\n\nThe procedure begins by learning an initial graph $W_{stage1}$. The key refinement is how we interpret the pattern counts. Instead of reacting to raw counts, we first establish a data-driven baseline. We iterate through every edge in the learned graph $W_{stage1}$ and count the patterns associated with each edge. This gives us an empirical distribution of pattern counts specific to the lea. From this distribution, we define the \"normal\" level of patterns ($n_{baseline}$) using a robust statistic like the median or the 75th percentile. For a prior-enforced edge $\\pi$, we then compute its excess pattern count: $n_{excess, \\pi} = \\max(0, n_{observed, \\pi} - n_{baseline})$. The strength of the adjustment (SA) is made adaptive by normalizing this excess count against the maximum count observed in the graph ($n_{max}$):\n$$\\text{SA} _{\\pi}=\\frac{n _{excess, \\pi}}{ n _{max} - n _{baseline} +1}$$\n\nThis factor, representing the anomaly score of the pattern count, can then be used in a straightforward update rule, such as $\\alpha_{\\pi, new} \\leftarrow \\alpha_{\\pi, old} \\times (1 - \\text{SA}_{\\pi})$.\n\nThis refined procedure is a natural extension of our theory. It is robust to graph density because the baseline is empirically derived from the graph's own structure. Furthermore, it is a non-oracle method, as both the signal and the adjustment strength are determined automatically from the data. This demonstrates the flexibility of the PGAP framework and its straightforward applicability to the practical challenges. \n\nTo empirically validate this proposed non-oracle procedure, we have already conducted a set of preliminary experiments, the results of which are presented in the accompanying table. The results consistently demonstrate that both PGAP variants, using either the median or the 75th percentile to establish a data-driven baseline, achieve a significant performance improvement over the standard method, with this robust improvement holding across challenging dense graph scenarios (d=40 with 8d edges, i.e. average degree = 16). This empirical evidence affirms that our proposed refinement is not only theoretically sound but also a practical and effective solution. \n\n**Table 1:** Performance of PGAP in the improved adjustment scenario based on empirically normalized adaptive. \n\n|        |          Node          |              |     d=20     |              |              |               |     d=40      |               |               |\n| :----: | :--------------------: | :----------: | :----------: | :----------: | :----------: | :-----------: | :-----------: | :-----------: | :-----------: |\n|        |       Graph&Edge       |    ER&4d     |              |    SF&4d     |              |     ER&8d     |               |     SF&8d     |               |\n| Noise  |    Prior Proportion    |     0.2      |     0.4      |     0.2      |     0.4      |      0.2      |      0.4      |      0.2      |      0.4      |\n|  Exp   |    NOTEARS-Softmax     |   69.2±8.4   |   64.0±6.3   |   54.5±7.7   |   49.2±6.9   |   296.0±7.8   |   259.0±11    |   218.3±0.8   |   180.0±13    |\n|        |     +PGAP(median)      | **62.3±7.9** | **56.7±6.0** | **51.5±8.5** | **42.3±7.3** | **282.0±4.7** | **233.8±7.9** | **213.5±4.6** | **162.8±10**  |\n|        | +PGAP(75th percentile) | **62.2±7.6** | **59.0±6.0** | **52.2±8.2** | **43.2±7.0** | **282.7±5.2** | **231.2±8.3** | **209.2±5.4** | **159.7±11**  |\n| Gauss  |    NOTEARS-Softmax     |   67.8±6.2   |   60.8±4.5   |   53.5±9.2   |   47.3±5.9   |   298.2±7.1   |   258.5±8.0   |   229.5±8.2   |   179.8±8.3   |\n|        |     +PGAP(median)      | **67.5±4.8** | **57.8±6.7** | **47.7±7.8** | **43.3±8.1** | **284.8±4.5** | **238.7±6.8** | **218.5±2.9** | **165.2±9.0** |\n|        | +PGAP(75th percentile) | **66.7±4.5** | **55.7±8.3** | **48.8±9.5** | **43.2±7.0** | **286.8±4.3** | **238.5±8.6** | **214.0±6.1** | **159.2±11**  |\n| Gumbel |    NOTEARS-Softmax     |   71.8±6.1   |   63.3±5.3   |   53.2±8.9   |   47.8±9.4   |   294.5±6.3   |   256.8±7.6   |   222.0±5.9   |   181.5±10    |\n|        |     +PGAP(median)      | **66.3±6.7** | **59.0±6.7** | **47.8±8.5** | **40.0±8.5** | **280.5±6.5** | **238.7±5.9** | **219.0±1.8** | **164.2±9.2** |\n|        | +PGAP(75th percentile) | **64.5±5.5** | **57.7±4.7** | **48.7±7.5** | **40.7±8.3** | **286.8±2.1** | **234.5±8.0** | **217.8±2.8** | **161.8±9.5** |\n\n**Q3:**\nThank you for this question, which is crucial for understanding the broader applicability of our work. This point was also raised by another reviewer, and we have formulated a detailed response. For the sake of brevity and to avoid repetition, we respectfully refer the reviewer to our response to **Reviewer 7Uc5, Q2**. In that response, we outline a concrete path for extending PGAP to nonlinear models. We discuss how the core concepts of \"deviation\" and \"propagation\" can be generalized to a functional domain and argue that the same topological patterns serve as principled signals of error. We also use a local approximation perspective to provide a concrete intuition for the propagation mechanism and discuss why this phenomenon is fundamental to causal discovery.\n\n[1] C Deng, K Bello et.al. Markov Equivalence and Consistency in Differentiable Structure Learning."}, {"review_id": "s1uJn4DFPk", "reviewer": "Reviewer_xCbJ", "summary": "The paper studies the effect of using imprecise prior in structure learning for linear DAG when the prior knowledge supplied by domain experts is misspeficied. The paper shows enforcing such priors can create edge weight deviation, which then propagates and spawns spurious edges. Two structural patterns are identified: triangles and double collisions. The expected number of apearance of such patterns is derived. This observation is used in a two-stage framework called Pattern-Guided Adaptive Prior (PGAP). PGAP detects the patterns after an initial run of any differentiable methods, then adaptively down-weights the priors before a second optimization pass. Extensive synthetic benchmarks and experiments are conducted to support the proposal.", "questions": "- The proposed framework is based on heuristics. Is it going to hurt the performance if the prior is correctly specified? More formally, are the identified stuctural patterns sufficient and necessary conditions for imprecise priors?\n- In the experiments, what are the priors and how are they specified? Are they precise or imprecise?", "limitations": "Limitations are discussed in Appendix E.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strength:**\n\n- The paper provides a fresh and quantitative viewpoint on using priors during structure learning. The observations come from solid theoretical analysis of the consequence of imprecise priors.\n- PGAP is a light wrapper around any continuous DAG learner thus has algorithmic simplicity. The pattern detection is local and cheap.\n- The experiments are comprehensive and support the effectiveness of the proposed framework and show consistent empirical benefits.\n\n**Weakness:**\n\n- One of the main contributions is the proposal of PGAP framework. However, it is only briefly described in words in the main paper. The full details are burried in the appendix. It is unclear how does the framework work by solely looking at Section 4.3.\n- The proposed framework is inspired by the observation from analysis, while is based on heuristics. There is not consistency guarantee of the framework, see questions.\n- The theoretical analysis of deviation relies on linear SEM thus might not extend to general DAG model. Although some exploratory experiments for nonlinear models are given in the appendix.", "was_revised": true, "final_justification": "The resposne from the authors addressed my questions on priors and experiments. I keep my score unchanged and am inclined to acceptance.", "rebuttal": "We are very grateful to you for your positive assessment and supportive comments on our work. We sincerely appreciate that you found our viewpoint to be \"fresh and quantitative,\" our analysis \"solid,\" and our experiments \"comprehensive.\" We will address the points you raised to further improve the paper and would be happy to answer any further questions you might have.\n\nBefore we address the specific questions, we would like to clarify a key distinction between \"accurate'' and \"precise'' priors, as this concept is fundamental to both of the reviewer's questions and to the core motivation of our paper. In our work, and in the broader literature, prior knowledge is typically assumed to be accurate. This means the prior correctly asserts the existence of a true relationship, for example that the edge $X_i \\rightarrow X_j$ exists in the ground truth graph. Our experiments strictly adhere to this principle by using only accurate priors. However, such priors are rarely precise because they do not specify the exact quantitative strength of that relationship. This strength is the numerical edge weight that a quantitative model like an SEM must learn. The central challenge we address stems from this gap between a qualitatively accurate assertion and the quantitatively precise value required by the model.\n\nWe now answer the specific questions raised.\n\n**Q1:**\nWe thank you for this insightful question regarding PGAP's behavior with a correctly specified prior. Our framework is indeed designed not to be negatively impacted in this ideal scenario. A truly \"correct\" prior (one that is both accurate and precise) would result in zero deviation ($\\gamma=0$). In this ideal case, our theory shows that no error propagation will be induced, and PGAP's mechanism for detecting special patterns is rarely triggered. We will clarify this important point in the revised manuscript to ensure this aspect of our method is clear.\n\nRegarding the relationship between the patterns and imprecise priors, while it is not a strictly necessary and sufficient condition, our paper's core contribution is to establish and leverage the strong, fundamental connection between them. Our argument is built on a three-part foundation of theoretical proof, expectation analysis, and empirical validation. Specifically, our analysis in Lemmas 2 and 3 provides a formal proof that imprecise priors, when they cause edge weight deviation, directly lead to the formation of these two specific patterns. Building on this, our expectation analysis in Lemmas 4 and 5 addresses their utility as a reliable signal. While it is possible for these patterns to occur naturally, our analysis demonstrates that their appearance is a rare event compared to their prevalence when induced by deviation. This makes their detection a strong indicator of a problematic prior integration. Finally, this theoretical linkage is robustly validated through empirical validation. Our experiments consistently show that adaptively responding to these structural signals leads to significant performance improvements, confirming their practical utility. Therefore, using these patterns as a signal is a theoretically-grounded and empirically-validated strategy for identifying errors caused by imprecise priors.\n\n**Q2:**\nIn all our experiments, the priors were generated by randomly selecting a proportion of edges from the true, data-generating DAG. Therefore, all priors used are accurate because they correspond to a true edge in the ground truth graph.\n\nHowever, a key point of our work is to distinguish this from being precise. While an accurate prior correctly identifies the existence of a causal link, it does not specify the exact numerical weight of this relationship, which is the quantitative value the SEM aims to learn. In our experiments, the prior is incorporated as a penalty that encourages a non-zero weight, thus satisfying the qualitative, accurate prior. But because the exact target weight is unknown and unspecified, this enforcement is inherently imprecise. This setup directly models the common real-world scenario where domain knowledge can confirm a causal link but not its precise strength, which is the central challenge our paper addresses.\n\n***\n\n**Regarding the Weaknesses Mentioned:**\n\n* On the clarity of the PGAP framework's description in the main paper, we agree that Section 4.3 is too brief. We will revise the main text to include a more detailed description of the two-stage procedure and the core mechanics of the adaptive adjustment.\n* As discussed in our response to Q1, our work focuses on establishing the strong, fundamental connection between imprecise priors and specific structural patterns, which istheoretically-grounded, and empirically-validated. \n* Finally, regarding the theory's reliance on linear SEMs, we have addressed this in detail in our response to **Reviewer 7Uc5, Q2**, where we outline a concrete path for generalization.\n\nWe hope these clarifications are helpful, and we thank the reviewer again for their encouraging and constructive feedback.\n\nAs a final point, we wish to share our response to another reviewer regarding the diversity of graph models. We conducted new experiments at their suggestion, but due to strict space constraints, we are unable to include the full analysis in that rebuttal. As these results strongly support our paper's claims of robustness and generalizability, we feel it is important to share this analysis with you as well. We have appended that full response below for your consideration and hope you find it informative.\n\nThese experiments generate random DAGs using three distinct models. The Watts-Strogatz (WS) model simulates social and biological networks with high clustering coefficients and short path lengths, where triangular patterns appear more frequently than in a general ER graph. The Stochastic Block Model (SBM) creates a non-uniform, blocky connection pattern by dividing nodes into different blocks, causing our target patterns to be unevenly distributed. Finally, Geometric Random Graphs (GRG) are formed by connecting nodes based on their proximity in a metric space.\n\nThe experimental settings are as follows: for WS, the lattice dimension $k \\in \\{1,2\\}$ and rewiring probability $p=0.3$; for SBM, the number of blocks $k \\in \\{5,10\\}$ and inter/intra block edge probability $p=0.1$; for GRG, the connection radius $k \\in \\{0.2,0.3\\}$. The experimental results under these diverse underlying graphs are shown in Table 1.\n\n**Table 1: Performance of PGAP in diverse underlying graphs and critical parameter settings. In the 'Lattice', 'Radius', and ’Block‘ column, (·) represents the number of edges under the node d and the critical parameter k.**\n\n| Graphs |             |          |     WS     |              |                |              |              |     SBM      |              |              |               |          |    GRG     |              |               |\n| :----: | :---------: | :------: | :--------: | :----------: | :------------: | :----------: | :----------: | :----------: | :----------: | :----------: | :-----------: | :------: | :--------: | :----------: | :-----------: |\n| Method |             | NT-Soft  |            |    +PGAP     |                |              |   NT-Soft    |              |    +PGAP     |              |               | NT-Soft  |            |    +PGAP     |               |\n|  Node  | **Lattice** |    F1    |    SHD     |      F1      |      SHD       |  **Block**   |      F1      |     SHD      |      F1      |     SHD      |  **Radius**   |    F1    |    SHD     |      F1      |      SHD      |\n|   30   | 1 **(60)**  | 53.5±5.0 |  50.7±6.6  | **56.5±4.7** |  **44.8±6.3**  |  5 **(71)**  |   57.1±3.0   |   55.3±4.2   | **58.0±6.5** | **49.8±5.7** | 0.2 **(43)**  | 61.1±3.2 |  26.0±2.8  | **62.9±3.9** | **23.0±3.3**  |\n|        | 2 **(147)** | 50.6±3.6 | 137.7±9.7  | **54.5±4.3** | **107.0±8.2**  | 10 **(64)**  | **58.9±5.4** | **44.5±3.0** |   58.3±8.4   |   44.8±9.3   | 0.3 **(101)** | 62.1±1.7 |  69.5±3.4  | **63.2±1.5** | **62.7±3.1**  |\n|   60   | 1 **(120)** | 56.8±1.4 | 104.7±4.9  | **58.8±1.5** |  **93.3±6.0**  | 5 **(118)**  |   56.0±8.4   |   98.3±9.9   | **57.1±5.3** | **83.5±7.6** | 0.2 **(195)** | 55.5±1.3 | 159.5±6.5  | **56.2±2.9** | **148.0±7.9** |\n|        | 2 **(333)** | 46.9±2.0 | 345.7±11.2 | **50.5±2.5** | **290.5±15.5** | 10 **(118)** |   56.9±3.5   |   90.3±9.3   | **62.5±2.5** | **73.8±3.2** | 0.3 **(368)** | 56.7±1.8 | 280.2±13.6 | **57.5±3.5** | **257.8±8.9** |"}]

Author rebuttal:

Meta-review: The paper analyzes how imprecise edge priors distort continuous DAG learning in linear SEMs, proving that prior-induced weight deviations propagate as first-order errors that manifest exactly as triangles and double collisions; it then introduces PGAP, a lightweight wrapper that detects these motifs after an initial run and adaptively down-weights the offending priors before re-optimization. After rebuttal, all reviewers lean accept, noting clearer statement of assumptions (linearity, acyclicity, noise models), stronger justification that the two motifs are exhaustive for first-order propagation, added real-data results (PaidSearch, Magic-niab, Ecoli70) and diverse graph families (WS, SBM, GRG), plus a data-driven tuning scheme that stabilizes performance on dense graphs; clarifications on “accurate vs. precise” priors and practical guidance also helped. The remaining limits are that theory stays in the linear/faithful setting, PGAP has no formal consistency guarantees, and dense-graph gains are more sensitive (though mitigated).
Decision: Accept (poster)