Title: Less Greedy Equivalence Search
Venue: NeurIPS 2025
Accepted: yes
Keywords: causality

reasoning under uncertainty

causal discovery

structure learning
Abstract: Greedy Equivalence Search (GES) is a classic score-based algorithm for causal discovery from observational data.
In the sample limit, it recovers the Markov equivalence class of graphs that describe the data.
Still, it faces two challenges in practice: computational cost and finite-sample accuracy.
In this paper, we develop Less Greedy Equivalence Search (LGES), a variant of GES that retains its theoretical guarantees while partially addressing these limitations.
LGES modifies the greedy step; rather than always applying the highest-scoring insertion, it avoids edge insertions between variables for which the score implies some conditional independence.
This more targeted search yields up to a $10$-fold speed-up and a substantial reduction in structural error relative to GES.
Moreover, LGES can guide the search using prior knowledge, and can correct this knowledge when contradicted by data.
Finally, LGES can use interventional data to refine the learned observational equivalence class.
We prove that LGES recovers the true equivalence class in the sample limit, even with misspecified knowledge.
Experiments demonstrate that LGES outperforms GES and other baselines in speed, accuracy, and robustness to misspecified knowledge.
Our code is available at https://github.com/CausalAILab/lges}{https://github.com/CausalAILab/lges.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a Generalized- (GGES) and two variants of Less Greedy Equivalence Search (LGES) algorithms that hold identifiability results in a generalized setting, by modify the edge insertions step of the well-known GES algorithm. In a first step the authors prove that under particular conditions choosing any score-increasing edge insertion within GES still leads to identifiability of the true Markov Equivalence Class (MEC), even if this does not correspond to the most score-increasing edge insertion. They, furthermore, reason that performing particular edge insertions between two variables, A-B, with the largest score increase might not be optimal in the presence of similar, score decreasing insertion operations on A-B under the same MEC. In consequence two approaches are presented: a 'conservative' variant that only selects edge insertions with no score-decreasing insert throughout the whole MEC, and a 'safe' variant the considers score-decreases from an arbitrary DAG sampled from the MEC.\n\nThe authors furthermore prove that GGES can start from arbitrary MEC, and also softly incorporate and correct for (possibly misspecified) prior knowledge, dividing given knowledge into four categories of (weakly) required and forbidden knowledge, that get prioritizes in the search, but ignored if inconsistent with the current MEC. Finally, they a variant for the incorporation of interventional data for LGES via a novel \"I-Orient\" algorithm is presented that identifies an the I-MEC, based on additional interventional data.\n\nExperiments are performed on Erdős–Rényi graphs with p variables 2p edges and up to 150 variables on data of randomly sampled additive linear Gaussian noise models. The presented experimental results on the synthetic ER graphs indicate consistent superior performance over the compared to GES and GIES algorithms for observational and interventional data in terms of inference performance (SHD, Precision, Recall) and overall runtime. Similarly, the authors show in the appendix on-par performance for real-world protein signaling data between their LGES and the standard GES algorithm. Similarly, the recovery from misspecified knowledge shows good recovery rates over GIES.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper provides overall significant theoretical results and insights which do result in seemingly large empirical performance increases, and might have lead to further developments on edge-selection strategies beyond this paper. While I do not believe, that conclusions drawn from experiments will largely change in qualitaty, my main critique is the omission of a possibly recent competitive xGES baseline and the decision to not consider the common choice of phase repetition within GES baselines, (but also the proposed LGES itself).\n\n**Strengths**\n\n1) The paper is generally well written. The authors provide clear and convincing arguments, towards weaknesses of the original GES algorithm, that limit its efficiency and performance. The modification towards the GGES and LGES variants, are well described. The corresponding proofs for the various theorems and propositions towards correctness and MEC identifiability of the algorithms are well described (as they partly align the original GES proofs) and, to the best of my understanding, seem to be correct.\n2) The theoretical proofs for MEC identifiability under arbitrary score-increasing edge insertion operations provides a novel and significant generalization for the choice of edge selection strategies of GES-class algorithms which might initiate to further developments on this topic.\n3) The experiments are well setup and follow standard procedure on Erdős–Rényi (ER) graphs with sufficient variation on the number of variables, applied metrics and repetitions. The authors are able to demonstrate superior performance of both LGES variants in comparison to the chosen baselines (PC,GES,GIES,NOTEARS) over all experiments in the main paper for MEC identifiability for observational and interventional data, as well as the recovery from misspecified knowledge, in terms of performance and runtime. An additional evaluation on real-world protein signaling data indicates the same performance as the GES algorithm.\n\n\n\n**Weaknesses**\n\n1) While the authors discuss and cite recent this recent GES variant, they do not compare to Extremely Greedy GES (xGES, [1]), which performs insert/delete/turning operations in an interleaved sequence and has also shown strong improvements over the original GES. While this approach is unlikely to improve in runtime, it might pose a more competitive baseline in terms of overall MEC inference.\n2) While the authors make use of mention the additional use of turning phase of [2], the work additionally suggests the repeated application of the forward, backward and turning phases until no further improvements in score can be observed. The presented pseudo code suggests that such repeated applications are not performed for LGES, and (if I did not miss the particular spot) the corresponding parameter in the code is set to false for all GES-like algorithm calls. While the presented results on a single iteration performance show remarkable performance increases, comparisons to GES/GIES with repeated phases might provide for a more competitive comparison. This particularly applies for the recovery of misspecified edges from prior knowledge, where the GES-Init algorithm can have the ability to insert novel edges after the, otherwise final, edge removal phase which is the otherwise only chance to eliminate erroneous edges.\n\n\n\n**Minor Points**\n\n* I would like to recommend to briefly mention the meaning of $Nd^G_Y$ in the background section.\n* Additionally, I believe the term $nd^G_Y$ in line 217 should be capitalized and made bold, similar to the other notation.\n\n\n\n[1] Nazaret, Achille, and David Blei. \"Extremely greedy equivalence search.\" *arXiv preprint arXiv:2502.19551* (2025).\n\n[2] Hauser, Alain, and Peter Bühlmann.  \"Characterization and greedy learning of interventional Markov  equivalence classes of directed acyclic graphs.\" *The Journal of Machine Learning Research* 13.1 (2012): 2409-2464.\n\nQUESTIONS:\n1) Could the authors provide further insights on the changes related to enabling repeated phase application with all LGES and GES-type baselines? In particular, can GES baselines overall improve and is the GES-Init algorithm more able to remove and recover from misspecified edges?\n2) Similarly, could the authors provide further insights in comparison to the performance under the possibly more competitive xGES algorithm? Could the presented LGES itself benefit from interleaved phase applications?\n3) In corollary 2, the authors utilize the property of SafeInsert to operate on arbitrary DAGs of the current MEC in order to consider possible score-decreases. I assume that the corresponding DAG is sampled randomly? Could the authors confirm this or explain their method for choosing a particular DAG?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-22"}

comments: SUMMARY:
This paper introduces Less Greedy Equivalence Search (LGES), which is an improved algorithm for causal discovery that improves upon the classic Greedy Equivalence Search (GES) method. It does so by doing doing edge selection less greedily, instead of greedily maximizing the score. LGES is shown to also leverage possibly misspecified prior assumptions and correct misspecified edges.

STRENGTHS AND WEAKNESSES:
Strength:  The paper carefully develops two new strategies for improving on GES. As the authors wrote, it is the first asymptotically correct score-based procedure for learning from interventional data that can scale to graphs with more than a hundred nodes. LGES is also robust in handling misspecified graphs. And the authors do compare with other algorithms such as PC and NOTEARS, and not jus showing that LGES is only an improvement over GES. 

Weaknesses: As one comment that stood out to me, I think the paper would be more compelling if a more extensive set of experiments was run. It would be more convincing if LGES was benchmarked on a real world dataset (perhaps Sachs which was considered in NOTEARS), as well as synthetic graphs of varying sparsity.

QUESTIONS:
I felt the paper was fairly clear and I liked how simulation was used to gain intuition on where GES may not do well. My only question is regarding a more extensive set of experiments, which in it of itself would reveal the algorithm's benefits. Finally, I will add that I'm also not super familiar with score based methods for CD and the associated literature, so perhaps other reviewers can champion the paper if they feel strongly about it.

LIMITATIONS:
Please see weakness.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-30

comments: SUMMARY:
The authors propose a variant of GES that changes the forward phase. Most importantly, they show that if an Insert operation between X and Y decreases the score, then (some) other Insert operations between those variables do not need to be considered. Further, the algorithm allows the incorporation of background knowledge (but will ignore this if it contradicts the data) and can utilise interventional data. Experiments show this new algorithm is both faster and more accurate than the original GES.

STRENGTHS AND WEAKNESSES:
This paper makes an important contribution, which is accompanied by theoretical proofs of correctness and thorough empirical evaluation. The paper is relevant both for researchers developing structure learning algorithms and for researchers wanting to apply them. Furthermore, the paper is clearly written.

QUESTIONS:
Q1: I could not quite follow the discussion in section 3.2 about a potential issue with ConservativeInsert. If I understand correctly, the issue is that sometimes, ConservativeInsert may discard the only score-increasing Insert(s). However, from Proposition 1, it seems that such score-increasing Inserts are not desirable because they create d-connections where we want d-separations. Can you give an example to show why it is nonetheless preferable to use a strategy that does consider such Inserts?

### Minor suggestions that don't impact my rating:
- on page 2, the period after "starts the search with $\mathcal{E}$" should be a comma
- I believe the "If not" in Strategy 2 should be "If so"
- in Example 3, $X$ should be $X_1$

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 4
ORIGINALITY: 3
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIt is a known practical issue that the GES algorithm may often produce denser graphs than truth with finite samples. This is partly because when an edge is added in the forward phase, only a single check of score increasing is performed. When this added edge is incorrect, the chance for it to be deleted in the backward phase is often small.\n\nTo deal with this issue, this paper proposes the algorithm \"Less Greedy Equivalence Search\" (LGES). Instead of adding an edge whenever a score increasing is obtained, LGES avoids adding edges even with increased scores whenever it implies **some (other)** conditional independences. Two variants of the insertion steps, named \"Conservative Inserts\" and \"Safe Inserts\" are proposed, one more aggressive with partial guarantees, and another with full guarantee.\n\nFurther, extensions to incorporate prior knowledge (even when some priors can be misspecified) and interventional data are provided.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1. The problem to tackle is quite relevant -- the practical issue of often-denser results given by GES indeed needs to be addressed.\n\n2. The key motivation is clear and natural: in each step of the progression, it doesn't have to be the best score to be moved to. As long as the score is increasing it is a valid move. This principle leaves with space to strategically choosing among all candidates to get a balance between greediness and robustness.\n\n3. The running examples used throughout is clear and helpful.\n\n\n**Weaknesses:**\n\n1. The implication of score decreasing to incorrect adjacency (Proposition 1) is sound and useful. **But is it complete (seems no)?** In other words, does the score decreasing in Proposition 1 capture **all** (or just some) possible conditional independence patterns that leads to non-adjacencies? Also, do the two insertion variants capture all what can be captured by Proposition 1? If the answers are no, the authors may need to justify why such heuristics is good enough in avoiding false adjacencies.\n\n2. **The claim for speed-up needs more justification.** Intuitively, such more robust decision for adding edges comes with more tests: instead of just testing between X and Y given Y's current parents, now all possible neighbor sets T need to be tested (e.g., in conservative strategy). Then, why is LGES finally faster? Is it because that less edge is inserted and the graph in progression is sparser? Some kind of complexity analysis is expected.\n\n3. **The backward phase (and the optional turning phase) is not optimized accordingly.** Intuitively the dual operations of choosing stable (not necessarily the best) neighbors can be applied to the backward phase to avoid false non-adjacencies. This also makes the whole algorithm more self contained and symmetric. I am curious why the authors didn't implement that. Is it because that it will usually hurt GES's performance?\n\n4. **The contributions on dealing with interventional data and prior knowledge seem a bit irrelevant.**\n   - For the interventional data, when one treats the experimental domain index as an additional root variable, the whole discovery method becomes nothing but to run plain GES (or any constraint based algorithms) on all variables including the domain index. Under this, I cannot see clearly why the authors need to separate this as a specific contribution. Or, are there any more specific optimization made to algorithmic procedure for interventional data?\n   - For incorporating prior knowledge, the idea of seeing prior knowledge as priority constraints instead of hard initialization is interesting. (Though it is still worth discussing whether data or prior knowledge should be prioritized, or should be customizable to the users). Could the authors confirm whether this idea is newly proposed in this paper, or it is also already used in some existing works? Usually how do other score based variants incorporate background knowledge?\n   - Overall, the technical discussions regarding these two aspects are too short in the main body. For example, the very important algorithm \"GETPRIORITYINSERTS\" is deferred to the appendix. And similarly many other references are deferred (e.g., Defs. A.3, A.4, A.5), making hyper links not work in main body. More detailed analysis of these two contributions are expected. They are now too vague.\n\n5. **Many existing literature on GES are ignored.** There are many existing GES variants aiming to address different aspects of issues. For example, \n   - \"Statistically Efficient Greedy Equivalence Search\" (http://proceedings.mlr.press/v124/chickering20a/chickering20a.pdf), \n   - \"Fast GES\" (https://www.ccd.pitt.edu/wp-content/uploads/2018/10/FGES1c-user-documentation-5_21_2016-sample-size.pdf; this one, together with the point above that interventional GES is nothing different than plain GES, make the claim on \"the first ... for learning from interventional data that can scale to ... more than a hundred node\" suspicious), \n   - \"Extremely Greedy Equivalence Search\" (https://proceedings.mlr.press/v244/nazaret24a.html; this is cited in the paper but not compared), etc.\n\n   Could the authors please have a subsection reviewing those existing variants of GES, and in the experimental section, compare with them all? This is important since the purpose of this paper is for practical gain.\n\n6. **The SafeInsertion strategy needs some more elaboration:**\n   - \"nd_Y^G\" seems undefined?\n   - Is it enough to pick up only one arbitrary DAG from the CPDAG, or should we iterate all DAGs (to find the highest score and to ensure the existence of a score increasing neighbor)?\n   - \"Check if G has a higher score .. if not, discard ..\": then what if the score is unchanged, or an unchanged score is provably impossible (even with possibly misspecified graphs)?\n   - At the first glance the SafeInsertion strategy looks very similar to the original strategy in GES: pick DAGs from CPDAG and then check if X is independent of Y conditioned on Y's current parents. Could the authors highlight the biggest difference there?\n\n\n\n7. (Minor) Sentences like \"This more targeted search yields up to a 10 -fold speed-up and 3 -fold reduction in structural error relative to GES\" seems inappropriate in abstract, without a clear scope and simulation setup.\n\nQUESTIONS:\n/\n\nLIMITATIONS:\n/", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}