Title: Learning to Condition: A Neural Heuristic for Scalable MPE Inference
Venue: NeurIPS 2025
Accepted: yes
Keywords: Most Probable Explanation

Probabilistic Graphical Models

Conditioning Heuristics
Abstract: We introduce learning to condition (L2C), a scalable, data-driven framework for accelerating Most Probable Explanation (MPE) inference in Probabilistic Graphical Models (PGMs)—a fundamentally intractable problem. L2C trains a neural network to score variable-value assignments based on their utility for conditioning, given observed evidence. To facilitate supervised learning, we develop a scalable data generation pipeline that extracts training signals from the search traces of existing MPE solvers. The trained network serves as a heuristic that integrates with search algorithms, acting as a conditioning strategy prior to exact inference or as a branching and node selection policy within branch-and-bound solvers. We evaluate L2C on challenging MPE queries involving high-treewidth PGMs. Experiments show that our learned heuristic significantly reduces the search space while maintaining or improving solution quality over state-of-the-art methods.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Learning to Condition (L2C), a neural heuristic framework for scalable Most Probable Explanation (MPE) inference in Probabilistic Graphical Models (PGMs). The core idea is to train an attention-based neural network to score variable-value assignments based on their utility for conditioning, balancing the preservation of optimal solutions with the simplification of inference. The model is supervised using a scalable data pipeline that extracts training signals from the search traces of existing MPE solvers. L2C is integrated with search algorithms, serving as a conditioning strategy before exact inference or as a branching and node selection policy within branch-and-bound solvers. Experiments on high-treewidth PGMs demonstrate that L2C significantly reduces the search space while maintaining or improving solution quality compared to state-of-the-art methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Innovative Integration: Combines neural networks with symbolic reasoning, using solver-guided feedback to ensure semantic correctness of generated constraints.\n- Data Efficiency: The supervision strategy leverages solver statistics without requiring exhaustive MPE enumeration.\n- Flexible Deployment: Functions as both a pre-processing conditioning strategy and a branching heuristic within branch-and-bound solvers.\nWeaknesses:\n- Binary Limitation: Assumes all variables are binary (\"w.l.o.g.\", Sec. 2). This restricts applicability to real-valued variables, limiting generalization beyond discrete domains.\n- MAXSAT/PB Solver Comparison: Lacks direct comparison with pseudo-Boolean (PB) solvers, missing an opportunity to contextualize novelty against established methods.\n\nQUESTIONS:\n- The conditioning problem resembles MAXSAT (weighted constraint optimization). How does L2C compare to state-of-the-art PB solvers (e.g., using cutting planes or conflict analysis) in efficiency?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces L2C, a novel neural method that accelerates MPE inference in PGMs. By learning from exact MPE solvers, L2C identifies strategic variable assignments that effectively balance problem simplification with solution optimality, enabling it to serve as both a powerful pre-processor and an intelligent heuristic within B&B algorithms. Empirical results demonstrate L2C's superior performance over traditional methods, significantly improving the efficiency and tractability of MPE problem-solving.\n\nSTRENGTHS AND WEAKNESSES:\n# Pros:\n\n1.The method is technically sound, features a cleverly designed training objective, and is strongly supported by comprehensive empirical results.\n\n2.L2C effectively addresses a critical challenge in MPE inference, making previously intractable problems solvable.\n\n3.The approach of learning conditioning strategies by combining deep learning with classical MPE solvers is highly novel.\n\n# Cons:\n1.If this is the first application of Attention to MPE problems, it would be very significant. However, I'd like to ask, how significant is the difference between L2C and the attention mechanisms in classical Graph Neural Networks?\n\n2.Reliance on computationally expensive 'oracle' solvers might limit its application to large-scale problems. Could we try a Bootstrap method to accelerate data collection?\n\n3.The core insight of the paper is: \"A key insight of our method is that solution ambiguity modulates conditioning risk.\" This statement is somewhat straightforward. It's quite intuitive, much like \"high risk, high reward\". Could you elaborate further?\n\n4.A more in-depth explanation of the specific information and contextual relationships captured within the learned variable-value pair embeddings would improve clarity. As I am not an expert in this field, it took me a considerable amount of time to understand the specific mechanism of the embeddings.\n\n5.The paper's discussion on L2C's applicability or extensions to other inference types or continuous variables is limited.\n\nQUESTIONS:\nSee Cons. Specifically:\n\nFirst question: Is this the first application of Attention to MPE problems?\n\nSecond: How significant is the cost of data collection?\n\nThird, this is a question.\n\nFourth, this is a suggestion.\n\nFifth, this is a suggestion.\n\nLIMITATIONS:\nThe authors only discuss limitations in the \"Future work\" section, specifically stating \"Future work will target L2C’s current limitations in scalability, solver signal quality, and the scope of learned guidance.\" (lines 396), which is insufficient for a comprehensive discussion of shortcomings.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a new data-driven framework called Learning to Condition (L2C) for tackling the Most Probable Explanation (MPE) inference problem in probabilistic graphical models. The MPE task in graphical models is known to be computationally intractable. Specifically, the proposed L2C method trains a neural network to score variable-value assignments based on their utility for conditioning. Essentially, L2C can generate cutset conditioning sets that once instantiated are likely to yield a much simplified MPE instance that can be solved efficiently by existing MPE solvers. In addition, the L2C approach can be used as bot variable ordering as well as value ordering heuristics for specialised search-based solvers for MPE such as MIP solvers or AND/OR search based solvers. The experimental evaluation is carried out on standard graphical models benchmarks. The results demonstrate the benefits of the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n---------\n\nThe MPE (or MAP) inference task in graphical models is a fundamental yet very difficult task with a wide range of real-world applications. Therefore, developing new methods to solving it more efficiently in practice is definitely called for.\n\nThe paper is fairly well written and organised. Therefore, the content is relatively easy to follow. The quality of the presentation is overall quite good.\n\nThe empirical evaluation, although fairly limited, is sound and the results are presented in a clear manner which makes it relatively easy for the reader to appreciate the benefits of the proposed method.\n\n\nWeaknesses\n----------\n\nThe empirical evaluation appears to be limited to a few classes of problems from the UAI Inference Challenge Benchmark (the BN and Grid instance are actually random instances) and the training dataset seems to be generated from the same distribution. Therefore, the performance gains highlighted in the experiments aren't actually surprising. The problem instances shown in Figure 2, although they have relatively high tree widths, are actually fairly easy for exact MPE/MAP inference.\n\nState of the art branch and bound algorithms for MPE/MAP exploit both variable as well as value ordering heuristics. Value ordering heuristics make a huge difference in the case of non-binary domains. However, the benchmarks considered in the experiments (as well as the presentation of the L2C methodology) only considers the case of binary variables.\n\nThe experimental evaluation ignores two classes of MPE algorithms: the local search MPE algorithms by Hutter et al, IJCAI-2005 and the exact soft arc-consistency based search algorithms from the toolbar2 suite. The latter use dynamic variable and value ordering heuristics and L2C would be more relevant for them instead of AOBB. The AOBB version considered in this paper, although quite powerful, is limited to a static variable ordering given by the pseudo-tree. \n\nFinally, the MIP formulation for MPE/MAP is known to be not very effective. MIP solvers are typically outperformed by dedicated solvers such as as toolbar and AOBB.\n\nQUESTIONS:\n- Regarding transferability, it would be interesting to train L2C on a given distribution (e.g., such as the one derived from the BN/Grid instances) and experiment with problem instances from a different distribution such as the pedigree or protein networks from the UAI Challenge Benchmark. What would be the performance gap in that case?\n\n- L2C looks very similar to the learning to branch approaches developed recently for mixed integer programming (see ref [31]). What is the conceptual difference between the two approaches?\n\nLIMITATIONS:\nThe proposed L2C method appears to be limited to binary variables.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work aims to scale the MPE inference by using a neural network-based conditioning algorithm. It includes first constructing the learning data by collecting subset variable assignments along with the computational costs. Then it uses these data to train a neural network that predicts 1) the likelihood of variable assignments being part of the MPE solution and 2) how efficient the candidate assignments are. At inference time, these scores are combined to select conditioning assignments. Experimental results on some PGMs are presented along with a comparison with a graph-based heuristic baseline and a full strong branching heuristic baseline.\n\nSTRENGTHS AND WEAKNESSES:\nThe problem of choosing a subset to condition as a way to speed up MPE inference on PGMs is important and challenging. The learning based approach in this work is novel to me. This paper is overall well written, with a thorough discussion on previous work, proposed methods clearly elaborated, and sufficient background for the readers to understand this work. Still, I have a few concerns:\n- It is unclear to me how the PGMs in the data collection process are generated. It seems to me that the learned heuristic only works well when the PGM at inference is similar to those in the data collection process; for example, if the PGMs for training data have relatively low treewidth and the PGM at inference time has high treewidth, very likely the learned heuristic won't perform well since this is out of distribution data.\n- The other thing unclear to me is how flexible the learned heuristics are. Can it be applied to various cases when the sizes of the conditioning sets are different, or does it require retraining for each case?\n- I think this work will benefit from having some toy example where the optimal MPE solution is known, and comparing how accurate the predictions from the learned heuristics are.\n\nQUESTIONS:\nIn addition to those raised in [Strengths And Weaknesses]:\n- How well does the learned heuristic perform for out-of-distribution data?\n- Many hyperparameters are involved: sampling parameter in the data collection process, the weights in the loss function, size of the conditioning set, etc. How are these hyperparameters chose,n and how sensitive is the performance to these hyperparameters?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-02"}