Title: Towards Better Branching Policies: Leveraging the Sequential Nature of Branch-and-Bound Tree
Venue: NeurIPS 2025
Accepted: no
Keywords: Mixed-Integer Linear Programming

Branch-and-Bound

Branching Variable Selection Policy

Generalization

Mamba
Abstract: The branch-and-bound (B\&B) method is a dominant exact algorithm for solving Mixed-Integer Linear Programming problems (MILPs). While recent deep learning approaches have shown promise in learning branching policies using instance-independent features, they often struggle to capture the sequential decision-making nature of B\&B, particularly over long horizons with complex inter-step dependencies and intra-step variable interactions. To address these challenges, we propose Mamba-Branching, a novel learning-based branching policy that leverages the Mamba architecture for efficient long-sequence modeling, enabling effective capture of temporal dynamics across B\&B steps. Additionally, we introduce a contrastive learning strategy to pre-train discriminative embeddings for candidate branching variables, significantly enhancing Mamba's performance. Experimental results demonstrate that Mamba-Branching outperforms all previous neural branching policies on real-world MILP instances and achieves superior computational efficiency compared to the advanced open-source solver SCIP. The source code can be accessed via an anonymized repository at https://anonymous.4open.science/r/Mamba-Branching-B4B4/.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents Mamba-Branching, a deep-learning-based branching strategy designed to enhance the Branch-and-Bound (B&B) algorithm for solving Mixed-Integer Linear Programming (MILP) problems. The key innovations include treating the branching path as a sequence to capture the influence of past decisions, using the Mamba architecture for efficient long-sequence processing, and applying contrastive learning to pre-train variable embeddings. The empirical results demonstrate significant improvements over existing neural branching strategies and even outperform SCIP’s default heuristic on challenging instances.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n- The paper systematically incorporates the sequential nature of B&B branching paths into a deep model, supported by clear theoretical motivation.\n- The introduction of the linear-complexity Mamba architecture accelerates the branching process.\n- Mamba-Branching shows consistent improvements across multiple metrics on the MILP-S and MILP-L test sets, even surpassing SCIP’s default relpscost heuristic on hard instances.\n- The paper provides detailed descriptions of the contrastive learning and sequence-modeling modules, each validated via ablation studies.\n\n### Weaknesses\n- The ablation table (Table 3) omits T-BranT’s performance, making direct comparison difficult.\n- The ablation studies are limited in scope and could be expanded to include evaluations of contrastive learning with standard sequence modeling and the effect of positional encodings within the sequence-modeling module.\n- The paper does not compare Mamba-Branching against other recent sequence-aware neural architectures, such as those introduced in “Exact Combinatorial Optimization with Temporo-Attentional Graph Neural Networks,” leaving a gap in the evaluation of its relative merits.\n- The paper acknowledges that Mamba-Branching does not universally outperform SCIP’s default relpscost heuristic, particularly on simpler instances, indicating that the approach may be more suitable for certain types of MILP problems.\n\nQUESTIONS:\n1. **Notation Clarification (Lines 197–198):** Could the authors confirm whether \\(o_t\\) should indeed be \\(o_T\\) in the definition provided in Lines 197–198?\n2. **Proof of Proposition:** Could the authors explain how to derive Proposition 1?\n2. **Inconsistent Baseline Performance (Table 1):** T‑BranT’s results are consistently worse than TreeGate’s, which seems to contradict the numbers reported in Tables 6–7 of \"Learning to Branch with Tree‑aware Branching Transformers”?\n3. **Missing Ablation Data (Table 3):** Why is T-BranT’s performance omitted from the ablation table, and could the authors provide this data to facilitate a more comprehensive comparison?\n4. **Incomplete Ablations.**\n    Only the presence or absence of contrastive learning is tested. It would be informative to also evaluate:\n   - Contrastive learning with standard (non‑Mamba) sequence modeling, and\n   - The specific effect of positional encodings within the sequence‑modeling module.\n\nLIMITATIONS:\n- The paper’s claims are primarily validated on two specific MILP test sets (MILP-S and MILP-L), which may limit the generalizability of the results to other types of MILP instances.\n- The paper does not explore the potential of reinforcement learning-based approaches, which could offer a different set of advantages and challenges in the context of branching policies.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents Mamba-Branching, a novel learning-based branching policy for branch-and-bound (B&B). The approach leverages the Mamba architecture to model the temporal dynamics across B&B iterations and incorporates a contrastive learning framework to pretrain the node embedding layer. Empirical evaluations on real-world MILP datasets show that Mamba-Branching consistently outperforms prior neural baselines and, in some cases, even surpasses the classical solver SCIP.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1. To the best of my knowledge, this is the first work to explicitly model step-wise temporal dependencies in B&B, addressing a significant gap in neural branching research.\n2. The proposed method demonstrates robust performance on large, heterogeneous MILP instances (MILP-L), where prior neural methods often struggle. In some settings, it even outperforms SCIP.\n3. The paper is well-written and well-structured, offering clear explanations of both the methodology and experimental results.\n\n\n**Weakness**\n1. The performance gains over strong baselines, such as TreeGate (on MILP-S) and T-BranT (on MILP-L), are relatively modest. As shown in Figure 2, Mamba-Branching leads in only about 50% of the instances. The paper would benefit from statistical significance testing (e.g., paired t-tests or Wilcoxon signed-rank tests) to validate the robustness of these improvements.\n2. There remains a large performance gap in node count compared to the relpscost heuristic. It is also unclear whether the reported advantages on MILP-L are sustained on less powerful hardware than the A100 GPU used.\n3. The **motivation for using a contrastive loss** is underdeveloped. While the loss function resembles Deep Graph Infomax (DGI) [2] more than CLIP [1], the paper provides **little justification** for why maximizing local-global mutual information is particularly suitable for node embeddings in B&B.\n4. The **connection to established contrastive learning approaches** in NLP or vision (e.g., CLIP) appears weak, especially considering the **supervised nature** of the downstream task.\n5. Given this supervision,**simpler pretraining strategies** (e.g., training a linear classifier on node labels and discarding it) should be considered and empirically compared as baselines.\n\n\n[1] Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision (2021), https://arxiv.org/abs/2103.00020\n\n[2] Petar Veličković, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. 2018. Deep Graph Infomax. arXiv:1809.10341\n\nQUESTIONS:\n1. **Regarding Table 2**, Mamba-Branching explores more nodes than the relpscost heuristic but achieves a smaller primal–dual gap.\n\n- Could this improvement be primarily attributed to **GPU acceleration**?\n\n- Have you evaluated the method’s **performance on less powerful hardware**, such as mid-range GPUs or even CPU-only settings?\n\n- Additionally, could you provide **per-instance comparisons** between Mamba-Branching and relpscost on the **largest few MILP-L instances**?\n\n2. Instead of contrastive pretraining, have you considered **standard supervised pretraining** for the embedding layer?\n\n- For example, by attaching **a linear classification head** to predict the best branching feature, training it to convergence, and then discarding the head.\n\n- How would such a supervised approach compare to contrastive pretraining in terms of **downstream branching performance**?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose Mamba-Branching, a method for enhancing heterogeneous MILP solving. Compared to Transformer- and GCNN-based baselines, Mamba-Branching demonstrates higher efficiency and stronger capability in capturing temporal dynamics across branch-and-bound (B&B) steps. The introduction of contrastive pretraining further boosts its performance. Experimental results show that Mamba-Branching excels particularly on hard instances—those unsolved within a 3600-second time limit—achieving the lowest primal-dual integral (PD Integral), even outperforming the expert policy used for imitation learning. However, on easier instances, its improvements are marginal over the baseline TreeGate, and it does not surpass the expert policy, relpcost.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1.  The paper is clearly structured and well-articulated, making it easy to follow the motivation, methodology, and results. The presentation effectively guides the reader through complex technical details without sacrificing clarity.\n2. The application of Mamba to model the trajectory of branch-and-bound (B&B) decisions is both novel and insightful. By capturing the temporal evolution of the search tree, the method leverages sequential dependencies across B&B steps to enhance branching decisions, demonstrating a deep understanding of the problem structure.\n\nWeaknesses:\n1. Inappropriate evaluation metric for easy instances: The paper uses the number of nodes explored as the evaluation metric for easy MILP instances, justifying it by stating that it \"directly impacts overall solving time. (https://arxiv.org/pdf/1906.01629)\" However, a more appropriate and direct metric for easy instances is the solve time itself. In such settings, node count may not reliably reflect actual solver efficiency.\n2. No advantage in solve time over baselines: From the provided code and result logs, it is evident that Mamba-Branching does not outperform the TreeGate baseline in terms of actual solving time. In fact, Mamba-Branching often requires longer wall-clock time, suggesting that its more complex modeling of B&B trajectories, while conceptually interesting, does not translate into practical speedup—especially in easy instances where fast convergence is crucial.\n\nQUESTIONS:\nQ1. Could the authors provide evaluation based on solve time, especially for easy MILP instances, as discussed in the Weakness section? From the code and result logs, it appears that Mamba-Branching is actually slower than the TreeGate baseline and probably much slower than relpcost expert stratery, suggesting no clear advantage in actual solver efficiency. The calculation could be 1-shifted geometric mean of the solving times in seconds, following https://arxiv.org/pdf/1906.01629.\n\n\nQ2. I understand that heterogeneous MILP solving with machine learning is a relatively challenging task. In fact, none of the recent works—including TreeGate, T-BranT, and the current Mamba-Branching method—have been able to outperform the expert branching policy relpcost. This is problematic because the whole idea behind imitation learning is to either:\n* (1) Approximate a powerful but slow expert policy in order to accelerate solving; for instance, in the context of ML for B&B, the GCNN paper (https://arxiv.org/pdf/1906.01629) uses strong branching—a very costly but high-quality decision policy—as the expert. They then train a neural network to imitate strong branching, achieving faster solving on homogeneous MILPs.\n* (2) Use imitation learning as a pretraining stage for reinforcement learning, with the goal of surpassing the expert policy via fine-tuning.\n\nHowever, TreeGate, T-BranT, and Mamba-Branching fail to realize either of these benefits. On easy MILP instances, their performance in terms of both node count and solve time is still worse than relpcost. But if the learned policy doesn't outperform the expert in any meaningful way, then just using relpcost directly is better.\n\nQ3. To be fair, this paper offers meaningful technical innovations. If the authors are interested in strengthening the work further or move forward to the next paper, I suggest two possible directions:\n\n(1) Apply Mamba to homogeneous MILP solving, following the setup in https://arxiv.org/pdf/1906.01629. That paper shows that a learned policy can outperform traditional methods—including strong branching and relpcost—on time efficiency. Your model could potentially improve decision quality due to Mamba’s ability to model temporal structure. The key is to balance the computational cost of Mamba with the gain in decision quality.\n\n(2) If the focus remains on heterogeneous MILP solving (as in this paper), I strongly encourage the authors to carefully revisit the hand-crafted features used in TreeGate. These features may not be well-suited for the type of expert policy being imitated. Specifically, relpcost is not a single coherent policy but a mixture of two strategies, i.e.,  strong branching and pseudo-cost branching. If you examine relpcost in detail, you'll find it blends strong and pseudo decisions. However, features extracted in TreeGate—such as SCIPgetAvgPseudocostScore in the average score vector—are biased toward pseudo branching. As a result, the learning model may simply overfit to pseudo-branch decisions because they are easier to imitate given the input representation. This partially explains why both TreeGate and T-BranT report high imitation accuracy, but not necessarily meaningful performance gains in solving. The expert actions the models are imitating are pseudo branching which is of lower quality than strong branching.\n\nTherefore, instead of relying on TreeGate’s features and mimicking relpcost, I suggest exploring the GCNN-based features and strong branching as the expert. Admittedly, this is difficult. As you have also shown empirically, GCNN + strong branching, which works well in homogeneous settings, can be hard to transfer to heterogeneous ones. This might be due to the limited expressiveness of GCNNs in capturing features across MILPs. Hence, future work could focus on strengthening the model architecture and leveraging large-scale pretraining to improve generalization.\n\n**Final remark:** Overall, this is a well-written paper with thoughtful modeling and clear motivation. However, because it follows the evaluation setup and assumptions of prior works that may not be rigorous—particularly in metric selection, input features and imitation target choice—I believe it falls short for acceptance at NeurIPS. The paper may be better suited for other venues.\n\nLIMITATIONS:\nSee the Questions.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the popular problem of using machine learning for branching in MILP solving. Different from many other papers, this paper pays attention to the features of the branch-and-bound tree instead of individual node features. To leverage tree features, it uses the sequence of node expansion and models it with the Mamba architecture. Empirical results show that the method expands fewer nodes than other state-of-the-art methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. Tree features are important in Branch-and-Bound. The paper leverages Mamba as a new architecture to inform branching decisions with those features. This is a novel aspect of the paper.\n2. Experiments are done on real-world instances from MIPLIB and CORAL and look promising. \n3. Source code is provided. \n\nWeaknesses:\n\n1. Experiment results are reported only on 8+73‎ = 81 instances. I can’t tell if the results are statistically significant. Especially on real-world instances, the runtime and number of nodes often has very high variance. Simply reporting the average of nodes and PD gap is not enough. I understand that difficult real-worlds instances are not that many. One way that you could obtain results on more instances is by rotating the train and testing sets. (For example, divide the instances into 4 groups and train on all combinations of three instance groups like similar to cross validation.) Also, you could evaluate the results on distributional settings like most other papers have done. You can obtain a lot more instances in distributional settings: https://sites.google.com/usc.edu/distributional-miplib/\n2. The first ever paper on learning to branch needs to be in your reference: Learning to Branch in Mixed Integer Programming. Khalil et al., AAAI 2016.\n\nQUESTIONS:\n1. What role does Proposition 1 play in the methodology? It seems to be simply a statement of a property that you wish to have. I don’t see how you achieve such property in the methodology. I.e., how do you enforce such a margin of \\delta? Contrastive learning is a method that can approximate such thing, but it doesn’t enforce it given a \\delta. \n2. Why is strong branching rarely applied in practical scenarios? Why does relpscost provide a more realistic expert representation? Can you provide reference or more explanation? Most existing literature seems to still rely on strong branching to collect expert demonstration\n\nLIMITATIONS:\nSee weakness 1.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies how to learn heuristics for guiding MIP solvers in their branch and bound process. Specifically, and as most works in this field, it focuses on the variable selection heuristic. Contrarily to most previous work, this approach takes inspiration from sequence modeling to solve the control problem of variable selection. The contribution lies primarily in the contrastive loss for representation learning of the token embedding, and in the usage of the previous Mamba architecture to process long sequences.\n\nSTRENGTHS AND WEAKNESSES:\nStrength. The paper is very well written and pleasant to read. The idea is sound and well presented.\n\nBoth strength and weakness. One strong advantage of sequence modeling for the resolution of MDPs is that it ignores the fact that the system to control might not be fully observable (and hence be a POMDP). For the case of variable selection in B&B solvers, this question is particularly important as the state variables have no guarantee of defining a Markov process. So using sequence modeling is an important good choice and a reason for acceptance of this contribution. However, the paper has little discussion on this topic: does the representation of Zarpellon et al. lead to an MDP? If so, then why would a sequence model perform better than a state-based policy? Conversely, if not, does the tree/variable token embedding preserve enough information for decision making (and why/how)? The sequence of token embeddings seems relevant to retrieve such necessary information but this receives little discussion of evaluation. There is a lot to be said here and the paper somehow lacks in this regard.\n\nWeakness. As many works in this field, the contribution relies on imitation learning of an already known good heuristic (pseudo-cost branching and its variants), and conjectures that the sequence model will be able to generalize across instances. Although I believe the sequence modeling aspect is a really good idea, this last (implicit) conjecture deserves better discussion in my opinion. It is not specific to this paper but I find it lacking in most works (including this one) and I believe such studies should bring more to the table than empirical evidence that imitation learning yields good policies. Similarly, one aspect that is overlooked is how this could be included in an optimization process to solve the optimal control problem (in a dynamic programming loop for instance, of in a policy gradient method) rather than doing supervised imitation.\n\nModerate strength. I believe the contrastive learning approach is an interesting contribution. Focussing on representation learning for such problems is often overlooked and this contribution brings the question to light. The empirical evaluation (in particular the ablation study) seems to indicate this part of the contribution is important. I claim no expertise in the contrastive loss itself but the discussion on the matter is interesting.\n\nBoth (moderate) strength and (moderate) weakness. Exploiting the Mamba architecture is very important for this contribution, as it requires to process fairly long sequences and hence strongly relies on the fact that Mamba's forward pass has a complexity that is linear in the number of tokens (contrarily to the quadratic complexity of a Transformer architecture). One could argue that Mamba is not a contribution of this very paper and hence the contribution is limited, but I believe stitching together good ideas is a practice that makes interesting papers and hence is a (moderate) argument for acceptance. However I was a bit disappointed by the resulting sequence lengths. The paper reports using a length of 24 tokens across all models. This seems like a really small number if the sequence is to represent a hidden state of the MDP and retrieve the Markov property, or at least enough information for efficient decision making. This makes me doubt the ability of the contribution to generalize to difficult instances and to larger problems. Why would 24 tokens keep enough memory to accurately represent the state of the B&B tree? The absence of perspective on this aspect makes the contribution less significant.\n\nWeakness. The empirical evaluation uses instances of MIPLIB and CORAL, which is good. However most works in this field use the Ecole benchmark as a testing ground. I believe Ecole's instances have important flaws and strongly appreciate seeing instances from MIPLIB. However, for evaluation fairness, reporting results on Ecole might provide interesting insights and strengthen the paper. So I believe the empirical evaluation is too weak in that regard (and also for the reasons that follow). Similarly, I find it surprising that the GCNN approach  the contribution is compared to is only that of Gasse et al. (2019). There have been alternate works, including that of Zarpellon et al., or Scavuzzo et al. (policy gradient method for branching policies), or Parsonson et al. (retrospective trajectories) and probably a few others that would yield a fair and insightful comparison. On the same page, if my memory is correct, Gasse et al. used strong branching as the expert to imitate, and fine-tuned their approach for this expert. Here the paper uses reliability pseudo-cost branching which is a stronger (and more complicated) heuristic. To fully and fairly assess empirically the importance of the sequence model and its ability to generalize (across experts and across instances), one would expect a comparison on several imitation experts. This is particularly important since most works that have tried to tackle both imitation learning or RL for such problems seem to struggle when it comes to imitating strong branching (or another heuristic). I also found the fact that average empirical results (e.g. Tables 1 and 2) are reported without standard deviation or any other variability metric across instances. This does not convince me that the proposed approach would perform well on all instances (despites its merits). And finally, since the central part of the contribution is the sequence model, I believe the paper would be much stronger if the properties of the model received more attention in the evaluation (and in particular the token sequence length as mentioned earlier). In particular, it seems quite surprising to train the sequence model on the very small number of instances contained in MIPLIB. This comes in stark contrast with the amount of data required to train such models generally and makes me doubt the relevance of the training loop. More details on why this training is sound and robust would be very welcome.\n\nLastly, the provided code does not seem to work. There's a \"from mamba_ssm import...\" in network.py but no mamba_ssm.py file anywhere to be found.\n\nOverall I hope this feedback will help the authors, regardless of the outcome for the paper. I found the idea interesting and believe this is a relevant research direction that deserves the investigation effort.\n\nQUESTIONS:\nPlease see questions along the strengths and weaknesses part.\n\nLIMITATIONS:\nYes (and mostly non-applicable).", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-14"}