Title: Computational Algebra with Attention: Transformer Oracles for Border Basis Algorithms
Venue: NeurIPS 2025
Accepted: yes
Keywords: Polynomial System Solving

Border Bases

Transformer

Computational Algebra

AI4Science

AI4Math
Abstract: Solving systems of polynomial equations, particularly those with finitely many solutions, is a crucial challenge across many scientific fields. Traditional methods like Gröbner and Border bases are fundamental but suffer from high computational costs, which have motivated recent Deep Learning approaches to improve efficiency, albeit at the expense of output correctness. In this work, we introduce the Oracle Border Basis Algorithm, the first Deep Learning approach that accelerates Border basis computation while maintaining output guarantees. To this end, we design and train a Transformer-based oracle that identifies and eliminates computationally expensive reduction steps, which we find to dominate the algorithm's runtime. By selectively invoking this oracle during critical phases of computation, we achieve substantial speedup factors of up to 3.5x compared to the base algorithm, without compromising the correctness of results. 
To generate the training data, we develop a sampling method and provide the first sampling theorem for border bases. We construct a tokenization and embedding scheme tailored to monomial-centered algebraic computations, resulting in a compact and expressive input representation, which reduces the number of tokens to encode an $n$-variate polynomial by a factor of $O(n)$. Our learning approach is data efficient, stable, and a practical enhancement to traditional computer algebra algorithms and symbolic computation.


Peer reviews:
comments: SUMMARY:
This paper uses a Deep Learning approach to speedup existing
Border basis based approaches to solving systems of polynomial equations.
This can
 serve as a  practical enhancement to traditional computer algebra algorithms.
In particular, they claim to obtain  3.5x speedup compared to Border basis computer algebra algorithm which does not use AI.
Additionally, introduce a novel sampling based method to generate training data.

STRENGTHS AND WEAKNESSES:
Strength:

1) Natural and central problem.The problem of solving systems of polynomial equations with finitely many solutions occurs in many fields, including cryptography, computational biology, algebraic geometry, etc. And furthermore Gröbner and border bases are part of the backbone of symbolic computation systems, although they are notoriously expensive. So, using ML to help compute Border bases is very natural. 

2) Using ML it in conjunction with traditional methods enables efficiency improvement without sacrificing correctness. This is a stroong benefit of the approach.

3) The speed up claimed will make the method useful in real computer algebra systems.

4 )
Tailoring tokenization and embeddings to algebraic input seems like a novel contribution in this field.


Weakness
Is 3.5 factor enough theoretically, can you do better?
how large is the experiment - I couldn't find the answer?

QUESTIONS:
What would be the plan going from Border bases to Grobner bases?

How does the transformer embed the polynomial input  ?

Empirical Breadth: A brief mention of the benchmarks, problem sizes, or types of systems tested would help assess practical significance.

LIMITATIONS:
Not much of a discussion
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces the Oracle Border Basis Algorithm (OBBA), the first deep learning approach that accelerates border basis computation for solving polynomial equation systems while maintaining mathematical correctness guarantees.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nSuccessfully combines deep learning with symbolic computation while maintaining mathematical correctness.\nMonomial embedding reduces token count by O(n) and attention memory by O(n²).\nClever insight that border basis computation's degree-by-degree structure enables more stable supervised learning vs. reinforcement learning.\n\nWeakness:\nRequires generating millions of training samples for each new problem domain.\nNo comparison with other acceleration techniques for symbolic computation.\nTraining only on degree 2 systems, though testing shows some generalization to degree 8\nFinite fields only: \"leaving characteristic-0 arithmetic and larger arity unexplored\" \nOnly 27 datasets constructed.\n\nQUESTIONS:\nSee Weakness and Limitation sessions\n\nLIMITATIONS:\nThis paper introducing the first deep learning approach that successfully accelerates border basis computation while maintaining mathematical correctness guarantees. The Oracle Border Basis Algorithm represents a pioneering achievement in AI-enhanced symbolic computation, demonstrating up to 3.5× speedup with rigorous mathematical proofs and novel monomial embedding techniques.\nHowever, several critical practical challenges severely limit the real-world applicability of the proposed solution. The method requires generating millions of training samples for each new problem domain, creating substantial computational overhead that may offset the runtime benefits in practice. The evaluation lacks comparison with other acceleration techniques for symbolic computation, making it difficult to assess the relative merit of the ML-based approach versus existing optimization methods. The training is limited to degree 2 systems while testing shows only modest generalization to degree 8, raising questions about scalability to higher-degree polynomial systems commonly encountered in real applications. The restriction to finite fields, \"leaving characteristic-0 arithmetic and larger arity unexplored,\" significantly constrains the method's applicability to many practical domains that require real or complex number computations. Additionally, the experimental evaluation is based on only 27 constructed datasets with limited parameter variations, providing insufficient evidence for the method's robustness across diverse problem structures and raising concerns about whether the observed performance gains will generalize to the broader landscape of polynomial system solving tasks.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
Solving systems of polynomial equations is a foundational problem in mathematics and many other domains. Classical methods like Gröbner and Border bases provide a way to search for solutions but are computationally intensive. The paper introduce the Oracle Border Basis Algorithm (OBBA), which provides a way to efficiently searching solutions for zero-dimensional cases. They use a lightweight Transformer oracle to predict and eliminates unhelpful polynomial reductions, which significantly reduce the computational overhead.

STRENGTHS AND WEAKNESSES:
Pros
1. The idea is novel s.t combining ML/DL with rigorous math problem is relatively new and rare in the domain.
2. They designed an interesting way to train a Transformer with customized tokenization and embedding.
3. The method shows a noticeable increasing in speed without compromising accuracy and can be generalized to OOD.

Cons
1. The method is only constrained to zero-dimensional problems and can’t be generalized to positive-dimensional cases.
2. The decision of when to invoke the oracle is heuristic and could benefit from theoretical analysis or learned policies.
3. Even though the experiment shows results up to 5 variables, but the scalability to higher-dimensional or real-world symbolic problems remains unclear. Has no theoretical guarantee.
4. It would be interesting to know if we randomly prune the search tree, how would the performance looks like.

QUESTIONS:
please see the weakness section.

LIMITATIONS:
yes.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents OBBA, an algorithm for solving zero‐dimensional systems, which integrates a Transformer oracle into border‐basis computations. The authors show that OBBA has reduced wall-clock runtime, while preserving nearly perfect accuracy. The speed-up is obtained by using the oracle to predict only the necessary expansions in the final steps of the border-basis algorithm.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n- The paper is well written, and contains a complete overview of the techniques used and comparison to state of the art.\n- Proposing new faster algorithms for symbolic computation is an important direction. The idea of integrating a Transformer-based oracle and analysing its complexity is nice and promising.\n\nWeaknesses:\n- My main concern is that the scope of this work might be a bit narrow.\n- To the best of my understanding, the theoretical proofs, while sound, they assemble previously known facts. I would be happy to change my assessment if the authors do not agree and can elaborate more on the originality of their proof techniques. \n- It seems therefore that the main contribution consists of the reduced wall-clock runtime presented in the experiments, along which the reduction of the input size for making the problem tractable by a Transformer. However, the experiments are for fairly small number of variables and degree. The empirical speedups, although measurable, occur on very small instances (wall-clock times on the order of 10⁻² s). It would be helpful to understand the limitations or bottlenecks when scaling to larger systems\n- It remains unclear which real-world applications benefit from these runtime improvements on such small systems; I invite the authors to elaborate more on these.\n\nQUESTIONS:\n- Can you comment on the originality of the proof techniques used?\n- Can you explain why you chose the number of variables and degree to be up to 5 and 4, respectively, and how you would expect the runtime gain to scale?\n- Can you comment on how you chose the Transformer architecture and on whether you did any ablation about that?\n- In Figure 3, what is the OOD accuracy achieved? Can you also comment on the trend of the number of fallbacks for degree 4 versus degree 8?\n\nI believe there are small typos in line 103 (T_n missing) and in line 94 (caligraphic I?).\n\nLIMITATIONS:\nThe algorithm proposed is limited to finite-sized polynomial systems, since it relies specifically on the border-basis algorithm.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-11"}