SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\left| \left| f \right| \right| _ {1} = \sum_ {i = 0} ^ {m} \left| a _ {i} \right|.
$$
Algorithm 1 SturmSequence
Input:  $f$ : a univariate polynomial
Output: sturm: the Sturm Sequence of  $f$
1:  $sturm \gets []$
2:  $h\gets f$
3:  $g\gets f^{\prime}$
4:  $r\gets -rem(g,h,x)$
5: while  $r \neq 0$  do
6: Append  $r$  to sturm
7:  $h\gets g$
8:  $g\gets r$
9:  $r\gets -rem(g,h,x)$
10: end while
11: return sturm
Algorithm 2 RootIsolation
Output:  $(a,b)$ :  $f$  has a real root in  $(a,b)$  where  $a, b$  are rational numbers
1:  $\overline{S} \gets \text{SturmSequence}(f)$
2:  $a\gets -||f||_1$
3:  $b\gets ||f||_1$
4: if  $V_{a}(\overline{S}) = V_{b}(\overline{S})$  then
5: return failure
6: end if
7: while  $V_{a}(\overline{S}) - V_{b}(\overline{S}) > 1$  do
8:  $c \gets \frac{a + b}{2}$
9: if  $V_{a}(\overline{S}) > V_{c}(\overline{S})$  then
10:  $b\gets c$
11: else
12:  $a\gets c$
13: end if
14: end while
15: return  $(a,b)$
(b)
Figure 6: The relationships between the number of cells and other important indicators. Figure 6a and Figure 6b respectively correspond to the relationships between the number of cells and computation time and the relationships between the number of cells and memory usage. Five hundred instances of 3-var are randomly selected from the random dataset to construct the two scatterplots.
A.2 Case Study: Discriminant
Actually, whether  $x^{2} + bx + c = 0$  has a real root is equivalent to checking the satisfiability of  $\exists x.x^{2} + bx + c = 0$ . Let's elaborate on why the quantified formula  $\exists x.x^{2} + bx + c = 0$  can be transformed into an equivalent quantifier-free formula  $b^{2} - 4c \geq 0$  via CAD techniques. By feeding the CAD algorithm with the set of polynomial  $\{x^{2} + bx + c\}$  and an order  $b \prec c \prec x$ , CAD decomposes  $\mathbb{R}^3$  into 9 cells:
$$
\begin{array}{l} \left\{ \begin{array}{l} b = b \\ c <   \frac {b ^ {2}}{4} \\ x <   - \frac {b}{2} - \frac {\sqrt {b ^ {2} - 4 a c}}{2}, \end{array} \right. \quad \left\{ \begin{array}{l} b = b \\ c <   \frac {b ^ {2}}{4} \\ x = - \frac {b}{2} - \frac {\sqrt {b ^ {2} - 4 a c}}{2}, \end{array} \right. \quad \left\{ \begin{array}{l} b = b \\ c <   \frac {b ^ {2}}{4} \\ x = - \frac {b}{2} + \frac {\sqrt {b ^ {2} - 4 a c}}{2}, \end{array} \right. \\ \left\{ \begin{array}{l} b = b \\ c <   \frac {b ^ {2}}{4} \\ x > - \frac {b}{2} + \frac {\sqrt {b ^ {2} - 4 a c}}{2}, \end{array} \right. \quad \left\{ \begin{array}{l} b = b \\ c <   \frac {b ^ {2}}{4} \\ - \frac {b}{2} - \frac {\sqrt {b ^ {2} - 4 a c}}{2} <   x <   - \frac {b}{2} + \frac {\sqrt {b ^ {2} - 4 a c}}{2}, \end{array} \right. \\ \left\{ \begin{array}{l} b = b \\ c = \frac {b ^ {2}}{4} \\ x <   - \frac {b}{2}, \end{array} \right. \quad \left\{ \begin{array}{l} b = b \\ c = \frac {b ^ {2}}{4} \\ x = - \frac {b}{2}, \end{array} \right. \quad \left\{ \begin{array}{l} \tilde {b} = b \\ c = \frac {b ^ {2}}{4} \\ x > - \frac {b}{2}, \end{array} \right. \quad \left\{ \begin{array}{l} \tilde {b} = b \\ c > \frac {b ^ {2}}{4} \\ x = x. \end{array} \right. \\ \end{array}
$$
Note that the sign of the polynomial  $x^{2} + bx + c$  is zero if and only if  $(b, c, x)$  belongs to cells:
$$
\left\{ \begin{array}{l} b = b \\ c = \frac {b ^ {2}}{4} \\ x = - \frac {b}{2}, \end{array} \right. \quad \left\{ \begin{array}{l} b = b \\ c <   \frac {b ^ {2}}{4} \\ x = - \frac {b}{2} - \frac {\sqrt {b ^ {2} - 4 a c}}{2}, \end{array} \right. \quad \left\{ \begin{array}{l} b = b \\ c <   \frac {b ^ {2}}{4} \\ x = - \frac {b}{2} + \frac {\sqrt {b ^ {2} - 4 a c}}{2}. \end{array} \right.
$$
So we know that there must exist  $x$  such that  $x^{2} + bx + c = 0$  when  $b^{2} - 4c > 0$  or  $b^{2} - 4c = 0$ , and it is impossible to find a  $x$  such that  $x^{2} + bx + c = 0$  when  $b^{2} - 4c < 0$ . So, we can conclude the quantified formula  $\exists x.x^{2} + bx + c = 0$  is equivalent to the quantifier-free formula  $b^{2} - 4c \geq 0$ . In fact, similar processes can be abstracted into a universal algorithm to solve the quantifier elimination problems in the real closed field. See [2] for more details.
A.3 Relation of #Cells And Efficiency
As in Figure 6, there is a strong correlation between the number of cells produced and the computation time, as well as the memory usage. When the number of cells increases, the computation time and the memory usage also increase.
Table 2: Parameters of randpoly function.
Parameter	Description
vars	List or set of variables
coeffs	Generator of coefficients
expons	Generator of exponents
terms	Number of terms
degree	Total degree for a dense polynomial
dense	The polynomial is to be dense
homogeneous	The polynomial is to be homogeneous
Table 3: Parameters of random dataset generation.
#vars	coeffs	expons	terms
3	rand(-100..100)	rand(0..2)	rand(3..6)
4	rand(-100..100)	rand(0..2)	3
5	rand(-100..100)	rand(0..2)	3
6	rand(-100..100)	rand(0..2)	3
7	rand(-100..100)	rand(0..2)	3
8	rand(-100..100)	rand(0..2)	3
9	rand(-100..100)	rand(0..2)	3
B Experiment Setup
B.1.1 Random Dataset
We generated the random polynomial set via the randpoly(args, opts) function in MAPLE, where opts are specifying properties like coefficients, exponents, terms, degree, dense, homogeneous. Table 2 lists the descriptions of parameters. We also show an example of random polynomial generation.
For example, randpoly([x1, x2, x3], terms = 4, exponents = rand(0..2), coefficients = rand(-100..100)) will generate a polynomial,
$$
5 6 x _ {1} x _ {2} ^ {2} x _ {3} ^ {2} - 4 x _ {1} ^ {2} x _ {2} x _ {3} + 3 7 x _ {1} x _ {2} ^ {2} x _ {3} - 3 2 x _ {1} x _ {2} x _ {3} ^ {2},
$$
where  $rand(a..b)$  is a random number generator in the range of  $[a, b]$ . Note that the random polynomial is sparse and non-homogeneous by default. We also ignore the parameter degree, because it is only valid in the case of dense random polynomial generation.
Table 3 lists the parameters for randpoly for generating the random dataset, where #vars = n corresponds to a list of variables  $[x_1, x_2, \dots, x_n]$ . The random number for terms is generated with Python's random library outside the MAPLE script. If the polynomial produced by MAPLE lacks a constant term, one is added, ranging from -100 to 100, using the same Python library.
B.1.2 SMT-LIB Dataset
The SMT-LIB dataset is built by the instances that only involve real constraints in the QF_NRA category of the SMT-LIB [17]. Using the Python interface of Z3[5], we parse the instances to extract the polynomial sets, discarding any instances that cannot be correctly parsed. Then we categorize the processed polynomial sets based on the number of variables and finally build the SMT-LIB dataset, containing instances ranging from 3 to 9 variables with counts \{5908, 1371, 131, 123, 318, 41, 24\}. Furthermore, we observed that there are numerous polynomial sets in the dataset that yield the same result after factoring out the irreducible factors, which are indistinguishable from CAD algorithms. So we cluster them and only one instance of each clustering is included in the dataset, resulting in \{1777, 387, 17\} instances for 3-var, 4-var to 6-var and 7-var to 9-var, respectively.
B.2 Neural Networks
Table 4 lists the hyper-parameters of the GNN encoder, the networks used in GRL-SVO(NUP)'s architecture to transform the original embeddings linearly and simulate the project process, actor network, critic network, and RL architecture.
C Ablation Experiments
C.1 The Effect of Features
There are 14 different features to characterize a variable listed in Table 5. We conduct an experiment on the effect of features. We make masks for these features, where the mask will set the features that we do not care about as zero.
Table 4: The hyper-parameters.
Category	Parameter	Value
GNN	The number of GNN layers	4
GNN	The number of Intermediate layer features	256
GNN	Aggregation method	mean
GNN	Use bias	True
NUP_transform	The size of MLP layer features	[14, 256, 128, 64]
NUP_simulate	The size of MLP layer features	[128, 512, 256, 64]
Actor	The size of MLP layer features	[256, 512, 128, 1]
Critic	The size of MLP layer features	[256, 512, 128, 1]
RL	Batch size	32
RL	Learning rate	2e-5
RL	Training maximum epoch	100
RL	Reward normalization factor (M)	50000
Table 5: The original embedding of a variable in an associated graph.
Symbol	Description
E1(x)	Number of other variables occurring in the same polynomials
E2(x)	Number of polynomials containing x
E3(x)	Maximum degree of x among all polynomials
E4(x)	Sum of degree of x among all polynomials
E5(x)	Maximum degree of leading coefficient of x among all polynomials
E6(x)	Maximum number of terms containing x among all polynomials
E7(x)	Maximum degree of all terms containing x
E8(x)	Sum of degree of all terms containing x
E9(x)	Sum of degree of leading coefficient of x
E10(x)	Sum of number of terms containing x
E11(x)	Proportion of x occurring in polynomials
E12(x)	Proportion of x occurring in terms
E13(x)	Maximum number of other variables occurring in the same term
E14(x)	Maximum number of other variables occurring in the same polynomial
1. One-hot masks (test the effect of a single feature), for example, to test the effect of  $E_{1}$ , the corresponding one-hot mask is  $(1,0,0,0,0,0,0,0,0,0,0,0,0)$ . Multiplying with input feature will result in a feature vector with only  $E_{1}$  while others are 0.
2. Operation masks (test the effect of different operations in features) will group features according to their operation type (maximum, sum, and proportion). Note that we treat  $E_{1}, E_{2}$  as sub-features utilizing sum operation.
Max:  $E_{3},E_{5},E_{6},E_{7},E_{13},E_{14}$
Sum:  $E_{1},E_{2},E_{4},E_{8},E_{9},E_{10}$
Prop:  $E_{11},E_{12}$
3. Object masks (test the effect of different objects in features) will group features according to their target objects (variable, term, and polynomials).
Var:  $E_{1},E_{3},E_{4},E_{13},E_{14}$
Term:  $E_5,E_6,E_7,E_8,E_9,E_{10},E_{12}$
- Poly:  $E_2, E_{11}$
4. Because degree is a common feature that most heuristics are concerned with, testing the effect of degree is necessary. Degree masks will group features according to whether they utilize degree.
- Degree:  ${E}_{3},{E}_{4},{E}_{5},{E}_{7},{E}_{8},{E}_{9}$
NoDegree:  $E_{1},E_{2},E_{6},E_{10},E_{11},E_{12},E_{13},E_{14}$
Table 6 and Table 7 show the results of different single, operation, object, and degree features. We can conclude that only one feature is not enough. The sum, term, and degree may be the most important factors, as using Sum/Term/Degree features will result in the largest difference in performance.
Table 6: The performance of different single features.
E1	E2	E3	E4	E5	E6	E7	E8	E9	E10	E11	E12	E13	E14
NUP	#SI	1459	1458	1467	1635	1560	1589	1461	1647	1613	1670	1462	1670	1459	1459
AVG.T	252.02	250.85	247.38	151.71	198.43	186.27	251.03	157.77	175.60	147.61	249.17	147.61	252.02	252.02
AVG.N	2874.94	2859.65	2852.56	2246.06	2642.66	2592.94	2861.09	2478.10	2515.79	2508.01	2857.07	2508.04	2874.94	2874.97
UP	#SI	1459	1540	1596	1693	1632	1627	1461	1687	1667	1706	1540	1706	1459	1459
AVG.T	252.33	211.51	184.73	121.36	163.03	165.65	251.22	137.31	145.16	129.87	211.53	129.86	252.23	252.24
AVG.N	2874.97	2743.79	2755.63	2174.76	2559.63	2531.44	2860.81	2394.39	2465.49	2386.24	2743.79	2386.24	2874.87	2874.90
Table 7: The performance of different feature classifications.
Operation	Object	Degree
Max	Sum	Prop	Var	Term	Poly	Degree	NoDegree
NUP	#SI	1640	1754	1670	1635	1726	1462	1723	1685
AVG.T	159.42	101.21	147.48	152.11	116.39	249.17	114.95	141.65
AVG.N	2444.09	2145.71	2496.74	2250.21	2300.32	2857.07	2177.08	2468.04
UP	#SI	1716	1788	1707	1698	1751	1540	1770	1715
AVG.T	119.80	81.93	129.53	119.32	102.07	211.55	91.31	125.53
AVG.N	2346.79	2082.22	2369.74	2184.08	2232.74	2743.79	2103.21	2359.62
C.2 The Effect of Network Size
We conduct an experiment on the effect of network size, and the results are listed in Table 8. The number of GNN layers (for short,  $\# \mathrm{G})\in \{1,2,3,4,5\}$  and the number of Intermediate layer features (for short,  $\# \mathrm{I})\in \{32,64,128,256,512\}$  where Actor and Critic will keep the same proportion 2:4:1. Note that the input dimension of Actor and Critic is the same as #I. For example, if  $\# \mathrm{I} = 32$  , then the dimension of the Actor and Critic are both [32, 64, 16, 1]. Elements in the following tables are #SI in the validation set and #SI in the testing set, respectively.
C.3 The Effect of Network Structure
We conduct an experiment on the effect of network structure. We build two models: one without embedding (NO_EMB), and the other without edge (NO_EDGE). Note that the GNN updating operator we used is  $\mathbf{x}_i' = \mathbf{W}_1\mathbf{x}_i + \mathbf{W}_2\sum_{j\in \mathcal{N}(i)}e_{j,i}\cdot \mathbf{x}_j + \mathbf{b}$ . As edges are not considered in NO_EDGE, actually, the operator will be  $\mathbf{x}_i' = \mathbf{W}_1\mathbf{x}_i + \mathbf{b}$ . So NO_EDGE is equivalent to MLP. For NO_EMB,  $\# SI = 1459$ ,  $AVG.T = 252.03$ ,  $AVG.N = 2874.91$ . Table 9 shows the result of NO_EDGE on testing set. The performance of NO_EMB drops dramatically while that of NO_EDGE is good, and GRL-SVO can still outperform such models. We explore the performance of NO_EDGE(NUP) under different sizes of parameters with GRL-SVO(NUP). MLP_4_512 has twice as many parameters as ours. GRL-SVO(NUP) can outperform all MLPs as shown in Figure 7. It is the advantage of the graph structure where a variable can grasp neighbor information.
C.4 The Effect of Reward Normalization Factor  $(M)$
We make an ablation experiment on  $M$  that we train the models with  $M = 10000, 20000, 50000$  (ours), 100000, and without  $M$ . Note that if there is no  $M$ , the reward (the number of cells) will be a relatively large integer. As shown in Table 10,  $M$  is necessary. The first number is the result of the validation set, while the second is the result of the testing set.
C.5 The Effect of GNN Architecture
We conduct experiments using different GNN architectures available in PyTorch Geometric that have similar formal parameters as the GNN architecture we used: ClusterGCNConv[41], EGConv[42],
Table 8: The performance (#SI) of GRL-SVO(NUP) and GRL-SVO(UP).
GRL-SVO(NUP)	GRL-SVO(UP)
32	64	128	256	512	32	64	128	256	512
1	1754,1737	1756,1741	1771,1765	1770,1765	1764,1765	1767,1759	1768,1758	1777,1774	1789,1789	1790,1793
2	1771,1764	1762,1747	1765,1763	1767,1766	1764,1768	1773,1763	1780,1777	1778,1776	1786,1793	1788,1794
3	1765,1758	1756,1744	1766,1758	1761,1766	1770,1769	1781,1767	1783,1778	1782,1785	1788,1794	1786,1793
4	1765,1761	1768,1761	1764,1762	1766,1765	1769,1765	1775,1766	1784,1775	1780,1779	1792,1794	1788,1793
5	1768,1762	1765,1763	1771,1759	1766,1761	1763,1762	1779,1779	1784,1781	1788,1797	1792,1790	1791,1790
Table 9: The performance of NO EDGE under different sizes.
MLP_2_256	MLP_3_256	MLP_4_256	MLP_4_512
#SI	1763	1763	1764	1756
AVG.T	97.89	98.43	97.68	99.08
AVG.N	2132.33	2140.18	2129.33	2148.64
(a) On training set
Figure 7: The training process of NO_EDGE and GRL-SVO(NUP).
(b) On validation set
FiLmConv[42], LEConv[42], GATConv [20], GATv2Conv [43], GeneralConv [44], ResGatedGraphConv [45], SageConv [46], TransformerConv [47]. We could observe from Table 11 that each graph neural network (GNN) is capable of effectively learning this problem. Our approach is not reliant on the specific GNN structure.
C.6 The Effect of Coefficient
With regard to the choice of variable order, the current works do not consider coefficient (both Expert-Based or Learning-Based heuristics). Some experts in Symbolic Computation believe that the reason may be related to the calculation of CAD projection: CAD projection uses two polynomials to make a resultant (Definition 3) to eliminate a common variable, so it first depends on the common variable set of these two polynomials (which does not involve coefficient); secondly, the amount of calculation generally depends on the degree of public variables because the degree determines the size of the resultant matrix. For practical instances, the variation range of the coefficients is uncontrollable, which will increase the difficulty of Learning-Based design and training the neural network. The impact of the coefficient on the problem is complex. We provide some intuition through Figure 8.
(a)  $\{x^3 y + 4x^2 +xy, - x^2 +2xy - 1\} ,x\prec y:13,y\prec x:89;$
(b)  $\{x^3 y + 8x^2 +xy, - x^2 +2xy - 1\} ,x\prec y:13,y\prec x:89;$
(c)  $\{x^3 y - 4x^2 + xy, -x^2 + 2xy - 1\}, x \prec y: 45, y \prec x: 125;$
(d)  $\{x^3 y + 4x^2 + xy, x^2 + 2xy - 1\}, x \prec y: 29, y \prec x: 101;$
(e)  $\{-x^3 y + 4x^2 + xy, -x^2 + 2xy - 1\}, x \prec y: 45, y \prec x: 97.$
Table 10: The performance of different  $M$
NO_M	1000	10000	50000 (ours)	100000
NUP	#SI	1677, 1666	1672, 1669	1764, 1769	1766, 1765	1766, 1765
AVG.T	133.06, 142.19	140.28, 142.56	93.03, 95.57	91.68, 97.78	93.46, 97.51
AVG.N	2230.65, 2214.20	2250.11, 2222.47	2134.50, 2136.30	2129.45, 2142.98	2147.43, 2157.56
UP	#SI	1182, 1196	1726, 1723	1789, 1795	1792, 1795	1791, 1793
AVG.T	387.16, 381.11	109.12, 115.13	73.86, 80.20	72.02, 79.57	73.68, 79.86
AVG.N	3125.50, 3101.32	2100.53, 2097.48	2058.08, 2081.92	2062.09, 2075.23	2058.05, 2070.75
Table 11: The performance of GRL-SVO(NUP) with different GNN architectures.
ClusterGCNConv	EGConv	FiLmConv	LEConv	GATConv
#SI	1767	1765	1760	1761	1726
AVG.T	94.26	98.53	96.49	96.76	119.72
AVG.N	2144.66	2177.32	2136.67	2151.75	2166.78
GATv2Conv	GeneralConv	ResGatedGraphConv	SageConv	TransformerConv
#SI	1762	1768	1759	1765	1769
AVG.T	104.14	95.17	96.25	96.74	94.45
AVG.N	2184.86	2132.81	2146.59	2131.22	2134.51
(d)
Figure 8: The results of slight changes of coefficients.
The number of cells of (a) and (b) are the same, while (c), (d), and (e) are different. But the best variable order is the same  $(x \prec y)$  in these cases. To a certain extent, in these cases, the coefficient mostly affects the number of cells. We conduct an experiment on coefficient that we have randomly modified the coefficients (in [-100, 100]) of 1000 instances randomly selected from a 3-var testing set. Since the coefficients were the only modification made, we used the original variable order generated from the unaltered instances. Our models continue to outperform other heuristics, as demonstrated by Table 12. We observe a slight decline in the performance of all heuristics, indicating that the coefficient also plays a significant role as a parameter (although it may not be the most crucial one).
D.1 Results Under Other Criteria
Assume  $T$  and  $N$  denote the running time and number of cells. COMAVG.  $T$  and COMAVG.  $N$  denote the average of  $T$  and  $N$  of the instances that all heuristics solved within the time limit. Since the sotd and ndrr heuristics are not applicable to most instances with the number of variables more than 5, we remove the comparison with these two heuristics on the results of the new criteria.
There are 1325, 292, 142, 106, 84, 103, and 87 common instances that all heuristics except sotd and ndrr solved within the time limit for the random dataset from 3-var to 9-var; and for the SMT-LIB dataset, 1670, 354, 9 instances for 3-var, 4-var to 6-var and 7-var to 9-var, respectively. Table 13 shows the results and the best scores are bolded. We can observe that GRL-SVO still achieves a relatively good performance under the new criteria.
D.2 Performance of Fine-Tuning
As GRL-SVO(UP) has demonstrated strong generalization abilities and achieved the best performance on almost all datasets, there is still room for further improvement in the generalization capabilities of GRL-SVO(NUP). So, we further investigate the performance of GRL-SVO(NUP) after fine-
Table 12: The performance of different heuristics after the coefficients are randomly modified.
brown	triangular	EMLP	sotd	ndrr	gmods	GRL-SVO(NUP)	GRL-SVO(UP)
#SI	831	762	855	898	842	867	892	912
AVG.T	155.42	212.50	125.54	79.73	141.29	100.30	79.96	64.89
AVG.N	2380.67	2606.40	2353.11	2065.13	2288.08	2162.19	2100.57	2023.58
Table 13: The performance of different heuristics under the new criteria. The dash “-” indicates that the method does not support the category.
Categories	NUP	UP
EB	LB	EB	LB
brown	triangular	EMLP	PVO(brown)	PVO(triangular)	GRL-SVO(NUP)	gmods	GRL-SVO(UP)
3-var(test)	COMAVG.T	36.45	48.77	34.08	-	-	23.28	22.52	17.68
COMAVG.N	1678.48	1968.18	1656.93	-	-	1474.59	1489.72	1420.41
4-var	COMAVG.T	24.25	14.07	-	23.22	13.75	20.50	12.48	13.82
COMAVG.N	2286.79	2476.00	-	2227.53	2306.30	2018.79	1730.37	1739.16
5-var	COMAVG.T	27.90	32.51	-	27.36	30.26	31.36	19.33	15.82
COMAVG.N	4624.49	5222.65	-	4355.15	4866.49	4298.32	3415.94	3340.59
6-var	COMAVG.T	52.71	38.95	-	56.23	40.57	38.38	32.67	22.98
COMAVG.N	7687.28	6897.06	-	7761.30	6733.09	6281.09	5440.62	4278.75
7-var	COMAVG.T	58.55	51.96	-	-	-	50.21	38.41	30.15
COMAVG.N	9785.12	9055.57	-	-	-	8731.50	6826.24	5810.71
8-var	COMAVG.T	102.45	118.83	-	-	-	91.54	67.18	69.62
COMAVG.N	15484.69	17655.02	-	-	-	14730.24	11053.10	12075.62
9-var	COMAVG.T	137.26	169.25	-	-	-	123.67	98.77	96.90
COMAVG.N	20045.44	23814.45	-	-	-	18132.95	16094.36	15995.05
SMT-LIB (3-var)	COMAVG.T	10.41	12.07	31.72	-	-	10.66	9.47	9.67
COMAVG.N	3234.61	3746.72	6391.53	-	-	2906.43	2736.89	2815.23
SMT-LIB (4-var to 6-var)	COMAVG.T	39.75	41.06	-	36.23	39.41	34.21	28.00	29.42
COMAVG.N	16839.63	16815.20	-	14730.58	15160.26	12977.31	10847.26	12156.08
SMT-LIB (7-var to 9-var)	COMAVG.T	43.57	43.55	-	-	-	17.55	36.78	41.99
COMAVG.N	17169.67	17169.67	-	-	-	6149.89	12669.22	14919.67
Table 14: The performance of NUP heuristics, with the performance of GRL-SVO(NUP) after fine-tuning. The dash “-” indicates that the method does not support the category.
Categories	NUP
EB	LB
brown	triangular	EMLP	PVO(brown)	PVO(triangular)	GRL-SVO(NUP)
3-var(test)	#SI	1620	1504	1686	-	-	1772
AVG.T	170.63	227.60	140.06	-	-	94.07
AVG.N	2421.18	2663.09	2384.29	-	-	2157.92
4-var	#SI	415	376	-	408	392	456
AVG.T	352.87	394.71	-	360.33	376.71	298.74
AVG.N	5258.57	5539.73	-	5338.45	5536.28	5060.19
5-var	#SI	236	202	-	242	218	253
AVG.T	435.64	495.34	-	419.50	466.47	394.12
AVG.N	12465.41	13493.00	-	11932.31	12826.53	11586.65
6-var	#SI	175	149	-	180	160	204
AVG.T	500.62	551.15	-	489.57	526.80	432.64
AVG.N	20487.45	20068.22	-	20029.06	18993.53	19407.97
7-var	#SI	163	118	-	-	-	175
AVG.T	549.15	632.62	-	-	-	522.35
AVG.N	28552.12	29182.78	-	-	-	27779.48
8-var	#SI	173	138	-	-	-	177
AVG.T	601.17	653.60	-	-	-	594.77
AVG.N	39540.72	40902.42	-	-	-	38957.71
9-var	#SI	151	125	-	-	-	171
AVG.T	651.36	691.92	-	-	-	619.39
AVG.N	48860.36	50315.52	-	-	-	47406.84
SMT-LIB (3-var)	#SI	1770	1763	1675	-	-	1768
AVG.T	20.33	23.68	83.09	-	-	20.86
AVG.N	4449.79	5070.46	7654.04	-	-	4014.59
SMT-LIB (4-var to 6-var)	#SI	374	372	-	372	372	365
AVG.T	83.92	87.85	-	86.22	86.88	88.42
AVG.N	24514.73	24178.54	-	22953.99	22644.04	20910.95
SMT-LIB (7-var to 9-var)	#SI	13	12	-	-	-	12
AVG.T	308.14	377.32	-	-	-	341.21
AVG.N	53971.24	58675.94	-	-	-	52381.82
tuning. We conduct a case study on fine-tuning, utilizing 100 instances with 4 variables. All the hyperparameters employed during the training process were identical to those listed in Table 4.
As shown in Table 14, after fine-tuning, GRL-SVO(NUP) exhibits the best performance among all NUP methods across all categories of the random dataset. The  $\# SI$  indicator for GRL-SVO(NUP) showed enhancements ranging from  $1.00\%$  to  $14.38\%$  across the 4-var to 9-var categories of the random dataset, with slight improvements also observed in the SMT-LIB dataset. Although fine-tuning can help improve the efficiency of GRL-SVO(NUP), the approach is not fit for the instances with a relatively larger number of variables, as we need to run the most variable orders of each instance to build a training dataset. So, for high-dimensional instances, how to fine-tune is a promising direction.