RiteeshGrandhi/sciclaimeval-shared-task huggingface dataset:
Preliminary
We will introduce the background knowledge needed in the dual task and the official baseline based on graph neural network (GNN) [ 19 , 13 ] .
Dual Task
For mixed-integer linear programs (MILPs), a well-known solving method is branch-and-bound (B&B) [ 14 ] . One of the key issues of B&B is how to select variables for Branching . In ML4CO competition, the goal of the dual task is to promote the dual bound to increase faster by selecting proper branching variables. The final metric is the dual integral over time, shown in Figure 1 .
Branching Algorithms
Although making branching decisions has received little theoretical understanding to this day [ 16 ] , there are many heuristic algorithms. Strong Branching involves testing which of the candidate variable gives the best improvement to the objective function before actually branching on them. Strong Branching is a high-quality but computationally expensive method. In practice, modern B&B solvers rely on hybrid-branching [ 15 ] and reliability branching [ 1 ] , which only use Strong Branching at certain nodes of the search tree and use pseudo-cost branching [ 3 ] at other nodes.
Official Baseline
The baseline method [ 8 ] regards the decision of Strong Branching as expert knowledge and train models by imitating the expert knowledge. Specifically, we convert the combinatorial optimization problems to bipartite graphs, illustrated in Figure 2 . Then we use the GNN to obtain the embedding of each variable. These embeddings are used to generate branching strategies for the current state. We train this GNN by minimizing the cross-entropy loss between its output and the expert decision.
Bad Generalization Issue
Following the method in [ 8 ] , we summarize two issues that may reduce the final performance.

First, there is a major gap between offline training and the online deployment phase. In offline training, the data is collected beforehand. The GNN model achieves an accuracy of around 0.80 (validation). However, the accuracy shows a sharp decline when deploying a trained model. As shown in Figure 3 , when making branching decisions for a random online instance, the averaged accuracy of different steps is about 0.1 .

Second, the higher accuracy does not necessarily lead to a higher reward. As shown in Table 2 , comparing models in different epochs, the accuracy is not strongly related to the final reward.
DAgger
The major performance gap between training and deployment can be explained by the defects of Behavior Cloning [ 2 ] , which forces agents to learn expert behavior. There is a common problem that the distribution of collected data and the data the model encounters in the real scene may be inconsistent [ 4 , 17 ] . For example, a driving-related dataset is rich in driver’s operation data during normal driving. But when the car is about to hit the wall, there may be a lack of data to deal with the contingency. To make a more completely collected dataset, we need to constantly explore the unseen data and label these data with expert knowledge.

Dataset Aggregation (DAgger) [ 18 ] is such a method to reduce the gap between training and depolyment. Based on DAgger, we collect data iteratively and train new models constantly. Details are described in Algorithm 1 .
KIDA
As shown in Section 2 , in the Item Placement benchmark, higher accuracy does not lead to higher rewards. Experiments show that the performance of different models obtained in DAgger is not stable. Therefore, we use model ensemble [ 5 , 12 ] to improve the performance of our model further. A noticeable difficulty is that the practicality of the model is time-sensitive. Averaging the output of different models is time-consuming. We consider averaging the weights of different models [ 21 ] . Here, we define Knowledge Inheritance in Dataset Aggregation (KIDA) as building a new model \pi_{avg} by averaging the weights of trained models during the dataset aggregation process. The framework is shown in Figure 4 . The pipeline of KIDA is similar to Born-Again Neural Networks [ 6 ] . In KIDA, the training of the current model depends on the generation using the last model. The final model is the ensemble of trained models.

Formally, for models obtained from dataset aggregation with parameters (\theta_{0},\theta_{1},...,\theta_{n-1}) , the parameters of \pi_{avg} are obtained by: \theta_{avg}=\sum_{i=0}^{n-1}\theta_{i}/n.

Figure 5 shows the difference between KIDA and the popular epoch weight average method. KIDA focuses on models trained by different data from different iteration rounds of DAgger. Although the training data of models are different, these data are related. Common epoch weight average such as Snapshot ensembles [ 10 ] and SWA [ 11 ] focuses on models that are obtained in the same training process of different epochs.
Experiment
In ML4CO competition, there are three benchmarks, including Balanced Item Placement, Workload Apportionment, and Anonymous Problem 1 1 1 Details about each benchmark can refer to https://www.ecole.ai/2021/ml4co-competition/ which need to be evaluated separately. In this section, we discuss the performance of different models in each benchmark.
Anonymous Problem
We show the performance of models with different settings in the validation set in Figure 6 . With the same data size, the model using DAgger performs better. With the increase of data size, the performance increases. The model we finally submitted is a new DAgger model with Dropout [ 20 ] layers and a longer solver time when collecting data.
Balanced Item Placement
Overall Performance . The performance of DAgger at different iteration rounds is shown in Figure 7 . Unlike in the Anonymous benchmark, the performance of DAgger is unstable. But when we apply KIDA in this benchmark, the performance has significantly improved.

Detailed Evaluation. To further explore our method, we randomly process a problem from the validation set using Strong Branching and compare the output of different models with Strong Branching labels. The accuracy of different models is shown in Figure 8 . DAgger model still has the highest accuracy, and the accuracy of the baseline model is the worst.

Models used in KIDA are the top 3 performance models of DAgger. Table 3 shows the comparison between these models and the KIDA model. Although KIDA has lower accuracy in collected validation data, it has a lower loss value and higher reward.
Workload Apportionment
In Workload Apportionment, baselines are trained and compared with random policy. From the table, we can see that the top 1 accuracy of the baseline model is 45.6%. The accuracy of a classification task with more than one hundred labels shows that the model has learned much expert knowledge. However, random strategies that do not rely on prior knowledge can obtain higher cumulative rewards.
Discussion
Overall, we show that Strong Branching can not produce completely reliable labels. To further explore the performance of Strong Branching, we compare the variation of the dual bound using different methods.

As shown in Figure 9 , we select a random instance for each benchmark, the variation of the dual bound using different strategies. Even if we ignore the expensive time cost of Strong Branching, the dual bound improvement may still fall behind GNN models. This observation may be explained by dual degeneracy [ 9 ] . If there is a high dual degeneracy in LP solution, the product score of Strong Branching that SCIP uses to combine the improvements of the two-child nodes would be close to zero [ 7 ] . In that case, Strong Branching may fail, and the expert knowledge needs to be redesigned based on other information accumulated in the problem-solving process.
Conclusion
In this paper, we devise a knowledge inheritance method in the dataset aggregation process for the dual task of ML4CO competition. By dataset aggregation, the inconsistency between training and deployment is reduced. By averaging the weights of different models from the dataset aggregation process, learned knowledge is generalized to get better results. Our model gets great improvements on Item Placement and Anonymous. Further, our experiments show models closer to expert knowledge do not necessarily achieve better results, which indicates Strong Branching fails in some cases. More heuristic branching algorithms need to be taken into account to build more reasonable expert knowledge to facilitate learning to branch.
Figure 1: Evaluation Metric of the Dual Task . Our goal is to minimize the dual integral over time; Evaluation Metric of the Dual Task . Our goal is to minimize the dual integral over time
Figure 2: Copy from [ 8 ] . Left: the bipartite state representation \mathbf{s}_{t}=(\mathcal{G},\mathbf{C},\mathbf{E},\mathbf{V}) with n=3 variables and m=2 constraints. Right: the bipartite GCNN architecture for parametrizing baseline policy \pi_{\theta}(\mathbf{a}\mid\mathbf{s}_{t}); Copy from [ 8 ] . Left: the bipartite state representation \mathbf{s}_{t}=(\mathcal{G},\mathbf{C},\mathbf{E},\mathbf{V}) with n=3 variables and m=2 constraints. Right: the bipartite GCNN architecture for parametrizing baseline policy \pi_{\theta}(\mathbf{a}\mid\mathbf{s}_{t})
Figure 3: Accuracy when Interacting with SCIP Solver. During the deployment phase, the accuracy drops a lot; Accuracy when Interacting with SCIP Solver. During the deployment phase, the accuracy drops a lot
Figure 4: Framework of KIDA . We train each model based on the data generation using the last model. Finally, we average the parameter of trained models; Framework of KIDA . We train each model based on the data generation using the last model. Finally, we average the parameter of trained models
Figure 5: KIDA vs. Epoch Weight Average; KIDA vs. Epoch Weight Average
Figure 6: The Cumulative Reward of Anonymous in Validation Dataset . The performance of DAgger models is related to the number of iteration rounds; The Cumulative Reward of Anonymous in Validation Dataset . The performance of DAgger models is related to the number of iteration rounds
Figure 7: The Cumulative Reward of Item Placement in Validation set . KIDA has significant advantages over just applying DAgger; The Cumulative Reward of Item Placement in Validation set . KIDA has significant advantages over just applying DAgger
Figure 8: Accuracy when Interacting with the Solver . Overall accuracy of DAgger 20, KIDA and the baseline are 0.635, 0.182, 0.071 respectively.; Accuracy when Interacting with the Solver . Overall accuracy of DAgger 20, KIDA and the baseline are 0.635, 0.182, 0.071 respectively.
(a) Item Placement; (b) Load Balancing; (c) Anonymous; Dual Bound Improvements with Steps . Strong Branching may still fall behind GNN-based policy or even random policy
Leaderboard of the Dual Task in ML4CO NeurIPS 2021 competition
Performance Comparison of Different Epochs . The imitation accuracy is not directly related to model performance
Performance Comparison between DAgger models and the KIDA model . KIDA suffers from an accuracy drop but has a lower loss
Performance Comparison between Baseline and Random Policy . The baseline model gets great accuracy but fails to achieve higher rewards