NeuroSchedule: A Novel Effective GNN-based Scheduling Method for High-level Synthesis
Abstract
High-level synthesis (HLS) is widely used for transferring behavior-level specifications into circuit-level implementations.
As a critical step in HLS, scheduling arranges the execution order of operations for enhanced performance.
However, existing scheduling methods suffer from either exponential runtime or poor quality of solutions.
This paper proposes an efficient and effective GNN-based scheduling method called NeuroSchedule, with both fast runtime and enhanced solution quality.
Major features are as follows: (1) The learning problem for HLS scheduling is formulated for the first time, and a new machine learning framework is proposed.
(2) Pretraining models are adopted to further enhance the scalability for various scheduling problems with different settings.
Experimental results show that NeuroSchedule obtains near-optimal solutions while achieving more than 50,000 × improvement in runtime compared with the ILP-based scheduling method.
At the same time, NeuroSchedule improves the scheduling results by 6.10% on average compared with state-of-the-art entropy-directed method.
To the best of our knowledge, this is the first GNN-based scheduling method for HLS.
Introduction
Recently, FPGA has been playing an important role in accelerating computations from the cloud to the edge [1, 2, 3, 4, 5].
As a reconfigurable computing device, FPGA gains increasing popularity for its high performance and great energy efficiency.
However, developers are required to write programs using the hardware description language (HDL) to implement applications in FPGAs which is time-consuming and error-prone.
Therefore, the utilization of FPGA’s enormous computing power is limited by the cumbersome developing process.
As a result, the concept of high-level synthesis (HLS) is proposed in the EDA community to speed up the developing process [6, 7, 8].
HLS automatically transfers high-level specifications (written in highlevel languages like C/C++) to behavior-level implementations (written in HDLs like Verilog/VHDL).
Figure 1 presents the whole flow of HLS.
As shown in the figure, HLS mainly comprises three parts: compilation, synthesis, and generation.
For compilation, high-level languages are transferred to
Control Data Flow Graphs (CDFGs) with the assistance of modern compilers like GCC or LLVM.
During the synthesis process, hardware resources (e.g. functional units, storage, and wires) are first allocated.
According to the allocated hardware resources, the operations in CDFG (such as ADD and
MUL) are scheduled to certain time steps.
After scheduling, the allocated hardware resources are int a0, m1, a2, a3, m4, a5; a0 = x0 + x1; m1 = x2 * x3; a2 = x4 + x5; a3 = a0 + m1; m4 = m1 * a2; a5 = a0 + a3;
High-level language code int a0, m1, a2, a3, m4, a5; a0 = x0 + x1; m1 = x2 * x3; a2 = x4 + x5; a3 = a0 + m1; m4 = m1 * a2; a5 = a0 + a3;
High-level language code
Synthesis
Allocation
[Figure]
Storage
Allocation
[Figure]
Components
Storage
Binding
Functional
[Figure]
Storage
Connectivity Components
Binding
Functional
[Figure]
Storage
Connectivity Components
Hardware Library
Scheduling
[Figure]
[Figure] [Figure]
[Figure] [Figure]
Scheduling
[Figure]
Compilation
[Figure]
Optimization
[Figure]
Compilation
[Figure]
Optimization
[Figure]
Generation
[Figure]
Generation bound to corresponding operations and time steps. After synthesis, the generation process generates corresponding HDLs according to the synthesized results. Scheduling is one of the most critical stages in the HLS process, which arranges the execution order of operations, and thus determines the performance of generated circuits. As depicted in
Figure 1, according to the allocated hardware resources and the dependencies between operations, the operations in CDFG are arranged in different time steps.
For example, during the scheduling process, operation ADD0 is scheduled to step 0 and bound to the adder.
It is noteworthy that only one adder is allocated to the scheduling task.
So operation ADD2 could not be scheduled to step 0 same as operation ADD0.
Therefore, determining the scheduling priority for the operations is the key objective of the scheduling problem.
However, the scheduling problem has been proved to be NP-hard [9].
And existing algorithms suffer from either unbearable long runtime or poor quality of solutions.
Integer linear programming (ILP) formulation has been proposed to solve the scheduling problem optimally [10].
However, the exponential runtime of ILP is unbearable.
Therefore, various heuristic algorithms are proposed to obtain sub-optimal solutions within acceptable runtime.
List scheduling is one of the most famous heuristic algorithms [11].
List scheduling stores ready operations in a list and arranges them according to a heuristically defined priority function.
Based on the list scheduling algorithm, force-directed scheduling algorithm is proposed [12], which builds a force-distribution graph for representing the priority function.
Inspired by the force-directed scheduling algorithm,
Entropy-Directed Scheduling (EDS) algorithm
[13] replaces the priority function with the entropy of
CDFGs to be scheduled.
The utilization of the entropy function not only speeds up the scheduling process, but also improves the quality of solutions.
Different from the algorithms in the list scheduling family, an algorithm based on the system of difference constraints (SDC) is proposed to speed up the ILP-based algorithm.
SDC-based algorithm transfers the scheduling constraints into a system of difference constraints using heuristics, which greatly reduces the number of equations to be solved.
However, the SDC-based algorithm cannot guarantee the optimal solution.
Therefore, the aforementioned heuristic algorithms all trade quality of solutions for runtime, which leaves a large optimization space to be explored.
On the one hand, the complicated dependencies of operations in the CDFG bring great challenges to the scheduling process.
On the other hand, Graph Neural Networks (GNNs) [14, 15, 16] have great representational power of complicated graphs, which is suitable for extracting complicated relations between operations in CDFGs.
Therefore, a natural idea is to adopt GNNs to tackle the scheduling problem for both fast runtime and high-quality solutions.
This paper proposes the first
GNN-based scheduling algorithm called NeuroSchedule.
Inspired by algorithms in the list scheduling family, NeuroSchedule builds a GNN model to predict the priorities of operations in the CDFG.
The predicted priorities are then used to arrange the operations in CDFG to obtain the scheduling results.
Major contributions of this paper are summarized as follows:
• A learning problem for the HLS scheduling is first formulated, and then a GNN-based learning framework is proposed to effectively and efficiently solve the problem. • Pre-training models are adopted to further enhance the scalability for different scheduling problems with different settings.
• Experimental results indicate that NeuroSchedule obtains near-optimal solutions while achieving more than 50,000 × speedup compared with the ILP-based algorithm [10].
At the same time, NeuroSchedule improves the scheduling results by 6.10% on average compared with the state-of-the-art EDS algorithm [13].
The rest of this paper is organized as follows.
Section 2 gives a motivating example for the GNNbased scheduling method.
Section 3 gives preliminaries and the formulation of the GNN-based scheduling problem.
The proposed machine learning framework is presented in Section 4.
Section 5. introduces NeuroSchedule, as well as the pre-training models adopted for enhancing the scalability.
Section 6 presents the experimental results.
Finally, conclusion is drawn in Section 7.
[Table]
[Figure]
Motivation
As introduced in Section 1, the list scheduling algorithm stores ready operations in a list and schedules them according to a heuristically defined priority function.
Here, a ready operation means all of its predecessing operations are finished and the operation could be scheduled for execution immediately.
For example, in Figure 2(a), node 3 is ready when node 0 and node 1 are finished.
The process of list scheduling could be stated as follows.
First, the ready operations are stored in a list.
Next, the operation with the highest priority in the list is picked to be scheduled to available hardware resources (e.g. adders and multipliers) for execution.
After the operation is finished, the newly generated ready operations are stored in the list.
The above process is iterated until all the operations are successfully scheduled.
From the list scheduling process, defining precise priority functions is crucial to obtain good scheduling results.
A motivating example is given in Figure 2.
Figure 2(a) presents the CDFG to be scheduled, which consists of 6 operations with 2 types (ADD and MUL).
And the available hardware resources for scheduling are an adder (for ADD) and a multiplier (for MUL).
Figure 2(c) depicts the entropydirected scheduling process, which takes the entropy function for the priority computation.
In the figure, ready operations are colored blue and finished operations are colored green.
Since the entropy priority of node 3 is higher than node 2, node 3 is scheduled before node 2.
However, scheduling node 3 first delays the scheduling of node 4, which makes the multiplier idle for 3 cycles (i.e. time steps).
Consequently, the final scheduling result is not optimal.
In Figure 2(b), we assume there is an optimal manually designed priority function.
Based on the optimal priority function, node 2 is scheduled before node 3, and thus the hardware resources are fully utilized.
Therefore, the scheduling algorithm based on the manual function gives the optimal solution.
The success of the manually designed function motivates us to build a machine learning framework, which learns the priority function automatically.
GNNs are an effective framework for representation learning of graphs, which follow a neighborhood aggregation scheme.
In GNNs, representation vectors of nodes are computed by recursively aggregating and transforming representation vectors of their neighboring nodes [14].
Therefore, GNNs are suitable to learn the CDFG embeddings for obtaining the priority function.
As a result, a GNN-based machine learning framework is proposed to learn the priority function.
Experiments verify the effectiveness of the proposed scheduling method based on GNN.
Hardware Library Priority Predicted 𝓟
GNN
[Figure]
by ILP Train Loss Function
[Figure]
𝓛
List Scheduling 𝓓
S’
Schedule Solution
Input CDFG
[Figure]
Preliminaries and Problem Formulation
CDFG.
In high-level synthesis, the program in high-level language is first translated into CDFG, a graph-based representation defined as follows:
Definition 1 A CDFG G =
( V, E ) is a directed graph with node set V and edge set E .
Each node v ∈ V represents an operation, e.g. add or load.
Each directed edge e = ( v i , v j ) ∈ E, i, j ∈ 1 , ..., | V | represents a dependency relationship between operation v i and v j , i.e., operation v j must be executed after the execution of operation v i .
ASAP and ALAP.
The As-Soon-As-Possible (ASAP) algorithm and the As-Late-As-Possible (ALAP) algorithm give the information about node flexibility during the scheduling process.
For node v i ∈ V , the result of ASAP and ALAP can be formulated as follows:
[Equation]
Resource-constrained scheduling problem.
Based on the CDFG, we have to solve the resourceconstrained scheduling problem: given the number of resources, find the scheduling solution with the minimum number of execution cycles, while satisfying the given set of scheduling and resource constraints.
Given a functional unit set U , where u i ∈
U represents a functional unit, such as an adder.
Denote d u i as the cycle delay of functional unit u i .
Given a CDFG G =
( V, E ) , and let n = | V | , a schedule can be defined as S =
( t 1 , t 2 , ..., t n , f 1 , f 2 , ..., f n ) , where ∀ i ∈ 1 , 2 , ..., n , (1) t i ∈ N ∗ , denoting the start cycle of operation v i ∈ V , (2) f
i ∈ U , denoting the functional unit used by operation v i .
Let L = max i ∈ 1 ,...,n t i + d f i − 1 denote the total number of execution cycles.
The resource-constrained scheduling problem minimizes L , which can be defined as follows:
Definition 2 Find an optimal schedule S ′
=
( t ′ 1 , t ′ 2 , ..., t ′ n , f ′ 1 , f ′ 2 , ..., f ′ n ) , such that:
a) ∀ e
= ( v i , v j
) ∈ E, i, j ∈ 1 , 2 , ..., n , t ′ i + d f ′ i ≤ t ′ j b) ∀ i, j ∈ 1 , 2 , ..., n, i ̸
= j , f ′ i = f ′ j = ⇒ | t ′ i − t ′
j | ≥ d f ′ i c)
L = max i ∈ 1 ,...,n t ′
i + d f ′ i − 1 is minimized.
List scheduling.
As introduced in Section 2, list scheduling is an effective scheduling algorithm.
It divides the node set V into three sets, finished nodes V F , ready nodes V R , and unscheduled nodes
V U .
Given the node priority function F , where F ( v i ) <
F ( v j ) , v i , v j ∈ V R indicates that operation v i has a higher priority than operation v j .
At each step, list scheduling algorithm chooses the node v h ∈ V R to schedule, where F ( v h ) =
min v r ∈ V R F ( v r ) .
A better priority function in the list scheduling algorithm corresponds to a better scheduling solution.
As the ILP-based scheduling algorithm can get the optimal scheduling results, the scheduling result S I
=
( t I 1 , t I 2 , ..., t
In , ... ) is a good candidate for the priority function, where F ( v x ) =
t Ix , v x ∈ V .
However, for large scheduling problems, the runtime of the ILP-based algorithm is not acceptable.
Therefore, we propose the GNN model to predict the priority function.
Neural Network Model.
Denoting the input data as D , our model can be formulated as:
[Equation]
Our loss function can be formulated as:
[Equation]
[Figure] Details of our loss function will be introduced in Section 4.
Figure 3 shows the framework of our
NeuroSchedule algorithm.
The ILP-based scheduling algorithm is adopted to generate the training labels for our GNN model.
The output of the GNN model, i.e. P , is used by the list scheduling algorithm as the priority function, which obtains significantly enhanced scheduling solution.
Model Design
A machine learning framework is proposed to learn the priority function in the list scheduling algorithm.
In the proposed framework, the CDFG is taken as input, and a GNN model is trained to predict the priorities of the operations.
This section gives the model input, the model architecture, and the selection of loss function.
4.1 Model Input
The input of a GNN model includes an adjacency matrix representing graph connections and a tensor representing node features.
Different node features will be selected for different problems.
In the scheduling problem, the node features consist of operation type, number of cycles of the operation, along with the above-defined ASAP and ALAP.
Operation’s type and number of cycles.
As the operation type is categorical and discrete, inspired by the graph-based program representation [17], the operation type is encoded by one-hot encoding.
For the number of cycles of an operation, the cycle number is directly used in the encoding.
For example, if there are two types of operations (MUL, ADD) and the cycle number of MUL operation is 1, the node feature of MUL operation could be represented as [1, 0, 1].
Here, “10" from the first “1" and second “0" in the feature denotes MUL.
Operation’s ASAP and ALAP.
Except for operation’s type and number of cycles, ASAP and ALAP are also adopted in the node features.
As introduced in Section 3, ASAP and ALAP represent the earliest step and the latest step that an operation could be scheduled, respectively.
The ASAP and
ALAP of an operation indicate the operation’s flexibility during scheduling.
For example, the ASAP of node ADD2 in Figure 5 is step 0 and the ALAP is step 1, which means node ADD2 could be scheduled between step 0 and step 1.
Different from node ADD2, the ASAP and ALAP of node
ADD3 in Figure 5 are both step 1, which means node ADD3 could only be scheduled to step 1.
The reason why ASAP and ALAP are adopted in the node features is that the flexibility of an operation affects its priority during scheduling.
Operations with low flexibility during scheduling tend to be the bottlenecks [13], which should be scheduled with high priority to fully utilize the hardware resources.
Finally, the operation’s type, operation’s number of cycles, along with normalized ASAP and ALAP, are concatenated as a tensor representing node features.
For example, there are two operations in
ADD
MUL
Step Step
Step Step
ADD ADD
MUL
Step Step
Step Step
ADD
ADD0ADD0 MUL1MUL1 ADD2ADD
ADD3ADD
ADD5ADD
MUL4MUL
ADD0 MUL1 ADD
ADD ADD
MUL
Step Step
Step00
Step Step
Step
 Step
Step
Step0022
Step
Step Step
[Figure]
Figure 5, MUL and ADD, and the number of cycles of operations MUL and ADD are both 1.
So node ADD3 could be encoded as [0, 1, 1, 0.5, 0.5].
It is noteworthy that the ASAP and ALAP of an operation are normalized according to the maximum ASAP value among all the operations.
The normalization is conducted because the number of operations in the CDFG varies such that using the absolute ASAP and ALAP values will bring noises.
4.2 Model Architecture
Figure 4 presents the architecture of the proposed model.
As shown in the figure, the proposed model is composed of a GNN model and a feed-forward network (FFN) model.
The GNN model is adopted to learn the embedding of each operation from the operation features and graph structure, while the FFN model is utilized to conduct regression tasks for predicting operations’ priorities.
The workflow of the model is presented as follows.
First, operation features including the operation’s type, operation’s number of cycles, along with normalized ASAP and ALAP are concatenated into a feature tensor as introduced in Section 4.1.
Next, a GNN model is adopted to learn operation embedding from the feature tensor and graph structure.
Graph Isomorphism Network (GIN) is used as the GNN model for its strong power of learning embeddings from graph structures [14].
Finally, the learned operation embedding is fed into the FFN model for predicting the operations’ priorities.
In the FFN model, there are three fully-connected layers, and ReLU is adopted as the activation function [18].
4.3 Training Settings
As depicted in Figure 4, the proposed model outputs the operations’ priorities by regression.
To effectively train the proposed model, a dedicated training pipeline is proposed.
The proposed training pipeline mainly focuses on two questions: 1.
How to acquire the regression labels; 2.
How to select the training objective function (a.k.a loss function).
Training label.
As formulated in Section 3, training the priority prediction task requires a collection of data D
= { g i , p i , c i } , where g i denotes CDFG’s graph structure, p i denotes operation properties (such as ASAP and ALAP), and c i denotes the training label.
In the proposed training pipeline, the ILP-based method is used to generate the regression labels [10].
The details are presented as follows.
The ILP-based method gives the optimal solution, which arranges CDFG’s operations into suitable time steps.
Obviously, operations arranged in later time steps have lower priorities than those arranged in earlier time steps.
Therefore, in the proposed training pipeline, the time step obtained by the ILP-based method (i.e. t Ii in Section 3) is taken as the regression label c i .
As mentioned in
Section 4.1, the time step of an operation is also normalized based on the maximum time step, which reduces the impact of noises.
Figure 2 gives an example.
In the manually designed priority function, the time step could be solved by the ILP-based method.
As shown in the figure, the ILP-solved time steps [0 , 0 , 1 , 2 , 2 , 3] are assigned to corresponding operations.
After normalization, the ILP-solved time steps are computed as [0 , 0 , 0 .
Training objective function.
The mean square error (MSE) function is commonly adopted for training a regression task.
Given dataset D = { g i , p i , c i } , the output of our GNN model has the form
P ( i ) =
P ( g i , p i ) .
The MSE function (cid:80) i ( P ( i )
− c i ) 2 guides the model to accurately predict the operations’ priorities.
However, the scheduling task cares more about the operations’ relative order rather than the absolute priority values of the operations.
Moreover, it takes more effort for training a model to predict the absolute value than to predict the rank.
Therefore, inspired by AutoTVM [19], the following rank loss function is adopted to train the model for predicting the rank of all operations [20]:
[Equation]
During training, the above rank loss function keeps the relative order of operations’ priorities with reduced training efforts.
NeuroSchedule 5.1 Pre-training for Different Scheduling Settings
As introduced in Section 4.1, the input of our model consists of the operation’s type and operation’s number of cycles, along with normalized ASAP and ALAP.
It works well when the numbers of functional units are fixed.
However, there are various scheduling settings in real HLS cases including: (1) different operation types, (2) different number of functional units for each type, and (3) different number of cycles for different operation types.
Figure 5(d) gives a scheduling example with 2 adders, 1 multiplexer, and multi-cycle operations, i.e., the MUL operation takes 2 cycles.
Therefore, for scheduling problems with different settings, specified GNN models are required to get valid solutions.
However, it is infeasible to train a GNN model for each scheduling setting due to the unbearable training efforts.
Therefore, we build a dataset of CDFGs with different scheduling settings, and pre-train our model with the dataset.
To specify the scheduling setting, we add the number of the corresponding functional units into the node features of each operation.
While dealing with a specific scheduling setting, we fine-tune the model using the randomly generated data from the setting.
Compared with direct training, the fine-tuning takes fewer efforts with enhanced scheduling solutions.
5.2 NeuroSchedule Algorithm
Algorithm 1 gives an overview of the proposed NeuroSchedule algorithm, which is based on the traditional list scheduling algorithm and our neural network models.
The symbols in Algorithmare defined in Section 3.
We use trained network model P as priority function F (line 3), and run the loop until each node in the CDFG is successfully scheduled (line 7).
In each cycle, we try to schedule every available functional unit using the node v chosen with the highest priority.
After the node v chosen is selected from the ready set V R , we update the schedule S P , and the sets V R and V F (line 16).
Then we delete all the out edges ( v i , v chosen ) of v chosen (line 20).
If the in-degree of v i equals 0, we add v i into the ready set V R .
Finally, the scheduling solution is obtained in S P .
Experiments
The proposed GNN-based scheduler is implemented in Python (version 3.9.12), and the GNN model is designed and trained with Pytorch [21] (version 1.8.0) and pyG [22] (version 2.0.4).
To train the GNN model, we build a dataset including 50,000 CDFGs.
The operations in the CDFGs are annotated with the priorities solved by the ILP-based method (as introduced in Section 4.3).
For efficiency, Gurobi [23] is adopted as the ILP solver.
Details of dataset preparation are presented in supplementary materials.
We conduct all the experiments on a Ubuntu 20.04 LTS Linux Server with a CPU (Intel(R) Xeon(R)
Gold 5218 CPU@2.30GHz) and a GPU (NVIDIA Tesla V100).
The experiments are conducted to answer the following questions:
[Table]
% CHStone motion
[Table]
Vectors
[Table]
CHStone sha
[Table]
[Table] [Table]
[Table] [Table] [Table]
[Table] [Table]
MiBench
[Table] [Table]
[Table] [Table]
[Table]
[Table]
MiBench
[Table]
[Table]
MiBench idctcol
[Table]
[Table] [Table]
[Table]
MiBench write
[Table] [Table]
[Table] [Table]
Total
[Table]
[Table]
Rate
[Table]
Q1: Could NeuroSchedule obtain better solutions than the entropy-directed algorithm? Q2: Could NeuroSchedule obtain optimal solutions with respect to the ILP-based method?
Q3: How fast is the proposed algorithm compared against the ILP-based method? Q4:
How do the pre-training methods work in enhancing the algorithm’s scalability? Q5: What are the differences between different loss functions? 6.1 Quality of Solutions
To answer Q1 , we evaluate the performance of NeuroSchedule against state-of-the-art entropydirected scheduling algorithm [13] and ILP-based method [10] using the benchmarks from CHStone [24] and MiBench [25].
The experimental results are presented in Table 1.
As shown in the table, NeuroSchedule improves the scheduling results by 6.10% on average compared with EDS.
Moreover, compared with the ILP-based method, NeuroSchedule obtains optimal solutions in 88.89% of the benchmarks and near-optimal solutions in the rest, while EDS can only obtain 16.67% optimal solutions in all benchmarks.
To answer Q2 , we synthesize a benchmark suite including large CDFGs with complex dependencies among the operations.
The entangled dependencies make it hard for scheduling algorithms to find an optimal solution.
We evaluate NeruoSchedule, entropy-directed algorithm [13] and ILP-based algorithm [10] on the synthesized complex benchmark suite.
The results are presented in Figure 6.
Figure 6 gives the heatmaps for comparing the scheduling results among NeuroSchedule, entropydirected algorithm, and ILP-based algorithm.
As shown in Figure 6(a), NeuroSchedule obtains optimal solutions on 90% of the synthesized complex benchmarks.
While in Figure 6(b), the entropydirected algorithm fails to obtain optimal solutions on all of the benchmarks due to the complex operation dependencies.
Moreover, as shown in Figure 6(c), NeuroSchedule obtains better solutions on 94% of the synthesized complex benchmarks compared with the entropy-directed algorithm.
6.2 Execution Time
To answer Q3 , we evaluate the runtime of NeuroSchedule against state-of-the-art entropy-directed scheduling algorithm and ILP-based method [10] using benchmarks from CHStone [24] and
MiBench [25].
The results are presented in Table 1.
As shown in the table, NeuroSchedule achieves more than 50,000 × speedup compared with the ILP-based method, while obtaining near-optimal solutions.
Moreover, the ILP-based method fails to solve certain benchmarks within 10,000 seconds due to the exponential runtime overhead.
Besides, since NeuroSchedule needs to launch CUDA kernels to compute operation priorities, it runs a little slower than the entropy-directed algorithm. [Figure]
[Figure] Nevertheless, NeuroSchedule is still fast and the difference in runtime between NeuroSchedule and entropy-directed algorithm could be ignored.
6.3 Pre-training and Fine-tuning
To answer Q [Figure] we synthesize a small dataset that consists of 1000 CDFGs.
Details of the small dataset are presented in supplementary materials.
Based on the synthesized small dataset [Figure] NeuroSchedule is trained in two different ways: from scratch and fined-tuned from the pre-training model.
The synthesized benchmark suite in Section 6.1 is adopted to evaluate the two models trained in different ways.
Besides, the optimal rate is selected as the metric, which indicates the number of benchmarks in the synthesized suite with optimal solutions obtained by NeuroSchedule.
Figure 7 depicts the optimal rate curve during training.
As shown in the figure, NeuroSchedule fine-tuned from the pre-training model achieves 3% improvement in the optimal rate compared with that trained from scratch.
Moreover, NeuroSchedule fine-tuned from the pre-training model achieves a better optimal rate while costing reduced training efforts compared with that trained from scratch.
6.4 Training Settings
To answer Q5 , NeuroSchedule is trained with MSE loss and Rank loss [20] on the synthesized small dataset described in Section 6.3.
The synthesized benchmark suite in Section 6.1 is adopted to evaluate the two models trained with different loss functions.
Besides, the optimal rate is selected as the metric.
Figure 7 depicts the optimal curves during training.
As shown in the figure, rank loss behaves slightly better than MSE loss, which indicates that training a model for predicting the absolute value requires more effort than training a model for predicting the rank.
Therefore, the rank loss is chosen as the default training objective function.
Conclusion
This paper presents NeuroSchedule, a novel and effective GNN-based scheduling method for HLS.
NeuroSchedule learns to predict operation priorities using GNN models.
The predicted priorities are adopted in list scheduling process.
Besides, to enhance NeuroSchedule’s scalability for various scheduling problems with different settings, pre-training methods are adopted.
Experimental results indicate that NeuroSchedule achieves significantly enhanced solutions compared with the state-ofthe-art method.
Moreover, NeuroSchedule obtains near-optimal solutions while achieving more than 50,000 × speedup compared with the ILP-based algorithm.
Future work includes elaborations on the training details for better tuning, supporting operation chaining, and testing of NeuroSchedule on larger and more complicated benchmarks.
Acknowledgments and Disclosure of Funding
This work was supported in part by the Key R&D Program of China (No.2019YFB2205002), the
Key Program of NSFC (No. 62034005), the NSFC (No. 61974084) and Beijing Municipal Science & Technology Commission (No. Z191100007519015).
A Supplementary Materials A.1 Traditional List Scheduling Algorithm
Input: Priority function F , CDFG G
=
( V, E ) , functional unit set U , and delay array d Output: Generated schedule S ′ 1: V F ← ϕ , V R ← ϕ , V U ← V , S ′ ← ϕ , Time ←2: for v i in V do 3: if v i .inDegree
{+() = 
 then 4:+}
V R ← V R ∪ v i , V U ← V U \ v i 5: while | V F | < | V | do 6: for u j in U do 7:
M ← MAX _ FLOAT , v chosen ← ϕ 8: for v i in V R do 9: if F ( v i ) <
M and v i .canUse
( u j ) then 10: M ← F ( v i ) , v chosen ← v i 11: if v chosen = ϕ or u j .unavailable ( Time ) then 12: continue 13:
S ′ .insert
( v chosen , u j , Time ) 14: u j .setUnavailable ( Time, Time + d u j − 1) 15:
V R ← V R \ v chosen , V F ← V F ∪ v chosen 16: for v i in V R do 17: if ( v chosen , v i ) ∈ E then 18: E ← E \ ( v chosen , v i ) 19: if v i ∈ V U and v i .inDegree
{+() = 
+}
then 20: V R ← V R ∪ v i , V U ← V U \ v i 21: Time ← Time + Algorithm 2 gives an overview of the traditional list scheduling algorithm.
Our proposed NeuroSchedule method as shown in Algorithm 1 derives from Algorithm 2, while taking our trained neural network as the priority function. The loop (line 5) runs until each node in the CDFG is successfully scheduled. In each cycle, for every available functional unit (line 6), the algorithm schedules the operation v chosen with the highest priority to the functional unit (line 13). Note that only operations of the corresponding type of the current functional unit can be chosen (line 9).
After the chosen operation v chosen is scheduled, all the out edges ( v chosen , v i ) of v chosen are deleted (line 18). If the in-degree of v i equals 0, v i is added into the ready set V R (line 20). Finally, the scheduling solution is obtained in S ′ . In the traditional list scheduling algorithm, the most widely-used priority function is:
[Equation]
6) where ASAP and ALAP are defined as in Section 3.
The priority function F t describes the flexibility of each node in the CDFG.
This priority function follows a simple but effective rule: nodes with higher flexibility are scheduled prior to nodes with lower flexibility. Recently, the force-directed priority function [12] and the entropy-directed priority function [13] are designed for better scheduling solutions. In the proposed NeuroSchedule method, our GNN-based priority function shows its superiority to other methods.
It is necessary to acquire sufficient data for training a GNN model to predict the operations’ priorities.
We propose a CDFG generation algorithm, which generates CDFGs with a balanced structure and high diversity. Algorithm 3 presents the CDFG generation algorithm, where T denotes the number of operations in the generated CDFG, M denotes the minimum number of operations in a layer, N denotes the maximum number of operations in a layer, I denotes the rate that inter-layer operations are connected, U denotes the rate that an operation is not connected to a predecessor, and G = ( V, E ) denotes the generated CDFG. As shown in the algorithm, operations in the generated CDFG are arranged into several layers, and operations in the current layer are randomly connected to operations in previous layers.
Input: T, M, N, I, U
Output: Generated CDFG G = ( V, E ) 1: L 1 ← ϕ, L 2 ← ϕ, L 3 ← ϕ, V ← ϕ, E ← ϕ 2: while T > 0 do 3: C ← random.randint ( M, N ) 4:
T ← T − C 5: repeat 6: C ← C −7:
O ← newOperation () 8: O.optype ← randomOpType () 9: V ← V ∪ O 10: L 3 ← L 3 ∪ O 11: if random.random () > U then 12: if random.random () <
I then 13: O ′ ← selectRandomElement ( L 1 ) 14: else 15: O ′ ← selectRandomElement
( L 2 ) 16: E ← E ∪ ( O, O ′ ) 17: until C <18: L 1 ← L 1 ∪ L19: L 2 ← L20:
L 3 ← ϕ 21: return ( V, E )
A.3 Dataset Details
Two datasets are synthesized for training and evaluating the proposed model. Details of the synthesized datasets are presented as follows. First, we synthesize a large dataset including 50000 CDFGs to train our model. The CDFGs are randomly generated using the algorithm shown in Algorithm 3, and the labels are generated using the ILP methods as introduced in Section 4.
The numbers of functional units are randomly generated according to the size of CDFG, using the following function:
[Equation]
Second, a small dataset including 1000 CDFGs are built to evaluate our training settings in Section 5.2. In the small dataset, the number of each functional unit is fixed to 1. For better scalability on real applications, the dataset is extended to give more instructive information to our GNN model. The node features for each node v i are a 11-dimensional vector F , and the details are given as follows:
0-1 : Operation’s type (one-hot); 2 : Number of operation’s cycles; 3 : The scheduling result using ASAP method (normalized); 4 : The scheduling result using ALAP method (normalized); 5 : ASAP ( v i ) as introduced in Section 3; 6 : ALAP ( v i ) as introduced in Section 3; 7 : The scheduling result of traditional list scheduling algorithm using ALAP − ASAP as the priority function; 8 : max v i ∈ V ASAP ( v i ) ; 9 : The maximum number of cycles obtained by traditional list scheduling algorithm; 10 : The number of corresponding functional units u v i . Note that features F [3 , 4] are different from F [5 , 6] , because F [3 , 4] are the scheduling results using the idea of ASAP and ALAP, while considering the resource constraints. All the node features can be obtained efficiently by the traditional list scheduling algorithm using ALAP − ASAP as the priority function.
In our network model, we use a 5-layer GIN [14] to generate the learned operation embeddings. The dimension of each GIN layer is 64 . The learned operation embeddings are fed into a (64 , 64) dense layer, and then into a (64 , 1) dense layer.
The activation function of each layer in the model is ReLU.
There is a dropout layer between the 5-layer GIN model and the fully connected layers, while the dropout value is set to be 0 . 5 .
In our training process, we train our model in 2 stages, pre-training and fine-tuning. More specifically, we train our GNN model in the following way: Stage 1 : Pre-training: We use our generated large dataset including 50000 CDFGs to train our model.
The dataset is divided into the training set consisting of 49000 CDFGs, and the evaluation set consisting of 1000 CDFGs. The learning rate is set to be 0 . 0002 initially, which decays by lr = 0 . 9 × lr for each 10 epochs. We train the model for 2500 epochs.
Stage 2 : Fine-tuning: We use our generated small dataset including 1000 CDFGs to fine-tune our model. The learning rate is set to be 0 . 0002 initially, which decays by lr = 0 . 9 × lr for each 10 epochs. We train the model for 25 epochs.