Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes NeuroSchedule, a GNN-based scheduling method for HLS. At the core of NeuroSchedule is a novel priority function that predicts the priority (rank) of a ready operation, which is used for list scheduling. To improve the scalability of NeuroSchedule over different settings, pre-training methods are adopted, which can be fine-tuned for downstream tasks. NeuroSchedule effectively approximates the optimal ILP-based priority function at a 50,000x lower computation cost. This leads to an average of 6.10% improvement in cycles over the Entropy-Directed Scheduling (EDS) algorithm.
strengths_and_weaknesses: **Strengths**

* The approach of applying GNN to HLS scheduling is novel and clever.

* NeuroSchedule demonstrates competitive results to the optimal ILP algorithm at a tiny faction of computation cost.
  

**Weaknesses**

* The effectiveness of NeuroSchedule is demonstrated only for small-scale kernel benchmarks taking just tens of cycles in most cases.

* The relatively small average improvement over EDS (i.e., 6.10% *without* including those with 0.0% improvement) while taking some 6$\times$ longer to run.
questions: * How would NeuroSchedule handle more complex/realistic applications with non-deterministic events such as DRAM accesses, caching, bank conflicts, etc?

* In Equation (2), why ALAP includes the second definition with ASAP() for those vertices not connected to $v_i$? Can you elaborate on this?
limitations: I find this paper presents an interesting proposal and was a very enjoyable read. As far as I know, this is the first work that formulates the scheduling priority function as a learning task with GNN to effectively approximate the optimal ILP algorithm at a tiny fraction of computation cost. Indeed, it's a promising direction and the results demonstrate the effectiveness of the proposed idea.

While I like the direction of this work, my acceptance is *weak* for the following limitations. For benchmark selection, most of the benchmarks look pretty small in scale, taking tens of cycles in most cases. At this point, it is not clear how effective NeuroSchedule would be for more realistic/complex applications that may require off-chip DRAM accesses, caches, and other non-deterministic behaviors. Also, while delivering competitive results to the optimal ILP-based schedule in most cases, the average improvement over EDS is rather small. This makes me question about the practical value of NeuroSchedule as its average improvement over EDS would go below 5% if we include those optimal benchmarks omitted while it takes about 6$\times$ longer scheduling time.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes NeuroSchedule, an efficient and effective GNN-based scheduling method designed for the HLS. This paper formulate the learning problem of the operation scheduling in HLS and is able to embed the operation information in the GNN to aid learning the schedule. This paper further builds a dataset of CDFGs with different scheduling settings to pre-train the model. With final-tuning on randomly generated designs in different settings, the model's prediction is able to be generalized.
strengths_and_weaknesses: Strength:
1. This paper proposes a formulated and more likely to be standardized method to solve the seemingly intractable scheduling problem in HLS. This may open door and inspire more ML techniques to be effectively applied in the hardware design.
2. The methodology description along with the supplementary material is clear and easy for the readers to follow or potentially reproduce

Weakness:
1. The results' improvements are hard to be justified as significant, compromising the method's effectiveness: As shown in Table 1, as compared with Entropy-directed methods, overall the runtime latency is improved by 16.67% but the optimization run-time is 5.7 times more. Further compare with the eligible options in ILP-based, it is shown that Entropy-directed method is already near optimal. Then it is questionable if it is worth it for the relatively small latency improvement but with huge degradation on the scheduling run-time.
2. A little too simplified scheduling assumption based mainly on individual operation latency and dependency, but authors can clarify in the following questions to clear the concern.
questions: 1. HLS seems to be not that related in this article. This article seems more about the general operation scheduling. Is there any special reason or limitation why the authors decide the scope of HLS?
2. How do the authors solve the case of resource conflict, if the hardware modules are to be reused? Not only the operation dependency will influence the scheduling, whether the hardware module busy or not will also have impact. For instance in Fig5.a, what if there are some restrictions that the first two addition have to use the same adder
3. Will and how do the authors consider HLS's different pargmas in the scheduling, as they will decently impact the scheduled performance via changing the underlying hardware, such as buffer memory size/bandwidths?

limitations: Yes, it's addressed.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors propose NeuroSchedule, a GNN-based schedule for solving the high-level synthesis (HLS) problem in the FPGA development. NeuroSchedule is the first GNN-based scheduler in this domain. It is trained using supervised learning with data labeled by the ILP-based scheduler. NeuroSchedule has an extremely good runtime improvement over the ILP-based scheduler, while achieving near-optimal solutions. NeuroSchedule is also compared with a heuristic-based method. Its run-time is not as good as the heuristic method, but its solutions are generally better. 
strengths_and_weaknesses: Strengths:

-	The inference time of the proposed algorithm is 50,000 times faster than its teacher schedule (ILP-based optimizer).
-	The quality of solutions provided by NeuroSchedule are better than the state-of-the-art entropy-directed method by 6.10% on average.
-	By fine-tuning the pre-trained model, NeuroSchedule can get good results in different settings.
-	The rank-loss (relative priorities) helps the trained model to get a better optimal rate, compared with traditional MSE loss.
-	The experiment section is well organized. 

Weaknesses:

-	It would be better to conduct significant test to see whether the performance of NeuroSchedule is significantly better than the entropy-directed method.
-	It would be better to compare with some reinforcement learning based scheduling methods.
-	There may be a typo in one of the formulas. 

questions: -	Should “max” in Line 41 be “min”, since the list scheduling algorithm picks the operations with the highest priority first and smaller F values indicate higher priorities?
limitations: The experiment shows the proposed method is slower than the heuristic entropy-directed method.

It would be better to mention how long it took to label the training data set of 50,000 CDFGs by the ILP-based scheduler. Not necessary a limitation, but the readers may want to know.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper implements a task scheduler for HLS based on the GNN model, and the suitable method for HLS scheduling is proposed for the first time. In hardware programming, it is necessary to convert a high-level language into a circuit-level implementation, but this step is time-consuming and yields poor quality in traditional methods. According to the characteristics of many data dependencies in the generation of CDFG during the compilation process, combined with the GNN model that can perform well on complex graphs, the proposed scheduling method can speed up the execution time of the synthesis step and improve the quality of the generated circuit-level language.
strengths_and_weaknesses: Strengths
1. The paper gives a clear example to illustrate the targeted optimization algorithm, which can clearly explain the purpose of this article, and the writing logic is clear, the motivation and optimization point of this paper is clearly clarified.
2. It converts the overall priority to a relative order that is more convenient for calculation when performing operations, thereby reducing the complexity of the overall calculation.
3. The application scenario is consistent with GNN, and the priority of each operator is predicted through the GNN model, thereby reducing the execution time of the entire process.

Weaknesses
1. Some descriptions need to be more detailed. For example, when it is stated in the Introduction that operators of the same type in each step are mutually exclusive, a restrictive description of the execution should be given to avoid readers' guessing.
2. NeuroSchedule introduces GPU for calculation. The experimental results only compare the accuracy with the EDS method (in Section 6.1), emphasizing the improvement of the accuracy, and the final execution time is longer than that of EDS. The paper lacks the analysis of the benefit ratio of additional computing device.

questions: 1. It is not clear enough at Equations (1) & (2), considering that priority determination is the key to scheduling steps, more details need to explain ASAP/ALAP?
2. In line 138 the author indicates that the smaller value the higher priority is, but in line 140 it indicates the algorithm schedules the largest value, the description here is confusing.
3. The description of the encoding operator in line 181 is not clear, and further normalized features can reduce the impact of noises, which is not well explained in the paper.
4. In Section 5.2, it is mentioned that V_chosen on line 25 of the algorithm, but the algorithm has only 23 lines, which makes it difficult to understand.
5. What is the final time interval for different scheduling sequences in Figure 5 (b) and (c)? This supplement would give a intuitive explanation on the operation’s flexibility during scheduling.

limitations: yes
review_title: null

Meta-review: metareview: This paper presents NeuroSchedule, a GNN-based scheduling method for high-level synthesis. To enhance the scalability for various scheduling problems with different settings, the paper additionally adopts pre-training/fine-tuning methodology. For pre-training, the paper use data labeled by the ILP-based scheduler in supervised way fashion. Experimental results indicate that NeuroSchedule can obtain near-optimal solution with significant speedup compared with traditional method.

All reviews have similar opinion on the strength and weakness of the paper. On the strength side, the paper is well written, and details are clearly explained. Both the GNN-based scheduling method and pre-training/fine-tuning approach are novel and interesting. The experiments are reasonably sound. 

The concerns are 1). The testing benchmarks are small; and 2). The optimization run time is longer than the known Entropy-Directed scheduling method. During rebuttal period, the authors provide further explanation, partially addressing the concerns. 

Overall, I think it is a solid paper, particularly in the aspect of novelty. Therefore, the paper is recommended for acceptance.

confidence: Certain
Decision: Accept