Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes NN-driven model for designing novel traffic engineering techniques automatically. 
main_review: This paper proposes NN-driven model for designing novel traffic engineering techniques automatically. This paper is missing details on neural architecture, configuration parameters and experiment details. I had major difficult understanding the write up due to these missing difficulties. Let me elaborate:

1. No details on the model: paper does not provide the details of the NN; how many layers, what is the size of the input, how many neurons per layer. What is the loss function and how is it trained?

2. Experiment setup: Why is 10 simulation sufficient? Is random => uniform random? How many concurrent failures? For a large network, 10 simulations of failure events is not sufficient.

3. Model abstraction and retraining of the network: does the proposed model has to be retrained for a different network size and topology? how long does it take to retrain and what is the initial training time ?

4. Simulation: What traffic is being simulated? What are the datacenter applications that paper is using?

Overall the paper is very light on details and is not fit for publication at this time. 

Other issues:
--
1. SmartNICs cannot behave like switches so the NN inference they are planning to do has to be done on P4 switches only. Authors should elaborate on how they plan to deploy their model in practice?
2. Nocomparison with Contra (which is state-of-the-art, they do not show how their TE algorithm behaves against contra for the exact same performance goals)
summary_of_the_review: This paper is missing details on neural architecture, configuration parameters and experiment details. I am not sure "what" the novelty of the paper is. It is most probably due to the fact that the paper significantly lacks in details.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes an ML-based approach Mistill to help automatically deploy distributed protocol from a given TE policy. The approach learns forwarding decisions together with intermediate results such as exchange information and LinkState encoding from exemplary policies. The resulting network can later be deployed in switches to encode link-state and compute forwarding decisions. The technical contributions are the following. First, they leverage a reparameterization trick to handle categorical distributions when encoding link states into a binary vector. Second, they use the scaled dot-product attention method to help switches learn to exploit the encodings mentioned above and make the forwarding decision. In the evaluation part, the paper compares with three custom baselines, i.e., LCP, MinMax, WCMP, and exceeds all of them in terms of corresponding metrics.
main_review: Strengths.
1. The paper gives some insights in incorporating machine learning techniques into the Traffic Engineering area, including how to encode the link-state using reparameterization and how to communicate using attention techniques.

2. The paper gives a general analysis of the pros and cons of ML approaches. It is informative when evaluating the feasibility of applying ML on real switches.

Weaknesses.
1. The paper represents the challenges in a vague way. In Section 1 Introduction, the paper illustrates the necessity of customizing Traffic Engineering schemes for various applications. However, it lacks concrete analysis of challenges that lie in translating a TE policy into a distributed protocol. Only a general description in Paragraph 2, such as "it requires the specification of exchange data, the processing of the data and the algorithm ...", is not solid and analytical enough to demonstrate how these challenge the design of a better TE scheme. The paper should show how difficult it is that to process per-application-level data and to design an algorithm. Furthermore, they do not explain how incorporating machine learning techniques mitigate those challenges.

2. The paper presents unclear descriptions of technical details. 1) In the second paragraph of Section 2, they claim that the proposed approach will learn two aspects, i.e., process and exchange the local state of switches and map the exchanged state into forwarding decisions. However, later in the fourth paragraph, they mention that the model should learn more than these two aspects. The model should also learn how to react to changes in the network, such as node and link failures and changes in monitored measures. They do not explain how the latter aspect is related to the former two aspects. 2) In Section 3, paragraph 5, does "to select the HNSAs necessary for this" indicates that there might be more HNSAs? Also, it is unclear what "this" refers to.

3. The evaluation part is not complete enough. First, the paper does not show the comparison of computing overhead compared to baselines. As a result, we do not know if the proposed one can be deployed in actual data center deployment. How long will it take for training? What's the computing overhead for inference? Second, they do not use traffic features from various applications, and thus we do not know if the proposed approach can adapt to various traffic features. It is crucial in Traffic Engineering, as stated in Section 1 Introduction.

4. The paper does not discuss the possible technical defects of their approach. For example, with wrong forwarding decisions accumulating, network congestion may happen. How does the approach handle problems that arise from intrinsic errors of the Machine Learning algorithm?
summary_of_the_review: The paper gives good insights into combining machine learning techniques and traffic engineering. However, it lacks solid evaluation to demonstrate the overhead of this approach and lacks analysis of problem handling. Moreover, the representation of background and some of the technical details does not help readers to understand. Therefore, the paper should 1) add evaluations demonstrating the overhead of ML techniques, analyze the possible problems that may arise from prediction errors; 2) rewrite ambiguous sentences to provide more clear technical details; 3) show more concrete difficulties in translating traffic engineering policy into protocols.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This work is targeting the problem of traffic engineering in computer networks. It proposes a learning method called Mistill for making packet forwarding decisions in distributed network switches using data generated by a centralized policy. The paper assumes that a forwarding policy can be obtained in a centralized fashion with global information, and the goal is to train a neural network model that can make the correct forwarding decision in each switch without global states. The network switch can communicate its local state to other switches to improve the overall performance.

main_review: This work tries to apply existing machine learning techniques to solve an important computer network problem; however, I have a number of concerns with this work:

1. The first concern is related to the contribution of the paper. This paper is mainly applying existing neural network techniques for the traffic engineering problem. There is a very limited contribution towards the machine learning techniques. On the other hand, if we focus on the application side, it seems the compute network evaluation environment is not realistic enough since only simulation results are provided (more on this point later).

2. The authors have emphasized in the paper that the network switches have limited computing power and the forwarding model needs to run in the data plane, however, there is no evaluation to show that the designed model would be able to run sufficiently fast in such an environment. In fact, (Chen et al. 2018) have shown that it is challenging to directly use neural networks in the data plane to make forwarding decisions due to latency constraints. Neural network latencies are at a millisecond level, but In today’s data center (which is the main concern for this paper), switch need to make switching decisions at a microsecond level. It is important to evaluate this in a real environment to persuade the reader that this method is actually feasible. 

3. One of the challenges when designing TE techniques is the low latency monitoring of the network state (such as in Alizadeh et al. 2018). In this work, the authors propose sharing each switch’s local state in a broadcast manner (selective nodes broadcast), is this an efficient communication pattern, would it cause high latency and overhead.

4. One of the benefits of decentralized control is the scalability of the system. However, it seems the current design requires retraining the model when adding more switches to the network.

5. What is the traffic load used in the evaluation, would the algorithms still work when different traffic load is used to evaluate the system? The policies used in the evaluation are relatively simple, more realistic workloads would be more desirable.

6. Presentation issue: some of the images’ legends are overlapping with the curve.

summary_of_the_review: This paper applies existing machine learning techniques to solve the traffic engineering problem in computer networks. The contribution toward the machine learning methodologies seems to be limited in this work, and the evaluation needs to be conducted in a more realistic environment to demonstrate the feasibility of this proposed method for the targeted application.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper studies the use of ML to design protocols for a data-center application. In particular, a single-server data center, a Fat-Tree topology, and a single policy are assumed. A neural network is training to learn forwarding policies from examples.  A modular structure is proposed to minimize complexity. The NN is trained to be adaptive, but the model itself is fixed at the end of training.
main_review: The paper studies the use of ML to design protocols for a data-center application. In particular, a single-server data center, a Fat-Tree topology, and a single policy are assumed. A neural network is training to learn forwarding policies from examples.  A modular structure is proposed to minimize complexity. The NN is trained to be adaptive, but the model itself is fixed at the end of training.

The paper is a welcome addition to the sparse set of papers on applying ML to problems in communication networks (compared with image classification, for example). A challenge, of course, is availability of data. 

Overall, the paper reads well, and it contains detailed information on the architecture and training (perhaps a bit too much for the ICLR audience).

There are several issues that need clarification:

(1) Do all switches need to have MISTILL installed? Can this work in a hybrid architecture where some switches do not have MISTILL?

(2) The work is focused on data-center application; the assumption of Clos-topologies makes it inapplicable to packet-switched networks, which are the norm today.

(3) Is the topology assumed known? Does every node know its parents and children nodes?

(4) The specific policy learned here is that of dropping packets or flows if a node is unreachable due to link or node failures; this seems to be a simple problem given the fat tree structure. Does the policy also switch to a new route if the current route is unavailable? 

(5) Typically protocols must instantiate several policies, not just a single policy. Can the framework be extended to do that?

(6) Given that the topology is assumed to be a tree, it is not clear that the approach can be extended to making use of new links, as would be the case in an ad hoc mesh network (not just links and hence paths disappearing in a tree, but also new links forming or potentially forming).

(7) A \beta term is introduced in the modified loss function in (6) to ensure that embeddings corresponding to different nodes do not end up close to each other. But this is precisely what contrastive learning is supposed to achieve. It is not clear why the authors were not able to get NCE to work.

(8) Associated with nodes and links is context information which can also be used to differentiate between embeddings. This has been widely used in the social networks literature. This paper assumes that nodes are identical and have no context information. 

 (9) The paper makes a criticism of CONTRA that the latter relies on the use of a high-level policy language.  What is wrong with that approach? If a ML model is to learn a policy, it should be able to recognize and resolve conflicts in the examples provided (the conflict may be due to lack of full context being provided or used by the learner). A purely data-driven approach may not be able to offer insights on what has been learned, vs. a neuro-symbolic approach where rules might be learned, and possibly interpretable by a human.

(10) Figures 5 and 6 provide some hints. But it is not clear what the model has learned. 

(11) There is some discussion of overhead (top of page 5), but this is insufficient. Would like to get some more details. 

(12) Shortest path routes are mentioned several times; but there is only one cycle-free path on a tree. 

(13) The 2009 Al-Fares paper indicates that a k-fat tree has (k/2)^2 k-core switches and can support k^3/4 hosts. For k=8 this leads to 16 switches and 128 hosts, but the paper says 80 switches and 128 hosts. Please clarify. Also are there k=8 servers at the data center? 


Typos and such
- Please define acronyms before, not after, first use; e.g. HNSA in Sec 3


summary_of_the_review: The paper studies the use of ML to design protocols for a data-center application; it adds to the sparse set of papers on the application of  ML to problems in communication networks, and is a welcome change from papers dealing yet again with MNIST. 

My main concern relates to generalizability: Can the framework be extended to non-tree topologies (in particular, where links may appear as well as disappear, and where multiple routes exist), to protocols that must support multiple policies, and to incorporation of contextual information associated with nodes (perhaps also links). It is not clear what the model has learned, and some additional details about the test setup are needed.
review_title: null

Meta-review: Most of the reviewers thought this paper has issues where it could be improved.  There was a range of concerns. Most importantly, several reviewers felt the novelty in the paper was unclear as well as the requirement for more details in the experimental evaluations.
Decision: Reject