Mistill: Distilling Distributed Network Protocols from Examples
Authors: Patrick Krämer
Johannes Zerwas
Andreas Blenk
Venue: ICLR 2022 Submitted (2022)
Keywords: communication networks
distributed protocols

Abstract: New applications and use-cases in data center networks require the design of Traffic Engineering (TE) algorithms that account for application-specific traffic patterns. TE makes forwarding decisions from the global state of the network. Thus, new TE algorithms require the design and implementation of effective information exchange and efficient algorithms to compute forwarding decisions. This is a challenging and labor and time-intensive process. To automate and simplify this process, we propose MISTILL. MISTILL distills the forwarding behavior of TE policies from exemplary forwarding decisions into a Neural Network. MISTILL learns which network devices must exchange state with each other, how to process local state to send it over the network, and how to map the exchanged state into forwarding decisions. We show the ability of MISTILL to learn distributed protocols with three examples and verify their performance in simulations. We show that the learned protocols closely implement the desired policies.

Under review as a conference paper at ICLR 2022
MISTILL: DISTILLING DISTRIBUTED NETWORK PROTOCOLS FROM EXAMPLES
Anonymous authors Paper under double-blind review
ABSTRACT
New applications and use-cases in data center networks require the design of Traf- fie Engineering (TE) algorithms that account for application-specifi traffic pat- terns. TE makes forwarding decisions from the global state of the network. Thus, new TE algorithms require the design and implementation of effective informa- tion exchange and efficient algorithms to compute forwarding decisions. This is a challenging and labor and inte-intensive process. To automate and simplify this process, we propose Mi stil... Mi Mistill distills the forwarding behavior of TE policies from exemplary forwarding decisions into a Neural Network. Mi iill learns how to process local state to send it over the network, which network de- vices must exchange state with each other, and how to map the exchanged state into forwarding decisions. We show the abilities of Mistill by learning three exemplary policies and verify their performance in simulations on synthetic and real world traffic patterns. We show that the learned protocols closely implement the desired policies, and generalize to unseen traffic patterns.
INTRODUCTION
Data center workloads are diverse and range from client-server applications. machine learning, and web-traffic to high-pertie perrforrefome computing applications (Benson et al. (2010); Roy et al. (2015);
Zhang et al.] (2017)) Each workload has different requirements towards Traffic Engineering (TE). For example, optimizing the completion time of individual flows is desirable for client-server appli- cations (Dukkipati & McKeown (2006), but leads to decreased performance in machine learning workloads (Chowdhury et al. (2014). To achieve the best performance, each workload needs a TE scheme that is tailored towards its specific requirements. Indeed, new applications regularly trigger the development of new TE schemes in the networking community (Katta et al. (2016); Chowdhury ct al. (2014); Chen et al. 2018):Bensoo et al. (201T)).
Obtaining deployable distributed protocol from initial TE design is challenging. A design includes the specification of update messages, the processing of update messages, and the algorithm compute the forwarding decisions that result in the desired policy (Katta al. (016)). The design of each component is challenging since the final protocol must meet line-rate, exchange information millsseelso granularity, and cope with limited computational requirements. All steps must be efficient, and the design holistic, requiring the exploitation of patterns in the network and the traffic. For instance, CONGA integrates a substantial part of the calculation for the forwarding decision into the exchange of update messages (Hsu et al. (2020)). The complexity makes customizing TE schemes hardly feasible for smaller enterprises, potentially resulting in competitive disadvantages.
To automate and simplify the translation of TE policy to a distributed protocol, we propose Mistill. Mi Mistill distills the forwarding behavior of a TE policy from exemplary forwarding decisions into Neural Network (NN). Mi Mistill 11 learns the encoding of the switch local state in up- date messages, the exchange of update messages, and the computation update messages with Machine Learning (ML). Mistill removes the need manually design update messages, information exchange, and the calculation of forwarding decisions. ML allows 11111 to automatically detect helpful patterns in traffic and network and exploit the patterns to leam a protocol. Further, our NN design makes the learned patterns accessible to humans and can thus aid the manual design process.
forwarding decisions from
Mistill

---

Under review as a conference paper at ICLR 2022
genoratnn
Inpul for
producess
Centralized System
Fonwarding decisions
Machine Leariing
Trained Model
deplay network
Update
Port
Trained Modal
forward 124
Local
Updates
Packet
State
update
Figure 1: Mistill trains a NN with forwarding decisions from an initial centralized TE system. Mistill deploys the trained NN the switches, which query the NN for forwarding decisions.
Our main contributions in this work are: 1) The design of an ML model that can learn forwarding behavior, makes learned patterns accessible humans, and has the potential to be deployed on real hardware. 2) We show in simulations that Mistill can learn the forwarding behavior of three TE policies and generalizes to previously unseen traffic patterns. 3) We analyze the learned messages and their exchange. We find that Mi st.ll can learn distributed protocols that closely implement the desired policy and are robust towards distributional shift the input. We analyze the learned messages' content, and show that the NN learns information exchange resembling an edge-cover, which can be used to optimize the information exchange of other TE schemes.
The document is organized follows. Sec. describes the application scenarios and the require- ments towards ML models. Sec. 3 describes the neural architecture and justifies design choices. Sec. 4ec.4dest describes experiments and presents results and visualizations of the learned protocols, Sec. discusses related work, and Sec. concluds the paper.
2 APPLICATION SCENARIO
Mistill learns distributed protocol from examples. Fig. illustrates the process and expected outcome. The first component is a centralized implementation of a TE policy. For example, a packet or flow level simulation using an Integer Linear Program or heuristic algorithm compute forwarding decisions based on the entire network state. The forwarding decisions specify how a switch forwards traffic its neighbors, given the current state of the network. Usually, new TE schemes are evaluated in such setting first to establish their performance and correctness.
Here, Mistill hooks in and uses the forwarding decisions to train an ML model, i.c., a NN. Mistill trains the model offline, i.c., not on the network devices. The NN learns three aspects: 1) how to encode local state update messages, how exchange the update messages, 3) how to map the exchanged state into forwarding decisions.
After training, Mistill deploys the NN switches, e.g., with P4 (Siracusano Bifulco 2018 All switches in the network have the same NN and use it in inference mode, i.e., do forward passes only. The NN distinguishes switches through an identifier. During training, the learns how to condition the forwarding decisions on the identifiers, thus learning the correct behavior for each switch. To compute forwarding decisions, switches do not need to know the network topology.
This work does not try outperform existing TE strategies in their specific application scenarios. Instead, this work aims to facilitate the automatic generation of novel TE policies, taking direct ad- vantage of switch architectures like Taurus (Swamy et al. (2020)) without asking network operators to take the time or acquire the expertise to design highly optimized protocols and implement them, e.g., in P4 (Bosshart al. (2044).
2

---

Under review as a conference paper at ICLR 2022
Outputs
BRRBA
SI
Port
LinkStace
Module
Network State Module
All Nodes (0, Y6124
Forwarding
Location (0, xja0
Localization Module
fustination 1/24
Figure 2: The NN architecture consists of four modules: The LocMOD, the HNSAMOD, the NSMOD and the FWD MOD.
2.1 DATA CENTER NETWORK REQUIREMENTS
The goal of Mist st i 11 is to distill network protocols for data-center networks from exemplary data. We focus Clos topologies due to their popularity and simple structure (Zhang et al. (2019) See Appendix [ for details. TE protocols in data-centers exchange update messages at (sub-)milll muubb)mmllisconn scale (Katta et al. (2016). Thus, the updates must be small to keep the resulting overhead low. To effectively utilize the frequent updates, switches must make forwarding decisions frequently, c.g., for each new flow. Thus, the decision logic should reside in the data-plane (Alizadeh et al.) (2014)) This limits the computational demand the decision logic can express towards the switch. The learned decision logic must thus be computationally efficient meet the limited computational capacities.
2.2 WHY NEURAL NETWORKS?
Mistill uses a NN to lear a distributed protocol, which is good choice for three reasons.
1) NNs are general function approximators (Glorot et al. (201I). This important because for- warding decisions can depend on the network state in non-linear way. For example, minimizing the maximum link utilization results in forwarding decisions that not depend linearly on the net- work state. A small change in the utilization can lead to sudden change in the forwarding decision.
2) NNs can be executed with the limited computational resources the data-plane (Sanvito et al. 2018;; Swamy et al. (2020)). Here, the choice of activation function is important. The ReLU activation requires only check for negativity (Glorot et al. (201T)). A NN with ReLU activations uses addition, multiplication and check for negative values. Further, NNs be quantized to integer or binary computations (Gholami et al. (202T).
3) The execution of NNs is easy to parallelize and accelerators for NN inference already exist for network equipment (Luinaud et al. ( Swamy et al. (2020)). In addition, research efforts show that NNs can be executed in the data- data-plane at line rate already
today's ASICs (Siraeusano & Bifulco (2018), and on end-hosts with SmartNICs (Sanvito et al. (2018)). We thus believe that switches will be increasingly capable of executing NNs at line rate.
3 NEURAL ARCHITECTURE
The NN architecture consists four modules: the Localization Module (LocMoD), the HNSA Module (HNSAMOD), the Network State Module (NSMoD), and the Forwarding Module (FwD MOD). Fig. 2 shows how the modules interact and represents inputs and outputs with hashes. See Appendix C for sketches of technical realizations of Mi i il.

---

Under review as a conference paper at ICLR 2022
Output. The NN has three outputs: the out-port, Hidden Node State Advertisements (HNSAs) i.e., update messages that inform other network devices about the local state of the sending device, and State Interest (SI), signaling which devices need to exchange information with each other.
Input. The NN has four inputs: LinkState, AllNodes, Destinat ion and Location. LinkState is the switch local state, e.g., the availability and utilization of incident links. The NN transforms the Li st into HNSAs. Locat ion is the identifier of a switch, Dest iestinaaiin i the destination of flow, and AllNodes the identifiers of all switches. The NN uses Locati inn, Destinat ion, and AllNodes to learn how to condition forwarding decisions on specific switch and destination and which switches' HNSAs are needed to make forwarding decision.
Training, The NN is trained end-to-end. Training end-to-end tailors HNSAs and SI to for- warding policy. The loss function depends on the forwarding objective. We interpret ob- jectives as multi-label classification or as learning of multinomial distribution. We use: Lun -i= p(xz) p(ss) loss function for the multinomial distribution, and Lii -Ein (p(z;) logp(y:) + (1- ((;)) log(1 - p(y;)) as loss function for the multi-label classifi- cation. N corresponds to the number of outputs, p p(z;) is the ground truth and p(y:) is the prediction of the NN. The loss Cmn can leam distributional targets to split traffic over adjacent nodes. Cml can learn policies that have changes in the forwarding policy require a re-training of the model. For example, new networking hardware, changes to the physical network topology. As we will show, re-training is short com- pared to the deployment process of new hardware, and can be integrated into the roll-out process.
or multiple equally good paths. Changes in the network that lead to
Inference. After training finishes, the NN is deployed to the switches. The switches the NN for three tasks: 1) to compute HNSAS from their local state, 2) to select the HNSAS necessary to calculate the out-port, and 3) to calculate the out-port. The switches do not change the weights of the NN. The switch also never execute the full NN. To compute HNSAs, the switch executes the HNSAMOD To compute the out-port, the switch executes the LOcMOD, NSMOD and FWD MOD, but not the HNSAMOD. The switch uses the HNSAS it received from other nodes instead.
3.1
DATA FORMAT
The inputs Location, Destination and AllNodes are binary vectors in {0. 1] and are the last three octets of IPv4 addresses following the addressing scheme for Frt-Trees (Al-Fares et al. 2008)). ) We experimented with different node representations and enclose the results in Appendix D
The LinkState E R$ is real-valued vector and represents incident edges. The first four at- tributes are a one-hot enčoding of the link availability in each direction. For example, 10 10 means that both directions are up, and 10 01 means that one direction is up and one down. The fifth and sixth attributes are the weight of the link in each direction, representing, e.g., link utilization. The link state of a switch is the concatenation of the vectors of all its links, resulting in a vector from RC-d where is the maximum degree of switches in the network. Vectors switches with a smaller degree are pero-pddded to this length.
3.2 LEARNING THE HNSA CONTENT
The HNSAMOD transforms the Li Linkstate of switch into a binary vector. The binary vector the content of HNSA The vector corresponds to a hidden layer with binary activations. Since switches send the HNSA over the network, binary activations are better than real-valued activations, since a binary representation allows the NN to have more neurons for the same number of bits. For example, a binary vector of 128 bits corresponds to four floating-point numbers. Since NNs store information in a distributed representation, having more neurons with low precision is better than having fewer neurons with high precision (Glorot et al. (20IT).
We sample the binary activations from m independent, one-hot encoded categorical distributions of arity n: HNSA ~ Cnt 01) ... || Cat(n, fm)). The NN calculates the parameters 0, from the LinkState by passing the LinkState through a feed forward NN with one hidden layer and
In analogy to Link State Advertisements (LSAs) in ltnt-state routing protocols. Hidden refers to the fact that the content of the messages is the activations of hidden layer of the NN. Further, HNSAsdiffer from LSAs in that they are not per link hut per node.
4

---

Under review as a conference paper at ICLR 2022
ReLU activation. To keep the NN differentiable. we re-parametteriizaoon trick to sample from the categorical distributions (Jang et al. (2017); Maddison et al. (2017)). During training, the NN learns how to parameterize the categorical distributions from the LinkState to encode informa- tion in the binary hidden layer. The arity n and number distributions m are hyperparameters of the NN architecture and optimized during training.
3.3 LEARNING TO COMMUNICATE
The NSMOD learns: 1) the SI output, i.e., whose HNSAS needed make a forwarding decision on a specific switch and destination, and 2) how to process HNSAS to make forwarding decisions.
To learn the SI and process HNSAS, product attention (Vaswani et al. (2017)), which can be interpreted as a learned lookup table that retrieves HNSAs based on the switch that executes the NN and given destination. The attention mechanism has three inputs: Keys K, Queries Q and Values V. K, Q and V are matrices, where K and V must have the same number of rows (Vaswani et al. (2017)) K and Q are used to compute wetention-weights A, with which convex combination of V is computed. The attention scores A are A = gumbelSoftmax W (W&R)T). where W, and W. are a trainable linear trans- formation Q and K, is the number of rows K, and gumbelSoftmax the gumbel softmax reparameterization trick (Jang et al. (2017); Maddison et al, (2017)). The output of the attention layer A (W, V), where W. a trainable linear transformation. Q corresponds the concatenation of destination and Location. K to Al 1Nodes and V to HNSAs. We use Multi-Head Attention (MHA), i.e., parallel attention mechanisms, and concatenate their output. The concatenated output passed through hidden layer with ReLU activation, and then into the FWD MOD.
attention mechanism inspired by the scaled dot-
The are the attention scores of the individual attention heads. Non-zero entries signal that the cor- responding HNSA contributes information for the forwarding decision. The scores A make learned patterns explicit and interpretable. To enforce sparse patterss, we use the gumbelSoftmax trick to calculate attention scores (Jang et al. (2017); Maddison et al. (2017)). Thus, each attention head selects one HNSA. The number of attention heads an upper bound on the exchanged messages.
3.4 LEARNING TO FORWARD
The forwarding decision of the NN is based on the network state encoded in HNSAs, i.e., the output of the NSMOD, the destination and the switch that makes the forwarding decision. The NN learns to combine this information to react to changes in the state of the network, and to learn how forwarding decisions differ for switches and destinations.
The calculation of the out-port requires the execution of the NSMOD, the LOCMOD and the FWD MOD. The LocMOD and the FWD MOD each consist of one feed forward NN with one hidden layer and ReLU activation.
4 EXPERIMENTS
Mistill learns a TE policy protecting the data-center network from traffic whose destination is reachable. If route to host exists, then traffic dropped. The challenge lies in accounting for link failures in the up-link from hosts to switches, which existing TE schemes cannot handle. Dropping traffic can important for applications that send larger amounts of data with the User Datagram Protocol (UDP). Existing TE schemes would forward this traffic to the last hop of the network and thus waste bandwidth (Hsu et al. 2000);; Dutt (017)). a route exists, we consider three policies to forward traffic: 1) WCMP: Splits traffic over neighbors in proportion to the avail- able down-stream bandwidth, 2) LCP: forwards traffic along the path with the least weight, and 3) MinMax: Forwards traffic along the path with the smallest maximum link utilization. All policies should work in the presence of arbitrary node and link failures. We compare the quality of the learned protocols to Equal Cost Multi-Pathing (ECMP) and the actual forwarding decision.
5

---

Under review as a conference paper at ICLR 2022
HNSAMOD
LocMoD FCN
NSMoD
FWD MOD
FCN
#Distr.
Arity
#Heads
Wa
Wa
W
FCN
FCNs
MinMax
108
64
90
48
17
109
104, 84
LCP
63
64
90
14
48
48
50
125
102, 105
WCMP
48
64
2
90
14
62
62
27
110
76, 71
Table 1: Hyper parameters for the best models.
KL DeeDieengance
0.015
MISTILL ECMP
MISTILL ECMP
MISTILL ECMP
Stretch
Strotch
0.4
0.010
0.1
0.2
0.005
0.2
random
combine
fb-hadoop
fb-wveb
ronder
random
combine fb fb-hadoop
fb-web render
random
combine
fb-hadoop fb-web render
(a) LCP
(b) MinMax
(c) WCMP
Figure 3: Bar plots for TE policies and traffic patterns.
4.1 DATA GENER ATION AND NN TRAINING
We generate data for k = 8 Fat Tree having 128 hosts and 80 switches (Al-Fares et al. (2008). To make the protocols robust node and link failures, remove between zero and ten edges and zero and five nodes uniformly at random from the topology. For the remaining edges, we sample edge weights from uniform distribution. We then sample source and destination pair and compute each policy's forwarding decisions on the generated graph for the sampled pair.
The training set consists of 200 000 problem instances, the validation of 50000 problem in- stances. The final results are obtained from a packet-level simulation. We use the Asynchronous Successive Halving Algorithm (ASHA) (Li et al. (2018) implemented the Ray Tune (Liaw al. (2018)) library for hyperparameter optimization and select the best configuration for the evaluation. Thi.s the hyperparameter of the best performing models. For all policies, the HNSAMOO con- sists of one fully connected layer with ReLU activation and linear transformation that generates the logits for the gumbelSoftmax function. The LOCMOD consists one fully connected layer with ReLu activation. The NSMOD consists of one MHA module and the FWD MOD of two fully connected layers. We describe the training process and the hyperparameter search space in detail in Appendix A Training single model takes in the order of 5 hours Training 100 models in parallel with ASHA on three NVIDIA Tesla V100 GPUs takes 60 hours. We implemented the NN in C and CUDA evaluate the inference time and evaluate the LCP NN, the largest network, on an Intel(R) Core(TM) 17-7700HQ CPU with 2.80 GHz, and an NVIDIA Tesla V100 GPU, Inference time on single CPU core without parallelization is 4 ms and 0.047 ms on the GPU, which suffices to meet the inter-arrival time of flows in data-centers (Benson et al. (2010)).
NVIDIA Tesla V100 GPU.
4.2 ROUTING PERFORMANCE
We evaluate five traffic patterns and perform 100 simulations with random node and edge failures for each pattern. Each simulation samples between zero and ten edge failures and between zero and five node failures. The traffic patterns are random. combine and render (Delimitrou et al. (2012), ), and Eb-hadoop and fb-web (Roy et al.) (205))). In random, the utilization of each edge is sampled from a uniform distribution. The other patterns are created from traffic matrices of data- center applications. fb-hadoop is based on Hadoop cluster in Facebook data-center, fb-web based on the production traffic of Facebook data-center (Roy et al. (2015). combine results from a web-search, and render renders the search results for users (Delimitrou et al. (2012)) Note that combine, render, fb-hadoop and fb-web result utilization patterns in the network that strongly differ from the random utilization values the NNs were trained with. In simulation, packets traverse the network from each ToR to every server. For each packet, we compare the metric of the optimal path with the metric of the path from the NN and Equal Cost Multi Pathing (ECMP).

---

Under review as a conference paper at ICLR 2022
3.5
32
1.4
Loss
3.0
16
Ratio
ECMP MISTILL
1.2
2.5
2.0
50
100
150
8
16
32
Epoch
Number of Blocks
(a) Training Progress.
(b) Performance.
Figure 4: Training loss for varying number of blocks in the GS for the calculation of HNSAS.
LCP: Fig. 3a shows bar plots for the LCP policy, and compares Mistill ECMP. The Y-Axis shows the average stretch of the objective, i.e., the ratio that Mistill ill and BCMP are worse than the optimal solution. Error bars correspond to the 99 % confidence interval of the mean. Fig. Ba shows that Mistill is better for all traffic patterns, with paths whose weight close to the optimal weight. Further, Fig. 3a shows that the error of Mi Mistinn stlllll is similar across all traffic patterns. The NN thus generalizes to distributions of the inputs that it has never seen during training.
MinMax: Fig. 3b shows similar plot in Fig. 3a for the MinMax policy. Fig. 3b shows that
Mistill is always better than ECMP, except for fb-hadoop. For fb-hadoop, Mist and
ECMP are both almost zero. This is due the specific pattern fb-hadoop, which the up-links of hosts are the most loaded links, and the TE policy has thus almost no impact on the objective. As for LCP, the NN generalizes to unseen input distributions.
We use the Kullback-Leible Divergence (KLD) to evaluate WCMP. Fig, 3c shows the average KLD for still and ECMP for the traffic patterns. Error bars correspond the % confidence interval of the mean. Fig. 3c that Mistill illl is better than ECMP for the random traffic pattern. ECMP has slightly smaller average KLDs for the other patterns. This is because available bandwidth is almost equally distributed over paths. WCMP essentially becomes ECMP up to a small error. Note how the average KLDs of Mi Mistill decreases for the other traffic patterns. This indicates that the NN also generalizes in this case unseen input distributions.
WCMP:
In summary, our experiments show that Mistill can learn the forwarding behavior of three pop- ular TE policies from exemplary samples. Further, the results show that the NN generalizes to previously unseen input distributions.
4.3 HNSA SIZE
We investigate the impact of the number of categorical distributions the Mi MinMax policy and vary them in {8, 16, 32, 64]. Fig. 4a shows that more distributions results in smaller loss, and that the decrease loss strong during the initial stages of training and converges towards the end.
Fig. 4b compares the ratio between ECMP and the true solution to the ratio of Mistil 11. The solid line corresponds the mean, the shaded area the mean's 99 % confidence interval. Fig. 4b shows that 8 distributions result in a protocol that performs better than ECMP. The protocol with 8 distri- butions, i.e., 8 bits, mostly accounts for link failures and little for edge weights. The performance of Mistill istill improves with more distributions and approaches the ground truth and the confidence interval shrinks. The improvement between 8 and distributions is larger than between 16 and 64. With 64 distributions, Mistill selects paths with similar metrics to the ground truth. With 64 distributions, the probing overhead in a network supporting up to 022 hosts with no over sub- scription and millisccond updates is 1.22 % for 10 Gbits links, i.e., in a similar order existing TE schemes Katta et al. (2016)). See Appendix C for details.
4.4 VISUALIZATIONS
This section visualizes the learned HNSA messages and the attention weights. We compare the attention weights of the gumbelSoftmax activation with the weights a model trained with the sparsemax activation function (Martins & Astudillo (016)) for the Mi nMax policy.

---

Under review as a conference paper at ICLR 2022
pilo
Bi663
Edge0 Up Edge8
(a) Link failures.
Bito
Bit63 1/0 Edge0 Edge8 1/O
(b) Edge Weights.
Figure 5: Bit pattern in HNSA messages for edge availability and edge weights.
TORO
TORa 4000
A0031 go
ca
JoRo
TO8a Ag0e
ADO31 co
ca
10:00
Aocoy co cap
BMOL
DO0L
1880 doow
TOR
pod dest nution
TEDOY 00
Scutsss actsation
Co Cui
(a) gunbelSoftmax
(b) sparsemax
(c) Edge Cover
Figure 6: Attention weights. Each row contains the weights for one switch over all other switches.
HNSA content 3 shows HNSAs Mistill for the MinMax policy. Fig. Sa illustrates how link failures reflect in the HNSAs. Fig. 5b iilluutrtates how edge weights reflect in the HNSAs. The left part of Fig, (5aaand Fig. [b show the HNSAs. The right part shows the availability of edges in Fig. [aa and the edge weights in Fig. 5b White corresponds to value of zero, green to value of one.
Fig. 5a shows that a large portion of the bits encode the link availability. In fact, 56 out 64 bits can be associated with link availability. Some bits even directly correspond to the availability of edges. Bit59 and Bi:26 corresponds to Edge8, and Bit4 to Edge7.
Consequently, Fig. 5b shows that most bits are constant for different edge weights. The NN uses eight out of the bits to encode edge weights. The relation between bits and edge weights is more complex than between link availability and bits, and no clear correspondence is observable.
Attention Scores Fig. 6 Billustrates the attention scores of all switches in the network for a specific destination node. To evaluate the effect of the gumbelSoftmax activation, we additionally trained NN with the sparsemax activation function (Martins & Astudillo (2016). Further, handcrafted attention scores corresponding to an edge cover of all edges belonging to the shortest paths between a switch and the destination. Fig. (aa shows the attention scores of the gumbelSoftmax activation, Fig. 6b of the sparsemax activation. and Fig. 66c the edge cover. The attention scores of individual attention heads are summed up and clipped to a maximum value of one.
Fig. 6a shows that the attention scores of the gumbelSoftmax resemble the edge cover in Fig. 6c In contrast, the attention scores of the sparsemax cover all nodes. Fig. (a show that the NN relies on the state of the ToR switch- and the aggregation switches in the pod of the destination. Further, all switches rely on their state. Some activations in Fig. 6a are not intuitive. For example, it is unclear why core switch C10 relies the state of ToR switch in different pod than the destination.
8

---

Under review as a conference paper at ICLR 2022
Fig. ba and 6b show that the sparsemax activation results in dense pattern that results in large signaling overhead of the learned protocol. The resulting protocol would exchange one message for each non-zero entry. The gumbelSoftmax activation has clear advantage and requires exchange of at most 10 messages, which is in the order of the edge cover, which requires at most 9 messages.
In summary, the majority of the bits in the HNSAs encode the edge availability. The attention scores resemble an edge cover of edges that belong to the shortest paths to a destination. The detected pattern sparse and can be used optimize the sending of update messages. This behavior does also hold for the WCMP and LCP policy.
5 RELATED WORK
Routing and Traffic Engineering. TE solution for DC networks addressing different goals and challenges exist (Noormohammadoou & Raghavendral (2017)). Solutions range from using well
established protocols such as OSPF Chiesa et al. (2016); Michael & Tang (2014) over new cen-
tralized designs (AI-Fares et al. (2010); [ensson et al. (201 Zhang et al. (2016) to distributed protocols that operate directly the data plane (Hsu et al. (2220; Kandula et al. (2007 Alizadeh
et al. (2014); Katta et al. (2016). Closest to Mi st ill is CONTRA (Hsu et al. (2020)) which proposes a high-level policy language expressing TE goals, and uses a path-vector based protocol for network state exchange. In contrast previous work, Mistill is not yet another TE scheme and does not aim to outperform existing TE schemes in scenarios they are optimized for. Instead, Mistil 11 is method to automate the design of distributed protocols by distilling protocols from data. In contrast to CONTRA, Mi still does not require a policy language. Further, Mist i 11 is not limited by the capabilities of path vector protocol. Instead, Mi st 11 learns the content, the exchange and the processing of information from data, and automatically detects useful patterns.
ML and Networking. Combinations of ML and networking have been explored from different angles in the past (Boutaba et al.] (2018)). Closest to Mistill are work from (Geyer & Carle (2018) and (Xiao et al. (2020)) which rely on Recurrent Neural Networks to learn forwarding behavior. In contrast to previous work, Mistin iillll111 focuses on data-center applications. Proposals in (Geyer& Carle (2018); Xiao et al.j (2020)) focus on wide area networks and do not apply to data centers. Both proposals require the exchange of several messages between neighboring nodes, similar to a distance-vector algorithm, to accommodate a change in the network. This procedure is too slow for the fast-changing traffic conditions in data-center networks. Mistill does not suffer from this problem and maps state updates directly into forwarding decisions. Further, we are the first to design such a system with practical deployment in mind. Key parts of the design of Mistill can, in principle, be deployed in the network, e.g. using Taurus (Swamy et al. (2020))
6 CONCLUSION
This paper investigates if it is possible to learn distributed protocol in data-driven manner that implements a custom TE policy. Accordingly, this paper proposes Mistill an ML--based ap- proach that learns distributed protocols from exemplary forwarding decisions. Mistill learns the processing and exchange of local information with other network elements and the computation of forwarding decisions from the exchanged information. Further, design Mi sistt. illl with deploy- ment on hardware in mind. Mistill uses only calculations that are available on network devices.
To show the applicability of Mistill, we learn the forwarding behavior of three TE policies and evaluate the learned protocols in simulations on traffic patterns from four realistic data-center appli- cations. Our results show that it is possible learn distributed protocols from data that closely match the original forwarding decisions and generalize to previously unseen traffic patterns. Further, we analyze the learned representations and show that they are reasonable. In the appendices, we provide detailed information on how Mi Mistill can be implemented in practice. We believe that our work opens many research opportunities for the future: designing switch architectures that implement learned routing protocols, using Mist ill as the basis for Reinforcement Learning based protocol design, and using learned patterns to optimize existing TE policies, e.g., to reduce the number of exchanged update messages.

---

Under review as a conference paper at ICLR 2022
REPRODUCIBILITY STATEMENT
We make all source code as well as the trained models available for the reviewers. Upon acceptance, will make the code and the models publicly available. To improve reproducibility, we include scripts and manuals on how to recreate figures from the paper.
We will post the link the anonymous git repository using the discussion forum. Once the discus- sion forum opens, we will post a link to the repository there with restricted visibility to the reviewers and area chairs.
REFERENCES
Mohammad Al-Fares, Alexander Loukissas, and Amin Vahdat. A Scalable, Commodity Data Center Network Architecture. SIGCOMM Comput. Commun. Rev, 38(4):63-74, August 2008. ISSN 0146-4833, doi: 10.1145/1402246..44000 14C2946 1402967
acm. org/ 1145/
Mohammad Al-Fares, Sivasankar Radhakrishnan, Barath Raghavan, Nelson Huang, Amin Vahdat, et al. Hedera: dynamic flow scheduling for data center networks. In Nsdi, pp. 89-92, 2010.
Mohammad Alizadeh, Tom Edsall, Sarang Dharmapurikar, Ramanan Vaidyanathan, Kevin Chu, Andy Fingerhut, Vinh The Lam, Francis Matus, Rong Pan, Navindra Yadav, and George Vargh- ese. Conga: Distributed congestion-aware load balancing for datacenters. SIGCOMM Comput. Commun. Rev., 44())4003333)) 514, August 2014. ISSN 0146-4833. doi: 00.1145/2440770.266633
URL httptt:ss//.i..... ottp:///////oi100//////00//// 2141/27400777711 2626316
Theophilus Benson, Aditya Akella, and David A. Maltz. Network Traffic Characteristics of Data Centers in the Wild. In Proceedings of the 10th ACM SIGCOMM Conference Internet Mea- surement, IMC '10, 267-280, New York, NY, USA, 2010. ACM. ISBN 978-1-4503-0483- URL http: htt///doi... acm. org 11110111111111191110
2. doi: 1111441/1994118999999
1879175
Theophilus Benson, Ashok Anand, Aditya Akella, and Ming Zhang. MicroTE:: Fine Grained Traffic Engineering for Data Centers. In Proceedings of the Seventh COnference on Emerg- ing Networking EXperiments and Technologies, CoNEXT '11, 8: 1:18:1 New York, NY, USA, 2011. ACM. ISBN 1-7-1--4500-1041--3 doi: 10.1145/207929662773333 2079334. URL http: 7/doi acm. 1111:00911115/29995666. 2079304
Pat Bosshart, Dan Daly, Glen Gibb, Martin Izzard, Nick McKeown, Jennifer Rexford, Cole Schlesinger, Dan Talayco, Amin Vahdat, George Varghese, and David Walker. P4: Program- ming Protecol-Indeepeddett Packet Processors. 44(3):87-95, July 2014. ISSN 0146-4833. doi: 00.1145/2656877.266668)
Raouf Boutaba, Mohammad A Salahuddin, Noura Limam, Sara Ayoubi, Nashid Shahriar, Felipe Estrada-Solano, and Oscar M Caicedo. A comprehensive survey on machine learning for net- working: evolution, applications and research opportunities. Journal of Internet Services and Applications, 91):1199, 2018.
David Calavera and Lorenzo Fontana. O'Reilly Media, Inc., 2019. ISBN 9781492050227
Li Chen, Justinas Lingys, Kai Chen, and Feng Liu. AuTO: Scaling Deep Reinforcement Learning for Datacenter-scale Automatic Traffic Optimization. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication, SIGCOMM '1, pp. 191-205, New York, NY, USA, 2018. ACM. ISBN 978- 1778-1455 4503-5566744 doi: 10.1 10.1145732305443323003 3230551
URL
http:://di.. acm. wwtpwo..o.oo.o.o..oo.o. 1141/ /23044//333333333 3235533333333333355111
Marco Chiesa, Gábor Rétvári, and Michael Schapira. Lying your way better traffic engineering. In Proceedings of the 12th International on Conference on emerging Networking EXperiments and Technologies, pp. 391-398, 2016.
Mosharaf Chowdhury, Yuan Zhong, and Ion Stoica. Efficient Coflow Scheduling with Varys. In SIGCOMM '14, pp. 443 454, Chicago, IL, USA, August 2014, ACM. ISBN 978-1-4503-2836-4 doi: 10.1145/2619239 2626315.
10

---

Under review as a conference paper at ICLR 2022
Christina Delimitrou, Sriram Sankar, Aman Kansal, and Christos Kozyrakis. ECHO: Recreating network traffic maps for datacenters with tens thousands of servers. In IISWC, pp. 14-24, La Jolla, CA, USA, 2012. IEEE. doi: 10.1 111/111C..22222.600299999999
Nandita Dukkipati and Nick McKeown. Why Flow-completion Time is the Right Metric for Conges- tion Control. SIGCOMM Comput. Commun. Rev., 36(1):59 62, January 2006. ISSN 0146-4833. doi: 10.1145/1111322.111
Dinesh G. Dutt. BGP in the Data Center. O'Reilly Media, Inc., 2017. ISBN 9781491983409
Fabien Geyer and Georg Carle. Learning and Generating Distributed Routing Protocols Using Graph-Based Deep Learning. In Proceedings of the 2018 Workshop Big Data Analyties and Machine Learning for Data Communication Networks, Big-DAMA '18, pp. 40-45, New York, NY, USA, 2018. ACM. ISBN 978-1-4503-59904-, doi: 10. 00.114453229600.3229910 URL
http://doi. acm. 1ttg:0.111///22220333222222
Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W. Mahoney, and Kurt Keutzer. A survey of quantization methods for efficient neural network inference. CoRR, abs/2103.13630, 2021.
wwt.org//aaa.... abs
Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep sparse rectifier neural networks. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, volume 15 of Proceedings of Machine Learning Research, pp. 315-323, Fort Lauderdale, FL., USA, 11-13 Apr 2011. JMLR Workshop and Conference Proceedings.
Aditya Grover and Jure Leskovec. Node2vec: Scalable feature learning for networks. In KDD '16. pp. 855-864, San Francisco, CA, USA, 2016. ACM. ISBN 9781450342322. doi: 10.1145/ 2939672.2939754.
William L. Hamilton, Rex Ying, and Jure Leskovec. Representation learning graphs: Methods and applications. CoRR, Abs/1709.05584, 2017. URL 05584
Kuo-Feng Hsu, Ryan Beckett, Ang Chen, Jennifer Rexford, and David Walker. Contra: A pro- grammable system for performance-aware routing. In 17th {USENIX) Symposium on Networked Systems Design and Implementation ((NSDI) 20), pp. 701-721, 2020.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical Reparameterization with Gumbel-Softmax. In 5th International Conference on Learning Representations, ICIR 2017, Toulon, France,
April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL https:
//peezeview www.://euum/... http:/oorummd--kkEyyeeeee rkE3y85ee
Srikanth Kandula, Dina Katabi, Shantanu Sinha, and Arthur Berger. Dynamic load balancing with- out packet reordering. ACM SIGCOMM Computer Communication Review, 37(2):51-62, 2007.
Naga Katta, Mukesh Hira, Changhoon Kim, Anirudh Sivaraman, and Jennifer Rexford. HULA: Scalable Load Balancing Using Programmable Data Planes. In Proceedings of the Symposium on SDN Research, SOSR '16. pp. 10:1-10:12, New York, NY, USA, 2016. ACM. ISBN 978- 1-4503-421177. doi: 10. 0001145/28905552889000 URL nnttp:///ot acm. oro//1..11..555
2890955. 2890968
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URLhttp:: http:
7/ //axiiv... ow..org//// abs 10abs/144 111111111110 6980
Liam Li, Kevin G. Jamieson, Afshin Rostamizadeh, Ekaterina Gonina, Moritz Hardt. Benjamin Recht, and Ameet Talwalkar. Massively parallel hyperparameter tuning. CoRR, bbbs//80.00594, 2018. UR..tttp:/ :/a //w.x/va.vvvvv///v rxiv www.wwwww......a..e... abs 181 01110.000 05934
Richard Liaw, Eric Liang, Robert Nishihara, Philipp Moritz, Joseph E Gonzalez, and Ion Sto- ica. Tune: A research platform for distributed model selection and training. arXiv preprint arXivv1807.000 18, 2018.
11

---

Under review as a conference paper at ICLR 2022
Thomas Luinaud, Thibaut Stimpfling, Jeferson Santiago da Silva, Yvon Savaria, and J. M. Pierre Langlois. Bridging the Gap: FPGAs as Programmable Switches. In HPSR 2020, pp. 1-7, Newark, NJ, USA, May 2020. IEEE. doi: 10. 10.1109/HPS448589.000 0tt.//009..220.9999999
Chris J. Maddison, Andriy Mnih, and Yee Whye Teh. The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables. In 5th International Conference on Learning Repre- sentations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. Open- Review.net, 2017.
Andre Martins and Ramon Astudillo. From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Labe Classification. Maria Florina Balcan and Kilian Q. Weinberger (eds.), Pro- ceedings of The 33rd International Conference Machine Learning, volume 48 of Proceedings of Machine Learning Research, pp. 1614-1623, New York, New York, USA, June 2016. PMLR. html
www.mlrr...mmm wttp://pppppeeee//eeeeeeeeee URL httt:::/ppooceeeeeeeetee http:///..pooeeeeoooooo...eeeees
Nithin Michael and Ao Tang. Halo: Hop-by-hop adaptive link-state optimal routing. JEEE/ACM
Transactions on Networking, 23(6):1862-187) 2014.
Vinith Misra and Sumit Bhatia. Bernoulli Embeddings for Graphs. CoRR, abs/1 180309911, 2018.
Mohammad Noormohammadpour and Cauligi S Raghavendra. Datacenter traffic control: Under- standing techniques and tradeoffs. IEEE Communicattonn Surveys & Tutorials, 20(2): 2017.
Arjun Roy, Hongyi Zeng, Jasmeet Bagga, George Porter, and Alex C. Snoeren. Inside the Social Network's (Datacenter) Network. SIGCOMM Comput, Commun. Rev, 4534)1123-137, August 2015. ISSN 0133- doi: 0111145/2829988.2777777 URL 02188.78777 2787472
Davide Sanvito, Giuseppe Siracusano, and Roberto Bifulco. Can the Network Be the Accelera- tor?, 2018. URL
Giuseppe Siracusano and Roberto Bifulco. In-newwork Neural Networks. CoRR, abs/18011 180100531,, 2018.
Tushar Swamy, Alexander Rucker, Muhammad Shahbaz, and Kunle Olukotun. Taurus: An Intel- ligent Data Plane. CoRR, abs/2002.08987,, 2020. URL 08987 arXiv: 2002.08987.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is All you Need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neu- Curran Associates, Inc., 2017. URL http:://appe perapers.nips. hwtpc//apppr71111 http://pppprr..ippsssccppppp/spp http:///iii-aattttoooo-sssse -ou-alll-youeeeeeee pdd
ral Information Processing Systems 30, pp.
Marcos A. M. Vieira, Matheus Castanho, Racyus D. G. Pacífico, Elerson R. S. Santos, Eduardo P. M. Câmara Júnior, and Luiz F. M. Vieira. Fast Packet Processing with EBPF and XDP: Concepts, Code, Challenges, and Applications. ACM Comput. Surv., 53(1), February 2020. ISSN 0360- 0300. doi: 10. 145/3371038.
Marcel Waldvogel, George Varghese, Jon Turner, and Bernhard Plattner. Scalable high speed ip routing lookups. In SIGCOMM '97, SIGCOMM '97, pp. 25-36, Cannes, France, 1997. ACM. ISBN 089791905X. doi: 10. 105.26333..
Shihan Xiao, Haiyan Mao, Bo Wu, Wenjie Liu, and Fenglin Li. Neural Packet Routing. In Pro- ceedings of the Workshop on Network Meets Al &amp; ML, NetAl '20, pp. 28-34, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 978 --540-0043--00 doi: 10.1 3405671.3 1.6677,.440813.. URL htll:ttts//cc...oo org/10. 110...115555000 111114//455555 3405811 event-place: Virtual Event, USA.
Hong Zhang, Li Chen, Bairen Yi, Kai Chen, Mosharaf Chowdhury, and Yanhui Geng. CODA: Toward Automatically Identifying and Scheduling Coflows in the Dark. In Proceedings of the 2016 ACM SIGCOMM Conference, SIGCOMM 16, pp. 160-173 New York, NY, USA, 2016. ACM. ISBN 458-1-4503-441336 TTT/10..145/293447222
doi: 10.1 10.114529948772.29448888 URL htttp://do acm.
12

---

Under review as a conference paper at ICLR 2022
Mingyang Zhang, Radhika Niranjan Mysore, Sucha Supitayapomppong, and Ramesh Govindan. Understanding Lifecycle Management Complexity Datacenter Topologies. NSDI 19, pp- 235-254, Boston, MA. USA. February 2019. USENIX Association. ISBN 978-1-93197 -49-2.
Qiao Zhang, Vincent Liu, Hongyi Zeng, and Arvind Krishnamurthy. High-resolution Measurement of Data Center Microbursts. In Proceedings of the 2017 Internet Measurement Conference, IMC '17. pp. 78 85, New York, NY, USA, 2017. ACM. ISBN 978- 1 1131365.3131377 URL http://doi acm. 2tt///0..1111/331133333333333333 place: London, United Kingdom.
17881-4500-5118-88 doi: 10.1145/
event-
13

---

Under review as a conference paper at ICLR 2022
A HYPER PARAMETER OPTIMIZATION
Finding the parameters was stage process. Step one was finding parameters that able to reproduce the shortest path without any network state info. In the second step, we introduced the network state information.
A.1 SHORTEST PATH
To learn the shortest paths, we used only the LOcMoD. We experimented with different node representations (see Appendix D). as well as different NN architectures. We experimented with multi-head attention with the IP of the destination as query and the IPs of the neighbors as keys and values. We also tried simple feed forward neural networks. It turned out, that simple feed forward neural network with one hidden layer and the concatenation of destination the current location is enough to reliably learn the forwarding behavior in Fat- Tree topologies. We performed a parameter analysis of the hyper parameters with ASHA (Li et al. (018)) implemented in Ray Tune Liaw al. 2018). Hyper parameters were:
and the of
batch size: {64, ................................ }
hidden layer sizes: {8...., 100)3, i.e., up to three layers.
learning rate: [0.001, 0.0001].
samples for ASHA: 100.
We used the ADAM (Kingma & Ba 2015)) optimizer and ReLU activation. The best performing model has one hidden layer with 90 neurons. We thus used one hidden layer with 90 neurons for the LocMoD and kept them fixed when experimenting with network state.
A.2 NETWORK STATE
To learn forwarding behavior with network state, we performed extensive parameter sweeps with ASHA. We explored the following parameter space:
batch size: (64,..., 128)
final FCNs in the FWD MOD: 128]2.. two or three layers.
FCN for encoding links: {32...., 150}
Number of blocks: [1....,65)
Arity: 12.
Tempererature of gumbelSoftmax: 0.6
number of heads for MHA in NSMOD: {9...., 14)
dimension of FCN for MHA module in NSMoD: {70, 129}
hidden dimension of MHA heads: (20..... 64]
output dimension of MHA heads: {16, ..6.....44
lcarning rate: [10 4.3 10-3.5)
To obtain the final models, we fixed the arity of the categorical distributions in the HNSAMOD to 2 and the number of blocks to 64. We then trained 100 models with ASHA, resulting in the parameters in Tbl. [ To evaluate the impact of the number of blocks, we kept the resulting parameters in Tbl. fixed and varied the number of blocks. One area of future work can be a more thorough search of the parameter space to reduce the model size. We did not try reduce the number of parameters as much as possible. For example, we believe that multiple attention heads can be pruned, as well as the number of blocks be reduced. Those elements of the network showed large degree of redundancy.
14

---

Under review as a conference paper at ICLR 2022
TWO
T01
T10
T11
T20
T21
T30
T3
Figure 7: Fat Tree topology - scalable three-tier Clos network. The topology is a multi-rooted tree with high path redundancy between any pair of hosts.
B CLOS-IOPOLOGIES
Data center networks are a good to start with learning network protocols. They have regular and repetitive structures and their addressing schemes encode the location of network nodes. Clos networks are one such common data center topology. One special instance of Clos networks is the (Fat Tree Al-Fares al. (2008);; well-known example that employs a three-tier structure and can interconnect are large number of servers using simple commodity hardware. Fig. shows a k Fat Tree topology. Fat Tree topology has three layers of switches: Sop-of-the-Race aggregation and core switches. Hosts are connected to Top-of-Rack (ToR) switches, which in turn are connected to aggregation switches. ToR and aggregation switches are organized in pods. The Fat Tree in k² Fig. has four pods which are interconnected by four core switches. A Fat Tree can support I
server nodes with 5&? 4 k-port switches. For instance, a Fat Tree constructed from 8-port switches support 128 hosts with 80 switches. Clos networks provide rich, high-capacity connectivity matrix, i.e., 4 paths exist between any pair of servers in different pods, and paths between servers in the same pod that are connected to different ToRs. The Fat Tree in Fig. already provides 4 paths between servers different pods, and 2 paths between servers the same pod connected different ToRs.
Clos networks have two advantages: Simplified inventory Management and resilience to fail- ures (Al-Fares et al. (2008); Dutt (2017)). Inventory management is simplified because the net- work can build from fixed-form switches. This makes easy to stock spare devices, and replace failed ones, since each switch is the same. Resilience of the topology is increased through high path diversity. a link or device fails, enough alternative routes exist.
C DEPLOYING DISTILLED PROTOCOLS
This section outlines two potential practical implementations of Mistill Sec. C. outlines an implementation of ill on switches in the network. See C.2 Mistill in the edge of the network. Finally, Sec. and details how the resulting overhead can computed.
outlines an implementation of
Sec. C.3 describes how update messages can be send
C.I IN-NETWORK DEPLOYMENT
In this deployment, Mistill is implemented the network without touching the servers. Also, Mistill 11l does not change the data plane traffic. Thus, Mistill is transparent to the server nodes, and can be deployed incrementally in the network.
To be operable, Mi st needs routines and data structures switches that interface with the NN. Each switch has four entities: A Flow(let) Table (FT), the Network State Database (NSDB), the NN, and Receiver Database (RDB). Fig. g shows the entities and their interaction.
The NSDB stores HNSAs received from other switches in the network. The switch populates the NSDB with the content of HNSAS. Switches do only need HNSAS from parts of the network flows
15

---

Under review as a conference paper at ICLR 2022
HNSA B 3
10011010
Forwarded Packet
Switch A
Switch
Switches
NN
match
B
10010101
Switches
cache decision miss
NN
FT
1
NSDB
RDB
Destination IP
LM
Figure 8: Realization of Mist 111 on switch.
traversing the switch are traveling to. HNSAs from other parts of the network are irrelevant, Fig. illustrates.
The NN used in two functions the switch: 1) compute forwarding decisions, and 2) to compute the HNSA content. To compute forwarding decisions, the switch needs information from the packet. In our examples, this amounts the source and destination IP address. All other inputs reside in the memory of the switch. To compute the HNSA content, the switch monitors the required statistics, e.g., the link utilization or queue sizes. The switch executes the HNSAMOD with the current link state. The resulting HNSA is propagated through the network to other parts of the network as indicated by the RDB. The propagation of HNSAs is detailed in Sec. C.3
The FT caches forwarding decisions to avoid packet-reordering at the destination. We believe that is likely that the content of the NSDB updated frequently during the life-time of longer flows. As result, it is likely that packets of the same flow forwarded along different paths in the network. Thus, the switch caches the forward decision the NN produces for the first packet a flow(let) and re-uses this decision for subsequent packets. This approach additionally decreases the computational effort. Caching of forwarding decisions to avoid packet re-ordering is also performed by other in-dataplane TE solutions (Alizadeh al. (2014); Katta al. (2016); Hsu et al. (2020)).
The RDB controls to which nodes in the network HNSAS are send. The RDB is lazily populated based on the source IP addresses of packets the switch observes during operation. Sec. C.3 explains the sending of HNSAs in detail.
To guarantee delivery of packets, the FT can be used to configure static routes, e.g., implement ECMP. If there is a training artifact in the NN that leads to wrong forwarding decisions, then affected traffic can use the fallback routes once the Time to Live (TTL. reaches certain value. our experiments, we did not experience any loops, though.
In this way, Mistill can be deployed incrementally. For example, Mistill can be deployed first on the ToR and aggregation switch of individual pods. Depending on the results, the number of pods that are operated with Mist i ill can be gradually increased. This approach requires configuration of static rules to let legacy pods that do not run Mi Mistil interact with pods that run Mistill. This is necessary, since Mistill switches will not use the existing routing protocol. Thus, the legacy switches need default routes to those pods. Switches that run with Mistill do not need static routes, they can be learned during training.
C.2 EDGE-BASED BASED DEPLOYMENT
The in-network based deployment requires that switches have substantial computational require- ments. For example, switches that based FPGAs (Luinaud et al. (2020)), have hardware
16

---

Under review as a conference paper at ICLR 2022
Step 2
Step 3
Step 1
Step
Figure 9: Probing scheme for all node types.
support for executing ML models (Swamy et al. (2020). Such switches are not available in existing data-centers. Thus, Mistill stlll 11 cannot be deployed there. Instead, we suggest a hybrid implementa- tion, where only the HNSAs are calculated on the switches, and the forwarding logic is moved into the servers.
Our results show that the computation of the HNSAS does require complex NN. The forward pass of this network takes less than one millisecond on an Intel(R) Core(TM) 17-7700HQ CPU with 2.80 GHz. Thus, the NN could operate in the control plane of the switch. The switch can calculate the HNSA content there and send packets to the end-hosts to inform them about its state, Running this process the control plane of the switch might impact the frequency with which updates be send, since the communication between control- and data-plane takes time. Evaluating the potential performance one area of future work.
The computation of the forwarding decisions is moved into the end-hosts. For each new flow, the end-host then determines a path through the network. The end-host calculates the aggregation switch its ToR switch should send the packet to. Then, the host can calculate the core switch the aggregation switch should send the packet to. From core switch, only one path exists to the destination host. Forwarding of the packets can be realized, e.g., with the Multiprotocol Label Switching (MPLS) protocol. The end-hosts add MPLS labels to the packets that determine the path through the network. Switches forward the packets based on those labels. MPLS is broadly available feature in data center equipment.
The required functionality the end-hosts can be implemented with the extended Berkeley Packet Filter (eBPF) Calavera & Fontana 2019). eBPF is a standard Linux technology and also available in Windows starting with Windows 10 and Windows Server 2016. eBPF allows the execution of custom byte-code in the Linux kernel. The execution of eBPF bytecode is triggered by certain hooks in the kernel. In this way, the forwarding logic of Mi sti For packet of new flow(let), a new route is calculated and stored in the NSDB, If an entry exists, the calculated route is used instead. Similarly, HNSAs from the switches can processed in the kernel with eBPF. Further, SmartNICs that can accelerate the execution of NNs exist for end-hosts today (Sanvito et al. (2088). Thus, the calculation of the NN and the processing of HNSAS can be offloaded to the NIC (Vieira et al. (2020)).
be invoked for outgoing packets.
C.3 HNSA PROPAGATION
We design an efficient propagation mechanism based multi-casting that is inspired from the learned pattern in the and takes advantage the structure of Clos-topologies. The propagation mechanism keeps the number of HNSAs that circulate in the network in the order of O(N), where N is the number of switches in the network.
The propagation mechanism exploits two observations: 1) up-stream switches need update messages from down-stream switches to make forwarding decisions. 2) Switches in one pod need update messages from similar set of down-stream switches, since each switch is likely to see similar set of destination IPs. Thus, we propose to multi-cast a HNSA to all switches in the source pod. HNSAS have header with two fields 1) the IP of the switch from which the HNSA originates and 2) bit
17

---

Under review as a conference paper at ICLR 2022
array of length k where k is the number of pods. Each switch tracks the pods from which traffic passing them originates, i.c., the RDB. The source pod of packet can be deduced from the source IP-address (Al-Fares et al. (2008)) The switch indicates those pods in bit array stored in the HNSA header.
Fig. 9 shows exemplary HNSA propagation from ToR. First, the ToR sends its HNSAS to all neighboring aggregation switches (Step 1). The aggregation switches forward the probes to all ToR switches in the pod as well as all adjacent cores switches (Step 2). The ToR switches consume the probes and do not circulate them further. The core switches forward the HNSA to all adjacent aggregation switches in pods indicated by the bit array in the HNSA header (Step 3). The aggregation switches then forward the HNSA to only the incident ToR switches, i.e., do not send it back up into the core layer (Step 4). For this, the aggregation switches check for each HNSA that they receive whether the HNSA originated in their own pod. If it originated in a different pod, then the aggregation switch does not forward the probe to the core layer. The ToR switches do not further circulate the HNSA. The checks if HNSA originated in the same pod not can be performed by comparing the IP address of the source switch with the IP address of the switch that is currently processing the HNSA
Further, switches forward probes only, if they did not receive a probe from the switch identified by the source IP within configurable amount of time. This can be achieved at line rate (Katta et al. (2016). Thus, aggregation switches the source pod will forward only one probe to their connected ToR switches.
The overhead of this probing scheme is reasonable. the worst case, all pods communicate with each other. at most 4 messages will traverse one link. One message for every switch in the network. Assuming a probing frequency of I ms and a link bandwidth of 10 Gbits, the probing overhead is then:
5A2
5A I
(54 Byte + k Bit)
10 Bits 0.001
We assume a size of 46 Bytes plus k Bit for the update message, consisting of 14 Bytes Ethernet header, 20 Bytes IP header and 12 Bytes plus k Bit for the probe. The 12 Bytes of the probe consist of 4 Byte for the source IP, 8 Byte for the HNSA content, and k Bit to indicate the pods the HNSA should be forwarded to.
Thus, for a k = 8 Fat-Tree, the overhead of probing one one link would be 0.3 % of the available bandwidth. For k 16 Fat-Tree supporting more than 1 100 hosts, the overhead would still be 1.22 %.
However, the evaluation of the SI in Sec. 4.4 shows that e.g., the HNSAs of core switches are not requested by any other switch. Further, if not all pods communicate with each other, as e.g., the case in the render or comb i ne pattern, then the number of HNSAS will further decrease. In practice we expect the number of update message thus to be less.
D NODE ADDRESSING SCHEMES
Addresses in communication networks should follow a pattern to, e.g., keep routing tables small (Waldvogel et al. (1997)). IP addresses did not contain a useful pattern, routing tables would have to store forwarding decisions for each and every IP address. Since the addresses are written to the header of every packet, they should also be concise to reduce overhead.
IP addresses today geared towards the available technology, e.g., towards longest prefix match- ing. It is unclear if the way nodes are addressed today is a good fit for routing protocols learned with ML.
Thus in this section, we investigate whether ML based routing protocols can be learned with IP addresses, whether latent space models, as used in natural language processing graph learning, yield an addressing scheme that results in better performance, and whether the addressing scheme plays role at all for the ability to learn routing scheme.
18

---

Under review as a conference paper at ICLR 2022
D.1 LEARNING BINARY EMBEDDINGS OF NODES IN GRAPHS
Learning useful representations for nodes in graphs is subject to ongoing research (Hamilton et al. (0177). We use Bernoulli Embeddings (Misra & Bhatia (018)) learn node addresses. Bernoulli Embeddings map nodes in graph into a binary space that encodes the neighborhood relationship. We do not consider approaches such node2vec (Grover & Leskover (2016)) that embed nodes into a continuous latent space as the size the necessary floating point numbers is prohibitive. We do also not consider methods that use additional meta-data for generating embeddings. Meta-data on the networking level, e.g., MAC addresses, hardware information, etc. is either random or not correlated with the network structure.
A binary embedding is d-dimensioonal binary vector, i.e., the embedding of node is E, E {0, 1)a. The binary vector is sampled from d independent Bernoulli distributions, parameterized by ma- trix Rlvlxd The thh element in the embedding of node i is thus sampled according to: E Bernoulli(Qr.t) The embedding can be optimized by changing the parameterization of the independent Bernoulli distributions such that an appropriate loss function minimized.
The intuition behind the method in (Misra & Bhatia (2018)) is to encode neighborhood relation into the embeddings by casting the problem as a link prediction problem: The probability of an edge eij from node to node j is proportional to the hamming distance du between the embeddings E; and Ej of those nodes:
exp (-ddg (Es, E2))
p(e,, E,, ΣxEV expvexp(( (-adH (E,, Ex))
The parameter a is a scaling factor. The hamming distance dæ between two vectors is defined as: dæ 10, {0, 13dd N; - x y) + (1 ==)²;.
The embeddings can then be trained by optimizing the log probability, i.e.:
((G, θ) = Σ E (logp(es) Eg, Ey)le
(LI))E
assuming that edges are independent of each other given the embeddings of incident nodes.
Misra et al. (Misra & Bhatia (2018) use a continuous approximation to the objective in Eq. 3 to circumvent the expectation over a discrete valued variable. In contrast, we make use of the Gumbel-Softman trick (Jang et al. (017); Maddison et al. (201)). a reparameterization trick for discrete random variables, to optimize the objective in Eq. B). addition, Misra et al. (Misra & Bhatia (2018)) use Noise Contrastive Estimation (NCE) to avoid the summation over all nodes in the graph in the denominator of Eq. (2). We do not use NCE since optimization is feasible for the size of communication networks. In addition, we experienced that NCE makes learning of unique embeddings more difficult due to aliasing effects between nodes.
Addresses in communication networks must be unique which translates to the embeddings of the nodes. The optimization objective in Eq. (3) is not designed for this constraint and might assign the same embedding different nodes. In fact, for fully connected graph single embedding for all nodes would in fact be the optimal solution. Also in other networks nodes exists that are structurally equivalent and thus get the same embedding. For example all hosts connected in a Fat-Tree topology would get the same embedding. Similarly, all core switches in a fat-tree that connect to the same aggregation switches are structurally equivalent and would get the same embedding. We thus adapt the objective in order to avoid duplicate embeddings. Specifically, we use the following energy function for the model of edge probabilities:
ToR switch
(4) where ( is a scalar parameter. The intuition behind Eq. 4 is that probability one for an edge is obtained only, if the embeddings are different. The parameter 3 controls the scaling of the difference. For example, 3 = pushes the embeddings of adjacent nodes towards having a hamming distance of one.
L 0) = -a (dz (E,, E,) 3)².
D.2 RESULTS
The suitability of addressing schemes is evaluated by learning static shortest paths. An addressing scheme is suitable if the forwarding rules for static ECMP can be learned reliably, i.e., if we obtain
19

---

Under review as a conference paper at ICLR 2022
Optimal
Random
Leamed12
Learned66
Learned20
Learne24
Composed
0.3707
0.6210
0.5397
0.4835
0.5002
0.6541
0.4185
0.3921
Table 2: Training results on the shortest path training tasks for graphs and embeddings. For the learned embeddings, different vector sizes were evaluated. Optimal is the average loss value for the ground truth.
a low loss value. Dists the results obtained for this learning task. We use addresses of up to 24 bits. IP addresses follow the scheme in (Al-Fares et al. (2008)). We remove the first octet since it is constant and thus does not contribute any information.
The random embedding results in the highest, i.e., worst, loss value. The addresses do not contain any form of pattern, after all. The learned embeddings result in lower loss values. The dimensional- ity of the embedding does not have an effect on performance. No consistent increasing decreasing trend is observable. The IP address based embedding results in the lowest loss values. Good per- formance here is explainable, since the forwarding behavior can be perfectly computed from the IP address alone in case of the Fat Tree. The composed embedding for the Fat Tree achieves a loss value of 0.4185 and is competitive with the IP-based result of 0.3921. This hints filtering similar structures in the graph and assigning embeddings to those structures in an equal manner.
In summary, IP addresses well suited to learn forwarding decisions well structured topologies such as Fat Trees. This has the advantage that learned routing protocols are backwards compat- ible, be incrementally deployed, run side-by-side with traditional routing protocols. For Misti 11, we thus use the last 24 bits of the IP addresses to represent nodes.
20


Reviews:
[{"review_id": "asa5LpcNSLd", "reviewer": "Reviewer_hRA9", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This work tries to apply existing machine learning techniques to solve an important computer network problem; however, I have a number of concerns with this work:\n\n1. The first concern is related to the contribution of the paper. This paper is mainly applying existing neural network techniques for the traffic engineering problem. There is a very limited contribution towards the machine learning techniques. On the other hand, if we focus on the application side, it seems the compute network evaluation environment is not realistic enough since only simulation results are provided (more on this point later).\n\n2. The authors have emphasized in the paper that the network switches have limited computing power and the forwarding model needs to run in the data plane, however, there is no evaluation to show that the designed model would be able to run sufficiently fast in such an environment. In fact, (Chen et al. 2018) have shown that it is challenging to directly use neural networks in the data plane to make forwarding decisions due to latency constraints. Neural network latencies are at a millisecond level, but In today’s data center (which is the main concern for this paper), switch need to make switching decisions at a microsecond level. It is important to evaluate this in a real environment to persuade the reader that this method is actually feasible. \n\n3. One of the challenges when designing TE techniques is the low latency monitoring of the network state (such as in Alizadeh et al. 2018). In this work, the authors propose sharing each switch’s local state in a broadcast manner (selective nodes broadcast), is this an efficient communication pattern, would it cause high latency and overhead.\n\n4. One of the benefits of decentralized control is the scalability of the system. However, it seems the current design requires retraining the model when adding more switches to the network.\n\n5. What is the traffic load used in the evaluation, would the algorithms still work when different traffic load is used to evaluate the system? The policies used in the evaluation are relatively simple, more realistic workloads would be more desirable.\n\n6. Presentation issue: some of the images’ legends are overlapping with the curve.\n", "summary_of_the_paper": "This work is targeting the problem of traffic engineering in computer networks. It proposes a learning method called Mistill for making packet forwarding decisions in distributed network switches using data generated by a centralized policy. The paper assumes that a forwarding policy can be obtained in a centralized fashion with global information, and the goal is to train a neural network model that can make the correct forwarding decision in each switch without global states. The network switch can communicate its local state to other switches to improve the overall performance.\n", "summary_of_the_review": "This paper applies existing machine learning techniques to solve the traffic engineering problem in computer networks. The contribution toward the machine learning methodologies seems to be limited in this work, and the evaluation needs to be conducted in a more realistic environment to demonstrate the feasibility of this proposed method for the targeted application.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Ah7jDGPxlmT", "reviewer": "Reviewer_noo6", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths.\n1. The paper gives some insights in incorporating machine learning techniques into the Traffic Engineering area, including how to encode the link-state using reparameterization and how to communicate using attention techniques.\n\n2. The paper gives a general analysis of the pros and cons of ML approaches. It is informative when evaluating the feasibility of applying ML on real switches.\n\nWeaknesses.\n1. The paper represents the challenges in a vague way. In Section 1 Introduction, the paper illustrates the necessity of customizing Traffic Engineering schemes for various applications. However, it lacks concrete analysis of challenges that lie in translating a TE policy into a distributed protocol. Only a general description in Paragraph 2, such as \"it requires the specification of exchange data, the processing of the data and the algorithm ...\", is not solid and analytical enough to demonstrate how these challenge the design of a better TE scheme. The paper should show how difficult it is that to process per-application-level data and to design an algorithm. Furthermore, they do not explain how incorporating machine learning techniques mitigate those challenges.\n\n2. The paper presents unclear descriptions of technical details. 1) In the second paragraph of Section 2, they claim that the proposed approach will learn two aspects, i.e., process and exchange the local state of switches and map the exchanged state into forwarding decisions. However, later in the fourth paragraph, they mention that the model should learn more than these two aspects. The model should also learn how to react to changes in the network, such as node and link failures and changes in monitored measures. They do not explain how the latter aspect is related to the former two aspects. 2) In Section 3, paragraph 5, does \"to select the HNSAs necessary for this\" indicates that there might be more HNSAs? Also, it is unclear what \"this\" refers to.\n\n3. The evaluation part is not complete enough. First, the paper does not show the comparison of computing overhead compared to baselines. As a result, we do not know if the proposed one can be deployed in actual data center deployment. How long will it take for training? What's the computing overhead for inference? Second, they do not use traffic features from various applications, and thus we do not know if the proposed approach can adapt to various traffic features. It is crucial in Traffic Engineering, as stated in Section 1 Introduction.\n\n4. The paper does not discuss the possible technical defects of their approach. For example, with wrong forwarding decisions accumulating, network congestion may happen. How does the approach handle problems that arise from intrinsic errors of the Machine Learning algorithm?", "summary_of_the_paper": "The paper proposes an ML-based approach Mistill to help automatically deploy distributed protocol from a given TE policy. The approach learns forwarding decisions together with intermediate results such as exchange information and LinkState encoding from exemplary policies. The resulting network can later be deployed in switches to encode link-state and compute forwarding decisions. The technical contributions are the following. First, they leverage a reparameterization trick to handle categorical distributions when encoding link states into a binary vector. Second, they use the scaled dot-product attention method to help switches learn to exploit the encodings mentioned above and make the forwarding decision. In the evaluation part, the paper compares with three custom baselines, i.e., LCP, MinMax, WCMP, and exceeds all of them in terms of corresponding metrics.", "summary_of_the_review": "The paper gives good insights into combining machine learning techniques and traffic engineering. However, it lacks solid evaluation to demonstrate the overhead of this approach and lacks analysis of problem handling. Moreover, the representation of background and some of the technical details does not help readers to understand. Therefore, the paper should 1) add evaluations demonstrating the overhead of ML techniques, analyze the possible problems that may arise from prediction errors; 2) rewrite ambiguous sentences to provide more clear technical details; 3) show more concrete difficulties in translating traffic engineering policy into protocols.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "9mXtSR-NL3c", "reviewer": "Reviewer_wBxW", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper studies the use of ML to design protocols for a data-center application. In particular, a single-server data center, a Fat-Tree topology, and a single policy are assumed. A neural network is training to learn forwarding policies from examples.  A modular structure is proposed to minimize complexity. The NN is trained to be adaptive, but the model itself is fixed at the end of training.\n\nThe paper is a welcome addition to the sparse set of papers on applying ML to problems in communication networks (compared with image classification, for example). A challenge, of course, is availability of data. \n\nOverall, the paper reads well, and it contains detailed information on the architecture and training (perhaps a bit too much for the ICLR audience).\n\nThere are several issues that need clarification:\n\n(1) Do all switches need to have MISTILL installed? Can this work in a hybrid architecture where some switches do not have MISTILL?\n\n(2) The work is focused on data-center application; the assumption of Clos-topologies makes it inapplicable to packet-switched networks, which are the norm today.\n\n(3) Is the topology assumed known? Does every node know its parents and children nodes?\n\n(4) The specific policy learned here is that of dropping packets or flows if a node is unreachable due to link or node failures; this seems to be a simple problem given the fat tree structure. Does the policy also switch to a new route if the current route is unavailable? \n\n(5) Typically protocols must instantiate several policies, not just a single policy. Can the framework be extended to do that?\n\n(6) Given that the topology is assumed to be a tree, it is not clear that the approach can be extended to making use of new links, as would be the case in an ad hoc mesh network (not just links and hence paths disappearing in a tree, but also new links forming or potentially forming).\n\n(7) A \\beta term is introduced in the modified loss function in (6) to ensure that embeddings corresponding to different nodes do not end up close to each other. But this is precisely what contrastive learning is supposed to achieve. It is not clear why the authors were not able to get NCE to work.\n\n(8) Associated with nodes and links is context information which can also be used to differentiate between embeddings. This has been widely used in the social networks literature. This paper assumes that nodes are identical and have no context information. \n\n (9) The paper makes a criticism of CONTRA that the latter relies on the use of a high-level policy language.  What is wrong with that approach? If a ML model is to learn a policy, it should be able to recognize and resolve conflicts in the examples provided (the conflict may be due to lack of full context being provided or used by the learner). A purely data-driven approach may not be able to offer insights on what has been learned, vs. a neuro-symbolic approach where rules might be learned, and possibly interpretable by a human.\n\n(10) Figures 5 and 6 provide some hints. But it is not clear what the model has learned. \n\n(11) There is some discussion of overhead (top of page 5), but this is insufficient. Would like to get some more details. \n\n(12) Shortest path routes are mentioned several times; but there is only one cycle-free path on a tree. \n\n(13) The 2009 Al-Fares paper indicates that a k-fat tree has (k/2)^2 k-core switches and can support k^3/4 hosts. For k=8 this leads to 16 switches and 128 hosts, but the paper says 80 switches and 128 hosts. Please clarify. Also are there k=8 servers at the data center? \n\n\nTypos and such\n- Please define acronyms before, not after, first use; e.g. HNSA in Sec 3\n\n", "summary_of_the_paper": "The paper studies the use of ML to design protocols for a data-center application. In particular, a single-server data center, a Fat-Tree topology, and a single policy are assumed. A neural network is training to learn forwarding policies from examples.  A modular structure is proposed to minimize complexity. The NN is trained to be adaptive, but the model itself is fixed at the end of training.", "summary_of_the_review": "The paper studies the use of ML to design protocols for a data-center application; it adds to the sparse set of papers on the application of  ML to problems in communication networks, and is a welcome change from papers dealing yet again with MNIST. \n\nMy main concern relates to generalizability: Can the framework be extended to non-tree topologies (in particular, where links may appear as well as disappear, and where multiple routes exist), to protocols that must support multiple policies, and to incorporation of contextual information associated with nodes (perhaps also links). It is not clear what the model has learned, and some additional details about the test setup are needed.  ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "4kicc-KZpaA", "reviewer": "Reviewer_qhEy", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper proposes NN-driven model for designing novel traffic engineering techniques automatically. This paper is missing details on neural architecture, configuration parameters and experiment details. I had major difficult understanding the write up due to these missing difficulties. Let me elaborate:\n\n1. No details on the model: paper does not provide the details of the NN; how many layers, what is the size of the input, how many neurons per layer. What is the loss function and how is it trained?\n\n2. Experiment setup: Why is 10 simulation sufficient? Is random => uniform random? How many concurrent failures? For a large network, 10 simulations of failure events is not sufficient.\n\n3. Model abstraction and retraining of the network: does the proposed model has to be retrained for a different network size and topology? how long does it take to retrain and what is the initial training time ?\n\n4. Simulation: What traffic is being simulated? What are the datacenter applications that paper is using?\n\nOverall the paper is very light on details and is not fit for publication at this time. \n\nOther issues:\n--\n1. SmartNICs cannot behave like switches so the NN inference they are planning to do has to be done on P4 switches only. Authors should elaborate on how they plan to deploy their model in practice?\n2. Nocomparison with Contra (which is state-of-the-art, they do not show how their TE algorithm behaves against contra for the exact same performance goals)", "summary_of_the_paper": "This paper proposes NN-driven model for designing novel traffic engineering techniques automatically. ", "summary_of_the_review": "This paper is missing details on neural architecture, configuration parameters and experiment details. I am not sure \"what\" the novelty of the paper is. It is most probably due to the fact that the paper significantly lacks in details. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Most of the reviewers thought this paper has issues where it could be improved.  There was a range of concerns. Most importantly, several reviewers felt the novelty in the paper was unclear as well as the requirement for more details in the experimental evaluations.
Decision: Reject