Original: Learning to Configure Computer Networks with Neural Algorithmic Reasoning
Revised: Learning to Configure Computer Networks with Neural Algorithmic Reasoning

Original: Abstract
Revised: Abstract

Original: We present a new method for scaling automatic configuration of computer networks.
Revised: We present a new method for scaling automatic configuration of computer networks.

Original: The key idea is to relax the computationally hard search problem of finding a configuration that satisfies a given specification into an approximate objective amenable to learning-based techniques.
Revised: The key idea is to relax the computationally hard search problem of finding a configuration that satisfies a given specification into an approximate objective amenable to learning-based techniques.

Original: Based on this idea, we train a neural algorithmic model which learns to generate configurations likely to (fully or partially) satisfy a given specification under existing routing protocols.
Revised: Based on this idea, we train a neural algorithmic model which learns to generate configurations likely to (fully or partially) satisfy a given specification under existing routing protocols.

Original: By relaxing the rigid satisfaction guarantees, our approach (i) enables greater flexibility: it is protocol-agnostic, enables cross-protocol reasoning, and does not depend on hardcoded rules; and (ii) finds configurations for much larger computer networks than previously possible.
Revised: By relaxing the rigid satisfaction guarantees, our approach (i) enables greater flexibility: it is protocol-agnostic, enables cross-protocol reasoning, and does not depend on hardcoded rules; and (ii) finds configurations for much larger computer networks than previously possible.

Original: Our learned synthesizer is up to 490 × faster than state-of-the-art SMT-based methods, while producing configurations which on average satisfy more than 92% of the provided requirements.
Revised: Our learned synthesizer is up to 490 × faster than state-of-the-art SMT-based methods, while producing configurations which on average satisfy more than 92% of the provided requirements.

Original:  Introduction
Revised:  Introduction

Original: Configuring large-scale networks is a challenging and important task as network configuration mistakes regularly lead to massive internet-wide outages affecting millions (resp.
Revised: Configuring large-scale networks is a challenging and important task as network configuration mistakes regularly lead to massive internet-wide outages affecting millions (resp.

Original: billions 2 ) of
Revised: billions 2 ) of

Original: Internet users [35, 25].
Revised: Internet users [35, 25].

Original: Typically, network operators provide a router-level configuration W which, after applying protocols such as shortest-path routing, induces a certain forwarding behaviour F WD as illustrated in Figure 1.
Revised: Typically, network operators provide a router-level configuration W which, after applying protocols such as shortest-path routing, induces a certain forwarding behaviour F WD as illustrated in Figure 1.

Original: As this remains a challenging task, much recent research has focused on automating configuration by leveraging synthesis techniques [15, 5, 31]: A synthesizer is used to automatically generate a router-level configuration W that, after applying routing protocols results in forwarding behavior that satisfies a given specification S on how traffic should be routed.
Revised: As this remains a challenging task, much recent research has focused on automating configuration by leveraging synthesis techniques [15, 5, 31]: A synthesizer is used to automatically generate a router-level configuration W that, after applying routing protocols results in forwarding behavior that satisfies a given specification S on how traffic should be routed.

Original: SMT-based Synthesis Due to the hardness of the configuration synthesis problem [7], many effective tools in this domain [15, 14] resort to satisfiability modulo theory (SMT) solvers, which employ search-based procedures to find a solution to a set of first-order logic constraints.
Revised: SMT-based Synthesis Due to the hardness of the configuration synthesis problem [7], many effective tools in this domain [15, 14] resort to satisfiability modulo theory (SMT) solvers, which employ search-based procedures to find a solution to a set of first-order logic constraints.

Original: This enables comprehensive and exact synthesis by modelling network behavior in first-order logic.
Revised: This enables comprehensive and exact synthesis by modelling network behavior in first-order logic.

Original: However, these tools are typically protocol-specific, hand-coded, and can exhibit discrepancies in behavior when compared to actual router hardware [6].
Revised: However, these tools are typically protocol-specific, hand-coded, and can exhibit discrepancies in behavior when compared to actual router hardware [6].

Original: Most importantly, however, they can be very slow or fail to complete for large networks.
Revised: Most importantly, however, they can be very slow or fail to complete for large networks.

Original: For example, the state-of-the-art SMT-based tool NetComplete [15] requires more than 6 hours to synthesize a configuration for a network with 64 nodes, for which other SMT-based tools like SyNET [14] take even longer ( > 24 hours)
Revised: For example, the state-of-the-art SMT-based tool NetComplete [15] requires more than 6 hours to synthesize a configuration for a network with 64 nodes, for which other SMT-based tools like SyNET [14] take even longer ( > 24 hours)

Original: Non-SMT-based tools such as Propane [4] or Zeppelin [39] have achieved better performance, but at the cost of generality.
Revised: Non-SMT-based tools such as Propane [4] or Zeppelin [39] have achieved better performance, but at the cost of generality.

Original: A
Revised: A

Original: B
Revised: B

Original: C
Revised: C

Original: D
Revised: D

Original: E
Revised: E

Original: F
Revised: F

Original: Forwarding Plane Protocol (Graph Algorithm, e.g. OSPF)
Revised: Forwarding Plane Protocol (Graph Algorithm, e.g. OSPF)

Original: Check Consistency Configuration
Revised: Check Consistency Configuration

Original: A
Revised: A

Original: B
Revised: B

Original: C
Revised: C

Original: D
Revised: D

Original: E
Revised: E

Original: F 3fwd(A,B,N1)not fwd(C,B,N1)
Revised: F 3fwd(A,B,N1)not fwd(C,B,N1)

Original: reachable(D,A,N1)isolation(A,B,N1,N2)
Revised: reachable(D,A,N1)isolation(A,B,N1,N2)

Original: Specification S
Revised: Specification S

Original: ConfigurationSynthesis
Revised: ConfigurationSynthesis

Original: [Figure]
Revised: [Figure]

Original: Addressing the scalability barrier The reason for the wide-spread use of SMT in configuration synthesis is the inherent computational complexity of the underlying synthesis problem, parts of which have been shown to be NP-hard [7, 16, 44].
Revised: Addressing the scalability barrier The reason for the wide-spread use of SMT in configuration synthesis is the inherent computational complexity of the underlying synthesis problem, parts of which have been shown to be NP-hard [7, 16, 44].

Original: This means that any exact synthesis method is bound to run into scalability issues (as do all SMT-based methods).
Revised: This means that any exact synthesis method is bound to run into scalability issues (as do all SMT-based methods).

Original: However, to be practically useful, a synthesizer must scale to the size of real-world networks and the frequency at which configurations are updated.
Revised: However, to be practically useful, a synthesizer must scale to the size of real-world networks and the frequency at which configurations are updated.

Original: For example, in a Tier-1 ISP, network operators modify their configurations up totimes per day, on average [36].
Revised: For example, in a Tier-1 ISP, network operators modify their configurations up totimes per day, on average [36].

Original: We argue that one way to address this scalability barrier, is relaxing the configuration synthesis problem to admit approximate solutions with high utility – configurations which may not always satisfy all requirements of a given specification but may satisfy almost all of them.
Revised: We argue that one way to address this scalability barrier, is relaxing the configuration synthesis problem to admit approximate solutions with high utility – configurations which may not always satisfy all requirements of a given specification but may satisfy almost all of them.

Original: Such a configuration would be a much better starting point for a network operator than having no automated support whatsoever.
Revised: Such a configuration would be a much better starting point for a network operator than having no automated support whatsoever.

Original: The core technical challenge then is coming up with a strategy likely to find solutions with high utility.
Revised: The core technical challenge then is coming up with a strategy likely to find solutions with high utility.

Original: This Work: enabling fast and scalable configuration via neural algorithmic reasoning We address this challenge and present the first learning-based framework for approximate configuration synthesis.
Revised: This Work: enabling fast and scalable configuration via neural algorithmic reasoning We address this challenge and present the first learning-based framework for approximate configuration synthesis.

Original: Our relaxed formulation allows us, for the first time, to apply end-to-end learning to the problem of network configuration synthesis, thereby enabling fast and scalable configuration synthesis with almost interactive response times (<90s even for large networks).
Revised: Our relaxed formulation allows us, for the first time, to apply end-to-end learning to the problem of network configuration synthesis, thereby enabling fast and scalable configuration synthesis with almost interactive response times (<90s even for large networks).

Original: Technically, we leverage the observation that routing protocols can often be formulated as Bellman-Ford style graph algorithms, a class of problems that has recently been studied in the area of neural algorithmic reasoning (NAR) [43].
Revised: Technically, we leverage the observation that routing protocols can often be formulated as Bellman-Ford style graph algorithms, a class of problems that has recently been studied in the area of neural algorithmic reasoning (NAR) [43].

Original: Connecting the two fields and building on ideas from NAR, we are able to train a graph-based neural model with a strong inductive bias to learn an inverse mapping from specifications back to network configurations: Our model learns how to perform synthesis from a dataset of (specification, configuration) pairs obtained by simulating the involved protocols and observing the computed forwarding state.
Revised: Connecting the two fields and building on ideas from NAR, we are able to train a graph-based neural model with a strong inductive bias to learn an inverse mapping from specifications back to network configurations: Our model learns how to perform synthesis from a dataset of (specification, configuration) pairs obtained by simulating the involved protocols and observing the computed forwarding state.

Original: With this method, we can support cross-protocol reasoning and do not have to manually provide any hardcoded synthesis rules.
Revised: With this method, we can support cross-protocol reasoning and do not have to manually provide any hardcoded synthesis rules.

Original: Concretely, we introduce a generic embedding scheme for topologies and configurations, making our method protocol-agnostic.
Revised: Concretely, we introduce a generic embedding scheme for topologies and configurations, making our method protocol-agnostic.

Original: During synthesis, given a specification, our model predicts distributions of network configurations from which we can sample possible results.
Revised: During synthesis, given a specification, our model predicts distributions of network configurations from which we can sample possible results.

Original: Main
Revised: Main

Original: Contributions
Revised: Contributions

Original: Our core contributions are:
Revised: Our core contributions are:

Original: • We formulate a relaxation of the exact configuration synthesis problem, which enables fast and scalable network configuration, amenable to learning-based techniques (Section 2).
Revised: • We formulate a relaxation of the exact configuration synthesis problem, which enables fast and scalable network configuration, amenable to learning-based techniques (Section 2).

Original: • We propose a NAR-based neural network architecture for learning synthesizer models that rely on a graph-based encoding of topologies and configurations, and a strong inductive bias towards an iterative synthesis procedure (Section 4).
Revised: • We propose a NAR-based neural network architecture for learning synthesizer models that rely on a graph-based encoding of topologies and configurations, and a strong inductive bias towards an iterative synthesis procedure (Section 4).

Original: • We conduct an extensive evaluation of our learning-based synthesizer with respect to both precision and scalability.
Revised: • We conduct an extensive evaluation of our learning-based synthesizer with respect to both precision and scalability.

Original: We demonstrate that our learned synthesizer is up to 490 × faster than a state-of-the-art SMT-based tool while producing high utility configurations which on average satisfy > 93% of provided constraints (Section 5).
Revised: We demonstrate that our learned synthesizer is up to 490 × faster than a state-of-the-art SMT-based tool while producing high utility configurations which on average satisfy > 93% of provided constraints (Section 5).

Original:  Configuration Synthesis: Exact and Learned
Revised:  Configuration Synthesis: Exact and Learned

Original: We first state the general configuration synthesis problem and explain why it is hard to solve.
Revised: We first state the general configuration synthesis problem and explain why it is hard to solve.

Original: We then present a rather different approach based on learning that addresses the scalability barrier of traditional synthesis.
Revised: We then present a rather different approach based on learning that addresses the scalability barrier of traditional synthesis.

Original: Forwarding Behavior and Specifications We focus on the level of the forwarding plane of a network.
Revised: Forwarding Behavior and Specifications We focus on the level of the forwarding plane of a network.

Original: This means we consider how a network forwards traffic, given a packet with a certain destination.
Revised: This means we consider how a network forwards traffic, given a packet with a certain destination.

Original: The forwarding plane is determined by a distributed computation that depends on the different routing protocols in use.
Revised: The forwarding plane is determined by a distributed computation that depends on the different routing protocols in use.

Original: More formally, we define the forwarding plane F WD as follows:
Revised: More formally, we define the forwarding plane F WD as follows:

Original: [Equation]
Revised: [Equation]

Original: Prot ( W ; T ) corresponds to the result of applying routing protocols to the network topology T and the configuration W (e.g. link weights).
Revised: Prot ( W ; T ) corresponds to the result of applying routing protocols to the network topology T and the configuration W (e.g. link weights).

Original: In the following, we omit T as it remains fixed in synthesis.
Revised: In the following, we omit T as it remains fixed in synthesis.

Original: The resulting forwarding plane F WD can be understood as a directed graph superimposed on topology
Revised: The resulting forwarding plane F WD can be understood as a directed graph superimposed on topology

Original: T .
Revised: T .

Original: It specifies a subset of links that are used to forward packets.
Revised: It specifies a subset of links that are used to forward packets.

Original: To illustrate consider Figure 1: applying the routing protocols yields forwarding plane F WD which corresponds to the subset of links ( C,A ) , ( B,A ) , ( A,E ) , ( D,F ) .
Revised: To illustrate consider Figure 1: applying the routing protocols yields forwarding plane F WD which corresponds to the subset of links ( C,A ) , ( B,A ) , ( A,E ) , ( D,F ) .

Original: The other links of the network are not part of the forwarding plane and will thus not be used to forward traffic.
Revised: The other links of the network are not part of the forwarding plane and will thus not be used to forward traffic.

Original: Given F WD , we consider a forwarding specification S ∶=
Revised: Given F WD , we consider a forwarding specification S ∶=

Original: { R i } i as an input to the synthesis problem.
Revised: { R i } i as an input to the synthesis problem.

Original: Each requirement in S is modelled as a function R i , where R i ( F WD )
Revised: Each requirement in S is modelled as a function R i , where R i ( F WD )

Original: = 1 if F WD satisfies the requirement and 0 otherwise.
Revised: = 1 if F WD satisfies the requirement and 0 otherwise.

Original: Practical example requirements include reachability, traffic isolation or specifying concrete forwarding paths.
Revised: Practical example requirements include reachability, traffic isolation or specifying concrete forwarding paths.

Original: Exact Configuration Synthesis We formulate the general configuration synthesis problem as the following optimization objective:
Revised: Exact Configuration Synthesis We formulate the general configuration synthesis problem as the following optimization objective:

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: P ( W ) denotes the set of all possible configurations and C ( F WD ,S ) is the specification consistency of a forwarding plane F WD w.r.t specification S .
Revised: P ( W ) denotes the set of all possible configurations and C ( F WD ,S ) is the specification consistency of a forwarding plane F WD w.r.t specification S .

Original: In traditional, exact configuration synthesis, this objective is solved by limiting the search to globally-optimal configurations such that C ( F WD ,S ) = 1 . 0 , i.e. all requirements must be satisfied.
Revised: In traditional, exact configuration synthesis, this objective is solved by limiting the search to globally-optimal configurations such that C ( F WD ,S ) = 1 . 0 , i.e. all requirements must be satisfied.

Original: Such exact methods typically resort to SMT solvers because of the hardness of the underlying problem: configurations comprise a large number of tunable parameters, where the execution of several interacting protocols yields the overall forwarding state.
Revised: Such exact methods typically resort to SMT solvers because of the hardness of the underlying problem: configurations comprise a large number of tunable parameters, where the execution of several interacting protocols yields the overall forwarding state.

Original: Even worse, parts of the configuration synthesis problem have been shown to be NP-hard [7, 16, 44]:
Revised: Even worse, parts of the configuration synthesis problem have been shown to be NP-hard [7, 16, 44]:

Original: For example, already the subproblem of finding link weights that yield a given set of forwarding paths under shortest-path routing is NP-hard [7].
Revised: For example, already the subproblem of finding link weights that yield a given set of forwarding paths under shortest-path routing is NP-hard [7].

Original: This makes scaling exact synthesis to real-world networks extremely challenging.
Revised: This makes scaling exact synthesis to real-world networks extremely challenging.

Original: Learning-Based Synthesis To enable fast and scalable configuration synthesis, we propose to relax both the optimality as well as the rigid satisfaction requirements w.r.t the specification S .
Revised: Learning-Based Synthesis To enable fast and scalable configuration synthesis, we propose to relax both the optimality as well as the rigid satisfaction requirements w.r.t the specification S .

Original: Concretely, we relax the set of admissible solutions to include configurations that are not optimal, but still satisfy a large number of provided requirements.
Revised: Concretely, we relax the set of admissible solutions to include configurations that are not optimal, but still satisfy a large number of provided requirements.

Original: Note that this is not the same as merely allowing solutions with C ( F WD ,S ) < 1 . 0 , because maximum satisfiability does not relax the hardness of the problem.
Revised: Note that this is not the same as merely allowing solutions with C ( F WD ,S ) < 1 . 0 , because maximum satisfiability does not relax the hardness of the problem.

Original: Instead, we propose to search for near-optimal, good solutions and rely on the value of specification consistency C as a measure of quality.
Revised: Instead, we propose to search for near-optimal, good solutions and rely on the value of specification consistency C as a measure of quality.

Original: An approximate synthesis formulation relaxes the hardness of the problem, however, it also leads to the difficult technical challenge of finding solutions with high utility (e.g., where many requirements are satisfied).
Revised: An approximate synthesis formulation relaxes the hardness of the problem, however, it also leads to the difficult technical challenge of finding solutions with high utility (e.g., where many requirements are satisfied).

Original: To address this challenge, we propose a rather different approach where we learn synthesis from data.
Revised: To address this challenge, we propose a rather different approach where we learn synthesis from data.

Original: Concretely, we learn an inverse mapping that attempts to predict approximate solutions ˆ W ⋆ with high utility, as guided by the following objective:
Revised: Concretely, we learn an inverse mapping that attempts to predict approximate solutions ˆ W ⋆ with high utility, as guided by the following objective:

Original: [Equation]
Revised: [Equation]

Original: This can be implemented as a synthesizer model M Syn which produces a solution given just the topology T and the specification S :
Revised: This can be implemented as a synthesizer model M Syn which produces a solution given just the topology T and the specification S :

Original: [Equation]
Revised: [Equation]

Original: We propose a learning-based approach for training such synthesizer models based on neural algorithmic reasoning (cf.
Revised: We propose a learning-based approach for training such synthesizer models based on neural algorithmic reasoning (cf.

Original: Section 4).
Revised: Section 4).

Original: First, however, we discuss the routing protocols that make up function
Revised: First, however, we discuss the routing protocols that make up function

Original: Prot and how they relate to graph algorithms and by extension to NAR.
Revised: Prot and how they relate to graph algorithms and by extension to NAR.

Original:  Routing Protocols as Graph Algorithms
Revised:  Routing Protocols as Graph Algorithms

Original: In practice, the routing protocols defining Prot are implemented as distributed systems in which multiple peers communicate to determine the network’s forwarding state.
Revised: In practice, the routing protocols defining Prot are implemented as distributed systems in which multiple peers communicate to determine the network’s forwarding state.

Original: However, theoretical work on routing algebras [21] has shown that the underlying computation can be understood as a traditional message-passing graph algorithm.
Revised: However, theoretical work on routing algebras [21] has shown that the underlying computation can be understood as a traditional message-passing graph algorithm.

Original: As a consequence, many routing protocols can be formulated as Bellman-Ford (BF) style propagation processes.
Revised: As a consequence, many routing protocols can be formulated as Bellman-Ford (BF) style propagation processes.

Original: This class of problems has also recently been subject to work on NAR [43, 41] and algorithmic alignment [45].
Revised: This class of problems has also recently been subject to work on NAR [43, 41] and algorithmic alignment [45].

Original: The authors of these works demonstrate that neural networks are capable of closely imitating BF-style algorithms when provided with a suitable inductive bias.
Revised: The authors of these works demonstrate that neural networks are capable of closely imitating BF-style algorithms when provided with a suitable inductive bias.

Original: Based on this insight, NAR proposes to replace traditional algorithms with neural networks to learn improved algorithmic procedures or extend existing algorithms to be applicable to raw data [41].
Revised: Based on this insight, NAR proposes to replace traditional algorithms with neural networks to learn improved algorithmic procedures or extend existing algorithms to be applicable to raw data [41].

Original: Following the idea of NAR, we implement a synthesizer model for network configurations as an iterative Graph Neural Network (GNN) to learn Prot − 1 by relying on a
Revised: Following the idea of NAR, we implement a synthesizer model for network configurations as an iterative Graph Neural Network (GNN) to learn Prot − 1 by relying on a

Original: Bellman-Ford style inductive bias.
Revised: Bellman-Ford style inductive bias.

Original: Synthesis Setting Our approach is general for the domain of networks, but we focus on two widelyused routing protocols: (1) Open Shortest Path First (OSPF)
Revised: Synthesis Setting Our approach is general for the domain of networks, but we focus on two widelyused routing protocols: (1) Open Shortest Path First (OSPF)

Original: [30] – it uses link weights to route traffic along the shortest path towards the destination, and the (2) Border Gateway Protocol (BGP) [32], used to exchange reachability information, mostly on the level of larger backbone networks.
Revised: [30] – it uses link weights to route traffic along the shortest path towards the destination, and the (2) Border Gateway Protocol (BGP) [32], used to exchange reachability information, mostly on the level of larger backbone networks.

Original: When using BGP, a routing destination announces its existence to other networks and routers by sending out BGP announcements.
Revised: When using BGP, a routing destination announces its existence to other networks and routers by sending out BGP announcements.

Original: Receivers of announcements then choose to pass them on to other peers, redistribute them internally and/or modify them according to a set of decision rules.
Revised: Receivers of announcements then choose to pass them on to other peers, redistribute them internally and/or modify them according to a set of decision rules.

Original: BGP and OSPF interact, for instance, BGP will consider the OSPF cost of internal destinations in its routing decisions.
Revised: BGP and OSPF interact, for instance, BGP will consider the OSPF cost of internal destinations in its routing decisions.

Original: This means, that effective BGP/OSPF synthesis tools must implement cross-protocol reasoning, configuring BGP and OSPF, such that together they yield the desired forwarding behaviour.
Revised: This means, that effective BGP/OSPF synthesis tools must implement cross-protocol reasoning, configuring BGP and OSPF, such that together they yield the desired forwarding behaviour.

Original: Example To provide a basic intuition about BGP/OSPF routing, consider the simple example network in Figure 2.
Revised: Example To provide a basic intuition about BGP/OSPF routing, consider the simple example network in Figure 2.

Original: We apply the two protocols to obtain the forwarding plane.
Revised: We apply the two protocols to obtain the forwarding plane.

Original: Physical OSPF edges are labelled with a corresponding link weight determining the OSPF cost of paths through the network.
Revised: Physical OSPF edges are labelled with a corresponding link weight determining the OSPF cost of paths through the network.

Original: Dotted lines represent designated BGP edges, used to propagate BGP announcements.
Revised: Dotted lines represent designated BGP edges, used to propagate BGP announcements.

Original: Our network imports multiple BGP announcements ∎ and ▲ from E and F respectively.
Revised: Our network imports multiple BGP announcements ∎ and ▲ from E and F respectively.

Original: Both represent a route to the same destination.
Revised: Both represent a route to the same destination.

Original: As shown, the announcements have a so-called BGP local preference of 2 (we ignore other BGP properties in this example).
Revised: As shown, the announcements have a so-called BGP local preference of 2 (we ignore other BGP properties in this example).

Original: The announcements are propagated through the network and the best route is selected according to a designated BGP decision procedure.
Revised: The announcements are propagated through the network and the best route is selected according to a designated BGP decision procedure.

Original: Figureshows the intermediate states of this propagation process.
Revised: Figureshows the intermediate states of this propagation process.

Original: As both announcements have the same local preference value, the decision cannot eliminate based on that.
Revised: As both announcements have the same local preference value, the decision cannot eliminate based on that.

Original: Instead, in round 2, node B
Revised: Instead, in round 2, node B

Original: Distributionof SynthesizedParameters
Revised: Distributionof SynthesizedParameters

Original: [Figure]
Revised: [Figure]

Original: selects ∎ over ▲ due to a lower OSPF cost (shorter path) of 2 via node A as compared to 3 via node D.
Revised: selects ∎ over ▲ due to a lower OSPF cost (shorter path) of 2 via node A as compared to 3 via node D.

Original: Node D selects ▲ over ∎ , since it learns this route directly from an external peer which is preferred in BGP.
Revised: Node D selects ▲ over ∎ , since it learns this route directly from an external peer which is preferred in BGP.

Original: After Round 3, the BGP propagation process converges to a stable state and we can derive the forwarding plane as shown on the left in Figure 2.
Revised: After Round 3, the BGP propagation process converges to a stable state and we can derive the forwarding plane as shown on the left in Figure 2.

Original: For completeness, we include the full BGP decision process in Appendix A.3.
Revised: For completeness, we include the full BGP decision process in Appendix A.3.

Original: Configuration Parameters For our purposes, we define the set of synthesized configuration parameters as follows:
Revised: Configuration Parameters For our purposes, we define the set of synthesized configuration parameters as follows:

Original: For OSPF, we synthesize link weights as explored in existing work [15, 16].
Revised: For OSPF, we synthesize link weights as explored in existing work [15, 16].

Original: For BGP, we focus on a setting, where we synthesize BGP import policies only.
Revised: For BGP, we focus on a setting, where we synthesize BGP import policies only.

Original: This means we synthesize the modifications required for BGP announcements when entering the network, to satisfy the routing specification.
Revised: This means we synthesize the modifications required for BGP announcements when entering the network, to satisfy the routing specification.

Original: Previous work has confirmed that this is a realistic configuration setting that applies to a majority of real-world networks [9, 12, 38].
Revised: Previous work has confirmed that this is a realistic configuration setting that applies to a majority of real-world networks [9, 12, 38].

Original: Based on the observation that routing protocols such as OSPF and BGP can be expressed as message-passing algorithms, we heavily rely on GNNs/NAR for the design of our synthesizer model as discussed next.
Revised: Based on the observation that routing protocols such as OSPF and BGP can be expressed as message-passing algorithms, we heavily rely on GNNs/NAR for the design of our synthesizer model as discussed next.

Original: In our evaluation, we then train such a model for the concrete case of BGP/OSPF and compare synthesis performance with a traditional SMT-based tool.
Revised: In our evaluation, we then train such a model for the concrete case of BGP/OSPF and compare synthesis performance with a traditional SMT-based tool.

Original:  Neural Configuration Synthesis Model
Revised:  Neural Configuration Synthesis Model

Original: We train a graph neural network (GNN) as the neural synthesis configuration model.
Revised: We train a graph neural network (GNN) as the neural synthesis configuration model.

Original: The model’s input consists of a topology, a specification, and a configuration sketch where to-be-synthesized parameters are omitted.
Revised: The model’s input consists of a topology, a specification, and a configuration sketch where to-be-synthesized parameters are omitted.

Original: Following the NAR paradigm, we first encode this synthesizer input in latent space.
Revised: Following the NAR paradigm, we first encode this synthesizer input in latent space.

Original: Then, we apply an iterative processor network based on the graph attention mechanism [42].
Revised: Then, we apply an iterative processor network based on the graph attention mechanism [42].

Original: Last, we apply a decoder network to predict the values of omitted configuration parameters, thereby synthesizing a configuration.
Revised: Last, we apply a decoder network to predict the values of omitted configuration parameters, thereby synthesizing a configuration.

Original: To remain protocol-agnostic, our model is generic with respect to the input format, using an intermediate representation based on Datalog-like facts.
Revised: To remain protocol-agnostic, our model is generic with respect to the input format, using an intermediate representation based on Datalog-like facts.

Original: Figure 3 provides an overview of our graph-based neural synthesizer architecture.
Revised: Figure 3 provides an overview of our graph-based neural synthesizer architecture.

Original: 4.1 Training Dataset of Inverse Pairs
Revised: 4.1 Training Dataset of Inverse Pairs

Original: A learning-based synthesizer model is formulated as a supervised learning problem.
Revised: A learning-based synthesizer model is formulated as a supervised learning problem.

Original: Thus, we can directly train a neural network to learn the inverse mapping, given a dataset of corresponding input-output pairs.
Revised: Thus, we can directly train a neural network to learn the inverse mapping, given a dataset of corresponding input-output pairs.

Original: To obtain such a dataset, we sample a random network configuration for some topology using a uniform generative process.
Revised: To obtain such a dataset, we sample a random network configuration for some topology using a uniform generative process.

Original: Then, we simulate the involved protocols using Prot , to obtain the corresponding forwarding plane.
Revised: Then, we simulate the involved protocols using Prot , to obtain the corresponding forwarding plane.

Original: Next, we extract a specification by randomly selecting properties that hold for the computed forwarding plane.
Revised: Next, we extract a specification by randomly selecting properties that hold for the computed forwarding plane.

Original: This leaves us with a pair of specification and topology as input, and a corresponding configuration as output.
Revised: This leaves us with a pair of specification and topology as input, and a corresponding configuration as output.

Original: The key to constructing the dataset is the implementation of Prot .
Revised: The key to constructing the dataset is the implementation of Prot .

Original: Even though Prot is protocolspecific, it turns out the overall implementation effort is comparatively low, especially when compared to SMT-based synthesis methods.
Revised: Even though Prot is protocolspecific, it turns out the overall implementation effort is comparatively low, especially when compared to SMT-based synthesis methods.

Original: Protocols are well-defined algorithms that can be easily simulated, whereas the alternative of implementing hardcoded synthesis rules directly often requires expert
Revised: Protocols are well-defined algorithms that can be easily simulated, whereas the alternative of implementing hardcoded synthesis rules directly often requires expert

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: knowledge of SMT solvers.
Revised: knowledge of SMT solvers.

Original: This process may also be adapted to rely on actual router hardware to compute the result of Prot , thereby capturing real-world behavior precisely
Revised: This process may also be adapted to rely on actual router hardware to compute the result of Prot , thereby capturing real-world behavior precisely

Original: [Figure]
Revised: [Figure]

Original: 4.2 Embedding Topologies, Specifications and Configurations
Revised: 4.2 Embedding Topologies, Specifications and Configurations

Original: To remain agnostic with respect to routing protocols
Revised: To remain agnostic with respect to routing protocols

Original: [Figure]
Revised: [Figure]

Original: our model architecture implements a generic graph-based encoding of Datalog-like facts, similar to knowledge graphs [33]: we first encode topologies, configurations and specifications as a set of Datalog-like facts and then employ a generic embedding scheme to embed these facts into latent space.
Revised: our model architecture implements a generic graph-based encoding of Datalog-like facts, similar to knowledge graphs [33]: we first encode topologies, configurations and specifications as a set of Datalog-like facts and then employ a generic embedding scheme to embed these facts into latent space.

Original: Fact Base A set of Datalog-like facts, as depicted in Figure 4, serves as the input fact base F to our synthesizer model.
Revised: Fact Base A set of Datalog-like facts, as depicted in Figure 4, serves as the input fact base F to our synthesizer model.

Original: F is a set of facts f ( a 0 ,...,a n ) where arguments may be constants (e.g. A or B) or integer literals.
Revised: F is a set of facts f ( a 0 ,...,a n ) where arguments may be constants (e.g. A or B) or integer literals.

Original: Each fact has a corresponding boolean truth value denoted as [ f ( a )] B .
Revised: Each fact has a corresponding boolean truth value denoted as [ f ( a )] B .

Original: For instance, from the fact base in Figure 4 we can derive [ fwd ( A,B,N 1 )]
Revised: For instance, from the fact base in Figure 4 we can derive [ fwd ( A,B,N 1 )]

Original: B = true and [ fwd ( B,A,N 1 )]
Revised: B = true and [ fwd ( B,A,N 1 )]

Original: B = false .
Revised: B = false .

Original: Synthesis as
Revised: Synthesis as

Original: Completion Task
Revised: Completion Task

Original: A network’s topology, protocol configuration, parameters, and the specification are all represented in a single fact base.
Revised: A network’s topology, protocol configuration, parameters, and the specification are all represented in a single fact base.

Original: To predict the value of unknown, to-besynthesized parameters, we support the notion of unknown parameters as illustrated at ➊ in Figure 4, where the link weight between router nodes A and C is omitted.
Revised: To predict the value of unknown, to-besynthesized parameters, we support the notion of unknown parameters as illustrated at ➊ in Figure 4, where the link weight between router nodes A and C is omitted.

Original: Based on this input format, synthesis corresponds to using our model to predict the value of unknown parameters in a provided fact base.
Revised: Based on this input format, synthesis corresponds to using our model to predict the value of unknown parameters in a provided fact base.

Original: Embedding To transform a fact base into a graph with node features, we apply a structurally-defined embedding scheme.
Revised: Embedding To transform a fact base into a graph with node features, we apply a structurally-defined embedding scheme.

Original: An example of this embedding scheme is given in Figure 5.
Revised: An example of this embedding scheme is given in Figure 5.

Original: As shown, we embed topology, specification and configuration all into a single graph.
Revised: As shown, we embed topology, specification and configuration all into a single graph.

Original: This places network routers, the related specification predicates as well as configuration parameters in close adjacency to each other, simplifying the synthesis procedure for the processor network.
Revised: This places network routers, the related specification predicates as well as configuration parameters in close adjacency to each other, simplifying the synthesis procedure for the processor network.

Original: In our scheme, both facts and constants are represented as distinct nodes.
Revised: In our scheme, both facts and constants are represented as distinct nodes.

Original: The relationships between facts and constants are encoded as node adjacency as defined by neighborhood functions
Revised: The relationships between facts and constants are encoded as node adjacency as defined by neighborhood functions

Original: N i .
Revised: N i .

Original: We use multiple neighborhoods, i.e. multiple types of edges, to encode the argument position of a constant occurring in a fact.
Revised: We use multiple neighborhoods, i.e. multiple types of edges, to encode the argument position of a constant occurring in a fact.

Original: To handle unknown parameters, we replace the corresponding embedding with a learned V hole embedding.
Revised: To handle unknown parameters, we replace the corresponding embedding with a learned V hole embedding.

Original: Overall, the embedding function E MB relies on a set of learned parameters, including V f ∈ R D per fact type in
Revised: Overall, the embedding function E MB relies on a set of learned parameters, including V f ∈ R D per fact type in

Original: F , W bool ∈ R D × 2 for boolean values, W f i ∈
Revised: F , W bool ∈ R D × 2 for boolean values, W f i ∈

Original: R D × N per integer argument of fact type f and V hole ∈ R D to represent unknown parameters.
Revised: R D × N per integer argument of fact type f and V hole ∈ R D to represent unknown parameters.

Original: D is the dimensionality of the latent space and N
Revised: D is the dimensionality of the latent space and N

Original: [Table]
Revised: [Table]

Original: specifies the number of supported integer values.
Revised: specifies the number of supported integer values.

Original: For the complete embedding scheme, please see
Revised: For the complete embedding scheme, please see

Original: Appendix A.1.
Revised: Appendix A.1.

Original: 4.3 A NAR-based Synthesizer Model
Revised: 4.3 A NAR-based Synthesizer Model

Original: Our synthesizer model employs an encode-process-decode architecture, inspired by neural algorithmic reasoning [41].
Revised: Our synthesizer model employs an encode-process-decode architecture, inspired by neural algorithmic reasoning [41].

Original: It employs four main components to produce the output distribution for an unknown parameter in a fact base F : The fact base embedding E MB , the encoder E NC GAT , the processor
Revised: It employs four main components to produce the output distribution for an unknown parameter in a fact base F : The fact base embedding E MB , the encoder E NC GAT , the processor

Original: P ROC GAT and the fact-type specific decoder network D EC f i .
Revised: P ROC GAT and the fact-type specific decoder network D EC f i .

Original: Overall, the model can be expressed as follows, where X j and H j refer to the intermediate node representations per node j in the graph constructed via our fact base embedding.
Revised: Overall, the model can be expressed as follows, where X j and H j refer to the intermediate node representations per node j in the graph constructed via our fact base embedding.

Original: [Equation]
Revised: [Equation]

Original: E NC GAT is a GNN relying on Graph Attention Layers (GAT) [42] for propagation.
Revised: E NC GAT is a GNN relying on Graph Attention Layers (GAT) [42] for propagation.

Original: We additionally apply noise to the latent node representation X j via z as a source of non-determinism, anticipating the fact that the synthesis problem often has more than one solution.
Revised: We additionally apply noise to the latent node representation X j via z as a source of non-determinism, anticipating the fact that the synthesis problem often has more than one solution.

Original: The processor P ROC GAT is modelled as an iterative process.
Revised: The processor P ROC GAT is modelled as an iterative process.

Original: It consists of a 6-layer graph attention module which we apply for a total of 4 iterations.
Revised: It consists of a 6-layer graph attention module which we apply for a total of 4 iterations.

Original: According to the NAR paradigm, this computational structure encodes an inductive bias towards an iterative solution to the synthesis problem.
Revised: According to the NAR paradigm, this computational structure encodes an inductive bias towards an iterative solution to the synthesis problem.

Original: In the encoder and processor GNNs, we rely on a composed variant of the graph attention layer as introduced by [42].
Revised: In the encoder and processor GNNs, we rely on a composed variant of the graph attention layer as introduced by [42].

Original: We employ multiple graph attention layers in lockstep, one per neighborhood N i defined by our graph embedding (cf.
Revised: We employ multiple graph attention layers in lockstep, one per neighborhood N i defined by our graph embedding (cf.

Original: Section 4.2), and combine the intermediate results after each step by summation.
Revised: Section 4.2), and combine the intermediate results after each step by summation.

Original: For details on the graph attention module, please refer to Appendix A.2.
Revised: For details on the graph attention module, please refer to Appendix A.2.

Original: Finally, an argument decoder D EC f
Revised: Finally, an argument decoder D EC f

Original: i is used, to produce the output distribution
Revised: i is used, to produce the output distribution

Original: O f j ( a 0 ,...,a i − 1 , ? ,...,a n ) for an unknown parameter at position i of fact f j ∈ F , corresponding to node j .
Revised: O f j ( a 0 ,...,a i − 1 , ? ,...,a n ) for an unknown parameter at position i of fact f j ∈ F , corresponding to node j .

Original: For this, the synthesizer model provides one decoder per integer argument, per supported fact type.
Revised: For this, the synthesizer model provides one decoder per integer argument, per supported fact type.

Original: For example, a model for the fact base in Figure 4 would provide a decoder network D conn∶ R D
Revised: For example, a model for the fact base in Figure 4 would provide a decoder network D conn∶ R D

Original: → R N to decode values for the third argument of a conn fact.
Revised: → R N to decode values for the third argument of a conn fact.

Original: Decoder networks are implemented as simple multi-layer perceptron models.
Revised: Decoder networks are implemented as simple multi-layer perceptron models.

Original: Supervision Signal
Revised: Supervision Signal

Original: During training, we mask all fact arguments that represent to-be-synthesized configuration parameters in our synthesis setting as unknown parameters.
Revised: During training, we mask all fact arguments that represent to-be-synthesized configuration parameters in our synthesis setting as unknown parameters.

Original: As a supervision signal we use the masked values as the ground-truth and apply a negative log-likelihood loss to the output distribution of the corresponding argument decoder networks.
Revised: As a supervision signal we use the masked values as the ground-truth and apply a negative log-likelihood loss to the output distribution of the corresponding argument decoder networks.

Original: Multi-Shot Sampling To sample values from the output distributions of unknown parameters, we apply a multi-shot sampling strategy.
Revised: Multi-Shot Sampling To sample values from the output distributions of unknown parameters, we apply a multi-shot sampling strategy.

Original: We sample values only for a subset of unknown parameters, insert them in the input fact base and run the synthesizer again.
Revised: We sample values only for a subset of unknown parameters, insert them in the input fact base and run the synthesizer again.

Original: We repeat until no more unknown parameters remain.
Revised: We repeat until no more unknown parameters remain.

Original: This multi-shot strategy allows the model to incorporate concrete values of previously synthesized parameters in its computation.
Revised: This multi-shot strategy allows the model to incorporate concrete values of previously synthesized parameters in its computation.

Original: In our evaluation, we compare this approach to sampling all parameters at once.
Revised: In our evaluation, we compare this approach to sampling all parameters at once.

Original:  Evaluation
Revised:  Evaluation

Original: In this section, we assess the performance of our model trained for BGP/OSPF synthesis.
Revised: In this section, we assess the performance of our model trained for BGP/OSPF synthesis.

Original: For this, we trained a synthesizer model on a dataset of 10 , 240 samples, constructed by randomly sampling
Revised: For this, we trained a synthesizer model on a dataset of 10 , 240 samples, constructed by randomly sampling

Original: [Table]
Revised: [Table]

Original: topologies, corresponding specifications and BGP/OSPF configurations as described in Section 4.1.
Revised: topologies, corresponding specifications and BGP/OSPF configurations as described in Section 4.1.

Original: For details on BGP/OSPF dataset generation and training, please see Appendix A.3.
Revised: For details on BGP/OSPF dataset generation and training, please see Appendix A.3.

Original: Lastly, we also evaluate our architectural design decisions in an ablation and parameter study in Appendix D.
Revised: Lastly, we also evaluate our architectural design decisions in an ablation and parameter study in Appendix D.

Original: Dataset, Metrics and Experimental Setup We compare using datasets Small (S), Medium (M), and Large (L).
Revised: Dataset, Metrics and Experimental Setup We compare using datasets Small (S), Medium (M), and Large (L).

Original: Each dataset comprises 8 real-world topologies taken from the Topology Zoo [29], where the number of nodes lies between 0-18, 18-39, and 39-153, respectively.
Revised: Each dataset comprises 8 real-world topologies taken from the Topology Zoo [29], where the number of nodes lies between 0-18, 18-39, and 39-153, respectively.

Original: To obtain random forwarding specifications we use the same generative pipeline as discussed in Section 4.1.
Revised: To obtain random forwarding specifications we use the same generative pipeline as discussed in Section 4.1.

Original: Regarding forwarding requirements, we implement support for three specification facts: fwd requirements to set/block forwarding paths, reachable to specify reachability and trafficIsolation to induce traffic isolation among traffic classes (no shared links).
Revised: Regarding forwarding requirements, we implement support for three specification facts: fwd requirements to set/block forwarding paths, reachable to specify reachability and trafficIsolation to induce traffic isolation among traffic classes (no shared links).

Original: In each topology, we do synthesis fordifferent traffic classes (routing destinations) at a time.
Revised: In each topology, we do synthesis fordifferent traffic classes (routing destinations) at a time.

Original: Overall, this results in 3 datasets x 8 topologies per dataset x 3 differently-size specifications = 72 synthesis tasks.
Revised: Overall, this results in 3 datasets x 8 topologies per dataset x 3 differently-size specifications = 72 synthesis tasks.

Original: To assess synthesis quality, we determine specification consistency as the relative number of specification facts in a fact base that are satisfied by the synthesized configuration.
Revised: To assess synthesis quality, we determine specification consistency as the relative number of specification facts in a fact base that are satisfied by the synthesized configuration.

Original: We run all experiments on an Intel(R) i9-9900X@3.5GHz machine with 64GB of system memory and an NVIDIA RTX 3080 GPU with 10GB of video memory.
Revised: We run all experiments on an Intel(R) i9-9900X@3.5GHz machine with 64GB of system memory and an NVIDIA RTX 3080 GPU with 10GB of video memory.

Original: 5.1 Synthesis Quality
Revised: 5.1 Synthesis Quality

Original: To assess the quality of synthesized network configurations, we examine specification consistency with increasingly large topologies and specifications.
Revised: To assess the quality of synthesized network configurations, we examine specification consistency with increasingly large topologies and specifications.

Original: For each synthesis task, we run our synthesizertimes using 4-shot sampling and report the network configurations with the highest overall consistency.
Revised: For each synthesis task, we run our synthesizertimes using 4-shot sampling and report the network configurations with the highest overall consistency.

Original: Table 2 documents the results.
Revised: Table 2 documents the results.

Original: On average, our synthesis model achieves > 93% specification consistency for all datasets and specifications.
Revised: On average, our synthesis model achieves > 93% specification consistency for all datasets and specifications.

Original: With few requirements, the synthesizer model even succeeds in producing fully-consistent configurations (cf.
Revised: With few requirements, the synthesizer model even succeeds in producing fully-consistent configurations (cf.

Original: Full Matches in Table 2).
Revised: Full Matches in Table 2).

Original: We observe a slight decrease in consistency with increasingly large topologies.
Revised: We observe a slight decrease in consistency with increasingly large topologies.

Original: Multi-Shot Sampling To determine the effect of multi-shot-sampling, we report consistency when using 1-shot, 4-shot and 8-shot sampling in Table 1.
Revised: Multi-Shot Sampling To determine the effect of multi-shot-sampling, we report consistency when using 1-shot, 4-shot and 8-shot sampling in Table 1.

Original: As a baseline, we also show consistency when sampling configurations from a uniform distribution (per parameter).
Revised: As a baseline, we also show consistency when sampling configurations from a uniform distribution (per parameter).

Original: This simple method can achieve suprisingly good results, as parts of a specification may be satisfied naturally by the mechanics of shortest-path routing.
Revised: This simple method can achieve suprisingly good results, as parts of a specification may be satisfied naturally by the mechanics of shortest-path routing.

Original: Still, competing with this baseline our synthesizer model shows clear improvements and multi-shot sampling further increases consistency.
Revised: Still, competing with this baseline our synthesizer model shows clear improvements and multi-shot sampling further increases consistency.

Original: Number of Samples We also consider the number of times we sample from our synthesizer model.
Revised: Number of Samples We also consider the number of times we sample from our synthesizer model.

Original: We observe that sampling more than one alternative configuration can lead to improved best-of specification consistency across all datasets.
Revised: We observe that sampling more than one alternative configuration can lead to improved best-of specification consistency across all datasets.

Original: Based on this observation, we boost synthesis peformance by sampling multiple times per synthesis task.
Revised: Based on this observation, we boost synthesis peformance by sampling multiple times per synthesis task.

Original: For each configuration that we obtain in this way, we simulate the routing protocols, obtain the forwarding plane and check specification consistency.
Revised: For each configuration that we obtain in this way, we simulate the routing protocols, obtain the forwarding plane and check specification consistency.

Original: This fully automated process allows us to determine the best result without consulting the user.
Revised: This fully automated process allows us to determine the best result without consulting the user.

Original: We select the best result as the overall output for synthesis, dismissing the other samples.
Revised: We select the best result as the overall output for synthesis, dismissing the other samples.

Original: We experiment how the number of times we sample affects the resulting consistency in Appendix B.
Revised: We experiment how the number of times we sample affects the resulting consistency in Appendix B.

Original: [Table]
Revised: [Table]

Original: Overall, sampling more than once is beneficial for all datasets.
Revised: Overall, sampling more than once is beneficial for all datasets.

Original: Average best consistency values appear to be reached after 4-5 samples for 3 × 16 BGP/OSPF requirements.
Revised: Average best consistency values appear to be reached after 4-5 samples for 3 × 16 BGP/OSPF requirements.

Original: Hence, we rely on 5 samples in all other experiments as a trade-off of fast synthesis time and good specification consistency.
Revised: Hence, we rely on 5 samples in all other experiments as a trade-off of fast synthesis time and good specification consistency.

Original: Unsatifiable Specifications
Revised: Unsatifiable Specifications

Original: In practice, network operators may sometimes provide unsatisfiable specifications.
Revised: In practice, network operators may sometimes provide unsatisfiable specifications.

Original: While exact methods will typically return an error for such inputs, our relaxed setting enables us to consider partial solutions, i.e. configurations that still achieve high specification consistency while ignoring unsat requirements.
Revised: While exact methods will typically return an error for such inputs, our relaxed setting enables us to consider partial solutions, i.e. configurations that still achieve high specification consistency while ignoring unsat requirements.

Original: This may be preferable in some scenarios, especially when perfect specification consistency is not critical anyway.
Revised: This may be preferable in some scenarios, especially when perfect specification consistency is not critical anyway.

Original: To simulate this scenario, we evaluate specification consistency for OSPF-only synthesis tasks, which were all verified to be unsatisfiable using the SMT-based synthesizer NetComplete [15].
Revised: To simulate this scenario, we evaluate specification consistency for OSPF-only synthesis tasks, which were all verified to be unsatisfiable using the SMT-based synthesizer NetComplete [15].

Original: We still observe a comparatively high average consistency of 0 .
Revised: We still observe a comparatively high average consistency of 0 .

Original: 90 for our learned synthesizer.
Revised: 90 for our learned synthesizer.

Original: This suggests that our model is indeed capable of handling unsatisfiable specifications, while still producing good, partial solutions.
Revised: This suggests that our model is indeed capable of handling unsatisfiable specifications, while still producing good, partial solutions.

Original: For more detailed unsat results and methodology, see Appendix B.2.
Revised: For more detailed unsat results and methodology, see Appendix B.2.

Original: 5.2 Comparison to SMT-based Synthesis
Revised: 5.2 Comparison to SMT-based Synthesis

Original: We compare the synthesis time of our learned synthesizer with the SMT-based, state-of-art configuration synthesis tool NetComplete [15].
Revised: We compare the synthesis time of our learned synthesizer with the SMT-based, state-of-art configuration synthesis tool NetComplete [15].

Original: We compare BGP/OSPF and OSPF-only synthesis time.
Revised: We compare BGP/OSPF and OSPF-only synthesis time.

Original: For each topology/specification, we report the total time of running our synthesizer 4 times using 4-shot sampling, whereas for consistency we report the best out of the 4 runs.
Revised: For each topology/specification, we report the total time of running our synthesizer 4 times using 4-shot sampling, whereas for consistency we report the best out of the 4 runs.

Original: If a fully consistent configuration is found, we do not continue to sample and only report time until then.
Revised: If a fully consistent configuration is found, we do not continue to sample and only report time until then.

Original: For
Revised: For

Original: NetComplete, we time out at 25 minutes and report that as a lower bound if exceeded.
Revised: NetComplete, we time out at 25 minutes and report that as a lower bound if exceeded.

Original: We restrict our comparison to specifications that only include forwarding paths (i.e. fwd facts), as the type of supported requirements in NetComplete does not fully align with our model.
Revised: We restrict our comparison to specifications that only include forwarding paths (i.e. fwd facts), as the type of supported requirements in NetComplete does not fully align with our model.

Original: Table 3 shows that our synthesizer model outperforms NetComplete by multiple orders of magnitude.
Revised: Table 3 shows that our synthesizer model outperforms NetComplete by multiple orders of magnitude.

Original: We observe a speedup of 20 − 490 × that increases with the size of topology/specification.
Revised: We observe a speedup of 20 − 490 × that increases with the size of topology/specification.

Original: The loss of precision due to approximation remains moderate, with the average consistency of the synthesized configurations being greater than 92% even for large topologies.
Revised: The loss of precision due to approximation remains moderate, with the average consistency of the synthesized configurations being greater than 92% even for large topologies.

Original: In comparison, NetComplete times out on more than half of the synthesis tasks, which means that we only observe a lower bound for speedup.
Revised: In comparison, NetComplete times out on more than half of the synthesis tasks, which means that we only observe a lower bound for speedup.

Original: Running our model on a GPU can result in even greater speedups (up to 900 × given enough
Revised: Running our model on a GPU can result in even greater speedups (up to 900 × given enough

Original: GPU memory, cf.
Revised: GPU memory, cf.

Original: Appendix B.3).
Revised: Appendix B.3).

Original: Comparing OSPF-only synthesis time, our synthesizer model achieves a speedup of 2-10x over NetComplete.
Revised: Comparing OSPF-only synthesis time, our synthesizer model achieves a speedup of 2-10x over NetComplete.

Original: With 16 requirements on dataset L this can also increase up to 500 × .
Revised: With 16 requirements on dataset L this can also increase up to 500 × .

Original: Further, our model produces fully-consistent OSPF configurations even more often than for BGP/OSPF.
Revised: Further, our model produces fully-consistent OSPF configurations even more often than for BGP/OSPF.

Original: See Appendix B.4 for the full OSPF-only synthesis comparison.
Revised: See Appendix B.4 for the full OSPF-only synthesis comparison.

Original: 5.3 Discussion
Revised: 5.3 Discussion

Original: We have shown that our learning-based synthesizer reaches a very high degree of consistency, often producing fully-consistent configurations.
Revised: We have shown that our learning-based synthesizer reaches a very high degree of consistency, often producing fully-consistent configurations.

Original: With respect to synthesis time, we outperform SMT-based methods by a large margin, especially for larger topologies.
Revised: With respect to synthesis time, we outperform SMT-based methods by a large margin, especially for larger topologies.

Original: However, this comes at a price: our model sometimes fails to produce fully consistent configurations, especially for large topologies with many requirements.
Revised: However, this comes at a price: our model sometimes fails to produce fully consistent configurations, especially for large topologies with many requirements.

Original: In comparison, SMT-based synthesis will always produce fully consistent configurations if and once it completes.
Revised: In comparison, SMT-based synthesis will always produce fully consistent configurations if and once it completes.

Original: We, therefore, observe a trade-off between consistency and synthesis time.
Revised: We, therefore, observe a trade-off between consistency and synthesis time.

Original: We also note that our evaluation considers real topologies (Topology Zoo [29]) but not real specifications.
Revised: We also note that our evaluation considers real topologies (Topology Zoo [29]) but not real specifications.

Original: This is due to the lack of a large, practical dataset thereof.
Revised: This is due to the lack of a large, practical dataset thereof.

Original: Still, we experiment with robustness (Appendix D) by introducing distribution shift regarding the size of specifications during training and achieve comparable performance.
Revised: Still, we experiment with robustness (Appendix D) by introducing distribution shift regarding the size of specifications during training and achieve comparable performance.

Original: Scaling to Even Larger Topologies
Revised: Scaling to Even Larger Topologies

Original: The Topology Zoo [29] as used for our evaluation, provides a good range of realistically-sized networks.
Revised: The Topology Zoo [29] as used for our evaluation, provides a good range of realistically-sized networks.

Original: However, if we consider synthesis at very large scale (e.g. thousands of routers), we note the following scalability limitations: (1) Our models consume a lot of video memory, reaching beyond the amounts available on current consumer GPUs ( > 12 GB).
Revised: However, if we consider synthesis at very large scale (e.g. thousands of routers), we note the following scalability limitations: (1) Our models consume a lot of video memory, reaching beyond the amounts available on current consumer GPUs ( > 12 GB).

Original: This limit is reached with networks of 150 or more routers, and we have to do inference on the CPU (as indicated in Table 3).
Revised: This limit is reached with networks of 150 or more routers, and we have to do inference on the CPU (as indicated in Table 3).

Original: If even faster synthesis is important at this scale, more than one such GPU is needed for inference.
Revised: If even faster synthesis is important at this scale, more than one such GPU is needed for inference.

Original: Further, (2) the distance that information is propagated in the synthesizer
Revised: Further, (2) the distance that information is propagated in the synthesizer

Original: GNN is finite due to the model’s fixed number of iterations.
Revised: GNN is finite due to the model’s fixed number of iterations.

Original: This means that in very large networks, our synthesizer model will only be capable of deriving solutions by local reasoning which is very likely to impact synthesis quality.
Revised: This means that in very large networks, our synthesizer model will only be capable of deriving solutions by local reasoning which is very likely to impact synthesis quality.

Original: While out of the scope of this paper, longer training with more synthesizer iterations and larger topologies may be necessary to obtain comparable results at very large scale.
Revised: While out of the scope of this paper, longer training with more synthesizer iterations and larger topologies may be necessary to obtain comparable results at very large scale.

Original: Nonetheless, we envision a wide range of practically-relevant applications for fast, approximate synthesis, including ML-guided synthesis, unsatisfiable specifications, and hybrid synthesizers, leveraging both learning and SMT solvers.
Revised: Nonetheless, we envision a wide range of practically-relevant applications for fast, approximate synthesis, including ML-guided synthesis, unsatisfiable specifications, and hybrid synthesizers, leveraging both learning and SMT solvers.

Original: We list a number of future directions in Appendix C.
Revised: We list a number of future directions in Appendix C.

Original:  Related Work
Revised:  Related Work

Original: Traditional Configuration Synthesis Next to methods based on compilation such as
Revised: Traditional Configuration Synthesis Next to methods based on compilation such as

Original: Propane/PropaneAT
Revised: Propane/PropaneAT

Original: [5], there is a number of exact configuration synthesis methods based on constraint solving, e.g. ConfigAssure [31], SyNet [14] and NetComplete [15].
Revised: [5], there is a number of exact configuration synthesis methods based on constraint solving, e.g. ConfigAssure [31], SyNet [14] and NetComplete [15].

Original: These tools are typically hand-coded, very protocol-specific, and can be slow due to the solvers they employ.
Revised: These tools are typically hand-coded, very protocol-specific, and can be slow due to the solvers they employ.

Original: In contrast, our approach is approximate but scales to much larger networks.
Revised: In contrast, our approach is approximate but scales to much larger networks.

Original: Further, as a side-product of our learning-based approach, we can easily adapt to new protocols and allow for transparent cross-protocol reasoning, merely by training on different protocol data.
Revised: Further, as a side-product of our learning-based approach, we can easily adapt to new protocols and allow for transparent cross-protocol reasoning, merely by training on different protocol data.

Original: GNNs and Networking DeepBGP [2] relies on GNNs and reinforcement learning to do configuration synthesis.
Revised: GNNs and Networking DeepBGP [2] relies on GNNs and reinforcement learning to do configuration synthesis.

Original: However, it is limited to BGP configuration and is slower than SMT-based synthesis.
Revised: However, it is limited to BGP configuration and is slower than SMT-based synthesis.

Original: In contrast, our learning-based framework is cross-protocol, does not rely on reinforcement learning and provides better synthesis times.
Revised: In contrast, our learning-based framework is cross-protocol, does not rely on reinforcement learning and provides better synthesis times.

Original: Apart from configuration synthesis, GNNs have also been applied to other problems in the networking domain.
Revised: Apart from configuration synthesis, GNNs have also been applied to other problems in the networking domain.

Original: For example, the authors of RouteNet [34] use GNNs to predict networking performance metrics.
Revised: For example, the authors of RouteNet [34] use GNNs to predict networking performance metrics.

Original: Other work focuses on learning improved protocols like
Revised: Other work focuses on learning improved protocols like

Original: Q-Routing [8] and Graph-Query Neural Networks [18].
Revised: Q-Routing [8] and Graph-Query Neural Networks [18].

Original: Neural Algorithmic Reasoning NAR [41] refers to the idea of replacing algorithms with neural networks to learn improved algorithmic procedures.
Revised: Neural Algorithmic Reasoning NAR [41] refers to the idea of replacing algorithms with neural networks to learn improved algorithmic procedures.

Original: Successful applications include graph algorithms [43], combinatorial optimization problems [10, 26, 37] and multi-task settings [23].
Revised: Successful applications include graph algorithms [43], combinatorial optimization problems [10, 26, 37] and multi-task settings [23].

Original: Our synthesis framework is the first application of NAR to the networking domain and relies on the NAR-native encode-process-decode architecture.
Revised: Our synthesis framework is the first application of NAR to the networking domain and relies on the NAR-native encode-process-decode architecture.

Original: In the hierarchy of NAR approaches in [10], our method is an algorithm-level approach, as we do not supervise on intermediate steps.
Revised: In the hierarchy of NAR approaches in [10], our method is an algorithm-level approach, as we do not supervise on intermediate steps.

Original: Although step-level methods promise better generalization, it is not clear what an intermediate result of a general synthesis procedure would be.
Revised: Although step-level methods promise better generalization, it is not clear what an intermediate result of a general synthesis procedure would be.

Original: Future work on a step-level approach may further improve our model.
Revised: Future work on a step-level approach may further improve our model.

Original:  Conclusion
Revised:  Conclusion

Original: We presented a learning-based method to enable approximate but scalable network configuration synthesis.
Revised: We presented a learning-based method to enable approximate but scalable network configuration synthesis.

Original: For BGP/OSPF routing, our neural synthesizer is up to 490 × faster than SMT-based methods, while producing configurations with very high specification consistency.
Revised: For BGP/OSPF routing, our neural synthesizer is up to 490 × faster than SMT-based methods, while producing configurations with very high specification consistency.

Original: We believe there are future research that can be explored in the direction of learning-based synthesis and ML-guided network configuration.
Revised: We believe there are future research that can be explored in the direction of learning-based synthesis and ML-guided network configuration.

Original: Ethical Issues
Revised: Ethical Issues

Original: This work does not raise any ethical issues.
Revised: This work does not raise any ethical issues.

Original: Acknowledgments This work was partially supported by an ETH Research Grant ETH-03 19-2.Checklist The checklist follows the references.
Revised: Acknowledgments This work was partially supported by an ETH Research Grant ETH-03 19-2.

Original: Please read the checklist guidelines carefully for information on how to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] , or [N/A] .

Original: You are strongly encouraged to include a justification to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.

Original: For example:

Original: • Did you include the license to the code and datasets?

Original: [Yes] We plan to publish our code and dataset under an open source license and make it available to others.

Original: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the

Original: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [N/A] (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]

Original: Appendix A Implementation and Model Details A.1 Structural Fact Base Embedding

Original: To transform a fact base into a graph with node features, we employ a structurally-defined embedding scheme.

Original: Given a fact base F , we apply the rules in Figure 6 to obtain node features and adjacency information.

Original: In the resulting graph, both a fact f and a constant c are represented as distinct nodes with features h c and h f , respectively.

Original: The relationships between facts and constants is encoded as node adjacency as defined by the neighborhood functions

Original: N i .

Original: We use multiple neighborhood functions, i.e. multiple types of edges, to encode the argument position of a constant occurring in a fact.

Original: Hence, the number of neighborhood functions corresponds to the maximum number of arguments in a single fact, e.g. 3 in Figure 4.

Original: Integer arguments and unary facts are directly accounted for in the respective node or fact embedding.

Original: To handle unknown parameters, we replace the corresponding emb ( ... ) term with the learned V hole embedding.

Original: Considering the embedding function emb , the learned parameters of our embedding are V f ∈ R D per fact type in

Original: F , W bool ∈ R D × 2 for boolean values, W f i ∈

Original: R D × N per integer argument of fact type f and V hole ∈ R D to represent unknown parameters.

Original: Hyperparameter D represents the dimension of the latent space the synthesizer model operates in and N specifies the number of supported integer values.

Original: A.2 Graph Attention Layer

Original: The recurrence relation of a node j ’s representation h j is defined as:

Original: [Equation]

Original: h ′ GAT i represents the output of a graph attention layer operating with neighborhood

Original: N i according to [42], h j the representation of node j at the previous step.

Original: Norm refers to batch normalization [24],

Original: Drop to dropout regularization [22] and Lin 1 and Lin 2 to linear layers with an inner dimension of 4 N .

Original: The mechanic of combining the results of multiple graph attention layers h

Original: ′ GAT i directly corresponds to how multiple attention heads are combined in the original Transformer architecture [40].

Original: [Equation]

Original: (learned fact type embedding)

Original: [Equation]

Original: (learned boolean embedding)

Original: [Equation]

Original: (learned integer embedding)

Original: [Equation]

Original: (learned hole embedding)

Original: [Equation]

Original: (constant embedding)

Original: [Equation]

Original: (fact embedding)

Original: [Equation]

Original: (integer arguments)

Original: [Equation]

Original: (constant node neighbors)

Original: [Equation]

Original: (fact node neighbors)

Original: A.3 Dataset Generation and Training

Original: To train a synthesizer model, we need a large number of fact bases encoding a variety of network topologies, configurations and forwarding specifications.

Original: Further, we must provide the model with target values for unknown parameters as a supervision signal.

Original: This section discusses how we construct such a dataset for BGP/OSPF synthesis and train a corresponding synthesis model.

Original: Dataset Generation To obtain network configurations and corresponding forwarding specifications we employ a generative process: We first randomly configure BGP and OSPF parameters for some topology.

Original: Then we simulate the OSPF and BGP protocol and compute the resulting forwarding plane.

Original: Based on this, we extract a forwarding specification which fixes a random subset of forwarding paths, reachability and isolation properties that are satisfied by the forwarding plane.

Original: Topologies For training, our dataset is based on random topologies with 16 - 24 routers.

Original: We generate the physical layout of these graphs by triangulation of uniformly sampled points in two-dimensional space.

Original: Simulation To compute the forwarding plane given an OSPF and BGP configuration, we simulate the protocols.

Original: For OSPF, we implement shortest path computation to obtain the forwarding plane.

Original: For BGP, we implement the full BGP decision process as shown in ??

Original: , except for the MED attribute.

Original: In addition to the basic networking model discussed in Section 3, we implement support for eBGP and iBGP, fully-meshed BGP session layouts as well as layouts relying on route reflection.

Original: For more details of these BGP concepts see [3].

Original: Fact Base Encoding We encode synthesis input using a small set of facts.

Original: For an example of a corresponding fact base see Figure 7.

Original: We use the router , external and network facts to mark routers, external peers and routing destinations respectively.

Original: Physical links between routers are encoded as connected(R1,R2,W) facts where W denotes the OSPF link weight.

Original: Further, we rely on ibgp , ebgp , route_reflector and bgp_route facts to represent different types of BGP sessions and imported routes.

Original: Using the notion of unknown parameters as discussed in Section 4.2, we encode configuration parameters, e.g., properties of imported routes or OSPF link weights.

Original: Specification Language We support a small specification language with three types of forwarding predicates:

Original: • fwd(R1, Net, R2) specifies that router R1 forwards traffic destined for Net to its neighbor R2 .

Original: • reachable(R1, Net, R2) specifies that traffic destined for Net that passes through R also passes through R2 before reaching its destination.

Original: • trafficIsolation(R1, R2, N1, N2) specifies that only one of [ fwd(R1,N1,R2) ]

Original: B and [ fwd(R1,N2,R2) ]

Original: B can be true at a time.

Original: In the negative case of each predicate, the opposite must hold true for the fact to be satisfied.

Original: This specification language can easily be extended by including new specification predicates in the dataset generation process.

Original: One must simply provide a method of extracting positive and negative cases of a specification fact from a given forwarding plane.

Original: Once included in the training dataset of a synthesizer model, the resulting synthesis system will be able to consider specifications relying on the new type of predicates.

Original: Training We generate a dataset of 10 , 240 input/output samples and instantiate our synthesizer model with a hidden dimension D = 64 and supported parameter values N = 64 .

Original: For optimization, we use the Adam optimizer [27] with a learning rate of 10 − 4 .

Original: We stop training after the specification consistency on a validation dataset no longer increases at ∼ 2800 epochs.

Original: [Table]

Original: [Figure]

Original: BGP Decision Process For completeness, we include the BGP decision process as assumed for our synthesis setting.

Original: The shown rules are applied in order until a single best BGP announcements remains.

Original: Highest Local preference 2.

Original: Lowest AS path length 3.

Original: Prefer Origin (IGP > EBGP > INCOMPLETE) 4.

Original: Lowest Multi-exit discriminator (MED) 5.

Original: External over internal announcements 6.

Original: Lowest IGP cost to egress 7.

Original: Lowest BGP peer id (tie breaker)

Original: Number of Samples

Original: A v e r a g e S p ec i fi ca ti on C on

Original: s i s t e

Original: n c y

Original: Small

Original: Medium

Original: Large

Original: B More Evaluation Results B.1 Number Of Samples

Original: Figure 8 shows the average best consistency across our three datasets with an increasing number of samples for 3 × 16 BGP/OSPF requirements.

Original: As the number of samples increases, the graph indicates the best consistency value reached so far.

Original: We observe that sampling more than once improves the average specification consistency across all datasets.

Original: With increasing size of the topology, more samples appear to be necessary for the resulting best specification consistency to converge.

Original: Therefore, we note that increasing the number of samples can improve consistency but it will also affect overall synthesis time as the model needs to be executed again for each sample.

Original: In our other experiments we rely on 4/5 samples per specification, which we consider a good trade-off of fast synthesis time and good specification consistency.

Original: B.2 Unsatisfiable Specifications

Original: To asses how our model performs in the presence of unsatisfiable specifications, we evaluate its synthesis performance on multiple datasets of unsatisfiable OSPF synthesis tasks.

Original: Specifications We construct the datasets U NSAT - N , where N ∈ { 2 , 4 , 6 , 8 , 10 , 12 , 14 , 16 } , by first generating 16 solvable OSPF forwarding path requirements per topology in Small, Medium and Large.

Original: Then we replace N of the requirements per topology with paths obtained under other, random link weights.

Original: We repeat this process up to five times, until we can verify that all variants of a topology as contained in the different U NSAT - N datasets are indeed unsatisfiable using the SMT-based synthesizer

Original: NetComplete [15].

Original: Overall, this leaves us with 8 U NSAT - N datasets of OSPF synthesis tasks, where we expect the maximum achievable specification consistency to decrease with increasing N .

Original: For some topologies, we were not successful in generating unsatisfiable variants for all N and therefore we removed them from the considered set of topologies for this part of our evaluation.

Original: We counttopologies per U NSAT - N dataset, where the number of nodes ranges between 16 and 153.

Original: Methodology For synthesis, we use the same sampling configuration as in Section 5.1 where we again report the best consistency value across 5 different runs of our synthesis model per synthesis task.

Original: We further configure our trained synthesizer model to do OSPF synthesis by only predicting link weights.

Original: Results We report the results of applying our synthesizer model to unsatisfiable OSPF specifications in Figure 9.

Original: For comparison, we also include the results of applying our synthesizer model to an

Original: U NSAT - 0 dataset, i.e. a dataset of satisfiable OSPF synthesis tasks.

Original: The average specification consistency drops with unsatisfiable specifications which is expected since the theoretical upper bound for specification consistency is lower with conflicting requirements.

Original: At the same time, specification consistency remains high, which suggests that our model still attempts to maximize specification consistency, even when provided with an unsatisfiable specification with many conflicting requirements.

Original: U NSAT - N datasets

Original: A v e r a g e S p ec i fi ca ti on C on

Original: s i s t e

Original: n c y

Original: In contrast, SMT-based synthesizers like NetComplete will not be able to produce any configurations for the synthesis tasks in our U NSAT - N datasets.

Original: B.3 BGP/OSPF Synthesis Time (GPU)

Original: In the following table we report BGP/OSPF consistency and synthesis time of our method (Neural) running on a GPU, compared with the SMT-based state-of-art synthesis tool NetComplete.

Original: The notation n/8TO indicates the number of timed out synthesis runs out of 8 (25+ minutes).

Original: Dataset Large is omitted due to GPU memory restrictions.

Original: BGP Synthesis Time (GPU, 5 samples)

Original: Synthesis Time (s)

Original: Accuracy (Neural) # Reqs.

Original: NetComplete

Original: Neural (GPU)

Original: Speedup ∅ Consistency ∅ No.

Original: Full Matches 2 reqs.

Original: S

Original: [Table]

Original: [Table]

Original: [Table]

Original: 44s

Original: [Table]

Original: 41.1x 1.

Original: [Table]

Original: 8/8 M 60.

Original: [Table]

Original: [Table]

Original: [Table]

Original: ± 0.

Original: 81.8x 0.

Original: [Table]

Original: 6/ 8 reqs.

Original: S

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: 7x

Original: [Table]

Original: 5/8 M >25m 8/8TO

Original: [Table]

Original: 23s

Original: [Table]

Original: [Table]

Original: 6x 0.

Original: [Table]

Original: 4/ 16 reqs.

Original: S 1416.83 s ± 235.25 7/8TO 1.55s ± 0.

Original: 913.3x 0.92 ± 0.

Original: 1/8 M >25m 8/8TO 1.65s ± 0.

Original: 906.5x 0.93 ± 0.

Original: B.4 OSPF Synthesis Time

Original: We also compare consistency and synthesis time of our method (Neural) with the SMT-based synthesis tool NetComplete doing OSPF-only synthesis.

Original: The notation n/8TO indicates the number of timed out synthesis runs out of 8 (25+ minutes).

Original: For this experiment, our neural synthesizer can run on the

Original: GPU, as OSPF-only synthesis consumes less GPU memory.

Original: Below, we report results of OSPF-only synthesis sampling from the synthesizer model 4 times andtimes respectively.

Original: We observe that sampling 4 times can be enough with smaller specifications to obtain full matches for all synthesis tasks.

Original: For larger specifications and networks, sampling more than 4 times can improve average consistency.

Original: Note however, that sampling 5 times for small specifications and topologies can lead to synthesis times that are longer than with NetComplete (0.7x with 2 reqs., 5 samples, dataset S).

Original: [Table]

Original: C Future Research Directions

Original: The key challenge with our learning-based system remains its imprecision, i.e. that it produces only partially-consistent configurations in some cases.

Original: On one hand, parts of the configuration synthesis problem are NP-hard [7, 16, 44], which means that practical and scalable synthesis tools have to compromise on precision unless P=NP.

Original: At the same time, satisfying the complete specification may be crucial, depending on the scenario.

Original: Therefore, our learning-based synthesis will not be applicable in the same way as precise, SMT-based synthesizers.

Original: Based on this insight, we envision a whole range of future directions for which imprecise, learning-based synthesis will be very useful:

Original: ML-assisted Configuration Imprecise synthesis may be used to enable ML-assisted configuration in the form of a semi-automated process: users provide a specification, apply the synthesizer and perform additional tweaking after they obtain a sufficiently consistent configuration.

Original: Optionally, our synthesizer model can also be applied again to predict alternative values for some of the configuration parameters, by masking only the desired parameters in a configuration (cf.

Original: Section 4.2).

Original: Recently, similar systems for code completion, like GitHub CoPilot [11, 20], have demonstrated that this query-predict-modify workflow can be highly effective at combining an imprecise ML-guided recommendation system with user interaction [13].

Original: Hybrid Systems Given that further tweaking of a configuration may be necessary, future work may also explore seeding SMT-based, partial synthesis methods [15] or configuration repair methods [17] with generated candidate solutions.

Original: This can further automate the process of obtaining fully consistent configurations while maintaining some of the performance benefits of a learning-based system.

Original: Existing work on partial synthesis has already shown that SMT-based synthesis can be much faster when a rough sketch of a configuration is already provided as an input [15].

Original: Imprecision in SMT-based synthesis Tweaking the result of a synthesizer is undesirable, but it is also a common practice with existing SMT-based synthesis, as it can be imprecise too.

Original: This can be caused by hand-coded SMT synthesis rules which do not always hold up in practice, given the complex behavior of real-world routers [6].

Original: Similarly, our learning-based method is imprecise, but could actually be trained on real-world data, possibly bridging the gap between an assumed formal model and real-world behavior.

Original: Future work is necessary to clarify how the configuration tweaking process differs in practice between SMT-based and learning-based synthesis.

Original: Optimality with Unsatisfiable Specifications As demonstrated in Appendix B.2, our learningbased system produces highly consistent configurations even in the presence of an unsatisfiable specification.

Original: In practice, this can be helpful as operators do not know whether their specification is unsatisfiable ahead of time.

Original: In these cases, a partially consistent configuration is the best one can hope for.

Original: In contrast, using SMT solvers in this scenario will not produce a configuration at all, while unsatisfiable specifications can be quite difficult to debug.

Original: Service Level Agreements Lastly, the notion of relaxed specification consistency is not entirely unbeknownst to the world of networking.

Original: Networks often operate in accordance with so-called

Original: Service Level Agreements (SLA), contracts that specify the guaranteed properties of service in terms of relative availability over time.

Original: Given this concept of SLAs, partially violated specifications in some scenarios can be tolerable, as explored by previous work like [38].

Original: D Additional Experiments

Original: In response to the reviews we carried out a number of additional experiments.

Original: We will integrate these results into our draft where applicable.

Original: D.1 Ablation and Parameter Study

Original: We conducted an ablation study to explore both the effectiveness of our architecture as well as the use of different components (e.g. different GNN layer modules).

Original: We compare across the following configurations.

Original: Configurations marked with (*) correspond to the configuration presented in the main body of the paper.

Original: Noise and Edge Types

Original: N O N OISE corresponds to our model without adding gaussian noise before applying the processor network.

Original: N O E DGE T YPES corresponds to our model with only one edge type.

Original: N OISE E DGE T Y (*) correspond to our model as presented in the paper.

Original: GNN Modules We also experiment with different GNN modules used internally by the encoder and processor network (cf.

Original: Section 4.3).

Original: In addition to a graph attention module (GAT, [42]), we also evaluate models that employ a simple message-passing layer (MPNN-max, [19]) and a graph convolutional layer (GCN-max, [28]), both aggregating by maximization.

Original: Hidden Dimensionality We also experiment with different values ( 16 , 32 , 64 and 128 ) for dimensionality D of the synthesizer model.

Original: Separate Latent Spaces

Original: According to our graph-based encoding, we embed information on topology, configurations and specification all in a common latent space.

Original: However, we also experiment with embedding the topology and configuration information into three separate latent spaces.

Original: For this, we train models with three different pairs of encoder+processor networks, one per type of facts/nodes relating to topology, configuration and specification respectively.

Original: At each layer and synthesizer iteration, the different encoder/processor GNNs can attend to the intermediate representations of the adjacent nodes according to the underlying fact base graph.

Original: [-As this induces a threefold increase in parameters, we compare the results of such synthesizer models S EP S PACE -16, S EP S PACEand our model C OM S PACE -
-]
Revised: [-As this induces a threefold increase in parameters, we compare the results of such synthesizer models S EP S PACE -16, S EP S PACEand our model C OM S PACE -
-]

Original: Due to the long training time of these models, we compare after training these configuration for 2000 epochs only.

Original: Sampling, Dataset and Metrics For the examined configurations, we train a synthesizer model using the same training setup as described in A.3.

Original: Using the resulting models, we perform synthesis for all datasets S, M, and L, relying on 5 samples per task.

Original: For N O N OISE and N OISE E DGE T Y (*) we additionally perform synthesis for the same datasets, but with 20 samples.

Original: Here, the test datasets

Original: S, M and L include all generated synthesis tasks of the specification size classes 3x2, 3x8 and 3x16 as used in other parts of our evaluation.

Original: Overall, this leads to 24 synthesis tasks on 8 different topologies per dataset S/M/L. We measure average best specification consistency, as well as number of full (Full) and partial, good ( > 90% ) matches in terms of specification consistency.

Original: Results

Original: The results of our experiments are provided in Table 4 and Table 5.

Original: For supporting multiple edge types in N OISE E DGE T Y , we observe a clear benefit over N O E DGE T YPES , both with respect to average best consistency as well as the total number of full and good matches.

Original: With just 5 samples, the effect of N O N OISE is not very pronounced.

Original: However when increasing the number times we sample from the synthesizer model per task, we observe a clear performance improvement.

Original: When sampling up to 20 times from the model, N OISE E DGE T Y (*) outperforms N O N OISE in almost all metrics (cf.

Original: Table 5), leading to higher average consistency and a higher number of full and good matches.

Original: We hypothesize that adding noise helps the model in producing a wider variety of solutions, eventually leading to better results when selecting the best one.

Original: Regarding the choice of GNN module, we observe that GAT is the most effective, but MPNN-max can also be a good choice.

Original: Regarding the choice of hidden dimension, we observe that 64 is a good balance of performance and memory usage, while 128 only brings slight improvements.

Original: For our purposes we selected D = 64 , since this can improve synthesis time significantly when running on a GPU (cf.

Original: Appendix B.3).

Original: For

Original: [Table]

Original: [Table]

Original: D = 128 this may not always be possible, especially when limited to

Original: GPU

Original: Lastly, we cannot observe a significant benefit of separating the latent spaces of topology, configuration and specification as described above using multiple encoder+processer networks.

Original: S EP S PACEappears to perform mostly on par with our C OM S PACE -64 configuration, while having significantly more parameters.

Original: [-S EP S PACE -16 is more comparable in terms of the number of parameters, but it performs worse than C OM S PACE -
-]
Revised: [-S EP S PACE -16 is more comparable in terms of the number of parameters, but it performs worse than C OM S PACE -
-]

Original: in most metrics.

Original: D.2 Dataset Statistics and Distribution Shift

Original: To examine the synthesis performance of our model with real topologies, we source our evaluation datasets from the Topology Zoo [29], a collection of real-world topologies.

Original: For training on the other hand, we only rely on synthetic data based on random topologies, configurations and specifications that are easy to generate by simulation (cf. A.3).

Original: As a consequence, we observe distribution shift between the synthetic data we train on and the closer-to-real-world data we evaluate on.

Original: Training and Evaluation Datasets Figure 10 illustrates parts of the distribution shift with respect to the number of nodes and specifications size.

Original: Our synthetic training dataset (Training Dataset) only contains samples of very limited size (15-25 nodes) with ∼ 20-40 different specification predicates per traffic class.

Original: In contrast, our Topology Zoo datasets used for evaluation contain topologies of much larger size (5-153) and larger specifications ( ∼ 1-60 predicates per traffic class).

Original: Considering this distribution shift and the good performance of learned synthesis on the evaluation datasets, our model appears to generalize well to larger topologies/specifications, even without having seen similarly-sized problem instances during training.

Original: Comparably strong generalization properties have also been previously observed with other NAR-based models [43].

Original: Specification Distribution Shift (SmallSpec)

Original: Our evaluation datasets rely on real topologies but have to resort to synthetic specifications due to the lack of a large, practical dataset in this space.

Original: Nonetheless, we want to provide some preliminary insight into the possible effect of a specification

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: distribution shift, as would be the case when applying our model to real-world synthesis tasks.

Original: For this, we construct a SmallSpec dataset (cf.

Original: Figure 10) which is similar to our training dataset but with smaller specifications.

Original: In Table 6 we report the results of training a synthesizer model on the

Original: SmallSpec dataset and comparing its performance with a model trained on our regular Training

Original: Dataset.

Original: Both models are evaluated on the same Topology Zoo evaluation datasets as in our evaluation.

Original: The resulting SmallSpec model achieves slightly lower but comparable synthesis performance.

Original: This provides some evidence regarding the robustness of our models when it comes to a distribution shift of the specifications used for synthesis.

Original: D.3 Varying Number of Samples

Original: In a previous revision of the paper, our evaluation relied on 5 samples to obtain consistency results and only 4 to determine synthesis times.

Original: In practice, this can be a sensible choice, trading off accuracy for speed.

Original: However, to rectify this discrepancy in our results, we carried out the respective dual experiments to also determine average synthesis times and consistency results with 5 and 4 samples respectively.

Original: The experiments in the main body of this revision all rely on 5 samples.

Original: Consistency We compare synthesis performance when sampling 4/5 times from our synthesis model.

Original: Below we list the results for average best consistency, number of full matches as well as the number of > 90% matches for each of the datasets S, M,

Original: L.

Original: The results correspond directly to the results in column "Overall" in Table 2, where we report the results for sampling 5 times.

Original: [Table]

Original: [Table]

Original: These results suggest that a higher number of samples can increase average consistency as well as the number of full and good, partial matches.

Original: Especially small specifications (cf. 3x2), seem to benefit from more samples.

Original: Overall, however there is not a very large difference between sampling 4/5 times.

Original: Synthesis Time We also compare synthesis time of sampling 4/5 times below.

Original: We report the synthesis time of running our synthesizer model with 4 samples as reported in the paper.

Original: Next, we report synthesis time of running our synthesizer model with 5 samples.

Original: Last, we report the speedup over NetComplete when running our synthesizer model with 4 and 5 samples, respectively.

Original: [Table]

Original: [Table]

Original: Overall, running our synthesizer model with 5 samples means that the model is invoked one more time per synthesis task.

Original: This is clearly reflected by the resulting synthesis times.

Original: However, as the number of samples is only a linear factor for overall synthesis time, the speedup over NetComplete remains very significant.