Patent Publication Number: US-2022239596-A1

Title: Dynamic sd-wan hub cluster scaling with machine learning

Description:
BACKGROUD 
     Today, SD-WAN hub-clustering is a networking mechanism akin to server load balancing. The hub clusters reside in datacenters and allow branch sites to access resources of the datacenter by connecting to one of the hubs of the cluster in a load-balanced fashion. Determining the number of hubs to be included in the cluster, however, remains a challenge faced by network designers, as the determination is based on multiple factors, such as the number of branches and network usage patterns of those branches. As a result, network operators tend to over-provision hubs, sometimes by as much as a factor of two. The lack of an easy solution can result in unsatisfied and frustrated customers who are increasingly using cluster solutions in cloud deployments and IaaS (infrastructure as a service). 
     BRIEF SUMMARY 
     Some embodiments of the invention provide a method of dynamically scaling a hub cluster in a software-defined wide area network (SD-WAN) based on current and historical traffic statistics. The SD-WAN includes a datacenter in which the hub cluster is located, a controller, and multiple branch sites that connect to the hub cluster in order to access resources of the datacenter. In addition to providing access to resources of the datacenter, the hub cluster captures and sends traffic statistics relating to flows to and from the branch sites, and provides these traffic statistics to the controller of the SD-WAN. The controller analyzes the received traffic statistics to identify traffic load fluctuations, and determines that a number of hubs in the hub cluster should be adjusted based on the identified fluctuations. The controller then adjusts the number of hubs in the hub cluster based on the determination. 
     In some embodiments, each branch site includes an edge forwarding node for sending and receiving traffic flows between the branch site and other elements of the SD-WAN (e.g., between the branch site and the hub cluster). In addition to the traffic statistics received from the hub cluster, the controller in some embodiments also receives additional traffic statistics from the edge forwarding nodes at the branch sites. In some embodiments, the additional traffic statistics received from the edge forwarding nodes include more fine-grained statistics compared to the traffic statistics received from the hub cluster. 
     The controller, in some embodiments, includes a traffic statistics storage, a learning engine, and a decision-making and evaluation engine. In some embodiments, the traffic statistics storage stores historical traffic statistics (i.e., traffic statistics received over time). The historical traffic statistics and the real-time traffic statistics received by the controller are used as input for the learning engine, according to some embodiments. For example, the real-time traffic statistics of some embodiments are input into the learning engine to identify real-time load fluctuations (e.g., fluctuations happening at short time scales), while the historical traffic statistics in addition to the current traffic statistics are input into the learning engine to identify cyclical load fluctuations (e.g., daily morning/night fluctuations) and seasonal load fluctuations (e.g., fluctuations during peak business operations). By identifying these fluctuations, the learning engine and decision making and evaluation engine of some embodiments are able to adapt and adjust the hub cluster size in response to real-time, cyclical, and seasonal load fluctuations. 
     In some embodiments, the traffic statistics (both historical and real-time) are divided into two sets. The first set includes flow traffic load statistics that form the traffic parameters used by the learning engine to identify the fluctuations described above, while the second set includes flow performance statistics that form the evaluation parameters of the decision-making and evaluation engine to determine whether to adjust the hub cluster size. Flow traffic load statistics, in some embodiments, include at least the number of branch sites in the SD-WAN, the rate of flow creation per branch site, the total number of flows per branch site, flows further categorized into applications, average bandwidth per flow, total number of fat flows per branch, and average bandwidth per fat flow. The flow performance statistics in some embodiments include at least packet re-transmits per flow, packet drops per flow, and mean opinion score (MOS) of real-time flows (e.g., VoIP calls). 
     In order to determine that a number of hubs in the hub cluster should be adjusted based on the identified fluctuations (e.g., real-time, cyclical, or seasonal fluctuations), the decision making and evaluation engine in some embodiments uses the identified fluctuations to predict the traffic load over a certain time interval (e.g., the next three hours) and to determine the hub cluster size that can handle the predicted load with the expected flow performance (e.g., based on prior tests on sizing of the hub cluster at different traffic loads). Based on the different between the current hub cluster size and required cluster size, the decision making and evaluation engine decides whether to scale the cluster up (i.e., add hubs) or to scale the cluster down (i.e., remove hubs). 
     In some embodiments, in addition to deciding whether to scale the cluster up or down, the decision making and evaluation engine evaluates the learning engine&#39;s performance by comparing the predicted traffic load and the actual traffic load. This comparison, in some embodiments, can trigger a re-training of the learning engine if the engine&#39;s predictions start to deviate significantly from the actual values. In some embodiments, re-training can also be triggered after a certain time-interval to account for new data. 
     The preceding Summary is intended to serve as a brief introduction to some embodiments of the invention. It is not meant to be an introduction or overview of all inventive subject matter disclosed in this document. The Detailed Description that follows and the Drawings that are referred to in the Detailed Description will further describe the embodiments described in the Summary as well as other embodiments. Accordingly, to understand all the embodiments described by this document, a full review of the Summary, the Detailed Description, the Drawings, and the Claims is needed. Moreover, the claimed subject matters are not to be limited by the illustrative details in the Summary, the Detailed Description, and the Drawings. 
    
    
     
       BRIEF DESCRIPTION OF FIGURES 
       The novel features of the invention are set forth in the appended claims. However, for purposes of explanation, several embodiments of the invention are set forth in the following figures. 
         FIG. 1  conceptually illustrates an example of an SD-WAN in some embodiments. 
         FIG. 2  conceptually illustrates an example of the machine-learning model of some embodiments. 
         FIGS. 3A-3B  conceptually illustrate an example of an SD-WAN before and after a machine-learning model has adjusted the hub cluster based on identified load fluctuations, according to some embodiments. 
         FIG. 4  illustrates a process performed by the controller in some embodiments to identify real-time load fluctuations. 
         FIG. 5  illustrates a process performed by the controller in some embodiments to identify cyclical load fluctuations. 
         FIG. 6  illustrates a process performed by the controller in some embodiments to identify seasonal load fluctuations. 
         FIG. 7  illustrates a process performed by the controller to evaluate the performance of the machine-learning model, in some embodiments. 
         FIG. 8  conceptually illustrates a computer system with which some embodiments of the invention are implemented. 
     
    
    
     DETAILED DESCRIPTION 
     In the following detailed description of the invention, numerous details, examples, and embodiments of the invention are set forth and described. However, it will be clear and apparent to one skilled in the art that the invention is not limited to the embodiments set forth and that the invention may be practiced without some of the specific details and examples discussed. 
     Some embodiments of the invention provide a method of dynamically scaling a hub cluster in a software-defined wide area network (SD-WAN) based on current and historical traffic statistics. The SD-WAN includes a datacenter in which the hub cluster is located, a controller, and multiple branch sites that connect to the hub cluster in order to access resources of the datacenter. In addition to providing access to resources of the datacenter, the hub cluster captures and sends traffic statistics relating to flows to and from the branch sites, and provides these traffic statistics to the controller of the SD-WAN. The controller analyzes the received traffic statistics to identify traffic load fluctuations, and determines that a number of hubs in the hub cluster should be adjusted based on the identified fluctuations. The controller then adjusts the number of hubs in the hub cluster based on the determination. 
     In some embodiments, each branch site includes an edge forwarding node for sending and receiving traffic flows between the branch site and other elements of the SD-WAN (e.g., between the branch site and the hub cluster). In addition to the traffic statistics received from the hub cluster, the controller in some embodiments also receives additional traffic statistics from the edge forwarding nodes at the branch sites. In some embodiments, the additional traffic statistics received from the edge forwarding nodes include more fine-grained statistics compared to the traffic statistics received from the hub cluster. 
     The controller, in some embodiments, includes a traffic statistics storage, a learning engine, and a decision-making and evaluation engine. In some embodiments, the traffic statistics storage stores historical traffic statistics (i.e., traffic statistics received over time). The historical traffic statistics and the real-time traffic statistics received by the controller are used as input for the learning engine, according to some embodiments. For example, the real-time traffic statistics of some embodiments are input into the learning engine to identify real-time load fluctuations (e.g., fluctuations happening at short time scales), while the historical traffic statistics in addition to the current traffic statistics are input into the learning engine to identify cyclical load fluctuations (e.g., daily morning/night fluctuations) and seasonal load fluctuations (e.g., fluctuations during peak business operations). By identifying these fluctuations, the learning engine and decision making and evaluation engine of some embodiments are able to adapt and adjust the hub cluster size in response to real-time, cyclical, and seasonal load fluctuations. 
     In some embodiments, the traffic statistics (both historical and real-time) are divided into two sets. The first set includes flow traffic load statistics that form the traffic parameters used by the learning engine to identify the fluctuations described above, while the second set includes flow performance statistics that form the evaluation parameters of the decision-making and evaluation engine to determine whether to adjust the hub cluster size. Flow traffic load statistics, in some embodiments, include at least the number of branch sites in the SD-WAN, the rate of flow creation per branch site, the total number of flows per branch site, flows further categorized into applications, average bandwidth per flow, total number of fat flows per branch, and average bandwidth per fat flow. The flow performance statistics in some embodiments include at least packet re-transmits per flow, packet drops per flow, and mean opinion score (MOS) of real-time flows (e.g., VoIP calls). 
     In order to determine that a number of hubs in the hub cluster should be adjusted based on the identified fluctuations (e.g., real-time, cyclical, or seasonal fluctuations), the decision making and evaluation engine in some embodiments uses the identified fluctuations to predict the traffic load over a certain time interval (e.g., the next three hours) and to determine the hub cluster size that can handle the predicted load with the expected flow performance (e.g., based on prior tests on sizing of the hub cluster at different traffic loads). Based on the different between the current hub cluster size and required cluster size, the decision making and evaluation engine decides whether to scale the cluster up (i.e., add hubs) or to scale the cluster down (i.e., remove hubs). 
     In some embodiments, in addition to deciding whether to scale the cluster up or down, the decision making and evaluation engine evaluates the learning engine&#39;s performance by comparing the predicted traffic load and the actual traffic load. This comparison, in some embodiments, can trigger a re-training of the learning engine if the engine&#39;s predictions start to deviate significantly from the actual values. In some embodiments, re-training can also be triggered after a certain time-interval to account for new data. 
       FIG. 1  illustrates an example embodiment of an SD-WAN (also referred to herein as a virtual network) for connecting multiple branch sites to each other and to a controller and at least one datacenter hub. As shown, the SD-WAN  100  includes a controller  110 , three branch sites  120 - 124  that each include an edge forwarding node  130 - 134  (also referred herein as edge nodes or nodes), a cloud gateway  140 , and a datacenter  150  with a hub  145 . 
     The edge nodes in some embodiments are edge machines (e.g., virtual machines (VMs), containers, programs executing on computers, etc.) and/or standalone appliances that operate at multi-computer locations of the particular entity (e.g., at an office or datacenter of the entity) to connect the computers at their respective locations other nodes, hubs, etc. in the virtual network. In some embodiments, the nodes are clusters of nodes at each of the branch sites. In other embodiments, the edge nodes are deployed to each of the branch sites as high-availability pairs such that one edge node in the pair is the active node and the other edge node in the pair is the standby node that can take over as the active edge node in case of failover. 
     An example of an entity for which such a virtual network can be established includes a business entity (e.g., a corporation), a non-profit entity (e.g., a hospital, a research organization, etc.), and an education entity (e.g., a university, a college, etc.), or any other type of entity. Examples of public cloud providers include Amazon Web Services (AWS), Google Cloud Platform (GCP), Microsoft Azure, etc., while examples of entities include a company (e.g., corporation, partnership, etc.), an organization (e.g., a school, a non-profit, a government entity, etc.), etc. In other embodiments, hubs like the hub  145  can also be deployed in private cloud datacenters of a virtual WAN provider that hosts hubs to establish SD-WANs for different entities. 
     In the example SD-WAN  100 , the hub  145  is a multi-tenant forwarding element that is deployed on the premises of the datacenter  150 . The hub  145  can be used to establish secure connection links (e.g., tunnels) with edge nodes at the particular entity&#39;s multi-computer sites, such as branch sites  130 - 134 , third party datacenters (not shown), etc. For example, the hub  145  can be used to provide access from each branch site  120 - 124  to each other branch site  120 - 124  (e.g., via the connection links  160  that terminate at the hub  145 ) as well as to the resources  155  of the datacenter  150 . These multi-computer sites are often at different physical locations (e.g., different buildings, different cities, different states, etc.), according to some embodiments. In some embodiments, hubs can be deployed as physical nodes or virtual nodes. Additionally, hubs in some embodiments can be deployed on a cloud (e.g., as a set of virtual edges configured as a cluster). 
     In the SD-WAN  100 , the hub  145  also provides access to the resources  155  of the datacenter  150  as mentioned above. The resources in some embodiments include a set of one or more servers (e.g., web servers, database servers, etc.) within a microservices container (e.g., a pod). Conjunctively, or alternatively, some embodiments include multiple such microservices containers, each accessible through a different set of one or more hubs of the datacenter (not shown). The resources, as well as the hubs, are within the datacenter premises, according to some embodiments. While not shown, some embodiments include multiple different SaaS datacenters, which may each be accessed via different sets of hubs, according to some embodiments. In some embodiments, the SaaS datacenters include datacenters for video conferencing SaaS providers, for middlebox (e.g., firewall) service providers, for storage service providers, etc. 
     Additional examples of resources accessible via the hub  145 , in some embodiments, include compute machines (e.g., virtual machines and/or containers providing server operations), storage machines (e.g., database servers), and middlebox service operations (e.g., firewall services, load balancing services, encryption services, etc.). In some embodiments, the connections  160  between the branch sites and the hub  145  are secure encrypted connections that encrypt packets exchanged between the edge nodes  130 - 134  of the branch sites and the hub  145 . Examples of secure encrypted connections used in some embodiments include VPN (virtual private network) connections, or secure IPsec (Internet Protocol security) connection. 
     In some embodiments, multiple secure connection links (e.g., multiple secure tunnels) can be established between an edge node and the hub  145 . When multiple such links are defined between a node and a hub, each secure connection link, in some embodiments, is associated with a different physical network link between the node and an external network. For instance, to access external networks in some embodiments, a node has one or more commercial broadband Internet links (e.g., a cable mode and a fiber optic link) to access the Internet, a wireless cellular link (e.g., a 5G LTE network), etc. The collection of the edge nodes, gateway, datacenter hub, controller, and secure connections between the edge nodes, gateway, datacenter hub, and controller form the SD-WAN  100 . 
     As mentioned above, the controller  110  communicates with each of the nodes  130 - 134  at the branch sites  120 - 124 , in some embodiments, to send information such as trace monitoring rules and receive information such as trace monitoring results via the connection links  170 A- 170 C. In some embodiments, the controller  110  also provides trace monitoring rules to, and receives trace monitoring results from, the gateway  140  via the connection link  170 D and the hub  145  via the connection link  170 E. While illustrated as individual connection links, the links  170 A- 170 E are sets of multiple connection links, according to some embodiments. 
     In addition to the connection links  170 A- 170 E and  160 , edge nodes  132  and  134  are connected via connection link  164 , while edge nodes  130  and  132  are connected to the gateway  140  via connection links  162 . The gateway  140  in this example is responsible for relaying information between edge nodes (e.g., edge nodes  120  and  122 , which do not share a direct connection). Also, the gateway  140  in some embodiments is used to set up direct edge-to-edge connections. In some embodiments, the gateway  140  can be used to provide the edge nodes with access to cloud resources (e.g., compute, storage, and service resources of a cloud datacenter). 
       FIG. 2  illustrates an example of the machine-learning model used in the embodiments described herein. The machine-learning model  200  in some embodiments is a component of the controller (e.g., controller  110 ). The machine-learning model includes the learning engine component  205  and the decision-making and evaluation component  210 . In some embodiments, the machine learning model  200  is a time-series model that captures real-time, cyclical, and seasonal load fluctuations. The model can be a single mathematical model that captures all three fluctuations (e.g., ARIMA (autoregressive integrated moving average model) in some embodiments, or it can include a collection of different mathematical models, each of which capture the relevant fluctuation in some embodiments. 
     When the model includes a collection of different mathematical models, the cyclical and seasonal parameters are learned from the time-series data, and subsequently subtracted from the time-series data to capture real-time fluctuations, as will be described in further detail below. In some embodiments, the model  200  is trained on historical data and updated periodically based on new data as well as based on the model&#39;s decisions are affecting system performance. 
     The decision-making and evaluation engine  210 , in some embodiments, uses fluctuations identified by the learning component  205  to predict traffic loads and adjust the hub cluster accordingly to account for the predicted loads. Additionally, the decision-making and evaluation engine  210  in some embodiments evaluates the performance of the machine learning model  200 , and triggers re-training if necessary, as mentioned above. 
     In some embodiments, the machine-learning model  200  makes both real-time and historical decisions based off of the received traffic statistics. For example, using parameters like the number of branch sites, packet re-transmits or drops per flow, MOS scores as input works by having a statistical co-relation of these parameters across all of the branch sites (e.g., branch sites  120 - 124 ), in some embodiments. While one, or a handful, of branch sites reporting increased retransmits or drops may not be indicative of much, the co-relation can indicate that the increase in these parameters are indeed connected to each other and thus probably have the same root cause, according to some embodiments. As a result, the machine-learning model in some embodiments can concur that the hub cluster requires scaling. 
     An example of a historical decision can include using parameters like flows per second, bandwidth consumed by the flows, and fat flow bandwidth. For example, an increase of fat flows when a few branch sites perform data backup at night can indicate that the cluster needs to be scaled up at time T on days D. As another example, the number of flows per second (i.e., rate of creation of flows) can be a significant parameter that impacts the operation of large networking devices. These flows may not consume bandwidth, but the sheer volume of flows impacts the scale. By using historical data, the hub can be scaled up to account for the sudden increase in the rate of flow creations. 
     In a third example, a sudden onslaught of traffic for an airport&#39;s SD-WAN when several flights land around the same time can bring a sudden burst of network users, thus degrading network connectivity in an unpredictable manner. Such a model is an easy learnable model to scale hubs beforehand to avoid degradation of network connectivity. 
     The real-time decisions made by the machine-learning model kick in after the problem has occurred, and are meant to mitigate in time to prevent further degradation, while the historical decisions kick in before the problem occurs to avoid the problem completely. As a result, most problems are initially corrected in real-time, while historical decisions will start to fix problems over the course of operation of the network. 
     In some embodiments, due to the nature the historical decisions, new hubs (e.g., VMs) can be spawned ahead of time and terminated when their need has ended. While spawning new hubs can be time consuming, it does not require other resources while not in use, which works well for historical decisions allow for enough time to spawn these hubs. Real-time needs, however, are more time-constrained and the thus spawning new hubs to solve real-time problems is untenable. Thus, some embodiments have a pool of backup hubs that do not occupy any compute resources, but are fully configured and ready to launch (e.g., from a disk/storage/configuration perspective). Additional details regarding the functionality of the machine learning model  200  will be described in further detail below. 
       FIGS. 3A-3B  illustrate an example embodiment of an SD-WAN before and after the hub cluster size has been altered. In these example, the SD-WAN  300  includes a datacenter  350 , branch sites  320 - 324 , and a controller  310 . The datacenter  350  includes hub clusters  345 A and  345 B, respectively. Like the hub cluster  145 , the hub cluster  345 A/B provides the branch sites  320 - 324  with access to the datacenter resources  355 . 
     As shown in  FIG. 3A , the hub cluster  345 A and edge nodes at the branch sites  320 - 324  provide traffic statistics to the learning engine  312  of the controller  310 . The learning engine then identifies fluctuations for the decision making and evaluation engine  314  to use to predict traffic load for a time period and determine the necessary hub cluster size for the predicted load. The decision making and evaluation engine  314  includes a sizing tests storage  316  for storing previous hub cluster size calculations, and a load comparisons storage  318  for storing comparisons of actual and predicted loads. 
     After the decision-making and evaluation engine  314  performs its calculations, it adjusts the hub cluster size by scaling the cluster up or down to ensure the hub cluster can handle the predicted traffic load and is not using unnecessary bandwidth (e.g., as illustrated by the encircled  3 ). In some embodiments, the decision-making and evaluation engine also evaluates the performance of the machine learning model (i.e., the performance of the learning engine and decision-making and evaluation engine). 
     As illustrated in  FIG. 3B , the hub cluster  345 B includes 5 hubs compared to the 3 hubs in the hub cluster  345 A as a result of the adjustment triggered by the decision-making and evaluation engine. Additionally, the hub cluster and branch sites are providing traffic statistics indicative of the actual load to the controller. 
     The decision-making and evaluation engine  314  then compares the actual load to the predicted load, in some embodiments, and then averages the performance of the machine-learning model over time (e.g., using data from the load comparison storage  318 ). When the calculations indicate significant deviations over time between the predicted loads and actual loads, model re-training is triggered, as illustrated by the encircled  3 . Additional details regarding the processes performed by the machine-learning model of the controller will be described in further detail below with reference to  FIGS. 4-7 . 
       FIG. 4  illustrates a process performed by the controller in some embodiments to adjust hub cluster size based on identified real-time load fluctuations. The process  400  starts, at  405 , with the controller receiving real-time traffic statistics from the hub cluster and branch sites. For example,  FIG. 3A  illustrates the learning engine  312  receiving traffic statistics from the edge nodes at the branch sites  320 - 324 , as well as from the hub cluster  345 A. In some embodiments, the learning engine only receives these statistics from the hub cluster and not from the branch sites. 
     After receiving the real-time traffic statistics at  405 , the controller analyzes, at  410 , the real-time traffic statistics along with historical traffic statistics using a time-series model to determine cyclical and seasonal parameters. The controller then subtracts, at  415 , the cyclical and seasonal parameters from the time-series data (i.e., the real-time and historical traffic parameters) to identify potential real-time load fluctuations. Real-time load fluctuations are traffic flow fluctuations that happen at short time scales (e.g., minutes to hours), according to some embodiments. For example, in some embodiments, a burst of users connecting to an application can cause a sudden burst of traffic. 
     The controller next determines, at  420 , whether any real-time load fluctuations have been identified. When the controller determines that no real-time load fluctuations have been identified, the process returns to  405  to receive real-time traffic statistics from the hub cluster and branch sites. Otherwise, when the controller determines at  420  that real-time load fluctuations have been identified, the process transitions to  425  to predict the traffic load for a selected time period (e.g., the next  3  hours). For example, after the learning engine  312  of the controller  310  identifies any load fluctuations, the decision-making and evaluation engine  314  uses the identified load fluctuations to predict traffic load for an upcoming time interval, in some embodiments. 
     Next, the controller determines, at  425 , a hub cluster size that can handle the predicted traffic load with the expected flow performance. In some embodiments, this determination can be based on prior tests on sizing of the hub cluster at different traffic loads. In the example of  FIG. 3A , the decision-making and evaluation engine  314  of the controller  310  makes this determination after predicting the traffic load. 
     After determining the needed hub cluster size at  425 , the controller determines at  430  whether to scale-up the hub cluster. For example, when the needed hub cluster size determined by the decision-making and evaluation engine is larger than the current hub cluster size, a scale-up decision is triggered. When the controller determines that the hub cluster should be scaled up, the process transitions to  435  to add hubs to the cluster. The process then returns to  405  to receive real-time traffic statistics. 
     Alternatively, when the controller determines that the hub cluster should not be scaled up, the process transitions to  440  to determine whether to scale down the hub cluster. For example, when the determined hub cluster size for the predicted traffic load is smaller than the current hub cluster size, a scale-down decision is triggered. Accordingly, when the controller determines at  440  that the hub cluster should be scaled-down, the process transitions to  445  to remove hubs from the hub cluster. The process then transitions back to  405  to receive real-time traffic statistics. 
     When the controller alternately determines at  440  that the hub cluster should not be scaled-down, the process returns to  405  to receive real-time traffic statistics. In some embodiments, this occurs when the determined hub cluster size is equal to the current hub cluster size, and thus the cluster does not need to be scaled up or down. 
       FIG. 5  illustrates a process  500  performed by the controller in some embodiments to adjust hub cluster size based on identified cyclical load fluctuations. The process  500  starts at  505  when the controller receives real-time traffic statistics from the hub cluster and branch sites. Next, at  510 , the controller analyzes the real-time traffic statistics along with historical traffic statistics using a time-series model to identify potential cyclical load fluctuations. In some embodiments, cyclical load fluctuations are traffic flow fluctuations that occur in a cyclical fashion at the time scale of business operations. For example, morning and night fluctuations can occur daily at an enterprise when users log into the enterprise network in the morning, and subsequently drop off from the network at night, according to some embodiments. 
     Following the analysis, the controller determines, at  515 , whether any cyclical load fluctuations were identified. When the controller determines that no cyclical load fluctuations were identified (e.g., a constant load), the process transitions back to  505  to receive real-time traffic statistics. Otherwise, when the controller determines at  515  that cyclical load fluctuations have been identified, the process transitions to  520  to predict the traffic load for a selected cyclical time period. For example, in some embodiments, the controller predicts the traffic load at the time that users log into the enterprise network in the example above. 
     Next, the controller determines, at  525 , the hub cluster size needed for the predicted traffic load. As described above, the controller can base this determination off of previous load test, in some embodiments. At  530 , the controller determines whether the hub cluster should be scaled-up. For example, if the determined hub cluster size is greater than the current hub cluster size, then the hub cluster needs to be scaled-up (i.e., for the cyclical time period). When the controller determines at  530  that the hub cluster needs to be scaled-up, the process transitions to  535  to add hubs to the hub cluster. The process then returns to  505  to receive real-time traffic statistics. 
     Alternatively, when the controller determines at  530  that the hub cluster should not be scaled up, the process transitions to  540  to determine whether the hub cluster should be scaled down. In other words, the process determines whether the determined hub cluster size is smaller than the current hub cluster size. When the controller determines that the hub cluster should be scaled down, the process transitions to  545  to remove hubs from the hub cluster. The process then returns to  505 . Otherwise, when the controller determines at  540  that the hub cluster should not be scaled down (i.e., the determined hub cluster size is equal to the current hub cluster size), the process returns to  505  without removing any hubs. 
       FIG. 6  illustrates a process  500  performed by the controller in some embodiments to adjust hub cluster size based on identified seasonal load fluctuations. The process  600  starts at  605  when the controller receives real-time traffic statistics from the hub cluster and branch sites. The controller then analyzes, at  610 , the real-time traffic statistics along with historical traffic statistics using a time-series model to identify potential seasonal load fluctuations. In some embodiments, seasonal load fluctuations are traffic flow fluctuations that occur on a seasonal basis (i.e., on the order of months). For example, in some embodiments, the network traffic load during peak business operations periods could be much higher than normal (e.g., increased online shopping during the months of November and December). 
     Next, the controller determines, at  615 , whether any seasonal load fluctuations were identified during the analysis. When the controller determines that no seasonal load fluctuations were identified (e.g., a constant load for seasonal periods), the process transitions back to  605  to receive real-time traffic statistics. Otherwise, when the controller determines at  615  that seasonal load fluctuations have been identified, the process transitions to  620  to predict the traffic load for a selected seasonal time period. In some embodiments, for example, the controller predicts the traffic load for shopping vendors during peak sales times each year. 
     At  625 , the controller determines the hub cluster size needed for the predicted traffic load. In some embodiments, the controller can base this determination off of previous load test, as mentioned above. Next, at  630 , the controller determines whether the hub cluster should be scaled-up. For example, if the determined hub cluster size is larger than the current hub cluster size, then the hub cluster needs to be scaled-up (i.e., for the seasonal time period). When the controller determines at  630  that the hub cluster needs to be scaled-up, the process transitions to  635  to add hubs to the hub cluster in order to scale up the cluster. The process then returns to  605  to receive real-time traffic statistics. 
     When the controller alternately determines at  630  that the hub cluster should not be scaled up (i.e., the determined hub cluster size is not larger than the current hub cluster size), the process transitions to  640  to determine whether the hub cluster should be scaled down. In other words, the process determines whether the determined hub cluster size is smaller than the current hub cluster size. When the controller determines that the hub cluster should be scaled down, the process transitions to  645  to remove hubs from the hub cluster. The process then returns to  605 . Otherwise, when the controller determines at  640  that the hub cluster should not be scaled down (i.e., the determined hub cluster size is equal to the current hub cluster size), the process returns to  605  without removing any hubs. 
     In some embodiments, the processes  400 - 600  are performed concurrently, while in other embodiments, these processes are performed separately or in groups. In addition to the steps described for the processes  400 - 600 , the controller of some embodiments evaluates the performance of the machine learning model by comparing the predicted traffic load to the actual traffic load (i.e., once it occurs).  FIG. 7  illustrates a process performed by the controller in some embodiments to evaluate the machine learning model&#39;s performance. 
     The process  700  starts, at  710 , by receiving data indicating actual traffic load during a particular time period (e.g., months during a seasonal load fluctuation). In some embodiments, the decision-making and evaluation engine of the machine learning model in the controller performs this process. The controller then compares, at  720 , the actual traffic load with the predicted traffic load. 
     Next, at  730 , the controller determines whether there is a significant deviation between the actual traffic load and the predicted traffic load. When the controller determines that there is no significant deviation, the process ends. Otherwise, when the controller determines at  730  that there is a significant deviation between the loads, the process transitions to  740  to calculate the average performance of the machine learning model over a particular time-interval. For example, the decision-making and evaluation engine  314  of the controller  310  includes a storage  318  for storing performance-related data. 
     Based on the calculations, the controller determines at  750  whether there have been significant deviations over time. When the controller determines that there have not been significant deviations over time, the process ends. Otherwise, when the controller determines that there have been significant deviations over time based on the calculations, the process transitions to  760  to trigger re-training of the machine-learning model. Following  760 , the process ends. In addition to the process  700 , the machine-learning model re-training in some embodiments can be triggered after a certain time-interval to account for new data. 
     Many of the above-described features and applications are implemented as software processes that are specified as a set of instructions recorded on a computer readable storage medium (also referred to as computer readable medium). When these instructions are executed by one or more processing unit(s) (e.g., one or more processors, cores of processors, or other processing units), they cause the processing unit(s) to perform the actions indicated in the instructions. Examples of computer readable media include, but are not limited to, CD-ROMs, flash drives, RAM chips, hard drives, EPROMs, etc. The computer readable media does not include carrier waves and electronic signals passing wirelessly or over wired connections. 
     In this specification, the term “software” is meant to include firmware residing in read-only memory or applications stored in magnetic storage, which can be read into memory for processing by a processor. Also, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while remaining distinct software inventions. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that together implement a software invention described here is within the scope of the invention. In some embodiments, the software programs, when installed to operate on one or more electronic systems, define one or more specific machine implementations that execute and perform the operations of the software programs. 
       FIG. 8  conceptually illustrates a computer system  800  with which some embodiments of the invention are implemented. The computer system  800  can be used to implement any of the above-described hosts, controllers, gateway and edge forwarding elements. As such, it can be used to execute any of the above described processes. This computer system includes various types of non-transitory machine readable media and interfaces for various other types of machine readable media. Computer system  800  includes a bus  805 , processing unit(s)  810 , a system memory  825 , a read-only memory  830 , a permanent storage device  835 , input devices  840 , and output devices  845 . 
     The bus  805  collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of the computer system  800 . For instance, the bus  805  communicatively connects the processing unit(s)  810  with the read-only memory  830 , the system memory  825 , and the permanent storage device  835 . 
     From these various memory units, the processing unit(s)  810  retrieve instructions to execute and data to process in order to execute the processes of the invention. The processing unit(s) may be a single processor or a multi-core processor in different embodiments. The read-only-memory (ROM)  830  stores static data and instructions that are needed by the processing unit(s)  810  and other modules of the computer system. The permanent storage device  835 , on the other hand, is a read-and-write memory device. This device is a non-volatile memory unit that stores instructions and data even when the computer system  800  is off. Some embodiments of the invention use a mass-storage device (such as a magnetic or optical disk and its corresponding disk drive) as the permanent storage device  835 . 
     Other embodiments use a removable storage device (such as a floppy disk, flash drive, etc.) as the permanent storage device. Like the permanent storage device  835 , the system memory  825  is a read-and-write memory device. However, unlike storage device  835 , the system memory is a volatile read-and-write memory, such as random access memory. The system memory stores some of the instructions and data that the processor needs at runtime. In some embodiments, the invention&#39;s processes are stored in the system memory  825 , the permanent storage device  835 , and/or the read-only memory  830 . From these various memory units, the processing unit(s)  810  retrieve instructions to execute and data to process in order to execute the processes of some embodiments. 
     The bus  805  also connects to the input and output devices  840  and  845 . The input devices enable the user to communicate information and select commands to the computer system. The input devices  840  include alphanumeric keyboards and pointing devices (also called “cursor control devices”). The output devices  845  display images generated by the computer system. The output devices include printers and display devices, such as cathode ray tubes (CRT) or liquid crystal displays (LCD). Some embodiments include devices such as touchscreens that function as both input and output devices. 
     Finally, as shown in  FIG. 8 , bus  805  also couples computer system  800  to a network  865  through a network adapter (not shown). In this manner, the computer can be a part of a network of computers (such as a local area network (“LAN”), a wide area network (“WAN”), or an Intranet), or a network of networks (such as the Internet). Any or all components of computer system  800  may be used in conjunction with the invention. 
     Some embodiments include electronic components, such as microprocessors, storage and memory that store computer program instructions in a machine-readable or computer-readable medium (alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a variety of recordable/rewritable DVDs (e.g., DVD-RAM, DVD−RW, DVD+RW, etc.), flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and/or solid state hard drives, read-only and recordable Blu-Ray® discs, ultra-density optical discs, any other optical or magnetic media, and floppy disks. The computer-readable media may store a computer program that is executable by at least one processing unit and includes sets of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as is produced by a compiler, and files including higher-level code that are executed by a computer, an electronic component, or a microprocessor using an interpreter. 
     While the above discussion primarily refers to microprocessor or multi-core processors that execute software, some embodiments are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions that are stored on the circuit itself. 
     As used in this specification, the terms “computer”, “server”, “processor”, and “memory” all refer to electronic or other technological devices. These terms exclude people or groups of people. For the purposes of the specification, the terms “display” or “displaying” mean displaying on an electronic device. As used in this specification, the terms “computer readable medium,” “computer readable media,” and “machine readable medium” are entirely restricted to tangible, physical objects that store information in a form that is readable by a computer. These terms exclude any wireless signals, wired download signals, and any other ephemeral or transitory signals. 
     While the invention has been described with reference to numerous specific details, one of ordinary skill in the art will recognize that the invention can be embodied in other specific forms without departing from the spirit of the invention. For instance, several of the above-described embodiments deploy gateways in public cloud datacenters. However, in other embodiments, the gateways are deployed in a third party&#39;s virtual private cloud datacenters (e.g., datacenters that the third party uses to deploy cloud gateways for different entities in order to deploy virtual networks for these entities). Thus, one of ordinary skill in the art would understand that the invention is not to be limited by the foregoing illustrative details, but rather is to be defined by the appended claims.