COMPUTING NODE CLUSTERS SUPPORTING NETWORK SEGMENTATION

Examples described herein may include transition of a distributed computing system to using a segmented network configuration. An example method includes receiving a network segmentation request at a distributed computing system. In response to the network segmentation request and during normal operation of the distributed computing system, the method includes allocating IP addresses to computing nodes of the distributed computing system based on a number of segmented networks, and applying firewall rules to open service ports of the computing nodes. Further in response to the network segmentation request and during normal operation, the method includes updating network configuration information of the computing nodes. For a computing node of the computing nodes, the method further includes publishing the allocated IP addresses, and restarting services of the computing node. The method further includes applying the firewall rules to open a subset of the service ports of the computing nodes.

TECHNICAL FIELD

Examples described herein relate generally to distributed computing systems. Examples of virtualized systems are described. Examples of distributed computing systems described herein may facilitate transition to use of segmented network configurations.

BACKGROUND

A virtual machine (VM) generally refers to a software-based implementation of a machine in a virtualization environment, in which the hardware resources of a physical computer (e.g., CPU, memory, etc.) are virtualized or transformed into the underlying support for the fully functional virtual machine that can run its own operating system, and applications on the underlying physical resources just like a real computer.

Virtualization generally works by inserting a thin layer of software directly on the computer hardware or on a host operating system. This layer of software contains a virtual machine monitor or “hypervisor” that allocates hardware resources dynamically and transparently. Multiple operating systems may run concurrently on a single physical computer and share hardware resources with each other. By encapsulating an entire machine, including CPU, memory, operating system, and network devices, a virtual machine may be completely compatible with most standard operating systems, applications, and device drivers. Most modern implementations allow several operating systems and applications to safely run at the same time on a single computer, with each having access to the resources it needs when it needs them.

One reason for the broad adoption of virtualization in modern business and computing environments is because of the resource utilization advantages provided by virtual machines. Without virtualization, if a physical machine is limited to a single dedicated operating system, then during periods of inactivity by the dedicated operating system the physical machine may not be utilized to perform useful work. This may be wasteful and inefficient if there are users on other physical machines which are currently waiting for computing resources. Virtualization allows multiple VMs to share the underlying physical resources so that during periods of inactivity by one VM, other VMs can take advantage of the resource availability to process workloads. This can produce great efficiencies for the utilization of physical devices, and can result in reduced redundancies and better resource cost management.

DETAILED DESCRIPTION

This disclosure describes embodiments for transition to network segmentation in a distributed computing system. Network segmentation typically involves isolating certain classes of traffic from other classes of traffic. For example, management traffic (e.g., traffic transmitted to and received from sources outside the distributed computing system) may be segmented into a different network than backplane traffic (e.g., traffic contained within the distributed computing system). Segmentation of traffic may be desirable for security purposes and/or for purposes of predicting and managing network bandwidth usage. In some examples, the transition to segmented networks may be responsive to a received request for segmentation. The request may include one or more network interface definitions. Each network interface definition defines the associated class of traffic, and other parameters for setting up the network interface. A network manager on the computing nodes of the distributed computing system may be configured to manage transition to segmented networks. In some examples, the transition may be performed by the distributed computing system while the distributed system remains operational. This type of transition may employ a rolling update, where the computing nodes of the distributed computing system are updated in a sequential and ordered fashion. That is, during the rolling update, only one computing node is updated at a time, allowing the other computing nodes to remain operational during the update. To facilitate the network segmentation transition, firewall rules may be relaxed on open service ports on the computing nodes to allow communication within the system. The firewall rules may be reinstated after the update to provide protection against undesired traffic.

Various embodiments of the present disclosure will be explained below in detail with reference to the accompanying drawings. The following detailed description refers to the accompanying drawings that show, by way of illustration, specific aspects and embodiments of the disclosure. The detailed description includes sufficient detail to enable those skilled in the art to practice the embodiments of the disclosure. Other embodiments may be utilized, and structural, logical and electrical changes may be made without departing from the scope of the present disclosure. The various embodiments disclosed herein are not necessary mutually exclusive, as some disclosed embodiments can be combined with one or more other disclosed embodiments to form new embodiments.

FIG. 1is a block diagram of a distributed computing system100, in accordance with an embodiment of the present disclosure. The distributed computing system100generally includes a computing node102and a computing node112and storage140connected to a network122. The network122may be any type of network capable of routing data transmissions from one network device (e.g., the computing node102, the computing node112, and the storage140) to another. For example, the network122may be a local area network (LAN), wide area network (WAN), intranet, Internet, or a combination thereof. The network122may be a wired network, a wireless network, or a combination thereof.

The storage140may include local storage124, local storage130, cloud storage136, and networked storage138. The local storage124may include, for example, one or more solid state drives (SSD126) and one or more hard disk drives (HDD128). Similarly, the local storage130may include SSD132and HDD134. The local storage124and the local storage130may be directly coupled to, included in, and/or accessible by a respective the computing node102and/or the computing node112without communicating via the network122. Other nodes, however, may access the local storage124and/or the local storage130using the network122. Cloud storage136may include one or more storage servers that may be stored remotely to the computing node102and/or the computing node112and accessed via the network122. The cloud storage136may generally include any suitable type of storage device, such as HDDs SSDs, or optical drives. Networked storage138may include one or more storage devices coupled to and accessed via the network122. The networked storage138may generally include any suitable type of storage device, such as HDDs SSDs, and/or NVM Express (NVMe). In various embodiments, the networked storage138may be a storage area network (SAN). The computing node102is a computing device for hosting virtual machines (VMs) in the distributed computing system100.

The computing node102may be configured to execute a hypervisor110, a controller VM108and one or more user VMs, such as user VMs104,106. The user VMs including the user VM104and the user VM106are virtual machine instances executing on the computing node102. The user VMs including the user VM104and the user VM106may share a virtualized pool of physical computing resources such as physical processors and storage (e.g., the storage140). The user VMs including the user VM104and the user VM106may each have their own operating system, such as Windows or Linux. While a certain number of user VMs are shown, generally any suitable number may be implemented. User VMs may generally be provided to execute any number of applications which may be desired by a user.

The hypervisor110may be any type of hypervisor. For example, the hypervisor110may be ESX, ESX(i), Hyper-V, KVM, or any other type of hypervisor. The hypervisor110manages the allocation of physical resources (such as the storage140and physical processors) to VMs (e.g., user VM104, user VM106, and controller VM108) and performs various VM related operations, such as creating new VMs and cloning existing VMs. Each type of hypervisor may have a hypervisor-specific API through which commands to perform various operations may be communicated to the particular type of hypervisor. The commands may be formatted in a manner specified by the hypervisor-specific API for that type of hypervisor. For example, commands may utilize a syntax and/or attributes specified by the hypervisor-specific API.

Controller VMs (CVMs) described herein, such as the controller VM108and/or the controller VM118, may provide services for the user VMs in the computing node. As an example of functionality that a controller VM may provide, the controller VM108may provide virtualization of the storage140. Accordingly, the storage140may be referred to as a storage pool. Controller VMs may provide management of the distributed computing system100. Examples of controller VMs may execute a variety of software and/or may serve the I/O operations for the hypervisor and VMs running on that node. In some examples, a SCSI controller, which may manage SSD and/or HDD devices described herein, may be directly passed to the CVM, e.g., leveraging PCI Pass-through in some examples. In this manner, controller VMs described herein may manage input/output (I/O) requests between VMs on a computing node and available storage, such as the storage140.

The computing node112may include user VM114, user VM116, a controller VM118, and a hypervisor120. The user VM114, the user VM116, the controller VM118, and the hypervisor120may be implemented similarly to analogous components described above with respect to the computing node102. For example, the user VM114and the user VM116may be implemented as described above with respect to the user VM104and the user VM106. The controller VM118may be implemented as described above with respect to the controller VM108. The hypervisor120may be implemented as described above with respect to the hypervisor110. In some examples, the hypervisor120may be a different type of hypervisor than the hypervisor110, example, the hypervisor120may be Hyper-V, while the hypervisor110may be ESX(i). In some examples, the hypervisor110may be of a same type as the hypervisor120.

The controller VM108and the controller VM118may communicate with one another via the network122. By linking the controller VM108and the controller VM118together via the network122, a distributed network of computing nodes including the computing node102and the computing node112, can be created.

Controller VMs, such as the controller VM108and the controller VM118, may each execute a variety of services and may coordinate, for example, through communication over network122. Services running on controller VMs may utilize an amount of local memory to support their operations. For example, services running on the controller VM108may utilize memory in local memory142. Services running on the controller VM118may utilize memory in local memory144. The local memory142and the local memory144may be shared by VMs on the computing node102and the computing node112, respectively, and the use of the local memory142and/or the local memory144may be controlled by the hypervisor110and the hypervisor120, respectively. The local memory142and144may include a flash driver or some other removable form of memory installed on the computing node102and112, respectively. Moreover, multiple instances of the same service may be running throughout the distributed system—e.g. a same services stack may be operating on each controller VM. For example, an instance of a service may be running on the controller VM108and a second instance of the service may be running on the controller VM118.

Generally, controller VMs described herein, such as the controller VM108and the controller VM118may be employed to control and manage any type of storage device, including all those shown in the storage140, including the local storage124(e.g., SSD126and HDD128), the cloud storage136, and the networked storage138. Controller VMs described herein may implement storage controller logic and may virtualize all storage hardware as one global resource pool (e.g., the storage140) that may provide reliability, availability, and performance. IP-based requests are generally used (e.g., by user VMs described herein) to send I/O requests to the controller VMs. For example, user VM104and user VM106may send storage requests to the controller VM108using over a virtual bus. Controller VMs described herein, such as the controller VM108, may directly implement storage and I/O optimizations within the direct data access path. Communication between hypervisors and controller VMs described herein may occur using IP requests.

Note that controller VMs are provided as virtual machines utilizing hypervisors described herein—for example, the controller VM108is provided behind hypervisor110. Since the controller VMs run “above” the hypervisors examples described herein may be implemented within any virtual machine architecture, since the controller VMs may be used in conjunction with generally any hypervisor from any virtualization vendor.

Virtual disks (vDisks) may be structured from the storage devices in the storage140, as described herein. A vDisk generally refers to the storage abstraction that may be exposed by a controller VM to be used by a user VM. In some examples, the vDisk may be exposed via iSCSI (“internet small computer system interface”) or NFS (“network file system”) and may be mounted as a virtual disk on the user VM. For example, the controller VM108may expose one or more vDisks of the storage140and the hypervisor may attach the vDisks to one or more VMs, and the virtualized operating system may mount a vDisk on one or more user VMs, such as the user VM104and/or the user VM106.

During operation, the user VMs (e.g., the user VM104and/or the user VM106) may provide storage input/output (I/O) requests to controller VMs (e.g., the controller VM108and/or the hypervisor110). Accordingly, a user VM may provide an I/O request over a virtual bus to a hypervisor as an iSCSI and/or NFS request. Internet Small Computer system Interface (iSCSI) generally refers to an IP-based storage networking standard for linking data storage facilities together. By carrying SCSI commands over IP networks, iSCSI can be used to facilitate data transfers over intranets and to manage storage over any suitable type of network or the Internet. The iSCSI protocol allows iSCSI initiators to send SCSI commands to iSCSI targets at remote locations over a network. In some examples, user VMs may send I/O requests to controller VMs in the form of NFS requests. Network File system (NFS) refers to an IP-based file access standard in which NFS clients send file-based requests to NFS servers via a proxy folder (directory) called “mount point”. Generally, then, examples of systems described herein may utilize an IP-based protocol (e.g., iSCSI and/or NFS) to communicate between hypervisors and controller VMs.

During operation, examples of user VMs described herein may provide storage requests using an IP based protocol, such as SMB. The storage requests may designate the IP address for a controller VM from which the user VM desires I/O services. The storage request may be provided from the user VM to a virtual switch within a hypervisor to be routed to the correct destination. For examples, the user VM104may provide a storage request to hypervisor110. The storage request may request I/O services from controller VM108and/or the controller VM118. If the request is to be intended to be handled by a controller VM in a same service node as the user VM (e.g., the controller VM108in the same computing node as the user VM104) then the storage request may be internally routed within the computing node102to the controller VM108. In some examples, the storage request may be directed to a controller VM on another computing node. Accordingly, the hypervisor (e.g., the hypervisor110) may provide the storage request to a physical switch to be sent over a network (e.g., the network122) to another computing node running the requested controller VM (e.g., the computing node112running the controller VM118).

Accordingly, hypervisors described herein may manage I/O requests between user VMs in a system and a storage pool. Controller VMs may virtualize I/O access to hardware resources within a storage pool according to examples described herein. In this manner, a separate and dedicated controller (e.g., controller VM) may be provided for each and every computing node within a virtualized computing system (e.g., a cluster of computing nodes that run hypervisor virtualization software since each computing node may include its own controller VM. Each new computing node in the system may include a controller VM to share in the overall workload of the system to handle storage tasks. Therefore, examples described herein may be advantageously scalable, and may provide advantages over approaches that have a limited number of controllers. Consequently, examples described herein may provide a massively-parallel storage architecture that scales as and when hypervisor computing nodes are added to the system.

In some examples, the distributed computing system100may support network segmentation. That is, network traffic may be segmented to isolate different classes of traffic. For example, management traffic (e.g., traffic transmitted to and received from sources outside the distributed computing system100) may be segmented into a different network than backplane traffic (e.g., traffic contained within the distributed computing system100). Examples of management traffic may include traffic to and from computing devices or nodes over outside networks, such as WANs or the Internet (e.g., using secure shell (SSH), simple network management protocol SNMP, etc.). Management traffic may be transmitted by or received by the user VMs104,106,114,116, the controller VMs,108,118, the hypervisors110,120. The backplane traffic may include traffic for operation within the distributed system100, such as configuration changes, data storage, management of the distributed computing system100, etc. The backplane traffic may be primarily transmitted by or received by the controller VMs108,118. Network segmentation may be desirable for security purposes and/or for purposes of predicting and managing network bandwidth usage. For example, internal backplane traffic may be isolated from outside management traffic, which may prevent an outside actor from interfering with internal operation of the distributed computing system100. The network segmentation may be segmented differently and may include more than two segmentations without departing from the scope of the disclosure.

To support network segmentation, the controller VM108may include a network manager109and the controller VM118may include a network manager119. The network manager109and the network manager119are each configured to control/manage the network segmentation. For example, the network manager109and the network manager119may each receive a request and instructions for a network segmentation implementation, and may provision additional networks, provision network interface cards (NICs), retrieve assigned internet protocol (IP) addresses, look up assigned IP addresses for other components, and perform other operations associated with conversion to segmented networks. In some examples, the provisioned networks may include virtual networks, and provision of the NICs may include creation of virtual NICs for each individual network. That is, the communication through the network122may use the same physical hardware/conduit, with the segmentation of traffic achieved by addressing traffic to different vLAN identifiers (e.g., each associated with a different virtual NIC (vNIC) configured for each controller VM108,118for each class of network traffic).

Enabling/disabling network segmentation may be controlled by an administration system. For example, as shown inFIG. 1, the distributed computing system100may include or be connected to an administrator system158that is configured to control network segmentation on the distributed computing system100. The administrator system158may be implemented using, for example, one or more computers, servers, laptops, desktops, tablets, mobile phones, or other computing systems. In other examples, the administrator system158may be wholly and/or partially implemented using one of the computing nodes of the distributed computing system100. However, in some examples, the administrator system158may be a different computing system from the distributed computing system100and may be in communication with one or more controller VMs108,118of the distributed computing system100using a wired or wireless connection (e.g., over a network).

The administrator system158may host one or more user interfaces, e.g., user interface160. The user interface160may be implemented, for example, by displaying a user interface on a display of the administrator system. The user interface160may receive input from one or more users (e.g., administrators) using one or more input device(s) of the administrator system, such as, but not limited to, a keyboard, mouse, touchscreen, and/or voice input. The user interface160may provide input to the controller VM(s)108,118and/or may receive data from the controller VM(s)108,118. The user interface160may be implemented, for example, using a web service provided by the controller VM108or one or more other controller VMs described herein. In some examples, the user interface160may be implemented using a web service provided by the controller VM108and information from the controller VM108may be provided to the administrator system158for display in the user interface160.

In some examples, a user may interact with the user interface160of the administrator system158to set up particular network segmentation configurations on the distributed computing system100. In some examples, the user may create new networks interfaces, assign classifications of traffic to the new network interface, assign network parameters, such as firewall rules, subnets, network masks, virtual networks identifiers, address pools and ranges, service port numbers, etc. Based on the network parameter inputs, in some examples, software running on the administrator system158may assign IP addresses to the computing nodes102and112for each segmented network interface definition. In other examples, the IP addresses may be assigned by the distributed computing system100after receiving a request. The administrator system158may provide a network segmentation request, including the network segmentation configuration information, to the controller VM(s)108,118. In some examples, the network segmentation configuration information may be provided to a selected one of the controller VMs108or118and the selected one of the controller VMs108,118may provide the network segmentation configuration information to the other of the controller VMs108,118. The network managers109,119may be configured to set up hypervisor backplane interfaces for each segmented network to implement assigned network configurations for each segmented network.

In some examples, the network segmentation may be provisioned at the time of initial setup/installation of the distributed computing system100. In other examples, the network segmentation may be implemented while the distributed computing system100is operational (e.g., in normal operation), example, the administrator system158may provide instructions to the controller VMs108,118to enable network segmentation while the distributed computing system100remains in a normal operating mode. That is, the distributed computing system100may transition to a segmented network implementation without disruption of operation of the distributed computing system100(e.g., the transition may be transparent to the user VMs104,106and114,116and other applications and services running on the computing nodes101and112, respectively, such that they continue to communicate and operate with minimal or no disruption). This may be more efficient than a network segmentation implementation that involves disruption (e.g., stopping, restarting, reconfiguring, etc.) of normal operation of the user VMs104,106and114,116and other applications and services running on the computing nodes101and112, respectively, to implement the segmentation (e.g., non-normal operation. The distributed computing system100may utilize a rolling update where the computing nodes102and112are updated using an iterative update process. That is, the network managers109,119may implement a rolling process that includes opening of service ports on each segmented network, updating IP address mapping in a database, strategic publishing of IP address assignment information, converting the computing nodes102,112to segmented network operation sequentially, etc. Publishing of the network segmentation information may be via a distributed database. Thus, during the rolling process, one computing node (e.g., the computing node102) may be configured to receive traffic according to the defined segmented network configuration while other computing nodes (e.g., the computing node112) may remain configured for the non-segmentation network setup. To facilitate the network segmentation in order to relax communication restriction within the distributed computing system100.

FIG. 2is a block diagram of a distributed computing system200utilizing network segmentation, in accordance with an embodiment of the present disclosure. The distributed computing system200generally includes a computing node202, a computing node212, and a switch290. The distributed computing system100ofFIG. 1may implement the distributed computing system200, in some examples. The computing nodes202and212may communicate using the switch290over one or more segmented networks. The one or more networks may include any type of network capable of routing data transmissions from one network device (e.g., the computing node202, the computing node212, and the switch290) to another. The network may include a local area network (LAN), wide area network (WAN), intranet, Internet, or a combination thereof. The network include a wired network, a wireless network, or a combination thereof. In some examples, the networks may be virtual networks, such as virtual LANs (VLANs)

The computing node202may be configured to execute a hypervisor210, a controller VM208and one or more user VMs (not shown). The hypervisor210may be any type of hypervisor. For example, the hypervisor210may be ESX, ESX(i), Hyper-V, KVM, or any other type of hypervisor. The hypervisor210manages the allocation of physical resources (such as storage and physical processors) to VMs (e.g., user VMs and the controller VM208) and performs various VM related operations, such as creating new VMs and cloning existing VMs. Each type of hypervisor may have a hypervisor-specific API through which commands to perform various operations may be communicated to the particular type of hypervisor. The commands may be formatted in a manner specified by the hypervisor-specific API for that type of hypervisor. For example, commands may utilize a syntax and/or attributes specified by the hypervisor-specific API.

The computing node212may include user VMs (not shown), a controller VM218, and a hypervisor220. The controller VM218may be implemented as described above with respect to the controller VM208. The hypervisor220may be implemented as described above with respect to the hypervisor210. In some examples, the hypervisor220may be a different type of hypervisor than the hypervisor210. For example, the hypervisor220may be Hyper-V, while the hypervisor210may be ESX(i). In some examples, the hypervisor210may be of a same type as the hypervisor220.

Controller VMs (CVMs) described herein, such as the controller VM208and/or the controller VM218, may provide services for the user VMs in the computing node. As an example of functionality that a controller VM may provide, the controller VM208may provide virtualization of storage (e.g., the storage140ofFIG. 1). Controller VMs may provide management of the distributed computing system200. Examples of controller VMs may execute a variety of software and/or may serve the I/O operations for the hypervisor and VMs running on that node. In some examples, a SCSI controller, which may manage SSD and/or HDD devices described herein, may be directly passed to the CVM, e.g., leveraging PCI Pass-through in some examples. In this manner, controller VMs described herein may manage input/output (I/O) requests between VMs on a computing node and available storage.

The controller VM208and the controller VM218may communicate with one another using one or more segmented networks via the physical switch290. By linking the controller VM208and the controller VM218together via the one or more segmented networks, a distributed network of computing nodes including the computing node202and the computing node212, can be created.

Controller VMs, such as the controller VM208and the controller VM218, may each execute a variety of services and may coordinate, for example, through communication over one or more segmented networks. Services running on controller VMs may utilize an amount of local memory to support their operations. Moreover, multiple instances of the same service may be running throughout the distributed system200—e.g. a same services stack may be operating on each controller VM. For example, an instance of a service may be running on the controller VM208and a second instance of the service may be running on the controller VM218.

Note that controller VMs are provided as virtual machines utilizing hypervisors described herein—for example, the controller VM208is provided behind hypervisor210. Since the controller VMs run “above” the hypervisors examples described herein may be implemented within any virtual machine architecture, since the controller VMs may be used in conjunction with generally any hypervisor from any virtualization vendor.

During operation, user VMs operating on the computing nodes202,212of the distributed file system200may provide I/O requests to the controller VMs208,218and/or the hypervisors210,220using one or more of the segmented networks. Hypervisors described herein may manage I/O requests between user VMs in a system and a storage pool. Controller VMs may virtualize I/O access to hardware resources within a storage pool according to examples described herein. In this manner, a separate and dedicated controller (e.g., controller VM) may be provided for each and every computing node within a virtualized computing system (e.g., a cluster of computing nodes that run hypervisor virtualization software), since each computing node may include its own controller VM. Each new computing node in the system may include a controller VM to share in the overall workload of the system to handle storage tasks. Therefore, examples described herein may be advantageously scalable, and may provide advantages over approaches that have a limited number of controllers. Consequently, examples described herein may provide a massively-parallel storage architecture that scales as and when hypervisor computing nodes are added to the system.

As previously described, the distributed computing system200may support network segmentation for operational and security benefits. Without network segmentation, all external (e.g., outside of the distributed computing system200) and internal traffic (e.g., within the distributed computing system200) would be shared over a single network, which could expose the distributed computing system200to security risks. Network segmentation may also be desirable for purposes of predicting and managing network bandwidth usage. In the example ofFIG. 2, the distributed computing system200may utilize a first network interface ETH0(e.g., having a first VLAN VLAN1) for a first class of traffic, a second network interface ETH2(e.g., having second VLAN VLAN2) for a second class of traffic, and a third network interface ETH1(e.g., having a third VLAN VLAN3) for a third class of traffic. In one example, backplane traffic may be allocated to the VLAN1, management traffic may be allocated to the VLAN2, and intra-computing node traffic may be allocated to VLAN3. To support network segmentation, the controller VMs208,218may each include a respective network manager209,219. The network managers209,219may configure the respective controller VM208,218for network segmentation. For example, the network managers209,219may create vNICs for each of the ETH0, ETH2, and ETH1network interfaces, and assign a specified IP address to each vNIC. The network manager209may create vNICs203(0)-(2), for communication using ETH0(vLAN1), ETH2(vLAN2), and ETH1(vLAN3), respectively. Each of the ETH0(vLAN1), ETH2(vLAN2), and ETH1(vLAN3), respectively, may act as a respective vNIC(0)-(2).

The hypervisors210,220may include respective virtual switches vswitches214and224, and multiple NICs233and226, respectively. The multiple NICs233and226may include physical NICs, such as peripheral component interconnect (PCI) NICs (pNICs). While only two NICs233and226are shown, more NICs may be included without departing from the scope of the disclosure. The vswitches214and224may be configured to route traffic for associated with each of the vLAN1, vLAN2, and vLAN3. The vswitch214may be configured to route data/traffic between the vNICs203(0)-(2) and the NICs233. The vswitch224may be configured to route data/traffic between the vNICs213(0)-(2) and the NICs226. The routing by the vswitches214,224may be based on network identifiers, IP addresses, etc. The NICs233and226may be coupled to the switch290to transmit and receive traffic/data. For example, internal backplane traffic may be isolated from outside management traffic, which may prevent an outside actor from interfering with internal operation of the distributed computing system200. The network segmentation may be segmented differently and may include more than two segmentations without departing from the scope of the disclosure.

As previously described, the network manager209and the network manager219are each configured to control/manage the network segmentation. The network managers209,219may receive a request and instructions for a network segmentation implementation, and may provision the ETH0, ETH2, and ETH1network interfaces (e.g., the vNICs203(0)-(2),213(0-(2))), retrieve assigned internet protocol (IP) addresses, look up assigned IP addresses for other components. In some examples, the network segmentation may be implemented at the time of installation/setup of the distributed computing system200. In other examples, the network segmentation may be triggered while the distributed computing system200is operational.

Enabling/disabling network segmentation within the distributed computing system200may be controlled by an administrator system, such as the administrator system158ofFIG. 1. The administrator system may provide a request to initiate network segmentation, along with network segmentation configuration information, to the network managers209,219. The network segmentation configuration information may include a network interface definition and network segmentation parameters, such as firewall rules, subnets, network masks, virtual networks identifiers, IP address pools and ranges, service port numbers, assigned IP addresses, etc. In some examples, the network segmentation configuration information may be provided to a selected one of the network managers209,219/controller VMs208or218and the selected one of the network managers209,219/controller VMs208,218may provide the network segmentation configuration information to the other of network managers209,219/the controller VMs208,218. The network managers209,219may be configured to set up host interfaces for each segmented network to implement assigned network configurations for each segmented network.

In some examples, the network segmentation may be provisioned at the time of initial setup/installation of the distributed computing system200. In other examples, the network segmentation may be implemented while the distributed computing system200is operational. In some examples, the network managers209,219may initiate a rolling update process to enable network segmentation while the distributed computing system200remains operational in response to a network segmentation request. The rolling update process may include applying firewall rules to open of service ports on two or more of the ETH2, and ETH1network interfaces, updating IP address mapping in a database, strategic publishing of IP address assignment information, and sequentially restarting the controller VMs208,218on each node, etc. Thus, during the rolling process, one computing node (e.g., the computing node202) may be configured to receive traffic according to the defined segmented network configuration while other computing nodes (e.g., the computing node212) may remain configured for the non-segmentation network setup. Upon restart, each of the controller VMs208,218may publish a remote procedure call (RPC) handler to identify communication information for the controller VM208,218. To facilitate the update and prevent communication blockage, firewall rules may be relaxed on open service ports on the distributed computing system200. The firewall rules may be reinstated after the update to provide protection against undesired traffic.

FIG. 3is a flowchart of a method300for enabling network segmentation at a computing node of a distributed computing system in accordance with some embodiments of the disclosure. The method300may be performed by the distributed computing system100ofFIG. 1, the distributed computing system200ofFIG. 2, or combinations thereof. In a specific example, one or more network managers, such as the network managers109,119ofFIG. 1, the network managers209,219ofFIG. 2, or combinations thereof may implement the method300. During performance of the method300, the distributed computing system may remain operational. That is, the transition to network segmentation may be transparent to a user.

The method300may include receiving a network segmentation request, at310. The network segmentation request may be received from an administrator system, such as the administrator system158ofFIG. 1. The network segmentation request may include network segmentation configuration information. The network segmentation configuration information may include a request to assign a first class of data traffic to a first network interface and a request to assign a second class of data traffic to a second network interface, for example. Additional requests may be included without departing from the scope of the disclosure. Each network interface definition may include parameters pertaining to one or more of firewall rules, subnets, network masks, virtual networks identifiers, IP address pools and ranges, service port numbers, assigned IP addresses, etc.

In response to the network segmentation request and during normal operation of the distributed computing system, the method300may include performance of one or all of the steps320-370. That is, the transition may be transparent to the user VMs and other applications and services running on the computing nodes of the distributed computing system such that they continue to communicate and operate with minimal or no disruption (e.g., remain in a normal operating mode). For example, the method300may further include, allocating and assigning a plurality of internet protocol (IP) addresses to computing nodes of the distributed computing system based on a number of segmented networks defined in the network segmentation request, at320. If the number of segmented networks is set to two, then two IP addresses would be allocated and assigned. The assigned IP addresses for each node may be included in a database on the distributed computing system.

The method300may further include applying firewall rules to open a plurality of service ports of the computing nodes, at330. The service ports may be opened for one or both of the segmented networks defined in the request, such as opening ports for one or more of the vLAN1, vLAN2, or vLAN3ofFIG. 2. Application of the firewall rules may prevent communication blockage within the distributed computing system during the transition to network segmentation. The firewall rules may be dynamic for each service port type based on the current network state of the distributed computing system, the application in which the distributed computing system is being used, etc.

The method300may further include updating network configuration information of the computing nodes, at340. Updating the network configuration information may include updating a configuration for a particular class of traffic to specify a new subnet, network mask, and vLAN identifier for the particular class of traffic.

The method300may further include performing a rolling update of the computing nodes, at350. That is, the rolling update may include an update a first computing node of the distributed computing system, followed by updating a second computing node of the distributed computing system For each computing node, the rolling update may include publishing the allocated and assigned plurality of IP address, at352, and restarting services of the computing node, at354. Publishing the IP addresses may be to a service that stores currently assigned IP addresses. Publishing of the IP addresses may include updating of distributed database that maintains a list of current IP addresses. After publishing of the new IP address for a particular subnet, services that monitor current IP addresses to update communication. Restarting services may include restarting services running on the controller VM (e.g., any of the controller VMs108,118ofFIG. 1or the controller VMs208,218ofFIG. 2). The restart may include stopping of running services, updating IP addresses to newly assigned IP addresses, and rebooting the controller VM. Upon reboot, the controller VM may publish a remote procedure call (RPC) handler to identify communication information for the controller VM. Once all computing nodes have transitioned to the network segmentation, one or more of the computing nodes of the distributed computing system may provide confirmation of completion to an administrator system, for example.

After the rolling update has been completed on each of the computing nodes, the method300may further include applying the firewall rules to open a subset of the plurality of service ports of the computing node, at360. For example, the method may include applying firewall rules to only open service ports for one of the segmented networks, such as a segmented network associated with the backplane traffic.

The method300is exemplary. The method300may include fewer or additional steps for each transition to network segmentation departing from the scope of the disclosure.

FIG. 4is a flowchart of a method400for setting up a network segmentation interface for a distributed computing system in accordance with some embodiments of the disclosure.FIGS. 5A-Ginclude example user interface diagrams for setting up a network segmentation interface for a distributed computing system in accordance with some embodiments of the disclosure. The method400may be performed by an administrator system, such as the administrator system158ofFIG. 1.

The method400may include initiating a user interface to create a new network segmentation interface associated with a class of data traffic, at410. The diagram500ofFIG. 5Aprovides an example of a user interface for creating a new network segmentation interface. The new network segmentation interface (e.g., one of ETH0-2) may include allocating a specific class of traffic to a new network interface.

The method400may include adding selected details associated with the new network interface in response to received input, at412. The diagram510ofFIG. 5Bprovides an example a user interface for adding network interface details. The new network segmentation interface details may include a new network interface name, an identifier for the corresponding vLAN (vLAN Identifier), and an IP address pool. The IP address pool identifies a pool of IP addresses that may be used for the new network interface. In some examples, portions of the user interface may be disable in response to missing required information. For example, the “Next” button511may be disabled until an IP address pool is created or assigned to the new network interface, in some examples.

In some examples, the method400may include creating a new IP address pool, at420. Creating the new IP address pool may include adding IP pool details, at422. The diagram520ofFIG. 5Cprovides an example of a user interface for creating a new IP address pool and adding IP pool details. The IP pool details may include a pool name, a netmask, and a range of IP addresses. In some examples, an existing IP pool may be used.

The method400may include selecting an IP address pool, at430. The selected IP address pool may include an existing IP address pool, or a newly created IP address pool from steps420and422. In some examples, the selection of the IP address pool may be automatic if only a single IP address pool exists in a selection list. The diagram540ofFIG. 5Dprovides an example of the interface for creating the new network segmentation interface with the IP pool automatically selected.

The method400may include selecting additional features for the new network interface, at440. The diagram540ofFIG. 5Eprovides an example of an interface for selecting additional features. The additional features/options may include block services, guest tools, or other features.

The method400may include creating the new network interface, at450. The diagram540ofFIG. 5Eprovides an example of a user interface for selecting additional features. The additional features/options may include block services, guest tools, or other features. If certain features are selected, the user interface may update to request additional information. For example, the diagram550ofFIG. 5Fprovides an example of an update to the user interface shown in the diagram540ofFIG. 5Eto include an entry561for a virtual IP address in response to selection of at least one of the block services or guest tools features. The diagram560ofFIG. 5Gprovides an example of an interface for tracking progress of creation of the new network interface.

The method400may include determining whether creation of the new network interface is successful, at460. In response to a determination that creation of the new network interface was successful, the method400may further include providing a successful creation indication, at470. Determining whether creation of the new network interface was successful may be based on a notification of successful creation, appearance of the network interface as an option, lack of an error message in creation of the network interface, etc. In response to a determination that creation of the new network interface failed, the method400may further include providing a creation failed indication, at480. The failure may be caused by lack of necessary information, such as failure to select an IP pool or selection of an IP pool that is already in use for the system, selection of incompatible features, etc. The diagram540ofFIG. 5Eprovides an example of an interface for selecting additional features. The additional features/options may include block services, guest tools, or other features.

FIG. 6depicts a block diagram of components of a computing node600in accordance with an embodiment of the present disclosure. It should be appreciated thatFIG. 6provides only an illustration of one implementation and does not imply any limitations with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made. The computing node600may implemented as the administrator system158, the computing node102, and/or the computing node112ofFIG. 1, the computing node202and/or the computing node212ofFIG. 2, or any combinations thereof. The computing node600may be configured to implement the methods300and400described with reference toFIGS. 3 and 4, respectively, in some examples, to migrate data associated with a service running on any VM.

The computing node600includes a communications fabric602, which provides communications between one or more processor(s)604, memory606, local storage608, communications unit610, I/O interface(s)612. The communications fabric602can be implemented with any architecture designed for passing data and/or control information between processors (such as microprocessors, communications and network processors, etc.), system memory, peripheral devices, and any other hardware components within a system. For example, the communications fabric602can be implemented with one or more buses.

The memory606and the local storage608are computer-readable storage media. In this embodiment, the memory606includes random access memory RAM614and cache616. In general, the memory606can include any suitable volatile or non-volatile computer-readable storage media. The local storage608may be implemented as described above with respect to local storage124and/or local storage130. In this embodiment, the local storage608includes an SSD622and an HDD624, which may be implemented as described above with respect to SSD126, SSD132and HDD128, HDD134respectively.

Various computer instructions, programs, files, images, etc. may be stored in local storage608for execution by one or more of the respective processor(s)604via one or more memories of memory606. In some examples, local storage608includes a magnetic HDD624. Alternatively, or in addition to a magnetic hard disk drive, local storage608can include the SSD622, a semiconductor storage device, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, or any other computer-readable storage media that is capable of storing program instructions or digital information.

The media used by local storage608may also be removable. For example, a removable hard drive may be used for local storage608. Other examples include optical and magnetic disks, thumb drives, and smart cards that are inserted into a drive for transfer onto another computer-readable storage medium that is also part of local storage608.

Communications unit610, in these examples, provides for communications with other data processing systems or devices. In these examples, communications unit610includes one or more network interface cards. Communications unit610may provide communications through the use of either or both physical and wireless communications links.

I/O interface(s)612allows for input and output of data with other devices that may be connected to computing node600. For example, I/O interface(s)612may provide a connection to external device(s)618such as a keyboard, a keypad, a touch screen, and/or some other suitable input device. External device(s)618can also include portable computer-readable storage media such as, for example, thumb drives, portable optical or magnetic disks, and memory cards. Software and data used to practice embodiments of the present disclosure can be stored on such portable computer-readable storage media and can be loaded onto local storage608via interface(s)612.1/0interface(s)612also connect to a display620.