Patent ID: 12229574

DETAILED DESCRIPTION

Logical network platform installation and upgrade in a virtualized computing system is described. In embodiments described herein, a virtualized computing system includes a software-defined datacenter (SDDC) comprising a server virtualization platform integrated with a logical network platform. The server virtualization platform includes clusters of physical servers (“hosts”) referred to as “host clusters.” Each host cluster includes a virtualization layer, executing on host hardware platforms of the hosts, which supports execution of virtual machines (VMs). A virtualization management server manages host clusters, the virtualization layers, and the VMs executing thereon.

In embodiments, the virtualization layer of a host cluster is integrated with an orchestration control plane, such as a Kubernetes® control plane. This integration enables the host cluster as a “supervisor cluster” that uses VMs to implement both control plane nodes having a Kubernetes control plane, and compute nodes managed by the control plane nodes. For example, Kubernetes pods are implemented as “pod VMs,” each of which includes a kernel and container engine that supports execution of containers. In embodiments, the Kubernetes control plane of the supervisor cluster is extended to support custom objects in addition to pods, such as VM objects that are implemented using native VMs (as opposed to pod VMs). A virtualization infrastructure administrator (VI admin) can enable a host cluster as a supervisor cluster and provide its functionality to development teams.

The SDDC includes a SD network layer across the host clusters. The SD network layer includes logical network services executing on a virtualized network infrastructure. The server virtualization platform manages the virtualized network infrastructure and, in cooperation with the logical network platform, manages the logical network services deployed on the virtualized network infrastructure. A VI admin interacts with the server virtualization platform for both server virtualization and network virtualization, as opposed to multiple admins interacting with two separate platforms. In embodiments, a virtualization management server includes a network management service that cooperates with a network management server (referred to as a “network manager”) of the logical network platform to manage the lifecycle of logical network services thereof. The virtualization management server provides a common interface (e.g., user interface (UI) and/or application programming interface (API)) for managing compute, network, and storage.

In embodiments, the virtualization management server also orchestrates installation and upgrade of the logical network platform based on a declarative specification. The declarative specification describes the proposed state of the logical network platform in the virtualized computing system. For example, which management cluster in which to deploy the network manager, which host cluster in which to deploy host binaries, which cluster in which to deploy edge services, and the like. The virtualization management server then deploys the network manager, host binaries, and edge VMs to implement the logical network platform used to provide SD networking. The virtualization management server can determine various configuration settings maintained therein and perform various precheck operations to verify that the logical network platform can be correctly installed. For upgrade, the virtualization management server cooperates with the network manager to initiate an upgrade of host binaries, edge VMs, and the network manager itself. The virtualization management server can execute prechecks to verify the upgrade operation can proceed.

The techniques described herein allow the virtualization management server to orchestrate the entire lifecycle of the SD network layer, including: installing binaries; deploying VMs on hosts; configuring and unconfiguring the virtualized infrastructure supporting logical network services, upgrading the supporting virtualized infrastructure on demand, and performing automated backup, restore, and recovery of the supporting infrastructure from inevitable restarts and failures. The techniques further allow the virtualization management server to orchestrate the lifecycle of logical network services, including deploying, configuring, reconfiguring, and removing such logical network services. These and further advantages and aspects of the disclosed techniques are described below with respect to the drawings.

FIG.1is a block diagram of a virtualized computing system100in which embodiments may be implemented. System100includes a cluster of hosts120(“host cluster118”) that may be constructed on server-grade hardware platforms such as an x86 architecture platforms. For purposes of clarity, only one host cluster118is shown. However, virtualized computing system100can include many of such host clusters118. As shown, a hardware platform122of each host120includes conventional components of a computing device, such as one or more central processing units (CPUs)160, system memory (e.g., random access memory (RAM)162), one or more network interface controllers (NICs)164, and optionally local storage163. CPUs160are configured to execute instructions, for example, executable instructions that perform one or more operations described herein, which may be stored in RAM162. NICs164enable host120to communicate with other devices through a physical network180. Physical network180is a physical network that enables communication between hosts120and between other components and hosts120(other components discussed further herein).

In the embodiment illustrated inFIG.1, hosts120access shared storage170by using NICs164to connect to physical network180. In another embodiment, each host120contains a host bus adapter (HBA) through which input/output operations (IOs) are sent to shared storage170over a separate network (e.g., a fibre channel (FC) network). Shared storage170include one or more storage arrays, such as a storage area network (SAN), network attached storage (NAS), or the like. Shared storage170may comprise magnetic disks, solid-state disks, flash memory, and the like as well as combinations thereof. In some embodiments, hosts120include local storage163(e.g., hard disk drives, solid-state drives, etc.). Local storage163in each host120can be aggregated and provisioned as part of a virtual SAN (VSAN), which is another form of shared storage170.

A software platform124of each host120provides a virtualization layer, referred to herein as a hypervisor150, which directly executes on hardware platform122. In an embodiment, there is no intervening software, such as a host operating system (OS), between hypervisor150and hardware platform122. Thus, hypervisor150is a Type-1 hypervisor (also known as a “bare-metal” hypervisor). As a result, the virtualization layer in host cluster118is a bare-metal virtualization layer executing directly on host hardware platforms. Hypervisor150abstracts processor, memory, storage, and network resources of hardware platform122to provide a virtual machine execution space within which multiple virtual machines (VM) may be concurrently instantiated and executed. One example of hypervisor150that may be configured and used in embodiments described herein is a VMware ESXi® hypervisor provided as part of the VMware vSphere® solution made commercially available by VMware, Inc. of Palo Alto, CA In the example ofFIG.1, host cluster118is enabled as a “supervisor cluster,” described further herein, and thus VMs executing on each host120include pod VMs130and native VMs140. Some native VMs140shown as support VMs145, have specific functions within host cluster118. For example, support VMs145can provide control plane or data plane virtualized infrastructure. An embodiment of software platform124is discussed further below with respect toFIG.2.

Host cluster118is configured with a software-defined (SD) network layer175. SD network layer175includes logical network services executing on virtualized infrastructure in host cluster118. The virtualized infrastructure that supports the logical network services includes hypervisor-based components, such as resource pools, distributed switches, distributed switch port groups and uplinks, etc., as well as VM-based components, such as router control VMs, load balancer VMs, edge service VMs, etc. Logical network services include logical switches, logical routers, logical firewalls, logical virtual private networks (VPNs), logical load balancers, and the like, implemented on top of the virtualized infrastructure.

Virtualization management server116is a physical or virtual server of a server virtualization platform that manages host cluster118and the virtualization layer therein. Virtualization management server116installs a control plane (CP) agent (“CP agent152”) in hypervisor150to add a host120as a managed entity. Virtualization management server116logically groups hosts120into cluster118to provide cluster-level functions to hosts120, such as VM migration between hosts120(e.g., for load balancing), distributed power management, dynamic VM placement according to affinity and anti-affinity rules, and high-availability. The number of hosts120in cluster118may be one or many. Virtualization management server116can manage more than one host cluster118. As described further herein, virtualization management server116also orchestrates management of SD network layer175and all or a portion of shared storage170.

In an embodiment, virtualization management server116further enables host cluster118as a supervisor cluster101. Virtualization management server116installs additional CP agents152in hypervisor150to add host120to supervisor cluster101. Supervisor cluster101integrates an orchestration control plane115with host cluster118. In embodiments, orchestration control plane115is derived from Kubernetes. In supervisor cluster101, hosts120become nodes for use by orchestration control plane115. Virtualization management server116provisions one or more virtual servers as “master servers,” which function as management entities in orchestration control plane115. In the embodiment ofFIG.1, supervisor cluster101includes a supervisor Kubernetes master104that functions as such a master server. For purposes of clarity, supervisor Kubernetes master104is shown as a separate logical entity. For practical implementations, supervisor Kubernetes master104can be implemented as a support VM145(an optionally one or more pod VMs130) in host cluster118. Further, although only one supervisor Kubernetes master104is shown, supervisor cluster101can include more than one supervisor Kubernetes master104in a logical cluster for redundancy and load balancing. Although host cluster118inFIG.1is enabled as supervisor cluster101, the SD network orchestration techniques described herein can be employed on host clusters118that are not so enabled.

In an embodiment, virtualized computing system100further includes storage manager110. Storage manager110is a physical or virtual server that provisions virtual disks in shared storage170(or a VSAN formed from local storage163) as independent objects. That is, virtual disks that persist apart from the lifecycle of any VM or container. Various components can interact with storage manager110to provision persistent storage, such as virtualization management server116and supervisor Kubernetes master104. Storage manager110can operate independently from virtualization management server116(e.g., as an independent physical or virtual server). Alternatively, storage manager110can be a service in virtualization management server116.

In an embodiment, virtualized computing system100further includes a network manager112. Network manager112is a physical or virtual server of a logical network platform that manages logical network services of SD network layer175. In an embodiment, network manager112comprises one or more virtual servers deployed by virtualization management server116as VMs. Network manager112and/or virtualization management server116installs additional control plane agents152and data plane (DP) modules153in hypervisor150to add a host120as a managed entity, referred to as a transport node. In this manner, host cluster118can be a cluster103of transport nodes.

In embodiments, the logical network platform of virtualized computing system100comprises three functions: management, control, and data. The management function provides an entry point of the logical network platform in the form of an API. The management function is responsible for performing operational tasks on the control and data functions. The control function computes the runtime state of the SD network layer175based on configuration from the management function. The control function is also responsible for disseminating topology information reported by the data function and configurations into components of the data function. The data function includes components of SD network layer175that perform stateless forwarding and transformation of packets based on tables populated by the control function. In the nomenclature used herein, the management and control functions of the logical network platform are performed by a logical network control plane119, which includes software components in both virtualization management server116and network manager112. That is, virtualization management server116includes a component119-1, and network manager includes a component119-2, of logical network control plane119. In an embodiment, the management function is divided between components119-1and119-2, whereas the control function is implemented entirely within component119-2. The data function of the logical network platform is implemented in hypervisor150and support VMs145. One example logical network platform that can be configured and used in embodiments described herein is a VMware NSX® platform made commercially available by VMware, Inc. of Palo Alto, CA.

In embodiments, virtualized computing system100includes edge transport nodes178. Edge transport nodes178are physical or virtual servers in SD network layer175(shown logically separate by way of example) that provide egress to, and ingress from, external networks. In embodiments, edge transport nodes are VMs (e.g., support VMs145), formed into one or more logical clusters, which execute in one or more host clusters118. Edge transport nodes178are discussed further herein.

Virtualization management server116, network manager112, and storage manager110comprise a virtual infrastructure (VI) control plane113for host cluster118, shared storage170, and SD network layer175. Similar to the logical network platform, the server virtualization platform includes management, control, and data functions. The management and control functions are performed by a server virtualization control plane117executing in virtualization management server116. A storage virtualization platform likewise can include management and control plane functions executing virtualization management server116and/or storage manager110.

In an embodiment, system100further includes an image registry190and a container repository192. As described further herein, containers of supervisor cluster101execute in pod VMs130. Containers are spun up from container images. Container images are registered with image registry190, which manages a plurality of container repositories (one of which is shown inFIG.1as container repository192) in which images of all containers registered with image registry190are stored. During registration of a container image, image registry190collects authentication information and during subsequent requests to access the registered container images, authenticates the requester using the collected authentication information. Once the requester is authenticated, image registry190permits the requester to fetch the container images registered to the requester.

A VI admin can interact with virtualization management server116through a VM management client106. Through VM management client106, a VI admin commands virtualization management server116to form host cluster118, configure resource pools, resource allocation policies, and other cluster-level functions, configure storage and networking, enable supervisor cluster101and the like. In embodiments, VI admin interacts with virtualization management server116through VM management client106to provide a declarative specification describing a proposed state of SD network layer175. In embodiments, VI admin interacts with virtualization management server116through VM management client106to perform other functions, such as enabling host cluster118as supervisor cluster101, which in turn provide a declarative specification describing a proposed state of SD network layer175(e.g., a state necessary for operating a supervisor cluster).

Kubernetes client102represents an input interface for a user to supervisor Kubernetes master104. Kubernetes client102is commonly referred to as kubectl. Through Kubernetes client102, a user submits desired states of the Kubernetes system, e.g., as YAML documents, to supervisor Kubernetes master104. In embodiments, the user submits the desired states within the scope of a supervisor namespace. In response, supervisor Kubernetes master104configures supervisor cluster101to match the desired state by creating pod VMs130, creating native VMs140, connecting VMs to storage and logical networks, destroying pod VMs130and native VMs140, and the like. The resources can be deployed within the confines of supervisor namespaces. In this manner, the user interacts with supervisor Kubernetes master104to deploy applications in supervisor cluster101within defined supervisor namespaces.

As described herein, virtualization management server116orchestrates SD network layer175based on a declarative specification, which describes a proposed state of SD network layer175for host cluster118. In embodiments, host cluster118is enabled as a supervisor cluster101described above and virtualization management server116orchestrates SD network layer175to provide an SD network configuration for supervisor cluster101. In other embodiments, host cluster118is not enabled as a supervisor cluster101. In such case, some components inFIG.1can be inactive and/or omitted, including Kubernetes client102, supervisor Kubernetes master104, image registry190, container repository192, and pod VMs130. Virtualization management server116can orchestrate SD network layer175to provide an SD network for native VMs140executing in host cluster118. While supervisor cluster101and a Kubernetes system are described in various examples herein, the SD networking orchestration techniques are not limited to such examples and are broadly applicable to virtualization systems having one or more host clusters in various configurations.

FIG.2is a block diagram depicting software platform124according an embodiment. As described above, software platform124of host120includes hypervisor150that supports execution of VMs, such as pod VMs130and native VMs140. In an embodiment, hypervisor150includes a VM management daemon213, a host daemon214, a pod VM controller216, an image service218, network DP modules220, and network agents222. VM management daemon213is a control plane agent152of server virtualization CP117. VM management daemon213provides an interface to host daemon214for virtualization management server116. Host daemon214is configured to create, configure, and remove VMs (e.g., pod VMS130and native VMs140).

Pod VM controller216is a control plane agent152of orchestration control plane115for supervisor cluster101and allows supervisor Kubernetes master104to interact with hypervisor150. Pod VM controller216configures the respective host as a node in supervisor cluster101. Pod VM controller216manages the lifecycle of pod VMs130, such as determining when to spin-up or delete a pod VM. Pod VM controller216also ensures that any pod dependencies, such as container images, networks, and volumes are available and correctly configured. Pod VM controller216is omitted if host cluster118is not enabled as a supervisor cluster101.

Image service218is configured to download and extract container images to shared storage170such that the container images can be mounted by pod VMs130. Image service218is also responsible for managing the storage available for container images within shared storage170. This includes managing authentication with image registry190, assuring providence of container images by verifying signatures, updating container images when necessary, and garbage collecting unused container images. Image service218is omitted if host cluster118is not enabled as a supervisor cluster101.

Network agents222comprises control plane agents152of logical network CP119. Network agents222are configured to cooperate with network manager112to control network DP modules220to implement logical network services. Network agents222and network DP modules220configure the respective host as a transport node in a cluster103of transport nodes. Network DP modules220augment the network virtualization functionality of hypervisor150.

Each pod VM130has one or more containers206running therein in an execution space managed by container engine208. The lifecycle of containers206is managed by pod VM agent212. Both container engine208and pod VM agent212execute on top of a kernel210(e.g., a Linux® kernel). Each native VM140has applications202running therein on top of an OS204. Native VMs140do not include pod VM agents and are isolated from pod VM controller216. Container engine208can be an industry-standard container engine, such as libcontainer, runc, or containerd. Pod VMs130are omitted if host cluster118is not enabled as a supervisor cluster101.

FIG.3is a block diagram depicting a logical view of virtualized computing system100having applications executing therein according to an embodiment. In the embodiment, supervisor cluster101is implemented by an SDDC350. SDDC350includes a server virtualization platform302and a logical network platform303. Server virtualization platform302comprises host clusters118, a virtualization layer hypervisors150), and server virtualization control plane117(e.g., virtualization management server116). Logical network platform303comprises network manager112and associated components in the virtualization layer (e.g., CP agents and DP agents). Server virtualization platform302cooperates with logical network platform303to orchestrate SD network layer175. Server virtualization control plane117(e.g., virtualization management server116) provides a single entity for orchestration of compute, storage, and network.

In some embodiments, a VI admin interacts with virtualization management server116to configure SDDC350to implement supervisor cluster101and an SD network308in supervisor cluster101. SD network308includes deployed virtualized infrastructure (e.g., distributed switch, port groups, resource pools, support VMs145) and logical network services implemented thereon (e.g., logical switching, logical routing, etc.).

Supervisor cluster101includes orchestration control plane115, which includes supervisor Kubernetes master(s)104and pod VM controllers216. The VI admin interacts with Virtualization management server116to create supervisor namespaces312. Each supervisor namespace312includes a resource pool and authorization constraints. The resource pool includes various resource constraints on supervisor namespace312(e.g., reservation, limits, and share (RLS) constraints). Authorization constraints provide for which roles are permitted to perform which operations in supervisor namespace312(e.g., allowing VI admin to create, manage access, allocate resources, view, and create objects; allowing DevOps to view and create objects; etc.). A user interacts with supervisor Kubernetes master104to deploy applications310on supervisor cluster101within scopes of supervisor namespaces312. In the example, the user deploys an application310-1on pod VM(s)130, an application310-2on native VMs140and application310-3on both a pod VM130and a native VM140.

In embodiments, the user also deploys a guest cluster314on supervisor cluster101within a supervisor namespace312to implement a Kubernetes cluster. Guest cluster314is constrained by the authorization and resource policy applied by the supervisor namespace in which it is deployed. Orchestration control plane115includes guest cluster infrastructure software (GCIS) configured to realize guest cluster314as a virtual extension of supervisor cluster101. The GCIS creates and manages guest cluster infrastructure objects316to provide abstract and physical representations of infrastructure supporting guest cluster314. The GCIS executes in orchestration control plane115(e.g., in supervisor Kubernetes master104). A user can interact with the Kubernetes control plane in guest cluster314to deploy various containerized applications (an application310-4). Applications310can communicate with each other or with an external network through SD network308.

As noted above, in some embodiments, SDDC350is not enabled as a supervisor cluster101. In such case, SD network308is generally deployed in SDDC350for use, by the workloads executing therein. Supervisor cluster101, orchestration control plane115, supervisor namespaces312, guest cluster infrastructure objects316, guest cluster314, and pod VMs130can be omitted from the logical view shown inFIG.3. Thus, SDDC350can generally support execution of native VMs140, which utilize an SD network308orchestrated by server virtualization platform302as described herein.

FIG.4is a block diagram depicting networked host clusters in virtualized computing system100according to an embodiment. In the example shown, virtualized computing system100includes two host clusters118-1and118-2, each configured the same or similar as host cluster118shown inFIG.1. Each host cluster118-1and118-2includes VMs130/140executing therein. Each VM130/140includes one or more virtual network interfaces to port(s) on a virtual switch406. Virtual switch406includes ports coupled to NICs164. NICs164are coupled to physical switches408on physical network180. Physical network180includes one or more physical routers410. Physical routers410are coupled between physical network180and an external network412, such as a wide area network (WAN) (e.g., the public Internet).

In an embodiment, network manager112and virtualization management server116comprise VMs in a management cluster402. Management cluster402is a logical cluster implemented within a host cluster118. For example, management cluster402can be implemented within another host cluster118in addition to host cluster118-1and118-2. In another example, management cluster402can be implemented within one of host cluster118-1or118-2. Network manager112and virtualization management server116have virtual network interfaces coupled to ports on a virtual switch406same as VMs130/140.

In an embodiment, support VMs145that include edge transport nodes178form an edge cluster404. Edge cluster404is a logical cluster implemented within a host cluster118. For example, edge cluster404can be implemented in another host cluster118in addition to host cluster118-1and118-2. In another example, edge cluster404can be implemented within one of host cluster118-1or118-2. Support VMs145, including edge transport nodes178, have virtual network interfaces coupled to ports on a virtual switch406same as VMs130/140, network manager112, and virtualization management server116.

VMs130/140exchange data among themselves over physical network180within L2 networks (L2 broadcast domains) referred to herein as “segments.” A virtual local area network backed (VLAN-backed) segment (also referred to as VLAN network or VLAN) is an L2 broadcast domain that is implemented as a traditional VLAN on physical network180. In the example shown, physical network180includes three VLAN-backed segments: a management VLAN-backed segment (management VLAN414); an uplink VLAN-backed segment (uplink VLAN416); and an overlay VLAN-backed segment (overlay VLAN418). Ports on a virtual switch406can be associated with a specific VLAN-backed segment of physical network180.

For example, network manager112, virtualization management server116, and edge transport nodes178can be coupled to ports on respective virtual switches406that are associated with management VLAN414. This allows communication of management traffic among network manager112, virtualization management server116, and edge transport nodes178. Although not specifically shown, components in hypervisor150within each host120can be coupled to management VLAN414through a virtual switch406(e.g., control plane agents152, pod VM controllers216, etc.). Edge transport nodes178can also be coupled to ports on virtual switch406associated with uplink VLAN416. Traffic on uplink VLAN416is routable to external network412via physical routers410. Uplink VLAN416carries north-south traffic between host clusters118and external network412.

VMs130/140and edge transport nodes178can be coupled to ports on respective virtual switches406associated with overlay VLAN418. Overlay VLAN418carries east-west traffic between VMs130/140. Overlay VLAN418supports overlay-backed segments or “logical segments.” A logical segment is a logical L2 network between VMs using L2-over-L3 tunnels through overlay VLAN418. Example tunneling protocols include VXLAN and Geneve. A logical segment is realized by deploying a logical switch. Overlay VLAN418can carry traffic associated with a plurality of different logical segments, each being a different logical network in an SD network. To support logical segments, virtual switches406are part of a distributed switch420that spans the hosts for which communication is desired. In the example, distributed switch420includes virtual switches406in each of host cluster118-1,118-2, and edge cluster404. This allows VMs in host cluster118-1to exchange data with VMs in host cluster118-2through logical networks on the overlay network (overlay VLAN418). This also allows VMs in either host cluster118-1or host cluster118-2reach external network412through edge transport nodes178.

FIG.5is a block diagram depicting a distributed switch420according to an embodiment. Distributed switch420includes one or more port groups502and an uplink port group508. Uplink port group508includes one or more uplinks510. Each port group502can be associated with a VLAN-backed segment of physical network180. VMs connected to a port group502send and receive traffic over the particular VLAN-backed segment associated with that port group502. Each uplink510is associated with one or more NICs164. A port group502can be assigned an uplink510to allow ingress to and egress from hosts120in which distributed virtual switch406is deployed. Virtualization management server116pushes out port groups502and uplinks510to each virtual switch406. In transport nodes, virtual switches406that implement distributed switch420are augmented with distributed router (DR) modules504and logical switch modules506which are part of the data plane of the logical network platform. DR modules504implement one or more DRs of an SD network. A DR is an L3 logical device that routes between overlay-backed segments (logical networks). A logical switch implements an overlay-backed segment (logical network).

FIG.6Ais a block diagram depicting a virtualization management server116in communication with a network manager112according to an embodiment. Virtualization management server116includes an interface606, an orchestrator608, a supervisor cluster service612, virtualization management services610, a network management service614, and a database616. Network manager112includes an API634and an upgrade coordinator635. Interface606includes an API602and a user interface (UI)604.

A VI admin interacts with virtualization management server116through interface606, either using API602directly or through UI604(e.g., a web interface). Interface606allows VI Admin to perform various operations, such as create host clusters, add hosts to host clusters, VM lifecycle management, create resource pools, enable supervisor clusters, create VSANs, and the like. In embodiments, a VI admin can supply a declarative specification installation and/or upgrade of a logical network platform (e.g., network manager112, edge VMs422, CP agents152, DP modules153, etc.). In embodiments, orchestrator608is the primary component responsible for collecting user input and invoking various services in response thereto. Orchestrator608can invoke network management service614to manage installation/upgrade of the logical network platform. Orchestrator608can invoke various other virtualization management services610for other tasks (e.g., cluster creation, host management, VM lifecycle management, resource pool management, etc.). The various services of virtualization management server116store objects in database616. Example objects include VMs618, distributed switches620, resource pools622, and configuration parameters623. In embodiments, a user interacts with supervisor cluster service612through interface606to enable host cluster118as a supervisor cluster101. Supervisor cluster service612invokes orchestrator608to manage SD network layer175on its behalf.

Orchestrator608provides a declarative specification describing installation or upgrade of the logical network platform. In addition, orchestrator608can provide a declarative specification describing a proposed state of SD network layer175to network management service614(once the logical network platform has been installed or upgraded). Orchestrator608can receive a declarative specification directly from a VI admin or from another service, such as supervisor cluster service612. Network management service614is in communication with network manager112over management VLAN414and cooperates with network manager112through API634. Orchestrator608can invoke an API624of network management service614based on a declarative specification. In embodiments, network management service614provides a proxy for API calls to API634of network manager112. For example, if declarative specification calls for upgrade of the logical network platform, network management service614can generate the API calls to API634of network manager112to orchestrate the upgrade.

FIGS.6B and6Care block diagrams depicting alternative configurations of orchestrator608, network management service614, and supervisor cluster service612according to embodiments.FIG.6Bshows an example where orchestrator network functionality631is incorporated into network management service614, rather than as part of orchestrator608. Further, network management service614is incorporated into supervisor cluster service612. The VI admin interacts with supervisor cluster service612through interface606, which then deploys and manages supervisor clusters and associated SD networking.FIG.6Cshows an example where orchestrator network functionality631is incorporated into network management service614, rather than as part of orchestrator608. Network management service614remains apart from supervisor cluster service612. Supervisor cluster service612interacts with network management service614to orchestrate and manage the logical network platform. An VI admin can also interact with network management service614through interface606to manage the logical network platform.

FIG.7is a block diagram depicting a process for obtaining software to install or upgrade a logical network platform according to an embodiment. In embodiments, network management service614includes a download manager702. Download manager702is configured to communicate with a remote server708that includes logical network platform binaries706. Logical network platform binaries706include one or more images of software for installing or upgrading the logical network platform of virtualized computing system100. Download manager702requests one or more versions of logical network platform software from remote server708and obtains the appropriate logical network platform binaries706. In another embodiment, rather than directly obtaining logical network platform binaries706from remote server708, download manager702orchestrates the download to local server708. Network management service614can further include upload manager704, which is configured to communicate with local server708to obtain logical network platform binaries706that have been previously downloaded from remote server708.

FIG.8is a block diagram depicting a logical view of installing/updating a logical network platform in a virtualized computing system according to an embodiment. Network management service614obtains logical network platform binaries706as discussed above, which are stored on virtualization management server116. During an install process, network management service614uses logical network platform binaries706to deploy one or more VMs to execute network manager112. Network management service614deploys binaries804to hosts120, which can include CP agents152and/or DP modules153. Network management service614deploys one or more VMs to execute edge services (e.g., edge VMs422). During an upgrade process, network management service614communicates with network manager112to upgrade the upgrade coordinator802. Thereafter, network management service614cooperates with upgrade coordinator802to update the logical network platform, including network manager112, binaries804, and/or edge VMs422. During the install and upgrade processes, network management service614performs various precheck operations to satisfy preconditions of the install or upgrade. Network management service614can communicate with database616to obtain various data to assist the install/upgrade process, including obtaining configuration parameters623. Embodiments of the install process and the upgrade process are discussed below.

FIG.9is a flow diagram depicting a method900of installing a logical network platform in a virtualized computing system according to an embodiment. Method900can be performed by network management service614, which comprises software executing on CPU, memory, storage, and network resources managed by a virtualization layer (e.g., a hypervisor) or a host OS.

Method900begins at step902, where network management service614receives a declarative specification defining the install. The declarative specification can identify the management cluster in which to install the network manager, the host cluster in which to install binaries, and the cluster in which to install edge VMs. The declarative specification can also identify the version of the logical network platform to install.

At step904, network management service614obtains configuration parameters623from virtualization management server116(e.g., from database616). Network management service614retrieves configuration parameters623in order to inherit the settings from virtualization management server116. Configuration parameters623include, for example, identities of the management cluster, the host cluster, networks, datastores, and the like. Configuration parameters623can further include information for DNS servers, gateways, subnets, NTP configurations, syslog servers, and the like.

At step906, network management service614identifies the appropriate logical network platform binaries to be used for the install based on the declarative specification and the configuration parameters. For example, the declarative specification can specify a particular version of the logical network platform. The logical network platform can include different images for a given version based on the size of the host cluster being managed (e.g., small, medium, large, etc.). Network management service614can determine the size of the installation based on configuration parameters623and select the appropriate image for the specified version. If the declarative specification does not specify a version, network management service614can select the latest version of the logical network platform that is present on virtualization management server116.

At step908, network management service614executes one or more prechecks prior to installation. Various prechecks can be performed to ensure the selected logical network platform can be installed correctly in the particular configuration of virtualized computing system100. Prechecks include, for example, verifying the availability resources in the management cluster to support network manager112, verifying management/host cluster availability, verifying the datastore availability, verifying required features are enabled (e.g., distributed resource scheduling, high availability, etc.), verifying versions of existing software (e.g., hypervisor version, virtualization management server version, etc.), verifying network information and connectivity, verifying the installation binaries, and the like. If any prechecks fail, network management service614generates a notification so that the user can correct any deficiency prior to the installation process.

At step910, network management service614deploys network manager112. In embodiments, network management service614creates a resource pool in the management cluster and provisions one or more VMs in the resource pool to execute network manager112. At step912, network management service614cooperates with network manager112to install host binaries (e.g., CP agents152and/or DP modules153) on hosts120in host cluster118. This enables hosts120to cooperate with network manager112to implement SD network layer175. At step914, network management service614cooperates with network manager112to deploy edge VMs422(e.g., in host cluster118or in another cluster).

FIG.10is a flow diagram depicting a method1000of upgrading a logical network platform in a virtualized computing system according to an embodiment. Method1000can be performed by network management service614, which comprises software executing on CPU, memory, storage, and network resources managed by a virtualization layer (e.g., a hypervisor) or a host OS.

Method1000begins at step1002, where network management service614receives a declarative specification defining the upgrade. The declarative specification can identify the network manager to upgrade, the host cluster in which to upgrade binaries, and the edge VMs to upgrade. The declarative specification can also identify the version of the logical network platform to use in the upgrade.

At step1004, network management service614cooperates with network manager112to upgrade the upgrade coordinator635. Network management service614then cooperates with upgrade coordinator635to perform the remaining upgrade steps. At step1006, network management service614executes one or more prechecks prior to upgrade. Prechecks include, for example, verifying versions of existing software (e.g., hypervisor version, virtualization management server version, etc.), verifying network information and connectivity, verifying the installation binaries, and the like. If any prechecks fail, network management service614generates a notification so that the user can correct any deficiency prior to the upgrade process.

At step1008, network management service614cooperates with upgrade coordinator635to upgrade host binaries. In an embodiment, host binary upgrade is performed in parallel across a plurality of hosts120in host cluster118(step1010). For example, network management service614can upgrade five hosts at a time until all hosts have been upgraded (or any other number of hosts). In another embodiment, network management service614upgrades hosts120in host cluster118serially one host at a time until all hosts120have been upgraded (step1012). At step1014, network management service614cooperates with upgrade coordinator635to upgrade edge VMs422. At step1016, network management service614cooperates with upgrade coordinator635to upgrade network manager112.

The embodiments described herein may employ various computer-implemented operations involving data stored in computer systems. For example, these operations may require physical manipulation of physical quantities. Usually, though not necessarily, these quantities may take the form of electrical or magnetic signals, where the quantities or representations of the quantities can be stored, transferred, combined, compared, or otherwise manipulated. Such manipulations are often referred to in terms such as producing, identifying, determining, or comparing. Any operations described herein that form part of one or more embodiments may be useful machine operations.

One or more embodiments of the invention also relate to a device or an apparatus for performing these operations. The apparatus may be specially constructed for required purposes, or the apparatus may be a general-purpose computer selectively activated or configured by a computer program stored in the computer. Various general-purpose machines may be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations.

The embodiments described herein may be practiced with other computer system configurations including hand-held devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, etc.

One or more embodiments of the present invention may be implemented as one or more computer programs or as one or more computer program modules embodied in computer readable media. The term computer readable medium refers to any data storage device that can store data which can thereafter be input to a computer system. Computer readable media may be based on any existing or subsequently developed technology that embodies computer programs in a manner that enables a computer to read the programs. Examples of computer readable media are hard drives, NAS systems, read-only memory (ROM), RAM, compact disks (CDs), digital versatile disks (DVDs), magnetic tapes, and other optical and non-optical data storage devices. A computer readable medium can also be distributed over a network-coupled computer system so that the computer readable code is stored and executed in a distributed fashion.

Although one or more embodiments of the present invention have been described in some detail for clarity of understanding, certain changes may be made within the scope of the claims. Accordingly, the described embodiments are to be considered as illustrative and not restrictive, and the scope of the claims is not to be limited to details given herein but may be modified within the scope and equivalents of the claims. In the claims, elements and/or steps do not imply any particular order of operation unless explicitly stated in the claims.

Virtualization systems in accordance with the various embodiments may be implemented as hosted embodiments, non-hosted embodiments, or as embodiments that blur distinctions between the two. Furthermore, various virtualization operations may be wholly or partially implemented in hardware. For example, a hardware implementation may employ a look-up table for modification of storage access requests to secure non-disk data.

Many variations, additions, and improvements are possible, regardless of the degree of virtualization. The virtualization software can therefore include components of a host, console, or guest OS that perform virtualization functions.

Plural instances may be provided for components, operations, or structures described herein as a single instance. Boundaries between components, operations, and data stores are somewhat arbitrary, and particular operations are illustrated in the context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within the scope of the invention. In general, structures and functionalities presented as separate components in exemplary configurations may be implemented as a combined structure or component. Similarly, structures and functionalities presented as a single component may be implemented as separate components. These and other variations, additions, and improvements may fall within the scope of the appended claims.