Patent Publication Number: US-2022217050-A1

Title: Policy management across multiple cloud computing environments within a network

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This application is a continuation of U.S. patent application Ser. No. 17/101,892, entitled “MULTI-ENVIRONMENT NETWORKING MANAGEMENT SYSTEM,” filed on Nov. 23, 2020, the contents of which are incorporated by reference in their entirety for all purposes. 
    
    
     BACKGROUND 
     This disclosure relates in general to systems and methods for managing policies across multiple cloud computing environments within a network. 
     Different cloud computing environments use different languages, data sources, commands, and protocols. For example, each cloud computing provider may use a different method to allocate subnets and IP addresses to instances within the respective cloud computing environment. This may cause instances within a network having multiple cloud computing environments to have overlapping subnets and IP addresses. Further, each cloud computing provider may require different instructions for applying policies to instances within the respective cloud computing environment. 
     SUMMARY 
     Exemplary embodiments of the invention provide systems and methods for determining, enforcing, and managing policies across different cloud computing environments within a network. According to an aspect of the invention, a system includes a user interface that receives configuration settings to be applied to a plurality of first instances and a plurality of second instances. A plurality of collectors of the system that retrieve information from a first cloud computing environment and a second cloud computing environment, and a controller determines policies for the plurality of first instances and the plurality of second instances. A configurator of the system applies the policies to the plurality of first instances and the plurality of second instances, a first tester that inspects operations of the plurality of first instances and detects violations of the policies, and an enforcer responds to the detected violations by receiving a notification from the first tester that a first instance from the plurality of first instances violated a first policy. The controller instructs the configurator to apply the first policy to the first instance again, shut down the first instance or cut off communications with the first instance. 
     According to another aspect of the invention, a method may include receiving configuration settings to be applied to a plurality of first instances within a first cloud computing environment and a plurality of second instances within a second cloud computing environment. In one step, information from the first cloud computing environment and the second cloud computing environment is retrieved. The information comprises a plurality of functionalities of the first cloud computing environment and the second cloud computing environment. The method further includes determining policies for the plurality of first instances within the first cloud computing environment and the plurality of second instances within the second cloud computing environment as functions of the configuration settings and the information. The policies are applied to the plurality of first instances within the first cloud computing environment and the plurality of second instances within the second cloud computing environment. Operations of the plurality of first instances within the first cloud computing environment and the plurality of second instances within the second cloud computing environment are inspected and violations of the policies by the plurality of first instances within the first cloud computing environment and the plurality of second instances within the second cloud computing environment are detected. In addition, the method also includes responding to the detected violations by receiving a notification that a first instance from the plurality of first instances violated a first policy. A controller instructs a configurator to apply the first policy to the first instance again, shut down the first instance or cut off communications with the first instance. 
     According to another aspect of the invention, a system may include a user interface that receives configuration settings to be applied to a plurality of first instances within a first cloud computing environment and a plurality of second instances within a second cloud computing environment. The system may also include a plurality of collectors that retrieve information from the first cloud computing environment and the second cloud computing environment. The information comprises a plurality of functionalities of the first cloud computing environment and the second cloud computing environment. In addition, the system may include a controller that determines policies for the plurality of first instances within the first cloud computing environment and the plurality of second instances within the second cloud computing environment as functions of the configuration settings and the information. Further, the system may include a configurator that applies the policies to the plurality of first instances within the first cloud computing environment and the plurality of second instances within the second cloud computing environment, a first tester that inspects operations of the plurality of first instances within the first cloud computing environment and detects violations of the policies by the plurality of first instances within the first cloud computing environment, and an enforcer that responds to the detected violations by receiving a notification from the first tester that a first instance from the plurality of first instances violated a first policy. The controller instructs the configurator to apply the first policy to the first instance again, shut down the first instance or cut off communications with the first instance. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The present disclosure is described in conjunction with the appended figures: 
         FIG. 1  depicts a block diagram of an embodiment of a network; 
         FIG. 2  depicts a block diagram of an embodiment of a cloud computing environment that includes a control system; 
         FIG. 3  depicts a block diagram of an embodiment of a portion of the network shown in  FIG. 1 ; 
         FIG. 4  depicts a block diagram of a cloud Open Systems Interconnection (OSI) model for cloud computing environments; 
         FIG. 5  depicts a flowchart of an embodiment of a method; and 
         FIG. 6  depicts a flowchart of an embodiment of a portion of the method shown in  FIG. 5 . 
     
    
    
     In the appended figures, similar components and/or features may have the same reference label. Further, various components of the same type may be distinguished by following the reference label by a dash and a second label that distinguishes among the similar components. If only the first reference label is used in the specification, the description is applicable to any one of the similar components having the same first reference label irrespective of the second reference label. 
     DETAILED DESCRIPTION 
     The ensuing description provides preferred exemplary embodiment(s) only, and is not intended to limit the scope, applicability or configuration of the disclosure. Rather, the ensuing description of the preferred exemplary embodiment(s) will provide those skilled in the art with an enabling description for implementing a preferred exemplary embodiment. It is understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope as set forth in the appended claims. 
     Referring first to  FIG. 1 , a block diagram of an embodiment of a network  100  is shown. The network  100  may include a first cloud computing environment  140   a , a second cloud computing environment  140   b , and a third cloud computing environment  140   c  that communicate via a public Internet  125 . The first cloud computing environment  140   a , the second cloud computing environment  140   b , and the third cloud computing environment  140   c  may be public clouds. Some examples of the first cloud computing environment  140   a , the second cloud computing environment  140   b , and the third cloud computing environment  140   c  include Amazon Web Services (AWS)®, Google Cloud Platform (GCP)®, and Microsoft Azure®. Some or all of the first cloud computing environment  140   a , the second cloud computing environment  140   b , and the third cloud computing environment  140   c  may be different from each other. For example, the first cloud computing environment  140   a  may run Amazon Web Services (AWS)®, the second cloud computing environment  140   b  may run Google Cloud Platform (GCP)®, and the third cloud computing environment  140   c  may run Microsoft Azure®. Although three cloud computing environments are shown, any suitable number of cloud computing environments may be provided. 
     Each of the cloud computing environments may communicate with the Internet via a secure connection. For example, the first cloud computing environment  140   a  may communicate with the public Internet  125  via a virtual private network (VPN)  190   a , the second cloud computing environment  140   b  may communicate with the public Internet  125  via a VPN  190   b , and the third cloud computing environment  140   c  may communicate with the public Internet  125  via a VPN  190   c.    
     A plurality of enterprises  198  may also communicate with the public Internet  125  via a VPN  190   d . Some examples of the enterprises  198  may include corporations, educational facilities, governmental entities, and private consumers. In addition, the plurality of enterprises  198  may communicate with a plurality of first domain users  195   a  via a VPN  190   f , a plurality of second domain users  195   b  via a VPN  190   g , and a plurality of third domain users  195   c  via a VPN  190   h . Some examples of the first domain users  195   a , the second domain users  195   b , and the third domain users  195   c  may include individual users that are authorized to use computing resources of the enterprises  198 . 
     Further, a control system  185  may communicate with the public Internet  125  via a VPN  190   e . As discussed in further detail below, the control system  185  may configure, test, and enforce policies across the first cloud computing environment  140   a , the second cloud computing environment  140   b , and the third cloud computing environment  140   c . For example, the control system  185  may ensure that the policies are consistent across the first cloud computing environment  140   a , the second cloud computing environment  140   b , and the third cloud computing environment  140   c.    
     With reference to  FIG. 2 , a block diagram of an embodiment of a cloud computing environment  200  that includes the control system  185  is shown. The cloud computing environment  200  may be a private cloud. A firewall  235  may be provided for the control system  185 . Some examples of the firewall  235  may include a proxy firewall, a stateful inspection firewall, a unified threat management (UTM) firewall, a next-generation firewall (NGFW), a threat-focused NGFW, and a virtual firewall. 
     The control system  185  may include a plurality of user interfaces  205 . The user interfaces  205  may allow users to provide input to the control system  185 . Some examples of the user interfaces  205  may include a keyboard, a mouse, a touchpad or touch screen on a display, a scroll wheel, a keypad, and an audio input device. For example, the user interfaces  205  may receive configuration settings to be applied to instances within the first cloud computing environment  140   a , the second cloud computing environment  140   b , and/or the third cloud computing environment  140   c.    
     The control system  185  may also include a controller  215 , a plurality of collectors  220 , a configurator  255 , a plurality of testers  265 , an enforcer  270 , and a reporter  280 . The controller  215 , the configurator  255 , the testers  265 , the enforcer  270 , and the reporter  280  may be modules within a computing system or may be separate computing systems that are communicatively coupled. The computing systems may have various components such as processors, storage subsystems, and communications subsystems. Some examples of the computing systems may include personal computers, workstations, mainframes, server racks, and handheld portable devices. 
     The collectors  220  may retrieve information from the first cloud computing environment  140   a , the second cloud computing environment  140   b , and/or the third cloud computing environment  140   c . The information retrieved by the collectors  220  may include functionalities of the first cloud computing environment  140   a , the second cloud computing environment  140   b , and/or the third cloud computing environment  140   c , such as network configurations, a firewall rules, cloud application programming interfaces (APIs), resources, cloud service providers, and data sets. The information retrieved by the collectors  220  may also include data input types, data types, data sizes, or data ages of the first cloud computing environment  140   a , the second cloud computing environment  140   b , and/or the third cloud computing environment  140   c.    
     The controller  215  may determine policies for the instances within the first cloud computing environment  140   a , the second cloud computing environment  140   b , and/or the third cloud computing environment  140   c . For example, the policies may include firewall rules, forwarding rules, network configurations, cross-cloud routing rules, IP addressing rules, cross-cloud peering rules, security group management rules, storage bucket access rules, resource management rules, or subnet configurations. The controller  215  may determine general policies for all of the instances as functions of the configuration settings that are received from the user interfaces  205 . Because the first cloud computing environment  140   a , the second cloud computing environment  140   b , and/or the third cloud computing environment  140   c  may use different languages, data sources, commands, and protocols, the controller  215  may also translate the general policies into specific policies for instances within each different cloud computing environment. For example, the controller  215  may determine specific policies according to the information retrieved from the first cloud computing environment  140   a , the second cloud computing environment  140   b , and/or the third cloud computing environment  140   c.    
     The configurator  255  may receive the policies from the controller  215  and apply the policies to the instances. For each specific policy, the configurator  255  may retrieve a script from an API and execute the script in order to apply the specific policy to the instances within one of the cloud computing environments. The script may include instructions for the instances to implement the specific policy. Any number of specific policies may be applied to the instances within the cloud computing environment. In this example, the configurator  255  pushes the policies directly to the instances in order to update the configurations of the instances. 
     In another example, a metadata endpoint  250  may be provided for the configurator  255 . The metadata endpoint  250  may receive the policies from the configurator  255  and host changes to the instances according to the policies. In this example, the instances retrieve the changes from the metadata service endpoint and apply the changes. The instances may subscribe to the metadata endpoint  250  and periodically check the metadata endpoint  250  for any updates. In this example, the instances pull the policies from the metadata endpoint  250  in order to update the configurations of the instances. 
     The testers  265  may inspect operations of the instances and detect violations of the policies by the instances. In some examples, a different tester  265  may be provided for each cloud computing environment. In other examples, a single tester  265  may be provided for a plurality of cloud computing environments. As described in further detail below, the testers  265  may detect violations of one or more specific policies by any of the instances. 
     The enforcer  270  may respond to the violations in a variety of ways. For example, the enforcer  270  may send a notification of the detected violation to the controller  215 , which may direct the configurator  255  to apply the policy that was violated to the non-complying instance. In one example, if the policy requires the instances to have non-overlapping IP addresses and the testers  265  identify an overlap between the IP addresses, the enforcer  270  may direct the configurator  255  to request new IP addresses from at least one of the cloud computing environments. This procedure may be repeated until the number of overlapping IP addresses has been reduced or eliminated. 
     Alternatively or in addition, the reporter  280  may send a notification of the detected violation to at least one of the user interfaces  205 . The notification may identify the instance that violated the policy, the cloud computing environment in which the instance is located, and the policy that was violated. The reporter  280  may send the notification via short message service (SMS), email, API call, or another notification method. 
     Referring next to  FIG. 3 , a block diagram of an embodiment of a portion  300  of the network  100  is shown. The first cloud computing environment  140   a  may include a plurality of first instances  345   a , the second cloud computing environment  140   b  may include a plurality of second instances  345   b , and the third cloud computing environment  140   c  may include a plurality of third instances  345   c . Some examples of the first instances  345   a , the second instances  345   b , and the third instances  345   c  may include virtual machines that emulate computer systems. The virtual machines may run various software packages. The first instances  345   a , the second instances  345   b , and the third instances  345   c  may be examples of the instances discussed above with respect to  FIG. 2 . 
     A metadata endpoint  350  may be provided for one, some, or all of the instances, such as a fourth instance  345   d  within the third cloud computing environment  140   c . As discussed above, the metadata endpoint  350  may receive instructions from the configurator  255  and apply the policies to the fourth instance  345   d.    
     With reference to  FIG. 4 , a block diagram of a cloud Open Systems Interconnection (OSI) model  400  for cloud computing environments is shown. The cloud OSI model  400  for cloud computing environments partitions the flow of data in a communication system into six layers of abstraction. The cloud OSI model  400  for cloud computing environments may include, in order, an application layer  410 , a service layer  415 , an image layer  420 , a software-defined data center layer  425 , a hypervisor layer  430 , and an infrastructure layer  435 . Each layer serves a class of functionality to the layer above it and is served by the layer below it. Classes of functionality may be realized in software by various communication protocols. 
     The infrastructure layer  435  may include hardware, such as physical devices in a data center, that provides the foundation for the rest of the layers. The infrastructure layer  435  may transmit and receive unstructured raw data between a device and a physical transmission medium. For example, the infrastructure layer  435  may convert the digital bits into electrical, radio, or optical signals. 
     The hypervisor layer  430  may perform virtualization, which may allow the physical devices to be divided into virtual machines that can be bin packed onto physical machines for greater efficiency. The hypervisor layer  430  may provide virtualized compute, storage, and networking. For example, OpenStack® software that is installed on bare metal servers in a data center may provide virtualization cloud capabilities. The OpenStack® software may provide various infrastructure management capabilities to cloud operators and administrators, and may utilize the Infrastructure-as-Code concept for deployment and lifecycle management of a cloud data center. In the Infrastructure-as-Code concept, the infrastructure elements are described in definition files. Changes in the files are reflected in the configuration of data center hosts and cloud services. 
     The software-defined data center layer  425  may provide resource pooling, usage tracking, and governance on top of the hypervisor layer  430 . The software-defined data center layer  425  may enable the creation virtualization for the Infrastructure-as-Code concept by using representational state transfer (REST) APIs. The management of block storage devices may be virtualized, and end users may be provided with a self-service API to request and consume those resources without requiring any knowledge of where the storage is actually deployed or on what type of device. Various compute nodes may be balanced for storage. 
     The image layer  420  may use various operating systems and other pre-installed software components. Patch management may be used to identify, acquire, install, and verify patches for products and systems. Patches may be used to correct security and functionality problems in software. Patches may also be used to add new features to operating systems, including security capabilities. The image layer  420  may focus on the compute instead of storage and networking. The instances within the cloud computing environments may be provided at the image layer  420 . 
     The service layer  415  may provide middleware, such as functional components that applications use in tiers. In some examples, the middleware components may include databases, load balancers, web servers, message queues, email services, or other notification methods. The middleware components may be defined at the service layer  415  on top of particular images from the image layer  420 . Different cloud computing environment providers may have different middleware components. 
     The application layer  420  may interact with software applications that implement a communicating component. The application layer  420  is the layer that is closest to the end user. Functions of the application layer  420  may include identifying communication partners, determining resource availability, and synchronizing communication. Applications within the application layer  420  may include custom code that makes use of middleware defined in the service layer  415 . 
     Various features discussed above may be performed at one or more layers of the cloud OSI model  400  for cloud computing environments. For example, translating the general policies into specific policies for different cloud computing environments may be performed at the service layer  415  and the software-defined data center layer  425 . Various scripts may be updated across the service layer  415 , the image layer  420 , and the software-defined data center layer  425 . Further, APIs and policies may operate at the software-defined data center layer  425  and the hypervisor layer  430 . 
     Each of the different cloud computing environments may have different service layers  415 , image layers  420 , software-defined data center layers  425 , hypervisor layers  430 , and infrastructure layers  435 . Further, each of the different cloud computing environments may have an application layer  410  that can make calls to the specific policies in the service layer  415  and the software-defined data center layer  425 . The application layer  410  may have substantially the same format and operation for each of the different cloud computing environments. Accordingly, developers for the application layer  410  may not need to understand the peculiarities of how each of the cloud computing environments operates in the other layers. 
     Referring next to  FIG. 5 , a flowchart of an embodiment of a method  500  is shown. The method  500  begins at block  510  where configuration settings to be applied to instances within a plurality of different cloud computing environments are received. For example, the configuration settings may be received via the user interfaces  205  and provided to the controller  215 . The configuration settings may be intended for the first instances  345   a  within the first cloud computing environment  140   a  and the second instances  345   b  within the second cloud computing environment  140   b . The first cloud computing environment  140   a  and the second cloud computing environment  140   b  may be run by different providers and may use different languages, data sources, commands, and protocols. 
     The method  500  continues at block  515  where information from the different cloud computing environments is retrieved. For example, information from the first cloud computing environment  140   a  and the second cloud computing environment  140   b  may be retrieved by the collectors  220 . One of the collectors  220  may retrieve the information from the first cloud computing environment  140   a , and another one of the collectors may retrieve the information from the second cloud computing environment  140   b . Both the configuration settings and the information may be provided to the controller  215 . 
     The method  500  continues at block  520  where general policies are determined as functions of the configuration settings. For example, the controller  215  may determine general policies that apply to the first instances  345   a  within the first cloud computing environment  140   a  and the second instances  345   b  within the second cloud computing environment  140   b . The controller  215  may determine general policies that apply to all of the instances within some or all of the cloud computing environments within a network. This may ensure that the policies are consistent across the different cloud computing environments. For example, the policies may include firewall rules, forwarding rules, network configurations, cross-cloud routing rules, IP addressing rules, cross-cloud peering rules, security group management rules, storage bucket access rules, resource management rules, or subnet configurations. 
     The method  500  continues at block  525  where the general policies are translated to specific policies for instances within the different cloud computing environments. For example, the controller  215  may use the information about the first cloud computing environment  140   a  and the second cloud computing environment  140   b  to determine specific policies for instances within the first cloud computing environment  140   a  and the second cloud computing environment  140   b , respectively. This translation may be performed at the service layer  415  and the software-defined data center layer  425  of the cloud OSI model  400  for cloud computing environments. 
     The method  500  continues at block  530  where scripts corresponding to the specific policies for the instances within the first cloud computing environment  140   a  and the second cloud computing environment  140   b  are retrieved. For example, the configurator  225  may receive the specific policies from the controller  215  and retrieve scripts corresponding to the specific policies from an API. The scripts may be written in JavaScript Object Notation (JSON)®. For each of the first cloud computing environment  140   a  and the second cloud computing environment  140   b , a separate script may be retrieved for each of the specific policies. The following is an example of a script that may be used to establish a firewall rule for the first instances  345   a  within the first cloud computing environment  140   a , where the first cloud computing environment  140   a  is run by Amazon Web Services (AWS)®: 
     Further, the following is an example of a script that may be used to establish the same firewall rule for the second instances  345   b  within the second cloud computing environment  140   b , where the second cloud computing environment  140   b  is run by Google Cloud Platform (GCP)®: 
     
       
         
           
               
             
               
                   
               
             
            
               
                 C02XW1WCJGH6:~ username$ gcloud compute instances list --quiet --project example-project 
               
               
                 --account username@netskope.com --format=“json” 
               
               
                 [ 
               
               
                  { 
               
               
                   “canIpForward”: false, 
               
               
                   “cpuPlatform”: “Intel Broadwell”, 
               
               
                   “creationTimestamp”: “2020-05-07T21:42:51.931-07:00”, 
               
               
                   “deletionProtection”: false, 
               
               
                   “description”: “”, 
               
               
                   “disks”: [ 
               
               
                    { 
               
               
                     “autoDelete”: true, 
               
               
                     “boot”: true, 
               
               
                     “deviceName”: “example-sample”, 
               
               
                     “diskSizeGb”: “500”, 
               
               
                     “guestOsFeatures”: [ 
               
               
                      { 
               
               
                       “type”: “VIRTIO_SCSI_MULTIQUEUE” 
               
               
                      } 
               
               
                     ], 
               
               
                     “index”: 0, 
               
               
                     “interface”: “SCSI”, 
               
               
                     “kind”: “compute#attachedDisk”, 
               
               
                     “licenses”: [ 
               
               
                      “https://www.googleapis.com/compute/v1/projects/ubuntu-os- 
               
               
                 cloud/global/licenses/ubuntu-1604-xenial” 
               
               
                     ], 
               
               
                     “mode”: “READ_WRITE”, 
               
               
                     “source”: “https://www.googleapis.com/compute/v1/projects/example-project/zones/us- 
               
               
                 west1-b/disks/example-sample”, 
               
               
                     “type”: “PERSISTENT” 
               
               
                    } 
               
               
                   ], 
               
               
                   “displayDevice”: { 
               
               
                    “enableDisplay”: false 
               
               
                   }, 
               
               
                   “fingerprint”: “4GTjmojZZPQ=”, 
               
               
                   “id”: “123456789098765432”, 
               
               
                   “kind”: “compute#instance”, 
               
               
                   “labelFingerprint”: “L_2Mh2A8RIA=”, 
               
               
                   “labels”: { 
               
               
                    “env”: “test”, 
               
               
                    “team”: “data” 
               
               
                   }, 
               
               
                   “lastStartTimestamp”: “2020-05-07T21:42:56.861-07:00”, 
               
               
                   “machineType”: “https://www.googleapis.com/compute/v1/projects/example-project/zones/us- 
               
               
                 west1-b/machineTypes/custom-8-32768”, 
               
               
                   “metadata”: { 
               
               
                    “fingerprint”: “MMMMMMMMMMM=”, 
               
               
                    “kind”: “compute#metadata” 
               
               
                   }, 
               
               
                   “name”: “example-sample”, 
               
               
                   “networkInterfaces”: [ 
               
               
                    { 
               
               
                     “accessConfigs”: [ 
               
               
                      { 
               
               
                       “kind”: “compute#accessConfig”, 
               
               
                       “name”: “External NAT”, 
               
               
                       “natIP”: “252.1.2.3”, 
               
               
                       “networkTier”: “PREMIUM”, 
               
               
                       “type”: “ONE_TO_ONE_NAT” 
               
               
                      } 
               
               
                     ], 
               
               
                     “fingerprint”: “00_ffffffff=”, 
               
               
                     “kind”: “compute#networkInterface”, 
               
               
                     “name”: “nic0”, 
               
               
                     “network”: “https://www.googleapis.com/compute/v1/projects/example- 
               
               
                 project/global/networks/sample-example”, 
               
               
                     “networkIP”: “10.240.128.18”, 
               
               
                     “subnetwork”: “https://www.googleapis.com/compute/v1/projects/example- 
               
               
                 project/regions/us-west1/subnetworks/sample-example-priv-vpn-subnet” 
               
               
                    } 
               
               
                   ], 
               
               
                   “reservationAffinity”: { 
               
               
                    “consumeReservationType”: “ANY_RESERVATION” 
               
               
                   }, 
               
               
                   “scheduling”: { 
               
               
                    “automaticRestart”: true, 
               
               
                    “onHostMaintenance”: “MIGRATE”, 
               
               
                    “preemptible”: false 
               
               
                   }, 
               
               
                   “selfLink”: “https://www.googleapis.com/compute/v1/projects/example-project/zones/us- 
               
               
                 west1-b/instances/example-sample”, 
               
               
                   “serviceAccounts”: [ 
               
               
                    { 
               
               
                     “email”: “1000000000000-compute@developer.gserviceaccount.com”, 
               
               
                     “scopes”: [ 
               
               
                      “https://www.googleapis.com/auth/devstorage.read_only”, 
               
               
                      “https://www.googleapis.com/auth/logging.write”, 
               
               
                      “https://www.googleapis.com/auth/monitoring.write”, 
               
               
                      “https://www.googleapis.com/auth/servicecontrol”, 
               
               
                      “https://www.googleapis.com/auth/service.management.readonly”, 
               
               
                      “https://www.googleapis.com/auth/trace.append” 
               
               
                     ] 
               
               
                    } 
               
               
                   ], 
               
               
                   “startRestricted”: false, 
               
               
                   “status”: “RUNNING”, 
               
               
                   “tags”: { 
               
               
                    “fingerprint”: “1-888_XXXXX=”, 
               
               
                    “items”: [ 
               
               
                     “https-server”, 
               
               
                     “nspublic” 
               
               
                    ] 
               
               
                   }, 
               
               
                   “zone”: “https://www.googleapis.com/compute/v1/projects/example-project/zones/us-west1-b” 
               
               
                  }, 
               
               
                  { 
               
               
                   “canIpForward”: false, 
               
               
                   “cpuPlatform”: “Intel Broadwell”, 
               
               
                   “creationTimestamp”: “2020-04-17T14:54:58.261-07:00”, 
               
               
                   “deletionProtection”: false, 
               
               
                   “description”: “”, 
               
               
                   “disks”: [ 
               
               
                    { 
               
               
                     “autoDelete”: true, 
               
               
                     “boot”: true, 
               
               
                     “deviceName”: “test-vm”, 
               
               
                     “diskSizeGb”: “500”, 
               
               
                     “guestOsFeatures”: [ 
               
               
                      { 
               
               
                       “type”: “VIRTIO_SCSI_MULTIQUEUE” 
               
               
                      } 
               
               
                     ], 
               
               
                     “index”: 0, 
               
               
                     “interface”: “SCSI”, 
               
               
                     “kind”: “compute#attachedDisk”, 
               
               
                     “licenses”: [ 
               
               
                      “https://www.googleapis.com/compute/v1/projects/ubuntu-os- 
               
               
                 cloud/global/licenses/ubuntu-1604-xenial” 
               
               
                     ], 
               
               
                     “mode”: “READ_WRITE”, 
               
               
                     “source”: “https://www.googleapis.com/compute/v1/projects/example-project/zones/us- 
               
               
                 west1-b/disks/test-vm”, 
               
               
                     “type”: “PERSISTENT” 
               
               
                    } 
               
               
                   ], 
               
               
                   “displayDevice”: { 
               
               
                    “enableDisplay”: false 
               
               
                   }, 
               
               
                   “fingerprint”: “ZZZZZZZZZZZ=”, 
               
               
                   “id”: “5555555555555555555”, 
               
               
                   “kind”: “compute#instance”, 
               
               
                   “labelFingerprint”: “XBXBXBXBXBX=”, 
               
               
                   “labels”: { 
               
               
                    “env”: “test”, 
               
               
                    “team”: “webui” 
               
               
                   }, 
               
               
                   “lastStartTimestamp”: “2020-04-29T10:59:48.190-07:00”, 
               
               
                   “lastStopTimestamp”: “2020-04-29T10:59:30.707-07:00”, 
               
               
                   “machineType”: “https://www.googleapis.com/compute/v1/projects/example-project/zones/us- 
               
               
                 west1-b/machineTypes/custom-8-44544”, 
               
               
                   “metadata”: { 
               
               
                    “fingerprint”: “LMFFFFF-FFF=”, 
               
               
                    “items”: [ 
               
               
                     { 
               
               
                      “key”: “serial-port-enable”, 
               
               
                      “value”: “true” 
               
               
                     } 
               
               
                    ], 
               
               
                    “kind”: “compute#metadata” 
               
               
                   }, 
               
               
                   “name”: “test-vm”, 
               
               
                   “networkInterfaces”: [ 
               
               
                    { 
               
               
                     “accessConfigs”: [ 
               
               
                      { 
               
               
                       “kind”: “compute#accessConfig”, 
               
               
                       “name”: “External NAT”, 
               
               
                       “natIP”: “35.1.1.15”, 
               
               
                       “networkTier”: “PREMIUM”, 
               
               
                       “type”: “ONE_TO_ONE_NAT” 
               
               
                      } 
               
               
                     ], 
               
               
                     “fingerprint”: “BBBBBrIIIII=”, 
               
               
                     “kind”: “compute#networkInterface”, 
               
               
                     “name”: “nic0”, 
               
               
                     “network”: “https://www.googleapis.com/compute/v1/projects/example- 
               
               
                 project/global/networks/sample-example”, 
               
               
                     “networkIP”: “ 10.240.131.209”, 
               
               
                     “subnetwork”: “https://www.googleapis.com/compute/v1/projects/example- 
               
               
                 project/regions/us-west1/subnetworks/sample-example-priv-vpn-subnet” 
               
               
                    } 
               
               
                   ], 
               
               
                   “reservationAffinity”: { 
               
               
                    “consumeReservationType”: “ANY_RESERVATION” 
               
               
                   }, 
               
               
                   “scheduling”: { 
               
               
                    “automaticRestart”: true, 
               
               
                    “onHostMaintenance”: “MIGRATE”, 
               
               
                    “preemptible”: false 
               
               
                   }, 
               
               
                   “selfLink”: “https://www.googleapis.com/compute/v1/projects/example-project/zones/us- 
               
               
                 west1-b/instances/test-vm”, 
               
               
                   “serviceAccounts”: [ 
               
               
                    { 
               
               
                     “email”: “1000000000000-compute@developer.gserviceaccount.com”, 
               
               
                     “scopes”: [ 
               
               
                      “https://www.googleapis.com/auth/devstorage.read_only”, 
               
               
                      “https://www.googleapis.com/auth/logging.write”, 
               
               
                      “https://www.googleapis.com/auth/monitoring.write”, 
               
               
                      “https://www.googleapis.com/auth/servicecontrol”, 
               
               
                      “https://www.googleapis.com/auth/service.management.readonly”, 
               
               
                      “https://www.googleapis.com/auth/trace.append” 
               
               
                     ] 
               
               
                    } 
               
               
                   ], 
               
               
                   “startRestricted”: false, 
               
               
                   “status”: “RUNNING”, 
               
               
                   “tags”: { 
               
               
                    “fingerprint”: “1-888_XXXXX=”, 
               
               
                    “items”: [ 
               
               
                     “https-server”, 
               
               
                     “nspublic” 
               
               
                    ] 
               
               
                   }, 
               
               
                   “zone”: “https://www.googleapis.com/compute/v1/projects/example-project/zones/us-west1-b” 
               
               
                  }, 
               
               
                  { 
               
               
                   “canIpForward”: false, 
               
               
                   “cpuPlatform”: “Intel Broadwell”, 
               
               
                   “creationTimestamp”: “2020-04-27T16:32:35.259-07:00”, 
               
               
                   “deletionProtection”: false, 
               
               
                   “description”: “”, 
               
               
                   “disks”: [ 
               
               
                    { 
               
               
                     “autoDelete”: true, 
               
               
                     “boot”: true, 
               
               
                     “deviceName”: “example-vm-test”, 
               
               
                     “diskSizeGb”: “500”, 
               
               
                     “guestOsFeatures”: [ 
               
               
                      { 
               
               
                       “type”: “VIRTIO_SCSI_MULTIQUEUE” 
               
               
                      } 
               
               
                     ], 
               
               
                     “index”: 0, 
               
               
                     “interface”: “SCSI”, 
               
               
                     “kind”: “compute#attachedDisk”, 
               
               
                     “licenses”: [ 
               
               
                      “https://www.googleapis.com/compute/v1/projects/ubuntu-os- 
               
               
                 cloud/global/licenses/ubuntu-1604-xenial” 
               
               
                     ], 
               
               
                     “mode”: “READ_WRITE”, 
               
               
                     “source”: “https://www.googleapis.com/compute/v1/projects/example-project/zones/us- 
               
               
                 west1-b/disks/example-vm-test”, 
               
               
                     “type”: “PERSISTENT” 
               
               
                    } 
               
               
                   ], 
               
               
                   “displayDevice”: { 
               
               
                    “enableDisplay”: false 
               
               
                   }, 
               
               
                   “fingerprint”: “11111111111=”, 
               
               
                   “id”: “8978978978978978978”, 
               
               
                   “kind”: “compute#instance”, 
               
               
                   “labelFingerprint”: “42424242424=”, 
               
               
                   “lastStartTimestamp”: “2020-04-27T16:32:41.002-07:00”, 
               
               
                   “machineType”: “https://www.googleapis.com/compute/v1/projects/example-project/zones/us- 
               
               
                 west1-b/machineTypes/n1-standard-8”, 
               
               
                   “metadata”: { 
               
               
                    “fingerprint”: “MMMMMMMMMMM=”, 
               
               
                    “kind”: “compute#metadata” 
               
               
                   }, 
               
               
                   “name”: “example-vm-test”, 
               
               
                   “networkInterfaces”: [ 
               
               
                    { 
               
               
                     “accessConfigs”: [ 
               
               
                   
               
            
           
         
       
     
     The method  500  continues at block  535  where the scripts corresponding to the specific policies for the instances within the first cloud computing environment  140   a  and the second cloud computing environment  140   b  are confirmed and executed. For example, the configurator  225  may confirm that the first script quoted above will instruct the first instances  345   a  within the first cloud computing environment  140   a  to establish the desired firewall rule. The configurator  225  may then execute the first script in order to apply the firewall rule to the first instances  345   a  within the first cloud computing environment  140   a . The firewall rule may be applied to the first instances  345   a  within the first cloud computing environment  140   a  simultaneously or in sequence. The configurator  225  may then confirm that the second script quoted above will instruct the second instances  345   b  within the second cloud computing environment  140   b  to establish the desired firewall rule. The configurator  225  may then execute the second script in order to apply the firewall rule to the second instances  345   b  within the second cloud computing environment  140   b . Again, the firewall rule may be applied to the second instances  345   b  within the second cloud computing environment  140   b  simultaneously or in sequence. This procedure may be repeated until the firewall rule has been applied to all of the instances in the network. 
     The method  500  continues at block  540  where operations of the instances within the cloud computing environments are inspected. For example, the testers  265  may inspect operations of the first instances  345   a  within the first cloud computing environment  140   a  and the second instances  345   b  within the second cloud computing environment  140   b . One or more of the testers  265  may be provided for each of the cloud computing environments, or a single tester  265  may be provided for all of the cloud computing environments. 
     As one example of the testing procedure that may be performed at block  540 , the first tester  265  may determine whether the first instance  345   a  was correctly set up as a load balancer. For example, the first tester  265  may inspect the first instance  345   a  to determine whether it complies with various policies, such as whether the load balancer was set up, whether the load balancer is running correctly, whether the load balancer has enough processing power, and whether the load balancer has the correct IP address. More specific details of embodiments of the testing procedure are provided below. 
     Referring next to  FIG. 6 , a flowchart of an embodiment of block  540  of the method  500  described in  FIG. 5  is shown. The example shown in  FIG. 6  describes a method for testing a first instance for compliance with a first policy. However, this example may be expanded to test the first instance for compliance with a plurality of policies. This example may also be expanded to test additional instances for compliance with the first policy and/or the plurality of policies. Any such testing may be performed in parallel and/or in sequence. 
     The method  540  begins at block  610  where a first policy is identified for testing compliance of the first instance  345   a  within the first cloud computing environment  140   a . For example, a first tester  265  may identify the first policy by referencing a table that stores a list of each specific policy that has been applied to each instance within the network  100 . The table may be stored within a data storage component within the control system  185 . The first policy may be identified by a variety of methods, such as a random selection from the specific policies that have been applied to the first instance  345   a , or a selection of the specific policy that was applied to the first instance  345   a  at the earliest time. 
     The method  540  continues at block  615  where a testing schedule is retrieved for the first instance  345   a . For example, the testing schedule may indicate a frequency of testing for the first instance  345   a  for compliance with each of the specific policies. The frequency of testing may be constant or may change as a function of time. The testing schedule may also indicate a predetermined time at which to start testing the first instance  345   a  after each of the specific policies was applied to the first instance  345   a . The testing schedule may be the same for one or more of the specific policies, or may be different for one or more of the specific policies. The first tester  265  may retrieve the testing schedule from the data storage component within the control system  185 . 
     The method  540  continues at block  620  where a test module to test the compliance of the first instance  345   a  within the first cloud computing environment  140   a  with the first policy is retrieved. A separate test module may be stored within the data storage component for each of the specific policies. Each test module may be written for a specific cloud computing environment, and may be written as a JavaScript Object Notation (JSON)® script. For example, the first tester  265  may retrieve a first test module to test the compliance of the first instance  345   a  within the first cloud computing environment  140   a  for compliance with the first policy. 
     The method  540  continues at block  625  where it is determined whether a predetermined time has elapsed since the first policy was applied to the first instance  345   a  or the first instance  345   a  was last tested for compliance with the first policy. For example, the first tester  265  may refer to the testing schedule for the first instance  345   a , along with a table that stores a list of the times at which the first instance  345   a  was tested for compliance with the specific policies. If the first instance  345   a  has already been tested for compliance with the first policy, the first tester  265  may determine whether a predetermined time has elapsed since the first instance  345   a  was last tested for compliance with the first policy. If the predetermined time has not elapsed, the method  540  may proceed to block  630 . If the predetermined time has elapsed, the method may proceed to block  640 . Similarly, if the first instance  345   a  has not yet been tested for compliance with the first policy, the first tester  265  may determine whether another predetermined time has elapsed since the first policy was applied to the first instance  345   a . If the other predetermined time has not elapsed, the method  540  may proceed to block  630 . If the other predetermined time has elapsed, the method may proceed to block  640 . 
     At block  630  it is determined whether an error message has been received from the first instance  345   a . For example, after the first policy is applied to the first instance  345   a , the first instance  345   a  may send an error message to the control system  185  indicating that the first instance  345   a  was unable to implement the first policy or maintain compliance with the first policy. 
     At block  635  it is determined whether a notification of a change to the first cloud computing environment  140   a  has been received. For example, after the first policy is applied to the first instance  345   a , the first cloud computing environment  140   a  may send a message to the control system  185  indicating a change in its operation or configuration that may affect the compliance of the first instance  345   a  with the first policy. If no notifications have been received from the first cloud computing environment  140   a , the method  540  may return to block  625 . If a notification has been received from the first cloud computing environment  140   a , the method  540  may proceed to block  640 . 
     At block  640  the first instance  345   a  is tested for compliance with the first policy by calling the test module corresponding to the first policy. For example, the first tester  265  may call the test module that was retrieved at block  610  to test the first instance  345   a  for compliance with the first policy. The testing may determine that the first instance  345   a  is complying with the first policy or violating the first policy. 
     Returning to  FIG. 5 , the method  500  continues at block  545  where the control system  185  responds to any detected violations. For example, the enforcer  270  may receive a notification from the first tester  265  that the first instance  345   a  violated the first policy. The enforcer  270  may then send a notification of the violation to the controller  215 , which may receive the notification and require the first instance  345   a  to comply with the first policy. For example, the controller  215  may instruct the configurator  255  to apply the first policy to the first instance  345   a  again. Alternatively, the controller  215  may shut down the first instance  345   a  or cut off communications with the first instance  345   a . Alternatively or in addition, the reporter  280  may send a notification of the violation to at least one of the user interfaces  205 . For example, the reporter  280  may send the notification of the violation to the user interface  205  that provided the configuration settings for the first instance  345   a.    
     As another example of the testing and response procedures discussed with respect to  FIG. 5 , the first tester  265  may inspect the IP addresses that have been assigned to the plurality of first instances  345   a  within the first cloud computing environment  140   a  at block  540 . The first tester  265  may then determine whether there is a violation of a policy to have non-overlapping IP addresses for the plurality of first instances  345   a  at block  540 . If such a violation exists, the first tester  265  may send a notification to the enforcer  270 . The enforcer  270  may then send a notification of the violation to the controller  215 , which may receive the notification and request the allocation of new IP addresses from the first cloud computing environment  140   a  at block  545 . This testing and response procedure may be repeated until the first instances  345   a  have non-overlapping IP addresses, or until the number of overlapping IP addresses has been reduced below a threshold. For example, the threshold may be a percentage of the number of the first instances  345   a  within the first cloud computing environment  140   a . Alternatively, the controller  215  may shut down any instances having overlapping IP addresses or cut off communications with any instances having overlapping IP addresses. Alternatively or in addition, the reporter  280  may send a notification of the overlapping IP addresses, including a list of the affected instances, to at least one of the user interfaces  205 . 
     Further, this testing and response procedure may be conducted to inspect the IP addresses that have been assigned to all of the instances within the network  100 . For example, one or more of the testers  265  may inspect the IP addresses that have been assigned to the plurality of first instances  345   a  within the first cloud computing environment  140   a , the plurality of second instances  345   b  within the first cloud computing environment  140   b , and the plurality of third instances  345   c  within the third cloud computing environment  140   c . The remainder of the testing and response procedure may be the same as discussed above. 
     Specific details are given in the above description to provide a thorough understanding of the embodiments. However, it is understood that the embodiments may be practiced without these specific details. For example, circuits may be shown in block diagrams in order not to obscure the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments. 
     Implementation of the techniques, blocks, steps and means described above may be done in various ways. For example, these techniques, blocks, steps and means may be implemented in hardware, software, or a combination thereof. For a hardware implementation, the processing units may be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described above, and/or a combination thereof. 
     Also, it is noted that the embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a swim diagram, a data flow diagram, a structure diagram, or a block diagram. Although a depiction may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be re-arranged. A process is terminated when its operations are completed, but could have additional steps not included in the figure. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination corresponds to a return of the function to the calling function or the main function. 
     Furthermore, embodiments may be implemented by hardware, software, scripting languages, firmware, middleware, microcode, hardware description languages, and/or any combination thereof. When implemented in software, firmware, middleware, scripting language, and/or microcode, the program code or code segments to perform the necessary tasks may be stored in a machine readable medium such as a storage medium. A code segment or machine-executable instruction may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a script, a class, or any combination of instructions, data structures, and/or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters, and/or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc. 
     For a firmware and/or software implementation, the methodologies may be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. Any machine-readable medium tangibly embodying instructions may be used in implementing the methodologies described herein. For example, software codes may be stored in a memory. Memory may be implemented within the processor or external to the processor. As used herein the term “memory” refers to any type of long term, short term, volatile, nonvolatile, or other storage medium and is not to be limited to any particular type of memory or number of memories, or type of media upon which memory is stored. 
     Moreover, as disclosed herein, the term “storage medium” may represent one or more memories for storing data, including read only memory (ROM), random access memory (RAM), magnetic RAM, core memory, magnetic disk storage mediums, optical storage mediums, flash memory devices and/or other machine readable mediums for storing information. The term “machine-readable medium” includes, but is not limited to portable or fixed storage devices, optical storage devices, and/or various other storage mediums capable of storing that contain or carry instruction(s) and/or data. 
     While the principles of the disclosure have been described above in connection with specific apparatuses and methods, it is to be clearly understood that this description is made only by way of example and not as limitation on the scope of the disclosure.