DYNAMIC ROUTING OF APPLICATION PROGRAMMING INTERFACE (API) CALLS TO A CONFIDENTIAL COMPUTING VIRTUAL MACHINE

Techniques for sending commands to a container agent of a confidential virtual machine (VM) are disclosed. An example method includes sending a first command from a computing device to a container agent of a confidential VM running on a host computing system. The first command is sent to the container agent through a control plane of the host computing system and causes the container agent to communicate with a relying party to verify confidentiality of the confidential VM. The method also includes receiving network information for the container agent from the relying party and establishing a network connection with the container agent based on the network information received from the relying party. The method also includes sending a second command from the computing device to the container agent of the confidential VM via the network connection.

TECHNICAL FIELD

Aspects of the present disclosure relate to confidential distributed computing systems, and more particularly to dynamic routing of API calls to confidential virtual machines (VMs) in a distributed computing system.

BACKGROUND

A container orchestration platform is a platform for developing and running containerized applications and may allow applications and the data centers that support them to expand from just a few machines and applications to thousands of machines that serve millions of clients. Container orchestration engines may provide an image-based deployment module for creating containers and may store one or more image files for creating container instances. Many application instances can be running in containers on a single host without visibility into each other's processes, files, network, and so on. Each container may provide a single function (often called a “service”) or component of an application, such as a web server or a database, though containers can be used for arbitrary workloads. One example of a container orchestration platform is the Red Hat™ OpenShift™ platform built around Kubernetes.

Secure encrypted virtualization (SEV) is a technology that is designed to isolate VMs from the hypervisor and other code that may coexist on the physical host at the hardware level. In this way, SEV may protect VMs from physical threats as well as protect them from other VMs and even the hypervisor itself. SEV is useful in a variety of applications. For example, certain customers of a cloud service may want to secure their VM-based workloads from the cloud administrator to keep their data confidential and minimize their exposure to bugs in the cloud provider's infrastructure.

DETAILED DESCRIPTION

The present disclosure describes techniques for creating a secure communication channel for communicating with a confidential VM in a distributed (i.e., cloud) computing system, including container orchestration platforms such as OpenShift™ and Kubernetes.

A cloud computing system may provide a serverless or cluster-based framework for the performance of client applications. For example, the framework may execute functions of a client's web application. The framework may invoke one or more serverless resources to execute the functions for the client on one or more worker nodes of a computing cluster. The worker nodes may be physical computing systems or may be execution environments within a such as VMs and containers.

The cloud computing system may dynamically manage the allocation and provisioning of resources within a computing framework referred to herein as a container cluster. The container cluster may be managed by a host system referred to herein as a container-orchestration system.

Each container provides an isolated execution environment for processing tasks related to the client applications, sometimes referred to as workloads. To instantiate a new container, the container orchestration system uploads a container image that provides instructions for how to build the container. The container image describes the allocated computing resources and file systems for a container instance to be instantiated, such as the container's operating system, applications to be executed, processing tasks to be handled, etc. The container image may include various base files that are required for minimal functioning and are provided by the host, as well as client-specific files that are specific to the client's applications and processes.

In some cases, the owner of the workload (also referred to herein the tenant) may want to protect the confidentiality of their workloads, which includes preventing the host system from having access to those workloads. For that reason, confidential cloud computing systems have been developed.

A confidential cloud computing system is one that uses cryptographic technology to provide a level of isolation between the host systems and the tenant workloads. Examples of such cryptographic technology include Advanced Micro Devices (AMD) SEV and Intel® trusted domain extensions (TDX). The containers created by such a system may be operated within what is referred to as a confidential VM. The memory used by a container within a confidential VM is usually encrypted in the memory controller of the system's processors. This guarantees that the data in memory for the container will not be accessible to the host system.

However, the host system does have some access to the containers within secure VMs. For example, instructions to load a container image or initiate or stop the execution of a workload may be transmitted to the VM through communication channels operated by the host. Additionally, the host system may need to communicate with the confidential VM for implementing various orchestration functions, such as the allocation and provisioning of resources to be used by the VM. Such communications between the host and the VM are unencrypted, which may prevent a security concern from the standpoint of the tenant. For example, the host may have the ability to execute commands that allow it to monitor processes executing on the VM, access data logs, or even access the VMs memory.

To address this potential security concern, confidential cloud computing systems may be configured to block certain types of commands from being transmitted to the confidential VM. For example, commands for requesting performance metrics or historical logs from the VM can be blocked. However, if the tenant uses the same communication interface to communicate with the VM, blocking these commands also blocks the tenant from executing such commands and receiving potentially useful information. Nevertheless, this has been considered an acceptable tradeoff for the enhanced security that it provides.

Embodiments of the present disclosure provide techniques to create a separate and secure tenant interface for communicating with confidential VMs running in a host computing system. The tenant interface can be encrypted and can be configured to provide VM access to approved users. The tenant interface is separate from the host interface and provides isolation between host-related operations and tenant operations, further enhancing security. This technique improves the computing system by allowing tenants to access certain capabilities of their confidential VMs that would otherwise be blocked through the host interface. Accordingly, a tenant can have confidence in the security of their data while also having access to useful information such as performance metrics and historical logs, which were previously inaccessible.

FIG.1is a block diagram that illustrates an example computer system architecture, in accordance with some embodiments of the present disclosure. The computing system110may be a distributed computing cluster that serves as a cloud computing platform. In some embodiments, the computing system100may be a Kubernetes-based container orchestration platform.

Resources of the computing system110are provisioned on behalf of tenants by allocating and orchestrating available host resources. Computing system110includes container orchestration system112to instantiate and manage containers and container workloads across one or more nodes120of the computing system110. The nodes120may be physical host machines or VMs in communication with one another. For example, nodes120may each be a physical host machine. AlthoughFIG.1depicts only two nodes120, computing system110may include any number of nodes.

The computing system110may include a processing device130, memory135, and storage device140. Processing device130may also include one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), network processor, or the like. Memory135may include volatile memory devices (e.g., random access memory (RAM)), non-volatile memory devices (e.g., flash memory) and/or other types of memory devices. Storage device140may be one or more magnetic hard disk drives, a Peripheral Component Interconnect (PCI) solid state drive, a Redundant Array of Independent Disks (RAID) system, a network attached storage (NAS array, etc. Processing device130may include multiple processing devices, storage devices, or devices. Processing device130may include a complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets or processors implementing a combination of instruction sets. In some embodiments, the storage device may include an etcd database142. The etcd database142is a store of information that the cluster uses to control the computing system110. For example, the etcd database142can store information used to control various processes of the cluster such as container configuration, service discovery, task scheduling, access permissions, and others. Contents of the etcd database142may be indexed by container identifier (e.g., node, VM, and container identifiers). The etcd database142may be accessed by various processes executing on the container orchestration system112and the nodes120.

Each node120may execute one or more confidential VMs122for executing client workloads, shown inFIG.1as applications126. The confidential VM provides a secure environment in which the VMs memory is encrypted so that the workload data is accessible the client owner of the VM but not the host computing system. Each VM122may include one or more containers124that provide an isolated execution environment for the client's applications126. A container hosted within a confidential VM may be referred to as a confidential container.

The applications126may include any type of executable program, including operating system files and components of client applications such as databases, Web servers, and other services, functions, and workloads. In some embodiments, the containers are executed inside of Kubernetes pods (not shown), which provide for grouping of containers so that the containers within a single pod can share the same resources, allowing them to communicate between each other as if they shared the same physical hardware, while remaining isolated to some degree.

The container orchestration system112can include a control plane114that exposes applications to internal and external networks by defining network policies that control communication with containerized applications (e.g., incoming HTTP or HTTPS requests for services inside the cluster). For example, the control plane114may include REST APIs which expose objects as well as controllers which read those APIs, apply changes to objects, and report status or write back to objects. The control plane114manages workloads on the nodes120and also executes services that are required to control the nodes120to facilitate deployment, scaling, and management of containerized software applications. In some embodiments, the control plane114may include a container orchestration API (e.g., Kubernetes API server). Host users with suitable credentials may be able to communicate with the container orchestration API to facilitate management of the computing platform110. Client users with suitable credentials may be able to communicate with the container orchestration API to facilitate management of the client's confidential VMs.

The container orchestration system112may scale a service in response to workloads by instantiating additional containers with service instances in response to an increase in the size of a workload being processed by the nodes. In this way, the container orchestration system112may allow applications and the data centers that support them to expand from just a few machines and applications to thousands of machines that serve millions of clients.

The computing system110may be accessed by client computing devices170through the network160. The client computing device170may be owned and operated by a system administrator of the computing system110or by a tenant of the computing system110. The tenant may be the owner and operator of one or more confidential VMs122running on the host computing system110. The client computing device170may include one or more software components for communicating with the computing system110. For example, the client computing device170may include a host channel172for issuing host side commands and/or a tenant channel174for issuing control commands used to control tenant owned VMs, containers, and workloads. A user operating the computing device170may use either the host channel172or the tenant channel174depending on the type access allowed to the user, the type of commands that are to be issued, and/or the communication channel to be used to access the VM122. The host channel172and the tenant channel174may use different user credentials to gain access to the computing system110. The host channel172and the tenant channel174may be separate components or they may be included as components of a tenant user interface (UI)176as shown inFIG.1.

The host channel172and the tenant channel174may be command line interfaces. In some embodiments, the host channel172is a kubectl tool used for issuing commands that refer to the host API (e.g., kubectl delete). The host-side commands may include commands to create or reconfigure a confidential VM (e.g., kubectl apply), commands to destroy it (e.g., kubectl delete) or commands to monitor its status (e.g., kubectl describe). Host-side commands may be received from the host channel172through the control plane114, which may include an API server (e.g, kube-apiserver).

The tenant channel174may be a kubectl tool used for issuing kubectl commands to the confidential VM122. The control commands may include commands that enable a tenant of the computing system110to control and monitor the confidential VMs122under the tenants ownership and control. For example, control commands may include commands to obtain container logs (e.g., “kubectl logs”, to execute commands within containers (e.g, “kubectl exec”, copy files from containers (e.g., “kubectl cp”), and others. In some embodiments, some control commands may be received from the tenant channel174through the control plane114, e.g., through the control plane's API server. When the tenant interface is established, as described further below, most or all of the commands may be received from the tenant channel174through the tenant interface, bypassing the control plane114and the host interface.

The container orchestration system112may provide an image-based deployment module for creating containers. In some embodiments, container orchestration system112may pull container images155from a remote image repository150which is communicatively coupled to the container orchestration system112through a network160. Network160may be a public network (e.g., the internet), a private network (e.g., a local area network (LAN) or wide area network (WAN)), or a combination thereof. In one embodiment, network160may include a wired or a wireless infrastructure. In some embodiments, the network160may be an L3 network.

The container images155contain the instructions needed to build a container. For example, container images155may contain the operating system and other executable programs to be run inside the container124, as well as instructions for how to build the container124and what applications should be executed when the container is built. The container image155can hold the source code, libraries, dependencies, environment configuration, tools, and other files needed for the client's application to run. Container images155may be defined and created by the client and may determine specific tasks or workloads that should run on the container124. Container images may be encrypted by the client to preserve the confidentially of the container image's code from the host computing system110. The image repository150may be connected to the computing system110through the network160as shown or may also be included as a storage repository within the computing system110.

When a container124is to be instantiated one or more container images155may be pulled from the remote repository by the container orchestration system112in accordance with instructions from the client received through the control plane114. Before a container124is instantiated from the loaded container image, a verification process may be performed to verify that the VM122is properly configured as a confidential VM. This verification process may be referred to as remote attestation and is performed, in part through a relying party180, which is able to communicate directly with the VM122without going through the control plane114as described further in a relation toFIG.2. In this way, communications between the relying party180and the VM122can be performed in the encrypted domain rather than unencrypted domain of the host. The remote attestation process is described further in relation toFIG.2. The relying party180may be connected to the computing system110through the network160as shown or may also be included as a process running within the computing system110. For example, the relying party180may be a process running within one or more confidential VMs122.

In embodiments of the present disclosure, the relying party180is also configured to enable the creation of a secure communication channel that enables the client computing device170to communicate directly with the confidential VM122in the encrypted domain through a tenant interface that is separate from the host interface and bypasses the control plane114. Since the communications are not accessible to the host computing system110, commands that would normally be blocked by the VM122to ensure confidentiality can now be allowed. Example techniques for implementing the secure communication channel for a tenant interface are described further in relation toFIGS.2-4.

FIG.2is a block diagram that illustrates an example system200for implementing a secure channel for a tenant interface, in accordance with some embodiments of the present disclosure. The system200may implemented, at least partly, in a confidential distributed computing system, such as the computing system110ofFIG.1.

As described above, the confidential VM122may be running on one of the nodes120shown inFIG.1. Communications between the host computing system110and the VM122may be performed through a software stack running on the node120, which may include a node agent202, a container runtime interface204, a container runtime,206, a hypervisor212, and a socket214. These components may be collectively referred to as the host interface216. However, it will be appreciated that the host interface216may also be considered as including the control plane114, which may not be running on the same node. Host-side commands from the host channel172(FIG.1) may be sent to the VM122through the host interface216.

The node agent202is the primary software tool that controls the operation of each node. The node agent202receives instructions through the control plane114for instantiating and monitoring containers. In some embodiments, the node agent202may be a Kubernetes kubelet. The container runtime206is software responsible for running containers, and the container runtime interface204is an API that enables the container runtime206to coordinate with the node agent202. In some embodiments, the container runtime interface204may include a CRI-O container engine and containerd daemon. The container runtime206may be a kata container runtime and may also include a shim process that records encrypted container output.

The container runtime206communicates with the VM122through the hypervisor212and optionally through a socket214(e.g., VSOCK), which facilitates communication between VMs and their host. Communication between the socket214and the container agent210is shown as communication link230. After the tenant interface to the container agent210is established, the communication link230can be eliminated. It will be appreciated that the host interface216described herein is one example of a host interface that could be implemented in accordance with the disclosed techniques and that other arrangements are also possible.

Communications from the host are received at the VM122by the container agent210, which runs on top a kernel208running inside the VM122, the kernel208being the most privileged component of the VMs operating system. The container agent210can manage container processes inside the VM122, responsive to instructions received from the container runtime206running on the host. In some embodiments, the container agent210is a kata agent. The container agent210may be configured to block the processing of specific types of commands that could potentially be received from the host computing system110through the container runtime206. Additionally, as discussed further below, when the separate tenant interface has been established, the container agent210may block all commands received through the host interface216.

The tenant can send instructions to the host through the control plane114to create the VM122and instantiate containers124within the VM122to run specific tenant workloads. To generate the container124, the container agent210may be instructed to pull an encrypted container image218of the tenant's choosing from the image repository150into a storage device of the VM122. Before decrypting the stored container image218and instantiating the running container124, a verification process is performed to ensure that the VM122is configured properly to ensure confidentiality.

The verification process may be performed by a third party, referred to herein as relying party180, which includes an attestation server220and a key broker222. The attestation server220can communicate directly with the container agent210through a direct network connection that does not involve the host interface216or the host's control plane114. For example, the container agent210may have an Internet protocol (IP) network address and port number that is known to the relying party. The relying party180and the container agent210may also use digital certificates such as Transport Layer Security (TLS) certificates to encrypt the communications between them. Thus, communications between the attestation server220and the container agent210can be encrypted and are inaccessible to the host. During the verification process, the container agent210submits evidence to the attestation server220and the attestation server220processes the evidence to determine whether certain criteria met, such as whether the VM's memory is encrypted.

If the evidence provided by the container agent210meets the specified criteria for ensuring confidentiality, the relying party180may obtain a cryptograph key or other secret from the key broker222and send it to the container agent210. The container agent210is then able to use the received key to decrypt the stored container image218. The container agent210then decrypts, unpacks, and mounts the container image218to instantiate the running container124, which contains the tenant's applications126. The key broker222may also provide, through the direct network connection, additional confidential information that the application126may need to operate, such as database passwords, and the like. In this way, the tenant can have confidence that the tenant's workloads are confidential from the host computing system110.

In accordance with presently disclosed techniques, the relying party180may also be used to establish the separate tenant interface that enables the tenant to communicate directly to the container agent210. Specifically, the relying party180has information (network address, port number, digital certificates, etc.) that enables it to communicate directly with the container agent210without going through the host interface216. This same information may then also be used to create a secure encrypted channel between the client's computing device170and the container agent210that bypasses the host interface216. The tenant interface may include a tenant API service190, which is a service that receives commands from the tenant channel174(FIG.1) and relays these commands to the container agent210through the same network connection that the relying party180uses for the remote attestation. The tenant API service190is then able to send commands to the container agent210in a more secure manner since the commands are no longer going through the host interface216and can also be encrypted. Thus, the container agent210can now be configured to accept commands that it would have blocked otherwise.

In some examples, the tenant API service190may be accessed after the tenant provides user credentials (e.g., username and password), which may be different from the credentials used to gain access to the host interface216through the control plane114(FIG.1). Example embodiments of the tenant API service190are described further in relation toFIGS.3and4. Depending on the details of a specific embodiment, the tenant API service190may be running on the client computing device170, the relying party180, or a node120of the computing system110.

In alternative embodiments, it may also be possible to send encrypted commands to the container agent210through the host interface216. For example, a tenant interface could be configured to inject encrypted commands directly into the container runtime206for delivery to the container agent210through the hypervisor212and the socket214. However, sockets such as VSOCK are generally not equipped to handle encrypted communications. Thus, to make such a solution viable could require a redesign of the socket214or only partial encryption of the commands sent from the tenant interface. The embodiments described in relation toFIGS.2-4avoid this potential drawback by bypassing the host interface216, including the hypervisor212and the socket214.

FIG.3is an example of a tenant interface, in accordance with embodiments of the present disclosure. The host interface216shown inFIG.3operates as described in relation toFIG.2, except that the communication link230(FIG.2) between the socket214and the container agent210has been eliminated and tenant commands from the tenant channel174now pass to the container agent210through the tenant API interface190. Tenant commands from the host channel172may still be delivered through the host interface216. For example, commands related to VM lifecycle may be sent to the host interface216to be processed by the container runtime206. As described further below, the tenant API service190may be running on the same computing device as the tenant UI176(e.g., computing device170) or on a separate computing device, such as the relying party180.

The API translator310transforms commands received from the tenant channel174into a style that is suitable for the container agent210. On the host side, host-side commands from the host channel172are processed by the API server of control plane114, the node agent202, the container runtime interface204, and the container runtime206. Each of these components may expose an API that uses different protocols (formatting, syntax, etc.) for receiving and issuing commands or other data. Accordingly, the format of the command received from the user interface may undergo various transformations along the chain of component APIs before reaching the container agent210through the host interface216. The API translator310processes commands from the tenant channel174to cause the same overall transformation that would be caused by the host interface216. This enables implementation of the disclosed embodiments without little or no changes to the programming of the tenant UI176or the container agent210.

In some embodiments, the API translator310may include copies of the components included in the host interface216. For example, the API translator310may have a similar software stack to that of the host interface216, including an instance of the same software used to implement the node agent202, container runtime interface204, and container runtime206. These components can perform substantially the same functions as their counterparts from the host interface216, resulting in the same overall transformation. In some embodiments, the API translator310may implement an algorithm or function that provides an equivalent overall transformation compared to the host interface216with fewer or no intermediates transformations. In other words, the API translator310may include a software component that replicates the overall transformation provided by two or more components of the host interface's software stack.

The API translator310may be configured to communicate directly with the container agent210in the encrypted domain. In embodiments in which the tenant API service190is operating on the client's computing device170, the relying party180(FIG.2) can share the relevant network information with the client computing device170, such as the IP address, and port number of the container agent210, the digital certificates used to encrypt and decrypt the communications, and others. This networking information may be stored, for example, to the tenant etcd database308.

Once the secure channel between the container agent210and the tenant API service190is established, any communications not related to the VM lifecycle can go through the tenant API service190. In Kubernetes, for example, the “kubectl logs” command causes the container agent to return a list of event logs generated by the VM122and/or the applications126running on the VM. This command may be received through the control plane114. However, for a confidential VM, the “kubectl logs” command is usually blocked by the container agent210to ensure that such log information cannot be exposed to the host computing system110. The “kubectl logs” command can be sent to container agent210through the tenant API service190without being blocked, and the resulting logs can be returned to the tenant through the tenant API service190in encrypted form without being exposed within the host computing system110.

Another example of a command that can be sent to the container agent210from the tenant API service190is the “ExecProcess” API resulting from the “kubectl exec” command. The “kubectl exec” command is used to manually execute a command with a container. The “kubectl exec” command can be used, for example, to perform a maintenance operation or to assist in a debugging procedure. However, as with the “kubectl logs” command, the “kubectl exec” is usually blocked by the container agent210to ensure that the host computing system110is not able to gain unauthorized access to the processes running in the container124. The “kubectl exec” command can be sent to container agent210through the tenant API service190without being blocked.

Other types of commands that may go through the tenant API service190include commands to instantiate a container or start or stop the execution of a container, commands for obtaining statistics about the container (metrics), configuring the guest networking, copying files, accessing container input and output (I/O), and others.

In some embodiments, the tenant can also access the container agent210through the host interface216and may also choose to send commands through the host interface216via the control plane114even if the secure connection has been established between the tenant API service190and the container agent210. This option may be chosen for commands that present little or no risk of compromising VM confidentiality, such as commands that relate to the lifetime of the VM122(e.g., commands to create or delete the VM122). In some embodiments, if the secure connection has been established between the tenant API service190and the container agent210, then some or all commands received from the host interface216may be blocked, so that any blocked commands would have to go through the tenant API service190.

In some embodiments, the client computing device170may be configured to determine whether to route a particular command to the container agent210through the control plane114of the host computing system110or through the tenant API service190. For example, the tenant UI176may be configured to have access to both the API server of the control plane114and the tenant API service190. Upon receiving a command from the tenant user, the computing device170can determine, based on the type of command, whether the command should be executed through the host (e.g., create or delete the VM) or though the tenant API service190(e.g., “kubectl logs” or “kubectl exec”). The computing device170may route the command accordingly. In other embodiments, the host interface216and the tenant API service190may be accessed separately through different user interfaces running on the computing device170.

In some embodiments, the tenant API service190may be operating on the relying party180, in which case the tenant channel174communicates with the API translator310over a network connection, and communication between the tenant channel174and container agent210are mediated by the relying party180. In such embodiments, the API translator310receives commands from the tenant channel174over a first network connection and sends commands to the container agent210over a second network connection. The second network connection may be the same connection used by the attestation server220to communicate with the container agent210(e,g, same IP address, port, digital certificates, etc.). The first network connection between the tenant channel174and the API translator310may be established by the tenant API service190using different network information (network addresses, ports, digital certificates, etc.). Communication between the tenant API service190and the tenant channel174may be encrypted using a different pair of digital certificates. However, even if communication between the tenant API service190and the tenant channel174is not encrypted, the host computing system110may not have access to the communication since the data and/or commands bypass the host interface216.

In some cases, the container agent210may be configured to establish an isolated network connection with the relying party180as opposed to exposing a publicly accessible network connection that is reachable by other computing devices. For example, the tenant may configure the container agent210to allow network communication from a single, pre-specified network address known to belong to the relying party180. The container agent210may be configured this way to reduce the possibility of a malicious actor gaining access to the confidential VM122. In such cases, sharing the network information with the client computing device170would not enable the client computing device170to successfully communicate with the container agent210. Deploying the tenant API service190in the relying party180ensures that the tenant API service190will be able to communicate with the container agent210using the relying party's access.

In some cases, the direct network connection between the client computing device170and the container agent210may be preferred over an indirect connection that uses the relying party180an intermediary. Accordingly, in some embodiments, different instances of the tenant API service190may be deployed on the client computing device170and the relying party180. The client computing device170and the relying party180may first attempt to establish a direct connection as described above. If the direct connection is established, the API translator310running on the client computing device170can send commands directly to the container agent210. If the direct connection fails, the indirect connection may be established, in which case the tenant UI176running on the client computing device170communicates over the network with API translator310running on the relying party180, and the API translator310send commands directly to the container agent210. For example, the relying party180may send network information of the container agent210to the tenant API service190running on the client computing device170. If the container agent210is not reachable by the tenant API service190in this manner, then the tenant API service190running on the client computing device170can inform the relying party180that the direct connection was not successful. The relying party180can then configure the separate network connection between the tenant UI176running on the client computing device170and the API translator310running on the relying party.

FIG.4is a process flow diagram for a method of sending commands to a container agent of a confidential VM, in accordance with some embodiments of the present disclosure. The method400may be performed by processing logic that may include hardware (e.g., circuitry, dedicated logic, programmable logic, a processor, a processing device, a central processing unit (CPU), a system-on-chip (SoC), etc.), software (e.g., instructions running/executing on a processing device), firmware (e.g., microcode), or a combination thereof. In some embodiments, the method400may be performed by the tenant API service190(FIGS.2and3) which may reside on the client computing device170shown inFIG.1. The method may begin at block402.

At block402, a first command is sent from a computing device to a container agent of a confidential virtual machine (VM) running on a host computing system. The first command is sent to the container agent through a control plane of the host computing system and causes the container agent to communicate with a relying party to verify confidentiality of the confidential VM. For example, the first command may instruct the container to load a container image and/or instantiate a container within the confidential VM based on the container image.

At block404, network information for the container agent is received from the relying party. The network information may be the same information that the relying party uses to communicate with the container agent to perform attestation, and may include, for example, a network address (e.g., IP address), port number, digital certificates, etc.

At block406, a direct network connection with the container agent is established based on the network information received from the relying party.

At block408, a second command is sent from the computing device to the container agent of the confidential VM via the direct network connection without accessing the control plane of the host computing system. For example, the second command may be a command to retrieve logs or metrics from the container agent or execute a function within a container instantiated by the container agent. The first and second commands may be kubectl commands input by a user into a kubectl command line interface.

It will be appreciated that embodiments of the method400may include additional blocks not shown inFIG.4and that some of the blocks shown inFIG.4may be omitted. Additionally, the processes associated with blocks402through408may be performed in a different order than what is shown inFIG.4. For example, the method may also include translating the command to a new format that is applicable to the container agent. The translation may be performed by a software stack residing on the computing device that processes the command in the same manner as a similar software stack that resides on the host computing system. The translation may also performed, at least in part, by a translator that generates an equivalent overall transformation of the second command that would be performed by the software stack that resides on the host computing system.

In some embodiments, commands may also be sent from the computing device to the container agent of the confidential VM via the control plane of the host computing system even after establishing the direct network connection with the container agent. Additionally, in some embodiments, commands may be sent from the computing device to the container agent of the confidential VM via the relying party. Such commands may be relayed to the container agent through the relying party if the container agent is not reachable from the computing device or if the network connection established between the computing device and the container agent fails.

FIG.5is a block diagram of a system for sending commands to a container agent of a confidential VM, in accordance with some embodiments of the present disclosure. The system500includes a processing device502operatively coupled to a memory504. The memory504includes instructions that are executable by the processing device502to cause the processing device502to send commands to a container agent of a confidential VM without going through the control plane of the host computing system on which the confidential VM resides.

The memory504includes instructions506to send a first command from the computing device to a container agent of a confidential virtual machine (VM) running on a host computing system. The first command is sent to the container agent through a control plane of the host computing system and causes the container agent to communicate with a relying party to verify confidentiality of the confidential VM.

The memory504also includes instructions508to receive network information for the container agent from the relying party. The memory504also includes instructions510establish a direct network connection with the container agent based on the network information received from the relying party. The memory504also includes instructions512to send a second command from the computing device to the container agent of the confidential VM via the direct network connection without accessing the control plane of the host computing system.

It will be appreciated that various alterations may be made to the process illustrated inFIG.5and that some components and processes may be omitted or added without departing from the scope of the disclosure.

The example computing device600may include a processing device (e.g., a general purpose processor, a PLD, etc.)602, a main memory604(e.g., synchronous dynamic random access memory (DRAM), read-only memory (ROM)), a static memory606(e.g., flash memory and a data storage device618), which may communicate with each other via a bus624.

Computing device600may further include a network interface device608which may communicate with a network620. The computing device600also may include a video display unit610(e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device612(e.g., a keyboard), a cursor control device614(e.g., a mouse) and an acoustic signal generation device616(e.g., a speaker). In one embodiment, video display unit610, alphanumeric input device612, and cursor control device614may be combined into a single component or device (e.g., an LCD touch screen).

Data storage device618may include a computer-readable storage medium628on which may be stored one or more sets of instructions622that may include a tenant API service630comprising instructions for carrying out the operations described herein, in accordance with one or more aspects of the present disclosure. The tenant API service630may also reside, completely or at least partially, within main memory604and/or within processing device602(e.g. within processing logic626) during execution thereof by computing device600, main memory604and processing device602also constituting computer-readable media. The tenant API service630may further be transmitted or received over a network620via network interface device608.

Unless specifically stated otherwise, terms such as “sending,” “receiving,” “establishing,” “translating,” “converting,” “generating,” “routing,” “updating,” “providing,” or the like, refer to actions and processes performed or implemented by computing devices that manipulates and transforms data represented as physical (electronic) quantities within the computing device's registers and memories into other data similarly represented as physical quantities within the computing device memories or registers or other such information storage, transmission or display devices. Also, the terms “first,” “second,” “third,” “fourth,” etc., as used herein are meant as labels to distinguish among different elements and may not necessarily have an ordinal meaning according to their numerical designation.