SOFTWARE-BASED HARDWARE SECURITY MODULE (HSM) FOR A VIRTUALIZED COMPUTING ENVIRONMENT

A software-based implementation of a hardware security module (HSM) includes a software-based HSM device that uses a hardware-protected secure environment to provide protection for data and for execution of code of the HSM device. The HSM device operates in a virtualized computing environment, and an interface to the security device enables an application running on a virtualized computing instance to access the security device. The execution of the code in the secure environment is a first security mode of operation, and the HSM device can switch between multiple different security modes of operation.

BACKGROUND

Unless otherwise indicated herein, the approaches described in this section are not admitted to be prior art by inclusion in this section.

Virtualization allows the abstraction and pooling of hardware resources to support virtual machines in a software-defined networking (SDN) environment, such as a software-defined data center (SDDC). For example, through server virtualization, virtualized computing instances such as virtual machines (VMs) running different operating systems (OSs) may be supported by the same physical machine (e.g., referred to as a host). Each virtual machine is generally provisioned with virtual resources to run an operating system and applications. The virtual resources may include central processing unit (CPU) resources, memory resources, storage resources, network resources, etc.

However, a virtualized computing environment having hosts that support VMs is often vulnerable to security threats, and so cryptographic techniques involving public/private keys are often used for security. A hardware security module (HSM) is one type of device that is used for such cryptographic techniques.

A HSM is a physical device that generates, protects, and manages digital keys for strong authentication, and may provide other functions (e.g., acceleration) that support cryptography. HSMs traditionally come in the form of a plug-in card or an external device that attaches directly to the host. HSMs are often used for their robust security properties—they provide an isolated hardware unit that performs secure cryptographic operations without exposing private keys to software on the system/host/network. HSMs can be used by public key infrastructure (PKI) services, key management services, financial payment applications, etc. in both public cloud environments and edge/Internet-Of-Things (IoT) environments, or other types of public/private environments.

While a dedicated hardware security solution such as HSMs may seem to offer robust solutions for a range of applications, there are several downsides to the use of such hardware-based solutions. For example, dedicated hardware solutions are often inflexible, lack interoperability, and can lead an organization to vendor lock-in (e.g., a user becomes dependent on a particular vendor for products and services, thereby making a switch to another vendor difficult or costly) . As another example, the use of dedicated hardware solutions complicates management in a cloud environment, such as in a software-defined datacenter (SDDC). For instance, VM migration and host maintenance become difficult to manage when applications demand dedicated hardware access. Still further, dedicated hardware requirements can add significant costs to security solutions.

DETAILED DESCRIPTION

The hardware-only implementation of the HSMs provides the advantages and disadvantages such as described above. An alternative to the hardware-only implementation of HSMs is a software-only implementation of HSMs that attempts to address some of the drawbacks of the hardware-only implementation of HSMs. An example of a software-only implementation of an HSM is the open source SoftHSM project. Such software-only implementations leverage software encryption and operating system isolation features to emulate hardware functionality. However, a problem with such software-only alternatives is that their security lacks robustness. For instance, while software-only solutions may offer flexibility and simplicity, such software-only solutions may be more vulnerable to attack by software processes running on the same system. For example, a compromised operating system (OS) may allow an attacker to extract private keys in the SoftHSM key store. As such, security guarantees in software-only solutions may be much weaker than their hardware counterparts.

The present disclosure addresses the above and other drawbacks of current hardware-only and software-only implementations of HSMs, by providing a software-based implementation of HSM that leverages the security protections provided by hardware, specifically secure enclaves provided by hardware. In this manner, the software-based HSM techniques disclosed herein receive the benefit of both worlds: the flexibility of software and the security robustness of hardware.

For example, the embodiments described herein combine a software-based implementation of HSM with trusted execution environment (TEE) technologies. TEE technologies allow applications to create an isolated resource environment into which code and data can be loaded and then executed in a protected manner, even from privileged system software (such as an operating system) running on the same platform. Embodiments of a TEE described herein enable an application to create protected memory regions, referred to as secure enclaves or other hardware-protected secure environment, through a hardware application program interface (API). HSM-related code and data stored and executed in a secure enclave are encrypted and protected from direct access attacks through hardware enforcement mechanisms built into the platform.

Accordingly, the embodiments described herein allow the security functionality of hardware security solutions (like HSM) to be effectively and securely implemented by hardware-protected software. A result is an embodiment of HSM that has the flexibility of software along with the stronger security protections of hardware.

Computing Environment For Software-Based HSM

To further explain the operations of the elements that may cooperate to provide and support a software-based/software-defined HSM that is protected by hardware, various implementations will now be explained in more detail usingFIG. 1, which is a schematic diagram illustrating an example virtualized computing environment100that can implement a software-based HSM in a hardware-protected secure environment. Depending on the desired implementation, virtualized computing environment100may include additional and/or alternative components than that shown inFIG. 1.

It is understood that the virtualized computing environment100is one example of a computing environment in which the methods described herein may be used. Methods to configure and use a hardware-protected software-based HSM may be implemented in other embodiments for other types of computing environments, including computing environments having physical endpoints (alternatively or in addition to endpoints that are comprised of virtualized computing instances) such as laptops, physical servers, mobile devices, desktop computers, etc. that are capable of using a software-based HSM.

In the example inFIG. 1, the virtualized computing environment100includes multiple hosts, such as host-A110A ... host-N110N that may be inter-connected via a physical network112, such as represented inFIG. 1by interconnecting arrows between the physical network112and host-A110A ... host-N110N. Examples of the physical network112can include a wired network, a wireless network, the Internet, or other network types and also combinations of different networks and network types. For simplicity of explanation, the various components and features of the hosts will be described hereinafter in the context of the host-A110A. Each of the other host-N110N can include substantially similar elements and features.

The host-A110A includes suitable hardware114A and virtualization software (e.g., a hypervisor-A116A) to support various virtual machines (VMs). For example, the host-A110A supports VM1118... VMY120, wherein Y (as well as N) is an integer greater than or equal to 1. In practice, the virtualized computing environment100may include any number of hosts (also known as computing devices, host computers, host devices, physical servers, server systems, physical machines, etc.), wherein each host may be supporting tens or hundreds of virtual machines. For the sake of simplicity, the details of only the single VM1118are shown and described herein.

VM1118may be a guest VM that includes a guest operating system (OS)122and one or more guest applications124(and their corresponding processes) that run on top of the guest OS122. VM1118may further include one or more guest software-based HSM components, as well as components associated with establishing and operating a secure enclave (SE) or other hardware-protected secure environment, both of which are configured to cooperate to enable usage of the software-based HSM components in the secure environment in a manner that will be further described in detail later below. Such guest software-based HSM components (as well as the components associated with establishing and operating the secure environment) are collectively depicted inFIG. 1as guest HSM/SE components126.

The guest HSM/SE components126may include agents, services, applications (which may be one of the applications124and/or other applications), modules, engines, objects or other data, virtual/logical ports, drivers, application program interfaces (APIs), libraries, daemons, other software or code that is stored on a computer-readable medium and executable by a processor, etc., all of which are generally referred to herein as component(s) and represented and described herein in the context of the guest HSM/SE components126. In some embodiments, one or more of the guest HSM/SE components126may be part of the guest OS122(e.g., reside in a user space and/or in a kernel space of the guest OS122), while one or more of the guest HSM/SE components126may be separate from and external to the guest OS122(e.g., reside in VM1118outside of the guest OS122) in some embodiments.

One or more of the guest OS122, the guest application(s)124, the guest HSM/SE components126, and other code and related data (including data structures) associated with operating VM1118may be stored in a guest memory space that is provisioned for VM1118and/or in some other storage location in host-A110A. For example, such guest memory may store a HSM library, a HSM driver, a trusted execution environment (TEE) application that generates and configures the secure environment, etc., whose operations will be described later below.

The hypervisor-A116A may be a software layer or component that supports the execution of multiple virtualized computing instances. The hypervisor-A116A may run on top of a host operating system (not shown) of the host-A110A or may run directly on hardware114A. The hypervisor116A maintains a mapping between underlying hardware114A and virtual resources (depicted as virtual hardware130) allocated to VM1118and the other VMs.

In one embodiment, HSM/SE components140may run on top of or within the hypervisor-A116A or elsewhere outside of the VMs. Analogous to the guest HSM/SE components126in VM1118, these HSM/SE components140reside outside of VM1118and collectively represent one or more components associated with establishing and operating a hardware-protected secure environment and one or more software-based HSM components that use the secure environment, which will be further described in detail later below.

The HSM/SE components140may include agents, services, applications, modules, engines, objects or other data, virtual/logical ports, drivers, APIs, libraries, daemons, other software or code that is stored on a computer-readable medium and executable by a processor, devices and hardware, the secure environment/enclaves themselves or parts thereof, etc., all of which are generally referred to herein as component(s) and represented and described herein in the context of the HSM/SE components140.

Hardware114A includes suitable physical components, such as central processing unit(s) (CPU(s)) or processor(s)132A; storage device(s)134A; and other hardware136A such as physical network interface controllers (NICs), storage disk(s) accessible via storage controller(s), etc. Virtual resources (e.g., the virtual hardware130) are allocated to each virtual machine to support a guest operating system (OS) and application(s) in the virtual machine, such as the guest OS122and the application(s)124(e.g., a word processing application, accounting software, a browser, etc.) in VM1118. Corresponding to the hardware114A, the virtual hardware130may include a virtual CPU, a virtual memory (including the guest memory138), a virtual disk, a virtual network interface controller (VNIC), etc.

A management server142of one embodiment can take the form of a physical computer with functionality to manage or otherwise control the operation of host-A110A...host-N110N. In some embodiments, the functionality of the management server142can be implemented in a virtual appliance, for example in the form of a single-purpose VM that may be run on one of the hosts in a cluster or on a host that is not in the cluster. The functionality of the management server142may be accessed via one or more user devices146that are operated by a system administrator. For example, the user device146may include a web client148(such as a browser-based application) that provides a user interface operable by the system administrator to access functionality of the management server142.

The management server142may be communicatively coupled to host-A110A ... host-N110N (and hence communicatively coupled to the virtual machines, hypervisors, agents, drivers, applications and modules, hardware, etc.) via the physical network112. The host-A110A ... host-N110N may in turn be configured as a data center that is managed by the management server142. In some embodiments, the functionality of the management server142may be implemented in any of host-A110A ... host-N110N, instead of being provided as a separate standalone device such as depicted inFIG. 1.

Depending on various implementations, one or more of the physical network112, the management server142, and the user device(s)146can comprise parts of the virtualized computing environment100, or one or more of these elements can be external to the virtualized computing environment100and configured to be communicatively coupled to the virtualized computing environment100.

Software-Based HSM In A Hardware-Protected Secure Environment

FIGS. 2 and 3are schematic diagrams illustrating some of the HSM and secure environment components in the virtualized computing environment100ofFIG. 1. More specifically,FIGS. 2 and 3show further details of the guest HSM/SE components126ofFIG. 1that reside in VM1118and of the HSM/SE components140ofFIG. 1that reside outside of VM1118(such as in the hypervisor116-A or elsewhere in the host-A110A).

Referring first toFIG. 2, the guest HSM/SE components126includes a SE device driver200that is configured to communicate with a SE device202via an API204. For example, the SE device202may be one of the HSM/SE components140that reside in or be provided by the hypervisor-A116A, and may be a virtual device that is presented/exposed to VM1118A via the API204. Among other things, the SE device202provides an abstraction of low-level details of a trusted execution environment (TEE) including those of one or more secure enclaves (SE)206or other hardware-protected platform in a SE environment208. The SE device202of various embodiments generates/configures and operates the enclaves206in the SE environment208. Further in some embodiments, the SE device202(and/or some other device in the host-A110A outside of VM1118) contains software-based HSM functionality.

By being an abstraction and a virtual device that is exposed via the API204, the SE device202may be accessed and used by applications124, utilities, tools, etc. of VM1118, such as for cryptographic or other security-related operations, via the SE device driver200. For instance, the applications124may employ a software developer kit (SDK)210to facilitate interaction with the SE device driver200via a SE port212and an API214. The SDK210and the SE port212may comprise parts of the guest HSM/SE components126shown inFIG. 1.

With respect to the HSM/SE components140, the SE environment208is accessible to the SE device202via an API216. According to some embodiments, the SE environment208(including the enclaves206) may reside in the hypervisor-A116A. In other embodiments, the SE environment may reside elsewhere in the host-A outside of the hypervisor-A110A.

Depending on various implementations, the SE environment208includes a secure monitor218that is supported by TEE hardware220in a SE backend222. The SE device202accesses the secure monitor218via the API216, and the secure monitor218in turn passes commands/calls/data/etc. between the SE device202and the enclaves206via an API224. In some embodiments, the SE backend222executes in the context of a user process and/or a kernel process of the hypervisor-A116-A, while the SE backend222of still further embodiments may execute outside of or independently of any process of the hypervisor-A110A.

Turning now toFIG. 3,FIG. 3more specifically shows the HSM-related components of the guest HSM/SE components126and of the HSM/SE components140ofFIG. 1. At VM1118according to some embodiments, a HSM library300resides or executes in the user space of the guest OS122, and a HSM driver302resides or executes in the kernel space of the guest OS122. Among other things, the HSM library300can store data, such as keys or other security-related objects/information, that are passed between the applications124and the enclaves206via an HSM device304.

The HSM driver302may be the same as the SE device driver200shown and described above with respect toFIG. 2, or the HSM driver302may be some other driver in VM1118. Analogously, the HSM device304residing in or provided by the hypervisor-A116A may be the same as the SE device202described above with respect toFIG. 2, or the HSM device304may be some other device in host-A110A outside of VM1118. In some embodiments, the HSM device304is software-based, and the HSM device304may be exposed by a VMX file306of the hypervisor-A116A for VM1118.

The HSM device304, as a software-based security device, executes at least some security-related operations inside of the secure enclave(s)206. Such security-related operations may include, for example, cryptography operations involving private key generation, signing, authentication, password verification, etc. Accordingly, such security-related operations can be executed with added secrecy/isolation by virtue of the hardware-based protections provided by the enclaves206.

In addition to execution of security-related operations inside of the enclaves206, the enclaves206may also store keys310, passwords, or other confidential information. As such, this confidential information also leverages the hardware-based security capability of the enclaves206for added protection.

FIG. 4is a schematic diagram illustrating interaction of an application with a secure environment for execution of code. For example, the application124may interact with the software (comprised of untrusted code400and trusted code402) of the software-based HSM device304. The untrusted code400may be code whose contents (including data) and execution processes are preferably protected by the functionality of the HSM device304. However, it is possible that privileged system code404(such as the guest OS122, virtual machine monitor (VMM) code, basic input/output system (BIOS) code, etc.) may get compromised by malicious code, thereby providing unauthorized access to the code400despite the level of protection provided by the HSM device304itself. Hence, such code400is considered to be untrusted code.

The trusted code402may be code whose contents (including data) and execution processes are preferably protected not only by the HSM device304but also by the enclave206. The trusted code402may comprise code (and data), for example, that need a higher level of protection against unauthorized access. Hence, such code402is considered to be trusted code since they require a higher level of protection and are therefore resident and executed inside the enclave206, so as to maintain the integrity/trustworthiness of such code.

As depicted inFIG. 4, the untrusted code400may include code to create the enclave206and code to call the trusted code402. For instance during the course of executing a portion of the untrusted code400, the execution sequence reaches a point in which a portion of trusted code needs to be called. Accordingly, the application124calls (shown at406) into the enclave206in order to have that portion of the trusted code402executed inside of the enclave206(e.g., executing the portion of the trusted code to process secret information, etc.). When the execution of that portion of the trusted code is completed, the execution returns (shown at408) to the untrusted code400outside of the enclave206to resume execution of subsequent portions of the untrusted code400.

As symbolically depicted at410inFIG. 4, the privileged system code404does not have visibility or access into the enclave206. Thus, the contents of the enclave206(e.g., keys, security-related code/operations and execution thereof, etc.) are protected against the privileged system code404in the event that such privileged system code404is compromised by a security threat.

FIG. 5is a flow diagram of an example interaction between an application, HSM components, and a secure enclave in the virtualized computing environment ofFIG. 1. More specifically,FIG. 5further illustrates the interaction(s) depicted inFIG. 4and described previously above, in the context of encryption, decryption, signing, etc., wherein trusted code operations are performed/executed inside of the enclave206while untrusted code operations are performed/executed outside of the enclave206.

Initially and not shown inFIG. 5, when the application124calls a key generation utility of the HSM device304, the utility calls into the enclave206, and the enclave206generates a key pair within the enclave206. The enclave206returns both the public key and the private key, sealed by an enclave sealing key (SK) referred to as an encrypted private key (EPK) back to the HSM library300for storage and for use in later lookup.

Then later when the application124calls a utility of the HSM device304to perform a signing operation, the utility obtains the EPK from the HSM library300, and passes the EPK back to the enclave206. These operations are generally depicted at500inFIG. 5.

The EPK is decrypted inside of the enclave206(shown at502inFIG. 5). The decrypted EPK is subsequently passed from the enclave206back to the HSM device304, so that operations can be performed to calculate a hash (e.g., generate a digest). The operations performed between the HSM device304and the application124to use the unencrypted private key to generate the digest are generally depicted at504inFIG. 5.

The HSM device304then sends the digest to the enclave206, which then performs operations to sign the digest. The signing operation inside of the enclave206is shown generally at506inFIG. 5.

The enclave206sends the signature to the HSM device304, which then sends the signature to the application124. The session with the HSM device304and the enclave206is then finalized/ended. These operations are generally depicted at508inFIG. 5.

The foregoing description with respect toFIGS. 1-5involve a first security mode of operation wherein at least some confidential/secret information and/or confidential/secret processes of the HSM device304are stored or executed inside of the hardware-protected enclave206, for purposes of added security. According to some embodiments, capability is provided to enable the virtualized computing environment100(e.g., the HSM device304) to switch between different security modes of operation.

FIGS. 6 and 7are schematic diagrams respectively illustrating other security modes of operation that may be used in the virtualized computing environment100ofFIG. 1. Depending on configuration decisions made by a user (e.g., a system administrator operating the management server142), the security mode of operation in the virtualized computing environment100can switch from one security mode of operation to another (different) security mode of operation. Such configuration decisions may be based on use cases, hardware resource availability, desired protection level, and/or other considerations.

FIG. 6for a second security mode of operation (as well asFIG. 7for a third security mode of operation) depicts some similar elements asFIG. 3that depicts the first security mode of operation. Thus, similar elements between these figures are labeled using the same reference numbers.

Beginning with the second security mode of operation ofFIG. 6, the keys310are stored in the HSM library300in the user space of the guest OS122of VM1118, while the security-related operations308(e.g., cryptography operations) are performed in user processes600at the hypervisor-A116-A. In this second security mode of operation, the protection level for the keys310may be classified as low, while the protection level for the security-related operations308may be classified as medium. Both of these protection levels may be relatively lower than those of the first security mode of operation wherein both the keys310and the security-related operations308have high protection levels by virtue of being located/executed in the enclave206.

For the third security mode of operation ofFIG. 7, a VM encryption utility (VMCrypt)700is used for encrypting VM1118and other VMs that run on the host-A110A. The VM encryption utility700encrypts the keys310as well, while the security-related operations308are performed in the user processes600at the hypervisor-A116-A. In this third security mode of operation, the protection level for the keys310may be classified as medium, while the protection level for the security-related operations308also may be classified as medium. Both of these protection levels may thus also be relatively lower than those of the first security mode of operation wherein both the keys310and the security-related operations308have high protection levels by virtue of being located/executed in the enclave206.

FIG. 8is a flowchart of an example method800to operate a software-based HSM device in the virtualized computing environment100ofFIG. 1. For example, the method800may be performed by the HSM device304or other similar software-based security device, in cooperation with the secure enclave(s)206(and/or some other hardware-protected secure environment), the application(s)124, the HSM driver302, the HSM library300, etc., for purposes of executing untrusted (first) code outside of the enclave206and executing trusted (second) code in the enclave206.

Example method800may include one or more operations, functions, or actions illustrated by one or more blocks, such as blocks802to812. The various blocks of the method800and/or of any other process(es) described herein may be combined into fewer blocks, divided into additional blocks, supplemented with further blocks, and/or eliminated based upon the desired implementation. In one embodiment, the operations of the method800and/or of any other process(es) described herein may be performed in a pipelined sequential manner. In other embodiments, some operations may be performed out-of-order, in parallel, etc.

Beginning at a block802(“ENABLE ACCESS, VIA A FIRST INTERFACE, TO THE SECURITY DEVICE BY AN APPLICATION”), access to the HSM device304by the application124is enabled via the API124and the SE device driver200.

At a block804(“ENABLE ACCESS, VIA A SECOND INTERFACE, TO A HARDWARE-PROTECTED SECURE ENVIRONMENT BY THE SECURITY DEVICE”), access by the HSM device304to the enclave206is enabled via the APIs216and224, and also by the secure monitor218.

The block804may be followed by a block806(“EXECUTE AN INITIAL PORTION OF FIRST CODE OUTSIDE OF THE SECURE ENVIRONMENT”). For example, the application124may send a first command to the HSM device304to perform encryption, decryption, signature, etc. operations. In response to the first command, the HSM device304may execute at least some of the untrusted code400shown inFIG. 4outside of the enclave206.

The block806may be followed by a block808(“EXECUTE A PORTION OF SECOND CODE INSIDE OF THE SECURE ENVIRONMENT”), wherein the HSM device sends a second command (such as a call depicted at406inFIG. 4) to the enclave206to execute at least some of the trusted code402inside of the enclave206. When the execution of the portion of the trusted code402inside of the enclave206is completed, execution returns (shown at408) to the untrusted code400.

The block808may be followed by a block810(“EXECUTE A SUBSEQUENT PORTION OF THE FIRST CODE OUTSIDE OF THE SECURE ENVIRONMENT”), wherein a subsequent portion of the untrusted code400is executed outside of the enclave206.

At a block812(“SWITCH SECURITY MODES OF OPERATION”), the security modes of operation may be switched for next processes. For example and as depicted inFIGS. 3, 6, and 7, the HSM device304may switch between the first, second and third security modes of operation depending on various use cases, hardware resources, desired protection levels, etc.

Computing Device

The above examples can be implemented by hardware (including hardware logic circuitry), software or firmware or a combination thereof. The above examples may be implemented by any suitable computing device, computer system, etc. The computing device may include processor(s), memory unit(s) and physical NIC(s) that may communicate with each other via a communication bus, etc. The computing device may include a non-transitory computer-readable medium having stored thereon instructions or program code that, in response to execution by the processor, cause the processor to perform processes described herein with reference toFIGS. 1-8. For example, computing devices capable of acting as host devices may be deployed in virtualized computing environment100.

Although examples of the present disclosure refer to “virtual machines,” it should be understood that a virtual machine running within a host is merely one example of a “virtualized computing instance” or “workload.” A virtualized computing instance may represent an addressable data compute node or isolated user space instance. In practice, any suitable technology may be used to provide isolated user space instances, not just hardware virtualization. Other virtualized computing instances may include containers (e.g., running on top of a host operating system without the need for a hypervisor or separate operating system; or implemented as an operating system level virtualization), virtual private servers, client computers, etc. The virtual machines may also be complete computation environments, containing virtual equivalents of the hardware and system software components of a physical computing system. Moreover, some embodiments may be implemented in other types of computing environments (which may not necessarily involve a virtualized computing environment), wherein it would be beneficial to provide hardware-based protected environments for the processes and data of software-based security tools.

The foregoing detailed description has set forth various embodiments of the devices and/or processes via the use of block diagrams, flowcharts, and/or examples. Insofar as such block diagrams, flowcharts, and/or examples contain one or more functions and/or operations, it will be understood that each function and/or operation within such block diagrams, flowcharts, or examples can be implemented, individually and/or collectively, by a wide range of hardware, software, firmware, or any combination thereof.

Some aspects of the embodiments disclosed herein, in whole or in part, can be equivalently implemented in integrated circuits, as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computing systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or as virtually any combination thereof, and that designing the circuitry and/or writing the code for the software and or firmware are possible in light of this disclosure.