Secure transport of encrypted virtual machines with continuous owner access

Managing encrypted datasets is illustrated. A method includes obtaining a first decryption key. The first decryption key is configured to be used to decrypt an encrypted dataset that has been encrypted using a first encryption mechanism. The first encryption mechanism is associated with the first decryption key that can be used to decrypt the dataset. The method further includes encrypting the first decryption key with a second encryption mechanism. The method further includes encrypting the first decryption key with a third encryption mechanism. The method further includes creating a package including at least the first decryption key encrypted with the second encryption method and the first decryption key encrypted with the third encryption method. The method further includes signing the package with a guardian signature and signing the package with a signature created from the first decryption key.

BACKGROUND

Background and Relevant Art

Interconnection of computing systems has facilitated distributed computing systems, such as so-called “cloud” computing systems. In this description, “cloud computing” may be systems or resources for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, services, etc.) that can be provisioned and released with reduced management effort or service provider interaction. A cloud model can be composed of various characteristics (e.g., on-demand self-service, broad network access, resource pooling, rapid elasticity, measured service, etc.), service models (e.g., Software as a Service (“SaaS”), Platform as a Service (“PaaS”), infrastructure as a Service (“IaaS”), and deployment models (e.g., private cloud, community cloud, public cloud, hybrid cloud, etc.).

Cloud and remote based service applications are prevalent. Such applications are hosted on public and private remote systems such as clouds and usually offer a set of web based services for communicating back and forth with clients.

In cloud computing environments, a tenant may include a user, company, department of a company or other entity that has rights to access one or more of the virtual machines deployed at a datacenter run by a cloud service provider. A tenant may wish to have virtual machines deployed on its behalf. However, the tenant may wish to prevent other entities (such as other tenants, eavesdroppers, or even datacenter administrators) from being able to access its virtual machines while those machines are in storage prior to being deployed on a host at a datacenter or when the virtual machines are being deployed to a host in the datacenter. To accomplish this, the virtual machine may be encrypted, such as by encrypting a virtual hard drive and/or the metadata for the virtual machine. Thus, an encrypted virtual machine will move in and out of offline storage and between hosts in a manner that preserves confidentiality and integrity of its contents. Further, the key used to encrypt the contents of the virtual machine may have to be rolled over (i.e. changed) with some regularity. This may present challenges for an owner of a virtual machine being able to access their encrypted virtual machines.

However, there may be some desire to have the virtual machine both able to be deployed at the host and available for retrieval and execution by the owner (e.g. the tenant) of the VM. This may be difficult given the protected nature of various encryption schemes. In particular, a goal of encryption is to prevent large numbers of entities from being able to decrypt an encrypted VM by protecting the secrets used to decrypt. Thus, there are challenges with having two different entities being able to decrypt an encrypted VM.

The subject matter claimed herein is not limited to embodiments that address any disadvantages or that operate only in environments such as those described above. Rather, this background is only provided to illustrate one exemplary technology area where some embodiments described herein may be practiced.

BRIEF SUMMARY

One embodiment illustrated herein includes a method that may be practiced in a computing environment. The method includes acts for managing encrypted datasets. The method includes obtaining a first decryption key. The first decryption key is configured to be used to decrypt an encrypted dataset that has been encrypted using a first encryption mechanism. The first encryption mechanism is associated with the first decrypt on key that can be used to decrypt the dataset. The me hod further includes encrypting the first decryption key with a second encryption mechanism. The second encryption mechanism is associated with a second decryption key used by a first entity, such that the second decryption key can be used by the first entity to decrypt the dataset by first decrypting the first key encrypted key using the second decryption key, and then using the decrypted first key to decrypt the dataset. The method further includes encrypting the first decryption key with a third encryption mechanism. The third encryption mechanism is associated with a third decryption key used by a second entity, such that the third decryption key can be used by the second entity to decrypt the dataset by first decrypting the first key encrypted key using the third decryption key, and the using the decrypted first key to decrypt the dataset. The method further includes creating a package comprising at least the first decryption key encrypted with the second encryption method and the first decryption key encrypted with the third encryption method. The method further includes signing the package with a guardian signature and signing the package with a signature created from the first decryption key.

DETAILED DESCRIPTION

A simple illustration of one embodiment of the invention is illustrated with reference toFIG. 1.FIG. 1illustrates an encrypted dataset101. The encrypted dataset101can be decrypted using a transport key102. In the example illustrated, multiple copies of the transport key102are encrypted using encryption schemes particular to particular entities. Thus, for example, a first copy102-0is encrypted for an owner such that an owner of the dataset101can decrypt the copy, but other entities cannot.FIG. 1further illustrates n additional copies where each of the n additional copies102-1through102-nare encrypted for custodians (such as entities that may host or store a dataset for the owner) such that each custodian can decrypt their copy, but other entities cannot. In some embodiments, this may be accomplished by using asymmetric encryption techniques. Thus, for example, the first copy102-0is encrypted using a public key for the owner such that the owner can use its private key to decrypt the first copy102-0. Similarly, the copy102-1may be encrypted using a public key for a first custodian, such that the first custodian can decrypt the copy102-1using the custodian's private key. Similar encryption and decryption may be done for the remaining custodian copies.

The copies are packaged together and signed with a guardian signature103using a cryptographic keyed hash or digital signature to ensure that the package has not been tampered with. The guardian, the entity that signs the copies, may be for example the owner or a custodian, and can change as the encrypted dataset101is transferred and/or re-encrypted. The guardian signature can cryptographically bind the package together to ensure that dataset101cannot be exposed to unauthorized entities. The package also includes a transport key signature104that is a form of authentication that is created by performing a cryptographic hash or other function (such as a message authentication code (MAC) function) of the transport key that can be used as proof of the guardian's knowledge of the transport key.

Illustrating now a more detailed example, some embodiments illustrated herein are directed to securely deploying encrypted virtual machines (VMs) to a host in a datacenter in a fashion that allows both the host to decrypt the encrypted VM and a tenant owner of the VM to able to decrypt the encrypted VM. This can be accomplished by the system encrypting the VM with a decryption key, and two (or more if desired) copies of the VM decryption key for the VM being created. One copy of the VM decryption key is encrypted using a tenant public key (allowing the tenant to decrypt the VM decryption key using the tenant private key, and then to decrypt the VM using the VM decryption key) while the second copy of the VM decryption key is encrypted using a key distribution service (KDS) public key (which allows the host to obtain the VM decryption key with help from the KDS). Additionally the two encrypted keys can be appropriately wrapped, such as using the guardian signature103and a transport key signature104as described above to ensure that tampering has not occurred.

While the above summary is in the context of deploying virtual machines in a datacenter environment, it should be appreciated that embodiments may be implemented in other scenarios as well

The virtual machine example is illustrated inFIG. 2, which illustrates a tenant200, a cloud service201, a virtual machine manager202, virtual machine storage203, a virtual machine204, a migration agent205, a KDS206, and a host207. In the illustrated example, the virtual machine204is encrypted as illustrated by the cross hatching of the virtual machine in the virtual machine storage203. The virtual machine storage203is a storage system maintained by a cloud service provider201. The tenant200provides VMs to the cloud service provider201, which can store them in the machine storage203. The tenant200may encrypt the VMs or the VMs may be encrypted by other entities.

The migration agent205(or another appropriate entity, e.g. the tenant200) provides two copies of the key needed to decrypt the encrypted virtual machine204. The keys may be provided to the migration agent by the tenant200or the key distribution service (KDS)206, or some other trusted entity. One copy209of the key208is encrypted using a public key of the tenant200, while the other copy210of the key208is encrypted using a public key of the KDS206. Various alternatives may be implemented.

The key208may be used to decrypt the VM204itself. Alternatively, the key208may be used to decrypt a virtual trusted platform module (vTPM) (as will be explained in more detail below) which can be used to decrypt the VM204. However, this still falls within the scope of what is described as the VM being encrypted. In some embodiments, the VM204may be encrypted by encrypting a virtual hard drive (VHD) and/or metadata of the VM204.

If the tenant200desires to retrieve and read the encrypted VM204, the tenant can simply request the encrypted VM204back from the virtual machine storage203and use its private key to decrypt the first copy209of the key208, and then use the key208to access the VM204.

To deploy the VM204to the host207, the encrypted VM is sent to the host207. Additionally, a package212including the encrypted copies209and210of the key208is sent to the host207.

The package212is signed with both guardian signature and a signature for the key208.

The host207sends a request211and the package212to the KDS206. Alternatively, along with sending the request211to the KDS206, the host207can also send the second copy210of the key208to the KIDS. Alternatively, the KDS206may receive the second copy210from another service and can store the second copy210in anticipation of the request211.

In some embodiments, a determination may be made that the host207meets certain qualifications, and if the host207meets the qualifications, then the key208can be accessed by the KDS206and sent back to the host207. For example, embodiments may require the host207to meet certain health requirements, such as the host may need to be running certain software (or certain version of software), have certain configuration settings, have an appropriate boot record, The host can then use this key208to unlock the VM204allowing the VM204to be deployed on the host207(as illustrated by the non-crosshatched version of the VM204).

The following now illustrates yet additional detail. In flows requiring the use of a Key Distribution Service, a data structure called a. “protection descriptor” or PD is used. The main function of PDs is cryptographic wrapping of an encryption key (e.g., the key208), called the Transport Key. This wrapping ensures that access to the key is only granted to authorized entities. The KDS206does not know or care what data the transport key protects.

By way of illustration, an owner of a Virtual Machine (VM) may wish to deploy the VM to a hoster. The VM comprises two sets of data a metadata section and a set of virtual hard drives (VHDs). The VHDs are encrypted using an appropriate encryption technology, such as BitLocker available from Microsoft Corporation of Redmond Wash. The full volume encryption key (FVEK) used to decrypt the primary VHD is protected by a virtual trusted platform module (vTPM), the state of which is encrypted and stored as part of the metadata, alongside the PD. The vTPM state is itself encrypted using a key wrapped by the PD. This allows the owner to protect the VM from unwanted access to the VM.

When a host at a hoster needs to launch a VM, it extracts the PD for the vTPM from the metadata and sends it to the KDS. If the host is authorized for the vTPM key, the KDS will return to a Trusted Execution Environment (TEE), such as a secure subsystem of the host, the key with which the vTPM can be decrypted. Various different secure subsystems may be jointly or alternatively be used. In one embodiment, this subsystem could be implemented as functions running in the kernel of the host VM. In another implementation it could be running in the hypervisor. In other embodiments it could be implemented as a separate address space enforced by the hypervisor using memory mapping capabilities of the processor (sometimes referred to herein as virtual secure mode (VSM)). In other embodiments it could be implemented as a separate execution area enforced by the processor such as the TrustZone of the ARM architecture, the emerging SGX capability described by Intel Corporation of Santa Clara, Calif., or Trusted Platform Module (TPM) technology). These different implementations can offer similar functionality, such as the capability to do cryptographic operations, store credentials, validate the integrity of code or data, and protect secrets. However, they may differ in the security properties they offer.

Alternatively, in some embodiments, if the VM is accessed in the VM owner's environment, the KDS is not involved in releasing the key as the PD contains a wrapping of the transport key that allows direct access by the owner of the VM. Notably, the KDS may be maintained by the cloud service201in some embodiments, but may be maintained by a third party in other embodiments.

The entities authorized to receive the key are either the “owner” or zero or more “custodians”. The main distinction between the two is that, in some embodiments, owners get to create the original PD; also, in some embodiments, only owners can be expressed in PDs using self-signed certificates. PD design pays attention to impostor detection: care is taken to ensure that an unauthorized entity cannot pose as the owner or a custodian of a PD. Finally, provisions are made for recovery from key compromise. For example, this may be accomplished by using different lifetime keys where the owner is represented by a long term key that is highly protected and that key is used to sign for shorter term owner keys that might not be offered the same degree of protection. The short term key becomes, effectively, a custodian (i.e. an owner as custodian.)

Embodiments may also include automatic key rolling. Every time a KDS206is asked to crack open a PD, it is because some entity (such as a fabric host) is trying to read an encrypted piece of data. Such an operation is associated with an “ingress” flow, such as migrating in a VM or creating a new protected VM from some encrypted “customization data”. The KDS206responds to such requests with the requested key as well as another PD—the one used in a subsequent “egress” flow, usually meaning migration of a. VM to another host213or to offline storage (e.g. machine storage203). This setup ensures that the KDS206has to be contacted only once—on ingress, and that is when the host health is evaluated. Additional host evaluation is not done on egress if the KDS is not contacted more than once. However, embodiments may allow multiple round trips to the KDS206as there is no harm in this occurring. Host evaluation may be performed each time embodiments contact the KDS. Thus, if the KDS is not contacted on egress, there is no host evaluation on egress either.

In some embodiments, at any point in time, exactly one entity, either the owner or a custodian, is designated the “Guardian” of the PD—that is the entity that has created and signed the PD (e.g. as illustrated at103). As the PD moves from the owner to a custodian and from one custodian to another, the guardianship will change. However, the guardianship will stay the same so long as the same KDS206handles the PD. Any custodian or owner can “take over” an existing PD (become its guardian) without the current guardian being involved.

The following illustrates a mathematically rigorous illustration of one example. The section will make use of the following notation;K0, Ki, ST, TT—entities involved in the flows (tenant K0, custodian Ki, source TEE ST, target TEE TT)NEPub, NEPri—public and private encryption keys of some entity N; NEis shorthand for NEPubNSPub, NSPri—public and private signing keys of some entity N; NSis shorthand for NSPubΣMSSNNK—certificate of an entity N (owner or custodian) specifying subject name of the entity SN and its public key (signing or encryption) NK, issued by entity M by signing with its private key MSPri(the signing key could sign the corresponding public key for a self-signed certificate)TKi, TKe—symmetric transport keys (e.g. TKimight be an ingress key and TKe—an egress key); in some embodiments, transport keys are not intended to be used directly but rather as inputs into a key derivation function to produce encryption and authentication keys for various portions of the overall protected VM payload, however, different key derivation schemes may be usedε(K)[M]—message M encrypted using key K; K might be symmetric or asymmetric, depending on contextα(K)[M]—Message M authenticated using symmetric key Kσ(K)[M]—Message M signed using asymmetric key KM1∥M2—Concatenation of messages M1and M2

Further, the shorthand in flow descriptions will use variables to represent more complex messages; specifically on ingress into a host one is likely to deal with messages described below:TK-e:=KDF(TK, “e”)—derived encryption key (KDF stand for “key derivation function”. “e” is an input to the KDF used to, e.g., differentiate derivation of encryption key from derivation of, e.g., signing key. However, it should be noted that this only one example, and any other appropriate input can be used just as well.)This is a symmetric key derived from TK for purposes of encrypting the payloadNote that several such encryption keys may have to be derived from a single transport key, corresponding to different parts of a protected VM payload (e.g. encrypted vTPM state is only one such part)TK-a:=KDF(TK, “a”)—derived authentication keyThis is a symmetric key derived from TK for purposes of authenticating the payload via e.g. an HMAC functionNote that it may be necessary to derive several such authentication keys from a single transport key for purposes of authenticated encryptionA:=ε(TK-e)[P]∥α(TK-a)[P]—encrypted and authenticated payload PAuthenticated and encrypted payload (such as vTPM state) created using an encryption key and its companion authentication key corresponding to a transport key TK

While in the example illustrated here, a separate authentication key and encryption key are illustrated, is should be appreciated that in other embodiments, some cryptographic algorithms allow the same key to be used for both encryption and authentication. Thus data structures illustrated herein can be simplified in other embodiments.

The PD will include multiple wrappings of the transport keys—one by the owner (below this will be called a message of type “B”) and zero or more wrappings by custodians, which are delegated by the owner (these are messages referred to as type “C”).B:=ΣK0SSK0K0S∥ΣK0SSK0K0E∥ε(K0E)[TK]—owner wrapped keyThis is the transport key encrypted to the owner of the VM; it allows the VM to be picked up by its owner at any point in timeOnly owner's entry can contain an undelegated (self-signed) certificate over the signing keyThere are three parts to this message—the certificate over the owner's signing key, an owner-issued certificate over the owner's public encryption key and the encryption of the transport key using the public encryption key (The certificate over the owner's signing key can be self-signed, but is does not need to be self-signed and could be issued by a certificate authority that is mutually trusted by all parties.)C:=ΣKiSSKjKjS∥ΣKjSSKjKjE∥ε(KjE)[TK]—custodian wrapped keyThis is the same transport key encrypted to a custodian of the VMKiis the “current” custodian to which the transport key is encryptedKiis the “previous” custodian in the chain (i==0 for owner), which delegates custodian authority to KjThere could be zero or more custodians for a VM, so zero or more messages of type C inside a PD

Different wrappings of the transport key are combined into a set, illustrated below as a matrix, where each row contains a number of concatenated entries corresponding to an owner or a custodian. The owners are identified as such (using the letter “o”). The current guardian is flagged as such with an asterisk. In the example illustrated below, the PD is in the shape it exists in when first created by the owner for consumption by the KDS206. The combination of different wrappings of the key is a message of type D. The message generated by the owner for a single custodian would look like:

D0⁢:=⁢⁢(*o⁢⁢ΣK0⁢SSK0⁢K0⁢S⁢ΣK0⁢SSK0⁢K0⁢E⁢ɛ⁡(K0⁢E)⁡[TK]--ΣK0⁢SSK1⁢K1⁢S⁢ΣK1⁢SSK1⁢K1⁢E⁢ɛ⁡(K1⁢E)⁡[TK])
The KDS206can consume this message, make itself the guardian and add additional custodian(s), resulting in a PD that looks as follows:

integrity and authenticity of the message of type D is provided by a message of type E. This message serves as proof that the guardian did in fact have the knowledge of the key protected by message of type D and did in fact author the message.Ei=α[TK-a] (Di)∥σ(GSPri)[Di]
where GSPriis the private signing key of the entry in message D which is marked as the “guardian”.

Finally, a message of type F is the actual “PD”—the concatenation of messages D and E:F:=D∥E

In some embodiments, the PD data format also includes the header (to communicate information such as the PD blob version number), as well as provisions for cryptographic agility (choice of cipher, mode and key size for each key).

A typical flow involving a PD over some payload includes the host sending a PD Fiof the ingress key TKito the KDS, and KDS responding with two pieces of information:1) A wrapping of two transport keys (ingress and egress—TKiand TKe) for consumption by the host's TEE. This wrapping involves authenticated encryption of a concatenation of ingress and egress keys:TW—wrapping key, generated by KDSTW-e, TW-a—encryption and authentication keys derived from TWH:=ε(STEPub)[TW]∥ε(TW-e)[TKi∥TKe]∥α(TW-a)[TKi∥TKe]2) An egress key PD Febuilt around TKeto be included with the egress payload. This is not targeted at the host's TEE and can only be cracked by the owners or one of the custodians.

In some cases, the host will provide multiple ingress protection descriptors. In that case, the KDS will ensure that all protection descriptors have the exact same owner (as evidenced from the self-signed certificate at the root of the delegation chain). The resulting egress protection descriptor will be a superset of all custodians from the ingress protection descriptors and message H will look like:

As mentioned already, the KDS206has no knowledge of the type of payload that the blob of type F protects. The payload could be vTPM state, provisioning information for a VM, or something entirely different. In some embodiments, the payload is encrypted using an encryption key derived from the “master” transport key, and authenticated using the corresponding authentication key (as illustrated by messages of type A). However, other key derivation arrangements count be constructed to achieve the same or a similar overall effect.

In some embodiments, each host (e.g. hosts207and213) completes attestation before it can either host a VM or participate in migration flows. Upon successfully completing attestation with a host attestation service (HAS)214, the host is issued a health certificate. The key in the certificate, in some embodiments, is a TEE public encryption key for a trusted subsystem of the host. The host subsequently presents the health certificate to the KDS206, which responds by encrypting sensitive data (e.g. a vTPM encryption key) to the host's TEE. Note that in this case, there is no “authentication” with the health certificate and no proof of possession required by the KDS206to authenticate the host. Simply, the host is free to present whatever health certificate it wants, but if it does not have the corresponding private key, it will not be able to understand the response it receives from the KDS.

Reference is now made toFIG. 3, which illustrates a flow300for one very specific example. With reference toFIG. 3:1. The host initiates attestation by contacting the attestation service2. The attestation service issues a request. In embodiments using TPM technology, this may be a Platform Configuration Register (PCR) read request. This is shown inFIG. 3as a single request/response exchange but in practice will likely take two legs: one to establish a session and another—to satisfy the request.Two different modes of attestation are possible in embodiments using a TPM 2.0 device—one is a traditional Trusted Computing Group (TCG) quote over a service-provided nonce, and another—a “direct PCR read” over an authenticated salted session. Note that is merely one example. In other embodiments, for example, TPM 1.2 could be used, although the direct PCR read functionality would not be available.3. The device satisfies the PCR read request by supplying values of the requested PCRs together with a TCG Log4. The device puts together an “attestation request” which comprises a response to the PCR read request5. The service inspects the supplied TCG log, the PCR values, and optionally consults policy to determine whether the contents of the TCG log supplied by the host are healthy (in addition to simply matching the PCR values)6. The service produces a health certificate, which internally encodes the TEE public key and a set of measurements of host health as determined by the health attestation service214.7. This health certificate is stored by the host for subsequent use in VM launch and migration flows.

Virtual Machine Hosting and Migration

VM migration is a complex flow containing several moving pieces. It involves the source host server, the target host server and a management service that orchestrates the move. This section will focus primarily on key management related to VM migration. VM migration may occur, for example, in one of the following cases:From one host to another under the purview of the same custodianFrom owner to custodianFrom one custodian to anotherFrom custodian back to owner

In each case, the VM is assumed to traverse hostile territory (network, storage, address space). The way of protecting payload and the PD itself described above achieves safe end-to-end transport of the VM and its key material.

In each VM migration scenario, the protected VM travels in its entirety from one host to another or to/from storage. in one embodiment, the VM comprises the following constituent blocks:vTPM encryption keysvTPM stateVM metadataOne or more VHDsVM memory stateVM device state

The vTPM encryption keys (e.g. transport keys or keys derived from the transport keys) are encrypted such that only an owner or custodian of a VM could gain access to them. The transport keys are then handed down to the TEE of the host authorized to see them.

In one illustrative example, the vTPM state is encrypted using the transport key or an encryption key derived from the transport key; the encryption is further authenticated using an authentication key, also derived from the same transport key. The vTPM state and the transport keys that protect it do not leave the TEE of the host unprotected. However, it should be appreciated that this is merely one example, and that there are virtually an unlimited number of ways that keys could be used to protect confidentiality and integrity that fall within the scope of embodiments of the present invention.

Sensitive portions of the metadata could also be authenticated, using a different authentication key, also derived from the transport key (in some embodiments, as a matter of good cryptographic practice, the authentication key used to validate vTPM encrypted state should not leave the TEE, so a separate key is used). More generally, in the illustrated example, all cryptography involving these secret keys is done using either the TEE or one of TEE extensions (such an extension may be the host OS's kernel protected using hypervisor-enforced code integrity). A key hierarchy may be built with the transport key at its root and various portions of the VM state are encrypted using keys in this hierarchy.

The VHD is presumed to be encrypted using the FVEK protected by vTPM, so migration architecture makes no additional attempt to protect it. In addition, it is possible for the tenant200to download from the service provider201a VM that belongs to that tenant and run it directly, i.e. without involving the KDS206.

Finally, VM memory and device state are encrypted using keys derived from the master transport key. These can be similarly authenticated using corresponding authentication keys.

Before flows involving protected guest VMs can begin, several prerequisites are satisfied.1. (OPTIONAL—only for cases where the VMM202orchestrates the migration flow) A management service VMM, such as SCVMM available from Microsoft Corporation of Redmond Wash., is up and running, available to both source and target server.2. The key distribution service206and host attestation service214are up.3. Source host server (e.g. host207) has a guest virtual machine204up and running.4. In case of migration, target host server (e.g. host213) is TEE capable and is up and running.

A series of steps are taken before any actual migration takes place. Due to extreme time constraints that may exist in some embodiments, hot migration in particular requires that these be taken (sometimes, well) in advance of the actual migration attempt. Note that the list below purposefully lists only new provisions and omits the steps that are well known, such as the source host querying the target for availability of resources to host the new guest VM, etc.1. Both source host (e.g. host207) and target host (e.g. host213) have completed attestation (e.g. such as described above)2. Target host (e.g. host213) satisfies the ingress migration policy, otherwise it is not eligible to receive the VM state—it is useful to establish this early on, although adherence of the host to the migration policy should be checked by the KDS206at the time of actual migration3. Successful attestation enables creation of secure communication between root TEEs of source and target hosts (specifically, the migration agent trustlet running inside the root TEE) and the KDS206by virtue of the encryption of data between the entities.

Converting Existing VM to Protected VM

A VM in a tenant's environment can be converted to a “protected VM” status by associating it with a vTPM. This is usually expected to be followed by migrating the VM into a service provider's fabric. In some embodiments, to create a protected VM out of a regular VM, the following takes place:1. A PD for the VM is created and populated with the correct owner and custodian(s)2. A vTPM is created and optionally certified by issuing an endorsement certificate signed by an appropriate authority, such as for example, the tenant's own certificate authority.3. An encryption technology, such as BitLocker is enabled on the VM and its VHDs fully encrypted

Once the VM's VHDs have been fully encrypted, the VM is “safe enough” to be migrated into the service provider's fabric.

Creation of a brand new PD (as opposed to re-encryption of an existing PD) is different from other flows because in other cases involving PDs, the host authenticates the KDS206by having it successfully crack an existing PD. In case of brand new PD creation, this original PD does not yet exist.

Some embodiments may bootstrap the flow around new PD creation by starting with a “null” PD—a known ingress key wrapped by the owner and every allowed tenant. This null PD's guardian is the owner itself. The PD is available to any of the tenant's hosts tasked with VM creation. To create a PD for the new VM, the host submits the null PD to the KDS, which returns two keys (a known key for ingress, plus a new egress key) as well as a PD around the egress key. This egress key is used by the host to migrate the VM out to machine storage203(or other storage) or to the service provider.

Following creation of the egress key, the host can create and certify a vTPM. Once the vTPM is created and certified, it can be attached to the VM's metadata. The VM's OS is then restarted and the new vTPM virtual device becomes exposed to it on next boot. At that point, the OS is free to encrypt its VHD, such as by using BitLocker.

VM Launch on a Host

Before a host can migrate out a VM, it first needs to load and run it. This is accomplished by downloading a VM from offline storage. The VM's VHD is presumed encrypted (e.g. with BitLocker), and the key is sealed inside the VM's vTPM. The vTPM state is wrapped using a transport key TKi(message Ai) and associated data to help the KDS unwrap the key TKifor the host (PD Fi). Both Aiand Fiare part of VM metadata.

With reference toFIG. 4, step-by-step, the VM launch flow400is:

1. Management service initiates VM launch on a host by contacting it and providing it to a link to the VM to be launched.2. The host receives the request3. The host reaches out to offline storage to download the VM4. After downloading the VM, the host looks inside VM metadata and proceeds to construct a request to the KDS:Extracts blobs Ai(wrapped vTPM state) and Fi(PD for vTPM encryption keys wrapped by the KDS)Sends to the KDS the PD Fiaccompanied by the host's health certificate, which contains its TEE public key STEPub5. The KDS receives the request and authenticates the health certificateCertificate is not expiredSignature on the certificate is valid (if the certificate is self-signed, the signature matches that of the attestation service; otherwise the certificate comes from a certificate authority (CA) that the KDS recognizes)(OPTIONAL) Issuance policies encoded in the certificate correspond to a healthy host per the current host health policyNOTE: The KDS does not validate that the request came from the host in possession of the private key corresponding to the health certificate; its response will be encrypted using the requestor's public TEE key and thus useless to an attacker6. The KDS cracks the blob Fiand computes the response:Processes the ingress PD to extract the ingress transport key:From Fi, extracts Diand EiFrom Di, extracts Bias well as zero or more messages of type CiAmong the messages Biand Cilocates the row corresponding to the KDSBuilds a certificate chain starting with the guardian's signing certificate, to make sure all signing key certificates in the chain roll up to the owner's signing certificate (which is the only one that can be self-signed, and if it is not self-signed, verifies that it is signed by a trusted authority such as e.g. Verisign of Reston, Va.)It may be beneficial to verify all of the messages Biand Ci.From the messages Biand Cithat correspond to the KDS, decrypts TKiFrom TKi, derives TKi-aUsing TKi-a, verifies HMAC inside Eias well as the guardian's signature over the PDGenerates the egress transport key TKeGenerates PD for TKe:From TEPub(obtained from Bi) and TKe, generates BeFrom KEPuband TKe, generates Ce(this is performed for all C messages, of which there may be zero or more)From Beand Ce, generates De; marks itself as the guardian while at itAuthenticates Dewith TKe-a, resulting in message EeSigns Deusing its signing keyConcatenates Deand Eeinto egress PD FePrepares to send TKiand TKeback to the hostGenerates TW and from that—TW-e and TW-aFrom TW, TW-a, TW-e, TKi, TKeand STEPEPub, generates HThe response to the host comprises concatenated messages: H∥Fe7. The host receives the response from the KDS and passes message H to TEE, alongside Ai8. TEE handles the response from the KDS:From H, using STEPri, decrypts TWFrom TW, derives TW-e and TW-aUsing TW-e, decrypts TKiand TKeAuthenticates the encryption of TKiand TKeusing TW-a, thus authenticating the KDS's ability to crack the PD (and thus the authenticity of KDS)From TKiderives encryption and authentication keys—TKi-e and TKi-aUsing TKi-e, decrypts vTPM state from AiUsing TKi-a, authenticates vTPM state encryption9. Once vTPM state is decrypted, TEE launches vTPMTEE holds on to the value of TKeuntil such time when it has to migrate the vTPM out (i.e. compose the message Aeand send Ae∥Feto the target)10. Now the host can complete the VM launch sequence

Actual Migration

In some embodiments, for forward secrecy reasons, each migration instance requires a unique key, good for that migration alone. An important security consideration that is accommodated by this design is that little or no trust is placed in the VMM202service.

One possible mechanism for migration involving TEE state is illustrated inFIG. 5. The flow500illustrated inFIG. 5utilizes several services: “management service” (such as SCVMM), “attestation service” and “key distribution service”. Attestation Service and Key Distribution Service can be co-located since they are equally trusted (and more trusted than SCVMM).

The actual migration starts with “key agreement” followed by actual data transfer.1. The management service initiates migration by contacting the source host and providing it with the virtual machine ID that is to be migrated and the identity of the target host.2. The source host forwards the migration request to the target host along with egress transport key PD Fe; from the standpoint of the target this is an ingress PD.3. Target host receives the migration request.4. Target host requests the KDS to unwrap the transport key for it by sending the message Feit received from the source host to the KDS alongside its host health certificate.5. The KDS validates the host health certificate presented by the target host in the same fashion as was described for the source host above.6. The KDS does similar processing of state encryption keys as was done for VM launch described above, only this time it returns keys TKeand TKe′instead of TKiand TKe.7. The target host receives this response and passes it on to its TEE for unwrapping.8. TEE of the target host decrypts the key TKeusing its private TEE key TTEpri9. Target host signals to source host its readiness to proceed with actual migration.

This completes the key agreement phase and sets the stage for one last step—actual secure data transfer.10. Finally, after all the steps above have been accomplished, the two servers can begin transfer of the state—it is encrypted on the source by the TEE migration trustlet, and passed through to the target TEE.11. TEE on source and target handle vTPM state encryption and encryption using derived keys to transfer the state from one to another.

Virtual Machine Backup and Restore

The backup and restore flow, from the key management and data flow perspective, is very similar to a migration out from the source server to some data-at-rest storage facility, followed (at some later time) by a migration into the target server of same data. The data at rest thus backed up does not require any additional protections, since the flow is already designed to traverse insecure territory. The only requirement is that the VM ID and the encrypted (by KDS) migration key be stored with the VM state, since these are used by the key distribution service to unseal the use of the decryption key necessary for restore.

The requirement for forward secrecy is best served if the host backing up the VM state obtains a new encryption key, and uses that to re-encrypt the VM state prior to backing it up.

Virtual Machine Failover Inside a Cluster

Failover of a VM inside a cluster can also be considered a special case of migration; in that case all nodes in a cluster share the same VM metadata for each VM ID hosted by the cluster, and can thus obtain the migration key from the KDS ahead of any failover. Nodes in a cluster share storage where the VM data is located; since all nodes in a cluster agree on which wrapped VM key to use, key agreement is likewise easy to establish to ensure fast and secure failover. In particular, failover scenarios do not require forward secrecy, so the key with which the vTPM is encrypted does not change for as long as a VM stays within the cluster.

Virtual Machine Migration Between Datacenters

So long as the PD for a VM contains both datacenters as custodians, migration is no different than in the case of tenant-to-service provider case—the receiving custodian cracks open the PD, generates a new PD and sets itself as that PD's guardian. The transport key in the new PD is encrypted to the owner and each custodian, as before.

Referring now toFIG. 6, a method600is illustrated. The method600may be practiced in a computing environment. The method includes acts for managing encrypted datasets. The method includes obtaining a first decryption key (act602). The first decryption key is configured to be used to decrypt an encrypted dataset that has been encrypted using a first encryption mechanism. The first encryption mechanism is associated with the first decryption key that can be used to decrypt the dataset.

The method600further includes encrypting the first decryption key with a second encryption mechanism (act604). The second encryption mechanism is associated with a second decryption key used by a first entity, such that the second decryption key can be used by the first entity to decrypt the dataset by first decrypting the first key encrypted key using the second decryption key, and then using the decrypted first key to decrypt the dataset.

The method600further includes encrypting the first decryption key with a third encryption mechanism (act606). The third encryption mechanism is associated with a third decryption key used by a second entity, such that the third decryption key can be used by the second entity to decrypt the dataset by first decrypting the first key encrypted key using the third decryption key, and then using the decrypted first key to decrypt the dataset.

The method600further includes creating a package comprising at least the first decryption key encrypted with the second encryption method and the first decryption key encrypted with the third encryption method (act608).

The method further include signing the package with a guardian signature (act610) and signing the package with a signature created from the first decryption key (act612).

Further, upon reaching various computer system components, program code means in the form of computer-executable instructions or data structures can be transferred automatically from transmission computer readable media to physical computer readable storage media (or vice versa). For example, computer-executable instructions or data structures received over a network or data link can be buffered in RAM within a network interface module (e.g., a “NIC”), and then eventually transferred to computer system RAM and/or to less volatile computer readable physical storage media at a computer system. Thus, computer readable physical storage media can be included in computer system components that also (or even primarily) utilize transmission media.