Patent Description:
This specification relates to storage and processing of data in cloud environments.

Cloud computing is network-based computing in which typically large collections of servers housed in data centers or "server farms" provide computational resources and data storage as needed to remote end users. To adopt the public cloud, customers must be willing to entrust their sensitive data and applications to cloud providers. To be able to make this decision, customers have to be convinced that their data and execution is safe; that they are protected from all type of attackers and are ultimately in control of their applications and data. Trust is thus one of the core enablers of cloud adoption but also an important consideration for adoption.

Traditionally, the trusted execution environments were created to support stand-alone computers and mobile devices to provide secure execution in the isolated trusted firmware based environments. However, these strategies are inadequate for the increased flexibility of the distributed computing and, in particular, do not meet the goals and scale of the public cloud Document <CIT> discloses a system for countering malware using a plurality of Trusted Execution Environments, including a trusted manager and a plurality of trusted agents, each linked to and responsible for measuring an application to attest its integrity.

In general, one innovative aspect of the subject matter described in this specification can be embodied in methods that include the actions of launching a root enclave; accessing an enclave manifest by the root enclave, wherein the enclave manifest specifies, for each of a plurality of component enclaves, a particular role for the respective component enclave; and instantiating each of the component enclaves, each component enclave configured to perform its respective role; wherein the root enclave and component enclaves form an enclave pod. Other embodiments of this aspect include corresponding systems, apparatus, and computer programs, configured to perform the actions of the methods, encoded on computer storage devices.

A second innovative aspect of the subject matter described in this specification can be embodied in methods that include the actions of launching a first root enclave and first component enclaves according to a first manifest; launching a second root enclave and second component enclaves according to a second manifest; providing first data to the first component enclaves; providing second data that is different from the first data to the second component enclaves. Other embodiments of this aspect include corresponding systems, apparatus, and computer programs, configured to perform the actions of the methods, encoded on computer storage devices.

In other words, this second innovative aspect can be embodied in methods that include the actions of launching a first root enclave; accessing a first enclave manifest by the first root enclave, wherein the first enclave manifest specifies, for each of a plurality of first component enclaves, a particular role for the respective first component enclave; instantiating each of the first component enclaves, each first component enclave configured to perform its respective role; wherein the first root enclave and first component enclaves form a first enclave pod; providing first data to the first component enclaves; launching a second root enclave; accessing a second enclave manifest by the second root enclave, wherein the second enclave manifest specifies, for each of a plurality of second component enclaves, a particular role for the respective second component enclave; instantiating each of the second component enclaves, each second component enclave configured to perform its respective role; wherein the second root enclave and second component enclaves form a second enclave pod; and providing second data to the second component enclaves, wherein the second data is different from the first data.

Another innovative aspect of the subject matter described in this specification can be embodied in methods that include the actions of receiving, from a plurality of enclave participants, a key fragment; constructing, from the key fragments, a master key for a rendezvous enclave; launching the rendezvous enclave using the master key; and executing, in the rendezvous enclave, data from the plurality of enclave participants. Other embodiments of this aspect include corresponding systems, apparatus, and computer programs, configured to perform the actions of the methods, encoded on computer storage devices.

A scalable and dynamic network of trusted execution environments addresses multiple use cases that involve customers' secrets and a growing demand to protect customers' data and code that are executed in the cloud. The network of trusted execution environments, called enclaves, can cryptographically prove to their parent root and to third party verifiers that they provide confidentiality and integrity. Running sensitive small execution tasks in the enclaves provides an additional level of protection, resiliency and high availability, and ensures that critical decisions cannot be be reverted by untrusted code or root-privileged adversary.

For rendezvous enclaves, the binding code and data to the distributed system of enclaves connected to the enclave that the customer controls reduce the complexity that would be required for explicit key management challenges for a set of trusted execution environments.

For hybrid enclaves, multiple use cases that involve customers' secrets and the growing demand to protect customers' data and code that shared with the cloud providers are met by the introduction of a scalable set of the trusted execution environments that can be used as a rendezvous place for untrusted multiple parties. The enclave can cryptographically prove to their multiparty participants that they provide confidentiality and integrity. Running sensitive execution tasks in the enclaves will provide an additional level of protection, resiliency, and high availability, to ensure that sensitive data and code cannot be accessed or exfiltrated by untrusted code or even by cloud high privileged insiders.

The enclaves can also facilitate the production of cryptographically signed or tamper evident logs detailing all actions and transactions. These logs can be provided or used as historical records or forensics.

An enclave pod is an isolated and secure execution environment in which customers may run their sensitive application payloads. Code and data that crosses the enclave boundary is encrypted. The enclave pod facilitates a chain of trust, enabled in hardware and/or software, that provides public interfaces to enable popular use cases like Certificate Authority hosting to issue Certificate Signing responses to the callers, or performing sensitive verification decisions on behalf of authentication or authorization systems, with the goal to ensure that the operation and decision that are made in the isolated secure execution environments cannot be altered or removed by unauthorized entities. The enclave pod thus enables confidentiality and integrity protection for customers' code and data running in the cloud against unwarranted disclosure and modifications.

In some implementations, an enclave pod assures confidentiality and integrity of code and data when the code and data is at rest, before it enters and after it leaves boundaries of the enclaves. The integrity of the enclaves can be remotely verified, as can the code and data that runs within the enclaves. Moreover, workloads can be distributed geographically over multiple regions or zones to ensure high availability and performance of their services, or due to data provenance requirements.

Because a set of enclaves communicating as peers or running trusted execution environments in respective standalone mode will not satisfy the complex use cases cloud providers need to address, an enclave pod employs a hierarchical model of trust between enclaves.

An enclave pod divides computationally sensitive task to multiple related subtasks and runs them in the set of trusted enclaves. For example, an enclave pod can perform multiple tasks expected from a Certificate Authority, such as authentication of a caller, validation of a request, issuance of the nonce/timestamp, etc., as a set of different children enclaves the descending from a root enclave. Similarly, for genomic research, even more complicated sets of operations can be executed by utilizing the hierarchical nature of enclaves in an enclave pod.

The root enclave can attest and verify the trust relationship between the children to be able to delegate work to them. The root enclave can also delegate the ability to establish the trust, and attest and verify their proofs to children enclaves to ensure that the system can scale and meet the demand of various use cases.

In some implementations, the execution of the sensitive and high-top secret tasks can be delegated to geographically distributed clusters and spawn across multiple geographic locations and regions to perform customer's' sensitive tasks within sub-enclaves hosted on these locations and orchestrated by a master enclave.

The enclave pods also enable user applications to create execution units that are protected from system software and, depending upon enclave technology being used, hardware attacks. The pod enclave architecture allows an enclave to prove its identity to a local or a remote verifier, obtain/generate secrets, and seal those secrets to its own identity. The enclave pod extends basic enclave attestation and sealing infrastructure to support system-level attestation and sealing. The hierarchal enclave pod is described with reference to <FIG> and <FIG> below.

In additional implementations, the enclave pods may be cooperatively distributed in a hybrid cloud, where a master enclave pod is operating on-premise of a customer or in a private cloud, and additional enclave pods are operating in the public cloud. Hybrid cloud enclaves are described with reference to <FIG> and <FIG> below.

In still further implementations, an enclave pod may be configured to enable sharing of sensitive data from multiple parties for the performance of common tasks. Access to the sensitive data by the multiple parties, however, is precluded. Such an enclave pods is referred to as a rendezvous enclave, and is described with reference to <FIG> and <FIG> below.

<FIG> is a system diagram of an enclave pod <NUM>. The enclave pod <NUM> is a manifest-based hierarchical aggregation of enclaves that define a system. The enclave pod <NUM> seals secrets (i.e., sensitive code, sensitive binaries, sensitive data, or any other data, instructions, code or information that a party deems sensitive or does not desire to disclose publicly) to a system description as well as an individual enclave identity. In operation, a secret is only accessible to an enclave in the enclave pod <NUM> if the enclave is a part of an enclave pod <NUM> and built according to a manifest, and only if the identified enclave in the enclave pod <NUM> is designated to have access to the secret.

The enclave pod <NUM> includes a root enclave <NUM>, a manifest <NUM>, and two or more component enclaves <NUM>. Each component enclave in the pod <NUM> may have a unique role, e.g., is assigned a particular task or process to perform.

The manifest <NUM> describes the enclave pod <NUM>. The manifest <NUM> lists the various roles in the pod <NUM>, and the identities of the enclaves <NUM> corresponding to each role. In some implementations, the manifest allows only one identity per role, and a role may not be assigned to two separate component enclaves. The manifest <NUM> is signed by a system builder, and has a purpose and version, which are described in more detail below. A manifest-verification key coupled with the manifest purpose and version define the identity of the manifest.

The root enclave <NUM> is responsible for verifying component enclaves and ensuring that their roles/identities match those listed in the manifest <NUM>. In other words, the root enclave <NUM> is responsible for enforcing the manifest <NUM>.

The component enclaves <NUM> communicate with each other over mutually authenticated and encrypted channels. The component enclaves <NUM> communicate with the root enclave <NUM> over unidirectionally authenticated and encrypted channel. The root-to-component channel is unidirectionally authenticated because only the root enclave <NUM> verifies the component enclave <NUM> against the manifest <NUM>. Verification can be done by any appropriate security verification process.

The component enclaves <NUM>, in some implementations, do not verify the root enclave <NUM>. In some implementations, the component enclaves <NUM> are agnostic to the manifest <NUM> itself, and they "blindly" rely on the root enclave <NUM> to enforce the manifest <NUM>.

The enclaves <NUM> and <NUM> can be instantiated by any appropriate enclave instantiation process. Once the pod of enclaves is established, the root enclave <NUM> can communicate with a remote verifier/secret provisioner <NUM> to obtain the secrets, e.g., binaries and data, for provisioning. The remote verifier <NUM> verifies the root enclave <NUM>, which then asserts the identity of the manifest <NUM> it is enforcing. If the verifier <NUM> determines the identity of the root enclave <NUM> and that of the manifest <NUM> the root enclave <NUM> it is enforcing is acceptable, the verifier <NUM> provisions the necessary secrets. The individual components of the enclave pod <NUM> can also generate additional secrets. In some implementations, the generated and provisioned secrets can be sealed to the manifest <NUM> and the component enclave identity.

Each component enclave <NUM> in a pod <NUM> has one role. A role is the meta-functionality implemented by the enclave <NUM>, and is an arbitrary string. Each component enclave <NUM> knows its own role, and knows roles of other component enclaves <NUM> it communicates with. Component enclaves <NUM>, in some implementations, do not know, nor are they required to know, the identities of the other component enclaves <NUM>. The role-to-identity mapping is provided by the manifest <NUM>, and is enforced by the root enclave <NUM>. Thus, when a component enclave <NUM> needs to communicate with another component enclave, the determination of which enclave to communicate with is role dependent.

The manifest <NUM> has three main purposes. First, the manifest <NUM> describe the enclave pod <NUM> to the remote verifier <NUM>. Second, the manifest <NUM> allows component enclaves <NUM> to communicate with each other based on their roles rather than their identities. Finally, the manifest <NUM> allows component enclaves <NUM> to seal secrets to the pod configuration, in addition to sealing them to their own identity.

The manifest <NUM> is signed by the system builder. The manifest signature itself is stored separately from the manifest <NUM>. The manifest signature may be derived, for example, from data in the manifest, such as the version, the size, a signature header, and any other data suitable for signature generation.

<FIG> is a timing diagram <NUM> of an inter-enclave communication. The timing diagram provides one example process for managing communications so that confidentiality and integrity are protected. Particular key-pair types are referred to below, but other key types may also be used. Likewise, other secure communication techniques and processes can also be used, however.

The communication channels between the component enclaves <NUM> are bidirectionally authenticated, while those between the root enclave <NUM> and a component enclave <NUM> are unidirectionally authenticated. This is achieved through an attach handshake. Each member of the enclave pod should have the possession of an ECDH key pair (see below) certified by the root enclave <NUM> ECDSA key (see below) to be able to securely communicate between each other. Additionally, each member of the enclave POD ensures that the peer's ECDH key is also certified by the same ECDSA key to be able to share any information.

According to the process of <FIG>, the root enclave <NUM> is launched first. In this particular example, the root enclave <NUM> generates two key-pairs for itself-rDSA and rDH. The rDSA is an Elliptic Curve Digital Signature Algorithm (ECDSA) key-pair, whereas rDH is an Elliptic Curve Diffie-Hellman (ECDH) key pair. The Root enclave <NUM> then signs rDH_pub as a channel-establishment key for the role "ROOT.

The component enclaves <NUM> are then launched. As the component enclaves <NUM> are launched, they go through an attach process that enables them to get their own channel-establishment and provisioning credentials. The attachment process works as follows. Each of the component enclaves <NUM> generates two key-pairs--cDH and cIES. The cDH is an ECDH key-pair that is used for inter-enclave channel establishment, which cIES is an Elliptic Curve Integrated Encryption Scheme (ECIES) key-pair that can be used by others to provision secrets to the enclave pod <NUM>.

The component enclave <NUM> then initiates the attach handshake with the root enclave <NUM>, and the attach handshake is performed according to the timing diagram of <FIG>. As depicted in <FIG>, the component enclave <NUM> sends to the root enclave <NUM> its manifest with the role ("Role"), its own component ECDH public key (cDH-pub) and ECIES public key (cIES_Pub).

The root enclave <NUM>, after receiving this request, issuing a challenge, and sends a packet to the component enclave <NUM> that initiated attach process, The packet includes the root's own Root ECDH public key (rDSA_pub), a challenge, and combination of ECDSA public key and the challenge, signed by the root enclave <NUM> ECDSA private key (rDSA_priv).

The component enclave <NUM> obtains the package, validates the signature with the root public ECDSA key that is part of the package, and sends back the attestation statement that includes: the component enclave <NUM> role as stated in the manifest; the component enclave <NUM> component ECDH public key (cDH_pub); the component enclave <NUM> component ECIES public key (cIES_pub); the root enclave ECDSA public key received in the previous step (rDSA_pub); and the challenge received in the previous step. The package is signed with the component enclave <NUM> ECIES private key and includes an attestation statement that will include the integrity measurement that describes the state of the component enclave <NUM>.

The root enclave <NUM> obtains the package, validates the signature with the component enclave <NUM> public ECIES key, verifies the attestation statement, and verifies the role against a known component manifest. After completion, the root enclave <NUM> stores the component enclave <NUM> public ECDH key (cDH_pub) and ECIES key (cIES_pub) together with the component enclave <NUM> role in the POD manifest.

As a result of "attachment process" completion the root enclave <NUM> then issues two statements. The first is a signed statement that may include: the component enclave <NUM> role; the type of the certified key "DH"; and the component enclave <NUM> ECDH public key cDH_pub. The statement is signed with the root enclave's <NUM> own rDSA Root enclave ECDSA private key. The second signed statement may include: the component enclave <NUM> role; the type of the certified key "IES"; and component enclave's <NUM> ECIES public key cIES_pub. The statement is signed with the root enclave's <NUM> own rDSA Root enclave ECDSA private key.

The, at the end of the attach handshake, each component enclave has an ECDH key-pair and an EC-IES key-pair that is certified by the ECDSA key of the root enclave. Thereafter, any two enclaves in the pod <NUM> can establish a secure connection using these certified ECDH keys. Each of the enclaves ensures that the peer's ECDH key is certified by the same ECDSA key.

Once the pod <NUM> is established through the attach process, the remote verifier <NUM> can provision secrets into this system. To minimize the exposure of the provisioned secrets, the remote verifier <NUM> wraps such secrets with cIES_pub key of individual component enclaves <NUM>. The root enclave <NUM> is within the trust boundary of such provisioned secrets, as it is the one that certifies the cIES_pub keys. Thus, the purpose of such wrapping is only to prevent inadvertent disclosure of these secrets.

Finally, each of the component enclaves <NUM> is responsible for sealing its own secrets. Each component enclave <NUM> does this by mixing its own sealing key with a role-and-manifest-specific key provided by the root enclave <NUM>. The role-and-manifest-specific key is generated by mixing root enclave's <NUM> sealing key with the role and the manifest identity.

In some implementations, the enclave pod <NUM> can produce secure logs based on all transactions or actions performed in the enclave. The logs are cryptographically signed with the key of the enclave.

Whilst the description of <FIG> above specifically concerns the use of elliptic curve cryptography by the use of the ECDH key pair and the ECDSA key, this is not essential to the invention, and other types of cryptographic keys may be used.

With a private cloud, the customer controls their on-premises environment and has existing workloads running within the on-premises environment. However, the public cloud may be more scalable and performant. Thus, hybrid cloud enclaves enable the customer to take advantage of the scalability of the public cloud while still offering the protection of on-premises environment for certain data and binaries.

When a customer is running hybrid cloud deployment, the customer may desire to protect their sensitive code or data from the public cloud provider. To accomplish this goal, the hybrid cloud allows customer to bind their secrets to the components they have full control over in their on-premises environment. This mechanism enables the customer to leverage the power of public cloud for other, less sensitive, code and data while keeping control of high-value data and code by running it locally. Likewise, the cloud provider may want to offload some sensitive tasks to run on the customer's data center or other cloud to comply with the regulations or offer customers more flexibilities, while still maintaining the trusted relationship with outsourced functionality by virtue of linking it to the customer's workloads that continue to reside in the public cloud.

<FIG> is a system diagram of multiple enclaves in a hybrid could. A hybrid cloud enclave <NUM> binds most sensitive code (relative to less sensitive code) and data to the isolated secure execution environment in an on-premise enclave pod <NUM> that is operated under the customer's control. The on-premise enclave pod <NUM> leverages the system of enclaves to enable strict isolation of the sensitive code execution or access to the sensitive data as specified by a customer, and less sensitive code and are processed in one or more cloud enclave pods <NUM>. The pods <NUM> (root enclave <NUM>, manifest <NUM>, and component enclaves <NUM>) and <NUM> (root enclave <NUM>, manifest <NUM>, and component enclaves <NUM>) are similar in the hierarchical architecture described with reference to the pod <NUM> of <FIG>.

The binding to the on-premise enclave pod <NUM> automatically enforces encryption whenever code or data leave the security boundary of the customer's enclaves. The on-premise enclave pod <NUM> also enables a customer to attest the enclaves, and overall gives the customer assurance that their sensitive code and data are processed on their terms, thus addressing the main risks that customers see in using a multi-tenant public cloud for their sensitive workloads.

<FIG> is a flow diagram of an example process for generating multiple enclaves in a hybrid cloud. The process <NUM> creates a master enclave in on-premise/private cloud environment (<NUM>). For example, a customer creates their master enclave, in empty form (without code or data) using a provided toolkit in on-premises/private cloud environment under their full control. The customer signs the master enclave its own private key and adds its public key to the package of the master enclave. The customer then activates an enclave management service from a public cloud provider, selects the task to provision for a cloud enclave pod, and specifies how the enclaves in the cloud are to be managed.

The process <NUM> uploads the empty enclave as a master enclave (<NUM>). The master enclave is uploaded to the cloud service. The master enclave, once instantiated, enables other cloud enclaves to validate the proof of possession of a key that a master enclave is signed with using a public key that is shared as part of the same transaction.

The process <NUM> validate public key for master enclave (<NUM>). The public key, which was shared offline (e.g., through another secure channel) with the cloud provider, is validated as well to ensure that a verifiable customer account is used to create the enclave pod.

The process <NUM> deploys code and data into master enclave (<NUM>). For example, the customer deploys the code and data into their master enclave on premise and validates functionality.

The process <NUM> provisions code and data into other enclaves from master enclave (<NUM>). For example, as the customer's enclave joins the system of enclaves in the cloud, code and data are deployed via the mater enclave to all other enclaves in the system as specified by the customer (regions, hierarchical relationship, availability, etc.) As each new enclave in a pod is instantiated, it has to attest its' origin and state to existing members of the pod based on the enclave's attestation flows. A customer can then interact with its system of enclaves to execute sensitive code or process sensitive data without separately managing encryption and complicated key management. The customer has assurance that their sensitive code and data are protected in the public cloud while in use, and confidentiality and integrity are enforced while at rest.

A customer can periodically run attestation of enclaves used in their system and the code that executes in their enclaves to validate the integrity of the code and data that run in the enclaves.

A rendezvous enclave enables the processing of data and code by multiple untrusted parties while ensuring that each party does not achieve access to data or code of other parties. This enables collaborative processing among parties with assurances of data integrity, security and confidentiality.

The rendezvous enclave provides customers with the option to run their private application payloads in the isolated secure execution environments where untrusted parties can safely share their data and perform their common tasks that benefit all participants. The rendezvous enclave also supports and maintains a chain of trust, rooted in hardware and/or software, and offers cryptographic attestation, verifiable by multiple parties, to ensure confidentiality and integrity of the tasks performed in the enclave.

In some implementations, a rendezvous enclave allows for the splitting and reconstruction of cryptographic keys in the enclave environment. The keys are escrowed by the set of untrusted parties to ensure that high impact authorization decision is performed with all parties' supervision and agreement, when offering their parts of the "key" to authorize the operation. This scheme is applicable to the cloud providers when other parties then cloud providers themselves are operating their facilities, and becoming the facility and data guardian entity. Similar approaches extend to not only cryptographic operations, but for the arbitrary type of operations, where untrusted multi-party can enforce an access control that required consent from all or subset of the parties. The rendezvous enclave thus ensures private operations and decisions that are made in the isolated secure execution environment cannot be altered, and personal sensitive data cannot be accessed or exfiltrated from the enclave.

<FIG> is a system diagram of a rendezvous enclave <NUM> for multiple parties. The enclave <NUM> (root enclave <NUM>, manifest <NUM>, and component enclaves <NUM>) is similar in the hierarchical architecture described with reference to the pod <NUM> of <FIG>. However, the rendezvous enclave <NUM> enables each component enclave <NUM> to be associated with a particular party of multiple parties that have agreed to form the rendezvous enclave <NUM>. Additionally, each party may also provide respective data <NUM> in encrypted form for processing in the enclave <NUM>. The rendezvous enclave <NUM> implements a straightforward key management and key handling mechanism to establish trust between the enclaves <NUM> to ensure security is not compromised. The rendezvous enclave <NUM> also provides for each party to independently remotely verify an attestation statement that depicts the state of the enclaves where their sensitive tasks being performed. The rendezvous enclave also, in some implementations, allows for quorum voting by use of the split keys to decide when to launch enclaves' execution and when to retire their execution, or perform other necessary enclave management tasks using the split key technique.

<FIG> is a flow diagram of an example process for generating a rendezvous enclave. The process <NUM> associates N customers with a rendezvous enclave (<NUM>). For example, assume multiple organizations agree to provide their confidential input to the cloud provider without revealing their data with the goal to compute their common algorithm on the combined data inputs and share the results. Each customer out of N group sign up for enclave management service from a public cloud provider, selects the task to provision cloud enclave pod (e.g., component enclave pods <NUM> for that customer), and specifies how the enclaves are to be managed, the configuration properties, etc..

As a result of this operation, every customer triggers creation of a respective manifest that describes the configuration of the enclaves pod they would like to deploy, a project identifier and storage bucket(s) identifier, and a set of encryption and signing keys associated with this enclave pod. The manifest is signed with the customer's private key that they shared offline with the cloud key management service.

The process <NUM> wraps each customer's key used to encrypt data by a customer supplied encryption key (<NUM>). Data in the cloud storage bucket is encrypted with the cloud KMS generated key that is wrapped by the customer supplied encryption key (KEK) to ensure that data cannot be accessed unencrypted without the owner's consent.

The process <NUM> verifies the public key shared with cloud provider (<NUM>). The public key that was shared offline with cloud provider is validated as well to ensure that a verifiable customer account is used to create enclave pod.

The process <NUM> deploys binaries (<NUM>). For example, customers specify the binaries they need to deploy into the enclave and specify the locations of the binaries.

The process <NUM> splits enclave signing key among the N customers to enable group voting to control enclave (<NUM>). For shared responsibility enclaves, a cloud key management system generates the enclave signing key and, using a key splitting technique, provides all N participants a respective portion or fragment of the entire key. The key splitting technique enables the participants to vote on the access to the rendezvous enclave pod <NUM> and allow them to control as a group the enclave use of their respective data from their storage bucket for the algorithm computation.

A cloud key management service split of a signing enclave private key into a N-of-N quorum fragments. For each customer, its respective nth fragment of the key is associated with a project ID of the customer and wrapped by the customer's supplied encryption key KEK. When customers agreed to start the process in the rendezvous enclave, they will use their nth fragment to sign the enclave manifest <NUM> and share the partially signed message with the cloud enclave management system. Since nothing in the resulting signature is revealing who is "voted", only after all N participants signed their challenges with their nth fragment of the key, the master key associated with the binary and enclave can be reconstructed and enclave can be launched. Thus, by signing the enclave manifest, all participants are agreeing to use their storage buckets as input into enclave computation. The same "voting" process can be applied when the enclave has to be retired or available for attestation, or results of computation made available for everyone. Only when N out of N customers "voted" to launch enclave with their portions of the keys, the enclave can be launched by the cloud and access the data shared by N customers.

In some implementations, every customer from the group can periodically run attestation of enclaves used in the collaborative matter after "voting" agree to enable this capability on the enclave pod <NUM>. As a result, all of the customer can obtain an integrity claim with the enclave measurements to be able to validate the integrity of the code and data that runs in the enclaves.

In some implementations, this capability may work based on a majority voting algorithm of the participants. In other implementations, a split and delegate model can be used when it is sufficient to use only M out of N split key fragments to regenerate the entire key to use it for the authorization of the main critical operation with enclaves, such as launching, retiring, and attestation.

Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on computer storage medium for execution by, or to control the operation of, data processing apparatus.

In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user's user device in response to requests received from the web browser.

Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front-end component, e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components.

The computing system can include users and servers. A user and server are generally remote from each other and typically interact through a communication network. The relationship of user and server arises by virtue of computer programs running on the respective computers and having a user-server relationship to each other. In some embodiments, a server transmits data (e.g., an HTML page) to a user device (e.g., for purposes of displaying data to and receiving user input from a user interacting with the user device). Data generated at the user device (e.g., a result of the user interaction) can be received from the user device at the server.

Claim 1:
A method for creating a secure execution environment, comprising:
receiving a request to create a master enclave, the master enclave being associated with a private key of a customer and a public key of the customer;
creating the master enclave in an on premises or private cloud environment;
instantiating the master enclave, as part of a public cloud service, such that it includes code and data provided by the customer;
distributing the code and data to one or more member enclaves according to a master manifest associated with the master enclave; and
operating the one or more member enclaves such that a first member enclave of the one or more member enclaves uses the public key to validate that a transaction is associated with the master enclave.