Patent Description:
Hardware privilege levels may be used by a processor to limit memory access by applications running on a device. An operating system runs at a higher privilege level and can access all memory of the device and define memory ranges for other applications. The applications, running a lower privilege level, are restricted to accessing memory within the range defined by the operating system and are not able to access the memory of other applications or the operating system. However, an application has no protection from a malicious or compromised operating system.

A TEE is enabled by processor protections that guarantee that code and data loaded inside the TEE is protected from access by code executing outside of the TEE. Thus, the TEE provides an isolated execution environment that prevents, at the hardware level, access of the data and code contained in the TEE from malicious software, including the operating system.

<NPL> discloses an architecture and implementation of Function-as-a-Service (FaaS) to provide strong security and accountability guarantees backed by Intel SGX with introduction of a new key distribution enclave and a transitive attestation protocol.

<CIT> discloses forming of an enclave pool including a plurality of enclaves. Each enclave may have a private enclave key and a public enclave key. A shared enclave pool key may be generated from or otherwise based on the public enclave key of each enclave of the enclave pool.

The invention is defined by the attached independent claims. Different embodiments are described by the dependent claims. Example methods and systems are directed to reducing latency in providing TEEs. In the most general sense, a TEE is any trusted execution environment, regardless of how that trust is obtained. However, as used herein, TEEs are provided by executing code within a portion of memory protected from access by processes outside of the TEE, even if those processes are running at an elevated privilege level. Example TEEs include enclaves created by Intel® Software Guard Extensions (SGX) and trust domains created by Intel® Trust Domain Extensions (TDX).

TEEs may be used to enable the secure handling of confidential information by protecting the confidential information from all software outside of the TEE. TEEs may also be used for modular programming, wherein each module contains everything necessary for its own functionality without being exposed to vulnerabilities caused by other modules. For example, a code injection attack that is successful against one TEE cannot impact the code of another TEE.

Total memory encryption (TME) protects data in memory from being accessed by bypassing a processor. The system generates an encryption key within the processor on boot and never stores the key outside of the processor. The TME encryption key is an ephemeral key because it does not persist across reboots and is never stored outside of the processor. All data written by the processor to memory is encrypted using the encryption key and decrypted when it is read back from memory. Thus, a hardware-based attack that attempts to read data directly from memory without processor intermediation will fail.

Multi-key TME (MKTME) extends TME to make use of multiple keys. Individual memory pages may be encrypted using the ephemeral key of TME or using software-provided keys. This may provide increased security over TME with respect to software-based attacks, since an attacker will need to identify the particular key being used by targeted software rather than having the processor automatically decrypt any memory that the attack software has gained access to.

Initializing a TEE requires multiple steps before the TEE can begin executing, causing latency in applications that repeatedly create and destroy TEEs. Besides workload-specific initialization, workload-independent initialization is performed, such as adding memory to the enclave. In function-as-a-service (FaaS) environments, a large portion of the TEE is workload-independent, and thus can be performed prior to receiving the workload.

FaaS platforms provide cloud computing services that execute application logic but do not store data. By contrast with platform-as-a-service (PaaS) hosting providers, FaaS platforms do not have a constantly running server process. As a result, an initial request to a FaaS platform may take longer to handle than an equivalent request to a PaaS host, but at the advantage of reduced idle time and higher scalability. Reducing the latency of handling the initial request, as described herein, improves the attractiveness of the FaaS solution.

Certain steps performed during enclave initialization are identical for certain classes of workloads. For example, each enclave in a class may use heap memory. Thus, the common parts of the enclave initialization sequence (e.g., adding heap memory) may be performed before the enclave is requested. When an enclave is requested for a workload in the class and the parts to specialize the enclave for its particular purpose are known, the final steps to initialize the enclave are performed. This reduces latency over performing all initialization steps in response to a request for an enclave.

A TEE may be initialized ahead of time for a particular workload. The TEE is considered to be a template TEE. When a TEE for the workload is requested, the template TEE is forked, and the new copy is provided as the requested TEE. Since forking an existing TEE is faster than creating a new TEE from scratch, latency is reduced.

A TEE may be initialized ahead of time for a particular workload and marked read-only. The TEE is considered to be a template TEE. When a TEE for the workload is requested, a new TEE is created with read-only access to the template TEE. Multiple TEEs may have access to the template TEE safely, so long as the template TEE is read-only. Since creation of the new TEE with access to the template TEE is faster than creating a new TEE with all of the code and data of the template TEE from scratch, latency is reduced.

In some example embodiments, as described herein, a FaaS image is used to create a TEE using an ephemeral key. When a TEE for the FaaS is requested, the ephemeral key is assigned an access-controlled key identifier, allowing for rapid provisioning of the TEE in response.

By comparison with existing methods of initializing enclaves, the methods and systems discussed herein reduce latency. The reduction of latency may allow additional functionality to be protected in TEEs, for finer-grained TEEs to be used, or both, increasing system security. When these effects are considered in the aggregate, one or more of the methodologies described herein may obviate a need for certain efforts or resources that otherwise would be involved in initializing TEEs. Computing resources used by one or more machines, databases, or networks may similarly be reduced. Examples of such computing resources include processor cycles, network traffic, memory usage, data storage capacity, power consumption, and cooling capacity.

<FIG> is a network diagram illustrating a network environment <NUM> suitable for servers providing functions as a service using TEEs, according to some example embodiments. The network environment <NUM> includes FaaS servers 110A and 110B, client devices 120A and 120B, and a network <NUM>. The FaaS servers 110A-110B provide functions to client devices 120A-120B via the network <NUM>. The client devices 120A and 120B may be devices of different tenants, such that each tenant wants to ensure that their tenant-specific data and code is not accessible by other tenants. Accordingly, the FaaS servers 110A-110B may use an enclave for each FaaS provided.

To reduce latency in providing functions, systems and methods described herein to reduce latency in TEE creation may be used. For example, a TEE for a function may be partially or wholly created by a FaaS server <NUM> prior to the function being requested by a client device <NUM>.

The FaaS servers 110A-110B and the client devices 120A and 120B may each be implemented in a computer system, in whole or in part, as described below with respect to <FIG>. The FaaS servers 110A and 110B may be referred to collectively as FaaS servers <NUM> or generically as a FaaS server <NUM>. The client devices 120A and 120B may be referred to collectively as client devices <NUM> or generically as a client device <NUM>.

Any of the machines, databases, or devices shown in <FIG> may be implemented in a general-purpose computer modified (e.g., configured or programmed) by software to be a special-purpose computer to perform the functions described herein for that machine, database, or device. For example, a computer system able to implement any one or more of the methodologies described herein is discussed below with respect to <FIG>. As used herein, a "database" is a data storage resource and may store data structured as a text file, a table, a spreadsheet, a relational database (e.g., an object-relational database), a triple store, a hierarchical data store, a document-oriented NoSQL database, a file store, or any suitable combination thereof. The database may be an in-memory database. Moreover, any two or more of the machines, databases, or devices illustrated in <FIG> may be combined into a single machine, database, or device, and the functions described herein for any single machine, database, or device may be subdivided among multiple machines, databases, or devices.

The FaaS servers <NUM> and the client devices <NUM> are connected by the network <NUM>. The network <NUM> may be any network that enables communication between or among machines, databases, and devices. Accordingly, the network <NUM> may be a wired network, a wireless network (e.g., a mobile or cellular network), or any suitable combination thereof. The network <NUM> may include one or more portions that constitute a private network, a public network (e.g., the Internet), or any suitable combination thereof.

<FIG> is a block diagram of the FaaS server 110A, according to some example embodiments, suitable for reducing latency of TEEs according to some example embodiments. The FaaS server 110A is shown as including a communication module <NUM>, an untrusted component <NUM> of an application, a trusted component <NUM> of an application, a trust domain module <NUM>, a quoting enclave <NUM>, a shared memory <NUM>, and a private memory <NUM>, all configured to communicate with each other (e.g., via a bus, shared memory, or a switch). Any one or more of the modules described herein may be implemented using hardware (e.g., a processor of a machine). For example, any module described herein may be implemented by a processor configured to perform the operations described herein for that module. Moreover, any two or more of these modules may be combined into a single module, and the functions described herein for a single module may be subdivided among multiple modules. Furthermore, according to various example embodiments, modules described herein as being implemented within a single machine, database, or device may be distributed across multiple machines, databases, or devices.

The communication module <NUM> receives data sent to the FaaS server 110A and transmits data from the FaaS server 110A. For example, the communication module <NUM> may receive, from the client device 120A, a request to perform a function. After the function is performed, the results of the function are provided by the communication module <NUM> to the client device 120A. Communications sent and received by the communication module <NUM> may be intermediated by the network <NUM>.

The untrusted component <NUM> executes outside of an enclave. Thus, if the operating system or other untrusted components are compromised, the untrusted component <NUM> is vulnerable to attack. The trusted component <NUM> executes within an enclave. Thus, even if the operating system or the untrusted component <NUM> is compromised, the data and code of the trusted component <NUM> remains secure.

The trust domain module <NUM> creates and protects enclaves and is responsible for transitioning execution between the untrusted component <NUM> and the trusted component <NUM>. Signed code may be provided to the trust domain module <NUM>, which verifies that the code has not been modified since it was signed. The signed code is loaded into a portion of physical memory that is marked as being part of an enclave. Thereafter, hardware protections prevent access, modification, execution, or any suitable combination thereof of the enclave memory by untrusted software. The code may be encrypted using a key only available to the trust domain module <NUM>.

Once the trusted component <NUM> is initialized, the untrusted component <NUM> can invoke functions of the trusted component <NUM> using special processor instructions of the trust domain module <NUM> that transition from an untrusted mode to a trusted mode. The trusted component <NUM> performs parameter verification, performs the requested function if the parameters are valid, and returns control to the untrusted component <NUM> via the trust domain module <NUM>.

The trust domain module <NUM> may be implemented as one or more components of an Intel® hardware processor providing Intel® SGX, Intel® TDX, AMD® secure encrypted virtualization (SEV), ARM® TrustZone, or any suitable combination thereof. In Intel® SGX, attestation is the mechanism by which a third entity establishes that a software entity is running on an Intel® SGX enabled platform protected within an enclave prior to provisioning that software with secrets and protected data. Attestation relies on the ability of a platform to produce a credential that accurately reflects the signature of an enclave, which includes information on the enclave's security properties. The Intel® SGX architecture provides the mechanisms to support two forms of attestation. There is a mechanism for creating a basic assertion between enclaves running on the same platform, which supports local, or intra-platform attestation, and then another mechanism that provides the foundation for attestation between an enclave and a remote third party.

The quoting enclave <NUM> generates attestation for enclaves (e.g., the trusted component <NUM>). The attestation is an evidence structure that uniquely identifies the attested enclave and the host (e.g., the FaaS server 110A), using asymmetric encryption and supported by built-in processor functions. The attestation may be provided to a client device <NUM> via the communication module <NUM>, allowing the client device <NUM> to confirm that the trusted component <NUM> has not been compromised. For example, the processor may be manufactured with a built-in private key using hardware that prevents access of the key. Using this private key, the attestation structure can be signed by the processor and, using a corresponding public key published by the hardware manufacturer, the signature can be confirmed by the client device <NUM>. This allows the client device <NUM> to ensure that the enclave on the remote device (e.g., the FaaS server 110A) has actually been created without being tampered with.

Both the untrusted component <NUM> and the trusted component <NUM> can access and modify the shared memory <NUM>, but only the trusted component <NUM> can access and modify the private memory <NUM>. Though only one untrusted component <NUM>, one trusted component <NUM>, and one private memory <NUM> are shown in <FIG>, each application may have multiple trusted components <NUM>, each with a corresponding private memory <NUM>, and multiple untrusted components <NUM>, without access to any of the private memories <NUM>. Additionally, multiple applications may be run with separate memory spaces, and thus separate shared memories <NUM>. In this context "shared" refers to the memory being accessible by all software and hardware with access to the memory space (e.g., an application and its operating system) not necessarily being accessible by all applications running on the system.

<FIG> is a block diagram <NUM> of prior art ring-based memory protection. The block diagram <NUM> includes applications <NUM> and <NUM> and an operating system <NUM>. The operating system <NUM> executes processor commands in ring <NUM> (Intel® and AMD® processors), exception level <NUM> (ARM® processors), or an equivalent privilege level. The applications <NUM>-<NUM> execute processor commands in ring <NUM> (Intel® and AMD® processors), exception level <NUM> (ARM® processors), or an equivalent privilege level.

The hardware processor prevents code that is executing at the lower privilege level from accessing memory outside of the memory range defined by the operating system. Thus, the code of the application <NUM> cannot directly access the memory of the operating system <NUM> or the application <NUM> (as shown by the "X" in <FIG>). The operating system <NUM> exposes some functionality to the applications <NUM>-<NUM> by predefining specific access points (e.g., by call gates, SYSENTER/SYSEXIT instructions on Intel® processors, SYSCALL/SYSRET instructions on AMD® processors, or any suitable combination or equivalent thereof).

Since the operating system <NUM> has access to all of memory, the applications <NUM> and <NUM> have no protection from a malicious operating system. For example, a competitor may modify the operating system before running the application <NUM> in order to gain access to the code and data of the application <NUM>, permitting reverse engineering.

Additionally, if an application is able to exploit a vulnerability in the operating system <NUM> and promote itself to the privilege level of the operating system, the application would be able to access all of memory. For example, the application <NUM>, which is not normally able to access the memory of the application <NUM> (as shown by the X between the applications <NUM> and <NUM> in <FIG>), would be able to access the memory of the application <NUM> after promoting itself to ring <NUM> or exception level <NUM>. Thus, if the user is tricked into running a malicious program (e.g., the application <NUM>), private data of the user or an application provider may be accessed directly from memory (e.g., a banking password used by the application <NUM>).

<FIG> is a block diagram <NUM> of enclave-based memory protection, suitable for reducing latency of TEEs according to some example embodiments. The block diagram <NUM> includes an application <NUM>, an enclave <NUM>, and an operating system <NUM>. The operating system <NUM> executes processor commands in ring <NUM> (Intel® and AMD® processors), exception level <NUM> (ARM® processors), or an equivalent privilege level. The application <NUM> and the enclave <NUM> execute processor commands in ring <NUM> (Intel® and AMD® processors), exception level <NUM> (ARM® processors), or an equivalent privilege level.

The operating system <NUM> allocates the memory of the enclave <NUM> and indicates to the processor the code and data to be loaded into the enclave <NUM>. However, once instantiated, the operating system <NUM> does not have access to the memory of the enclave <NUM>. Thus, even if the operating system <NUM> is malicious or compromised, the code and data of the enclave <NUM> remains secure.

The enclave <NUM> may provide functionality to the application <NUM>. The operating system <NUM> may control whether the application <NUM> is permitted to invoke functions of the enclave <NUM> (e.g., by using an ECALL instruction). Thus, a malicious application may be able to gain the ability to invoke functions of the enclave <NUM> by compromising the operating system <NUM>. Nonetheless, the hardware processor will prevent the malicious application from directly accessing the memory or code of the enclave <NUM>. Thus, while the code in the enclave <NUM> cannot assume that functions are being invoked correctly or by a non-attacker, the code in the enclave <NUM> has full control over parameter checking and other internal security measures and is only subject to its internal security vulnerabilities.

<FIG> is a block diagram of a database schema, according to some example embodiments, suitable for use in reducing latency of TEEs. The database schema <NUM> includes an enclave table <NUM>. The enclave table <NUM> includes rows 530A, 530B, 530C, and 530D of a format <NUM>.

The format <NUM> of the enclave table <NUM> includes an enclave identifier field, a status field, a read-only field, and a template identifier field. Each of the rows 530A-530D stores data for a single enclave. The enclave identifier is a unique identifier for the enclave. For example, when an enclave is created, the trust domain module <NUM> may assign the next unused identifier to the created enclave. The status field indicates the status of the enclave, such as initializing (created but not yet ready for use), initialized (ready for use but not yet in use), and allocated (in use). The read-only field indicates whether the enclave is read-only or not. The template identifier field contains the enclave identifier of another enclave that this enclave has read-only access to.

Thus, in the example of <FIG>, four enclaves are shown in the enclave table <NUM>. One of the enclaves is initializing, two are initialized, and one is allocated. Enclave <NUM>, of the row 530A, is a read-only enclave and is used as a template for enclave <NUM>, of the row 530B. Thus, the processor prevents enclave <NUM> from being executed but enclave <NUM> has access to the data and code of enclave <NUM>. Additional enclaves may be created that also use enclave <NUM> as a template, allowing multiple enclaves to access the data and code of enclave <NUM> without multiplying the amount of memory consumed. Enclaves <NUM>-<NUM>, of the rows 530B-530D, are not read-only, and thus can be executed.

<FIG> is a block diagram <NUM> of a sequence of operations performed by the trust domain module <NUM> in building a TEE, according to some example embodiments. As shown in <FIG>, the sequence of operations includes ECREATE, EADD / EEXTEND, EINIT, EENTER, and FUNCTION START. The ECREATE operation creates the enclave. The EADD operation adds initial heap memory to the enclave. Additional memory may be added using the EEXTEND operation. The EINIT operation initializes the TEE for execution. Thereafter, the untrusted component <NUM> transfers execution to the TEE by requesting the trust domain module <NUM> to perform the EENTER operation. The trusted FUNCTION of the TEE is performed, in response to the FUNCTION START CALL, by executing the code within the TEE.

As shown in <FIG>, the operations may be divided in at least two ways. One division shows that the ECREATE, EADD / EEXTEND, and EINIT operations are performed by the host application (e.g., the untrusted component <NUM>) and the EENTER operation transfers control to the TEE, which performs the FUNCTION. Another division shows that the creation of the TEE and the allocation of heap memory for the TEE can be performed regardless of the particular code and data to be added to the TEE ("workload-independent operations") while the initialization of the TEE and subsequent invocation of a TEE function depend on the particular code and data that are loaded ("workload-dependent operations").

A pool of TEEs may be pre-initialized by performing the workload-independent operations before the TEEs are requested. As used herein, a pre-initialized TEE is a TEE for which at least one operation was initiated before the TEE is requested by an application. For example, the TEE may be created by the ECREATE operation before the TEE is requested. In response to receiving a request for a TEE, the workload-dependent operations are performed. By comparison with solutions that do not perform the workload-independent operations before the request is received, the latency is reduced. In some example embodiments, operations for pre-initialization of a TEE are performed in parallel with receiving the request for the TEE. For example, the ECREATE operation for the TEE may begin and, before the ECREATE operation completes, the request for the TEE is received. Thus, the pre-initialization is not defined by completing the workload-independent operations a particular amount of time before the request for the TEE is received, but by beginning the workload-independent operations before the request for the TEE is received.

For a FaaS environment, each function may share a common runtime environment that is workload independent and is initialized before the workload-dependent operations are performed. The startup time of a FaaS function is an important metric in FaaS services, since shorter startup times allow for higher elasticity of the service.

<FIG> is a flowchart illustrating operations of a method <NUM> suitable for initializing and providing access to TEEs, according to some example embodiments. The method <NUM> includes operations <NUM>, <NUM>, <NUM>, and <NUM>. By way of example and not limitation, the method <NUM> may be performed by the FaaS server 110A of <FIG>, using the modules, databases, and structures shown in <FIG>.

In operation <NUM>, the trust domain module <NUM> pre-initializes a pool of enclaves. For example, the operations of creating the enclave and allocating heap memory to the enclave may be performed for each enclave in the pool of enclaves. In some example embodiments, the pool of enclaves includes <NUM>-<NUM> enclaves, <NUM> enclaves, <NUM> enclaves, or <NUM> enclaves.

In various example embodiments, the enclaves in the pool of enclaves are partially pre-initialized or fully pre-initialized. A fully pre-initialized enclave has at least one of the workload-specific operations performed before the enclave is requested. A partially pre-initialized enclave has only the workload-independent operations performed before the enclave is requested. Pre-initialized enclaves reduce the time-to-result for any enclave, but they are particularly valuable for short-lived ephemeral enclaves (e.g., FaaS workloads) where the initialization overhead dominates the overall execution time.

The pre-initialized enclaves may be created by forking or copying a template enclave. The template enclave is created first, with the desired state of the pre-initialized enclaves. Then the template enclave is forked or copied for each of the pre-initialized enclaves in the pool. In some example embodiments, the template enclave is itself part of the pool. In other example embodiments, the template enclave is read-only, not executable, and retained as a template for later use. The template enclave may include memory contents and layout for a FaaS.

Each enclave in the pool of enclaves may have its memory encrypted using a key stored in the processor. The key may be an ephemeral key (e.g., the TME ephemeral key) or a key with a key identifier that is accessible outside of the processor. The trust domain module <NUM> or an MKTME module may generate the key and assign it to the enclave. Thus, the key itself is never exposed outside of the processor. The enclave is assigned a portion of physical memory. Memory access requests originating from the physical memory of the enclave are associated with the key identifier, and thus the key, of the enclave. The processor does not apply the key of the enclave to memory accesses that originate from outside the physical memory of the enclave. As a result, memory accesses by untrusted applications or components (e.g., the untrusted component <NUM>) can receive only the encrypted data or code of the enclave.

In some example embodiments, a different ephemeral key without a key identifier (e.g., an MKTME key) is used to encrypt each enclave in the pool of enclaves. Thereafter, when an enclave for the FaaS is requested, the ephemeral key is assigned an access-controlled key identifier by the trust domain module <NUM>, allowing for rapid provisioning of the enclave in response.

The trust domain module <NUM>, in operation <NUM>, receives a request for an enclave. For example, the untrusted component <NUM> of an application may provide data identifying an enclave to the trust domain module <NUM> as part of the request. The data identifying the enclave may include a pointer to an address in the shared memory <NUM>, accessible by the untrusted component <NUM>.

The request may include a pre-computed hash value for the enclave and indicate a portion of the shared memory <NUM> (e.g., a portion identified by an address and a size included in the request) that contains the code and data for the enclave. The trust domain module <NUM> may perform a hash function on a binary memory state (e.g., the portion of the shared memory <NUM> indicated in the request) to confirm that the hash value provided in the request matches the computed hash value. If the hash values match, the trust domain module <NUM> has confirmed that the indicated memory actually contains the code and data of the requested enclave and the method <NUM> may proceed. If the hash values don't match, the trust domain module <NUM> may return an error value, preventing the modified memory from being loaded into the enclave.

In some example embodiments, the request includes an identifier of a template enclave. The trust domain module <NUM> creates the requested enclave with read-only permissions to access the template enclave. This allows the requested enclave to read data of the template enclave and execute functions of the template enclave without modifying the data or code of the template enclave. Accordingly, multiple enclaves may access the template enclave without conflict and the data and code of the template enclave is stored only once (rather than once for each of the multiple enclaves). As a result, less memory is copied during creation of the accessing enclaves, reducing latency.

In operation <NUM>, in response to the received request, the trust domain module <NUM> selects an enclave from the pre-initialized pool of enclaves. The trust domain module <NUM> may modify the selected enclave by performing, based on the data identifying the enclave received with the request, additional operations on the selected enclave, such as the workload-specific operations shown in <FIG>. The additional operations may include copying data or code from the address in the shared memory <NUM> indicated in the request to the private memory <NUM> allocated to the enclave.

In some example embodiments, the additional operations include re-encrypting the physical memory assigned to the enclave with a new key. For example, the pre-initialization step may have encrypted the physical memory of the enclave using an ephemeral key and the enclave may be re-encrypted using a unique key for the enclave, with a corresponding unique key identifier. In systems in which the key identifiers are a finite resource (e.g., with a fixed number of key identifiers available), using the ephemeral key for the pre-initialized enclaves may increase the maximum size of the pool of enclaves (e.g., to a size above the fixed number of available key identifiers). The additional operations may also include creating a secure extended page table (EPT) branch for the selected TEE that derives code mappings from the template TEE.

The trust domain module <NUM> provides access to the selected enclave in response to the request (operation <NUM>). For example, a unique identifier of the initialized enclave may be returned, usable as a parameter for a later request (e.g., an EENTER command) to the trust domain module <NUM> to execute a function within the enclave.

Thereafter, the trust domain module <NUM> may determine that execution of the selected enclave is complete (e.g., in response to receiving an enclave exit instruction). The memory assigned to the completed enclave may be released. Alternatively, the state of the completed enclave may be restored to the pre-initialized state and the enclave may be returned to the pool. For example, the template enclave may be copied over the enclave, operations to reverse the workload-specific operations may be performed, a checkpoint of the enclave performed before the workload-specific operations are performed, and a restore of the checkpoint performed after execution is complete, or any suitable combination thereof.

By comparison with prior art implementations that do not perform the pre-initialization of the enclave (operation <NUM>) before receiving the request for the enclave (operation <NUM>), the delay between the receiving of the request and the providing of access (operation <NUM>) is reduced. The reduction of latency may allow additional functionality to be protected in enclaves, for finer-grained enclaves to be used, or both, increasing system security. Additionally, when the enclave is invoked over the network <NUM> by a client device <NUM>, processor cycles of the client device <NUM> that are used while waiting for a response from the FaaS server <NUM> are reduced, improving responsiveness and reducing power consumption.

In operation <NUM>, the trust domain module <NUM> creates a template enclave marked read-only. For example, the enclave may be fully created, with code and data loaded into the private memory <NUM>. However, since the template enclave is read-only, functions of the template enclave cannot be directly invoked from the untrusted component <NUM>. With reference to the enclave table <NUM> of <FIG>, the row 330A shows a read-only template enclave.

In operation <NUM>, in response to the received request, the trust domain module <NUM> copies the template enclave to create the requested enclave. For example, the trust domain module <NUM> may determine that the data identifying the enclave indicates that the requested enclave is for the same code and data as the template enclave. The determination may be based on a signature of the enclave code and data, a message authentication code (MAC) of the enclave code and data, asymmetric encryption, or any suitable combination thereof.

The trust domain module <NUM> provides access to the selected enclave in response to the request (operation <NUM>). For example, a unique identifier of the initialized enclave may be returned, usable as a parameter to a later request (e.g., an EENTER command) to the trust domain module <NUM> to execute a function within the enclave.

By comparison with prior art implementations that create the enclave in response to receiving the request for the enclave (operation <NUM>) instead of copying the template enclave to create the requested enclave (operation <NUM>), the delay between the receiving of the request and the providing of access (operation <NUM>) is reduced. The reduction of latency may allow additional functionality to be protected in enclaves, for finer-grained enclaves to be used, or both, increasing system security. Additionally, when the enclave is invoked over the network <NUM> by a client device <NUM>, processor cycles of the client device <NUM> that are used while waiting for a response from the FaaS server <NUM> are reduced, improving responsiveness and reducing power consumption.

In operation <NUM>, the trust domain module <NUM> pre-initializes a first pool of enclaves of a first class and a second pool of enclaves of a second class. The pre-initializing is either complete or partial. For complete initialization, the enclaves in the pool are fully prepared for use and already loaded with the code and data of the enclave. Thus, all members of the class are identical. For partial initialization, the enclaves in the pool share one or more characteristics, such as the amount of heap memory used. The enclaves are initialized with respect to the shared characteristics, but the actual code and data of the enclave is not loaded during pre-initialization. Additionally, further customization may be performed at a later step. Accordingly, different enclaves may be members of the class when partial initialization is performed.

The trust domain module <NUM>, in operation <NUM>, receives a request for an enclave of the first class. For example, the untrusted component <NUM> of an application may provide data identifying an enclave to the trust domain module <NUM> as part of the request. The data identifying the enclave may include a pointer to an address in the shared memory <NUM>, accessible by the untrusted component <NUM>. The data identifying the class of the enclave may be included in the enclave or in the request.

In operation <NUM>, in response to the received request, the trust domain module <NUM> selects an enclave from the pre-initialized pool of enclaves for the first class. The trust domain module <NUM> may modify the selected enclave by performing, based on the data identifying the enclave received with the request, additional operations on the selected enclave, such as the workload-specific operations shown in <FIG>. The additional operations may include copying data or code from the address in the shared memory <NUM> indicated in the request to the private memory <NUM> allocated to the enclave.

By using different pools of different classes, enclaves that are similar but in relatively low demand can be put into a common class for partial pre-initialization, reducing latency while consuming resources only in proportion to their demand. At the same time, enclaves that are in high demand can be fully pre-initialized, reducing latency still further.

<FIG> is a block diagram <NUM> showing one example of a software architecture <NUM> for a computing device. The architecture <NUM> may be used in conjunction with various hardware architectures, for example, as described herein. <FIG> is merely a non-limiting example of a software architecture and many other architectures may be implemented to facilitate the functionality described herein. A representative hardware layer <NUM> is illustrated and can represent, for example, any of the above referenced computing devices. In some examples, the hardware layer <NUM> may be implemented according to the architecture of the computer system of <FIG>.

The representative hardware layer <NUM> comprises one or more processing units <NUM> having associated executable instructions <NUM>. Executable instructions <NUM> represent the executable instructions of the software architecture <NUM>, including implementation of the methods, modules, subsystems, and components, and so forth described herein and may also include memory and/or storage modules <NUM>, which also have executable instructions <NUM>. Hardware layer <NUM> may also comprise other hardware as indicated by other hardware <NUM> which represents any other hardware of the hardware layer <NUM>, such as the other hardware illustrated as part of the software architecture <NUM>.

In the example architecture of <FIG>, the software architecture <NUM> may be conceptualized as a stack of layers where each layer provides particular functionality. For example, the software architecture <NUM> may include layers such as an operating system <NUM>, libraries <NUM>, frameworks/middleware <NUM>, applications <NUM>, and presentation layer <NUM>. Operationally, the applications <NUM> and/or other components within the layers may invoke application programming interface (API) calls <NUM> through the software stack and access a response, returned values, and so forth illustrated as messages <NUM> in response to the API calls <NUM>. The layers illustrated are representative in nature and not all software architectures have all layers. For example, some mobile or special purpose operating systems may not provide a frameworks/middleware layer <NUM>, while others may provide such a layer. Other software architectures may include additional or different layers.

The operating system <NUM> may manage hardware resources and provide common services. The operating system <NUM> may include, for example, a kernel <NUM>, services <NUM>, and drivers <NUM>. The kernel <NUM> may act as an abstraction layer between the hardware and the other software layers. For example, the kernel <NUM> may be responsible for memory management, processor management (e.g., scheduling), component management, networking, security settings, and so on. In some examples, the services <NUM> include an interrupt service. The interrupt service may detect the receipt of an interrupt and, in response, cause the architecture <NUM> to pause its current processing and execute an interrupt service routine (ISR) when an interrupt is accessed.

The drivers <NUM> may be responsible for controlling or interfacing with the underlying hardware. For instance, the drivers <NUM> may include display drivers, camera drivers, Bluetooth® drivers, flash memory drivers, serial communication drivers (e.g., Universal Serial Bus (USB) drivers), Wi-Fi® drivers, NFC drivers, audio drivers, power management drivers, and so forth depending on the hardware configuration.

The libraries <NUM> may provide a common infrastructure that may be utilized by the applications <NUM> and/or other components and/or layers. The libraries <NUM> typically provide functionality that allows other software modules to perform tasks in an easier fashion than to interface directly with the underlying operating system <NUM> functionality (e.g., kernel <NUM>, services <NUM> and/or drivers <NUM>). The libraries <NUM> may include system libraries <NUM> (e.g., C standard library) that may provide functions such as memory allocation functions, string manipulation functions, mathematic functions, and the like. In addition, the libraries <NUM> may include API libraries <NUM> such as media libraries (e.g., libraries to support presentation and manipulation of various media format such as MPEG4, H. <NUM>, MP3, AAC, AMR, JPG, PNG), graphics libraries (e.g., an OpenGL framework that may be used to render two-dimensional and three-dimensional in a graphic content on a display), database libraries (e.g., SQLite that may provide various relational database functions), web libraries (e.g., WebKit that may provide web browsing functionality), and the like. The libraries <NUM> may also include a wide variety of other libraries <NUM> to provide many other APIs to the applications <NUM> and other software components/modules.

The frameworks/middleware <NUM> may provide a higher-level common infrastructure that may be utilized by the applications <NUM> and/or other software components/modules. For example, the frameworks/middleware <NUM> may provide various graphic user interface (GUI) functions, high-level resource management, high-level location services, and so forth. The frameworks/middleware <NUM> may provide a broad spectrum of other APIs that may be utilized by the applications <NUM> and/or other software components/modules, some of which may be specific to a particular operating system or platform.

The applications <NUM> include built-in applications <NUM> and/or third-party applications <NUM>. Examples of representative built-in applications <NUM> may include, but are not limited to, a contacts application, a browser application, a book reader application, a location application, a media application, a messaging application, and/or a game application. Third-party applications <NUM> may include any of the built-in applications <NUM> as well as a broad assortment of other applications. In a specific example, the third-party application <NUM> (e.g., an application developed using the Android™ or iOS™ software development kit (SDK) by an entity other than the vendor of the particular platform) may be mobile software running on a mobile operating system such as iOS™, Android™, Windows® Phone, or other mobile computing device operating systems. In this example, the third-party application <NUM> may invoke the API calls <NUM> provided by the mobile operating system such as operating system <NUM> to facilitate functionality described herein.

The applications <NUM> may utilize built-in operating system functions (e.g., kernel <NUM>, services <NUM> and/or drivers <NUM>), libraries (e.g., system libraries <NUM>, API libraries <NUM>, and other libraries <NUM>), frameworks/middleware <NUM> to create user interfaces to interact with users of the system. Alternatively, or additionally, in some systems, interactions with a user may occur through a presentation layer, such as presentation layer <NUM>. In these systems, the application/module "logic" can be separated from the aspects of the application/module that interact with a user.

Some software architectures utilize virtual machines. In the example of <FIG>, this is illustrated by virtual machine <NUM>. A virtual machine creates a software environment where applications/modules can execute as if they were executing on a hardware computing device. A virtual machine is hosted by a host operating system (operating system <NUM>) and typically, although not always, has a virtual machine monitor <NUM>, which manages the operation of the virtual machine <NUM> as well as the interface with the host operating system (i.e., operating system <NUM>). A software architecture executes within the virtual machine <NUM> such as an operating system <NUM>, libraries <NUM>, frameworks/middleware <NUM>, applications <NUM> and/or presentation layer <NUM>. These layers of software architecture executing within the virtual machine <NUM> can be the same as corresponding layers previously described or may be different.

Certain embodiments are described herein as including logic or a number of components, modules, or mechanisms. Modules may constitute either software modules (e.g., code embodied (<NUM>) on a non-transitory machine-readable medium or (<NUM>) in a transmission signal) or hardware-implemented modules. A hardware-implemented module is a tangible unit capable of performing certain operations and may be configured or arranged in a certain manner. In example embodiments, one or more computer systems (e.g., a standalone, client, or server computer system) or one or more hardware processors may be configured by software (e.g., an application or application portion) as a hardware-implemented module that operates to perform certain operations as described herein.

In various embodiments, a hardware-implemented module may be implemented mechanically or electronically. For example, a hardware-implemented module may comprise dedicated circuitry or logic that is permanently configured (e.g., as a special-purpose processor, such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC)) to perform certain operations. A hardware-implemented module may also comprise programmable logic or circuitry (e.g., as encompassed within a general-purpose processor or another programmable processor) that is temporarily configured by software to perform certain operations. It will be appreciated that the decision to implement a hardware-implemented module mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry (e.g., configured by software) may be driven by cost and time considerations.

Accordingly, the term "hardware-implemented module" should be understood to encompass a tangible entity, be that an entity that is physically constructed, permanently configured (e.g., hardwired), or temporarily or transitorily configured (e.g., programmed) to operate in a certain manner and/or to perform certain operations described herein. Considering embodiments in which hardware-implemented modules are temporarily configured (e.g., programmed), each of the hardware-implemented modules need not be configured or instantiated at any one instance in time. For example, where the hardware-implemented modules comprise a general-purpose processor configured using software, the general-purpose processor may be configured as respective different hardware-implemented modules at different times. Software may accordingly configure a processor, for example, to constitute a particular hardware-implemented module at one instance of time and to constitute a different hardware-implemented module at a different instance of time.

Hardware-implemented modules can provide information to, and receive information from, other hardware-implemented modules. Accordingly, the described hardware-implemented modules may be regarded as being communicatively coupled. Where multiple of such hardware-implemented modules exist contemporaneously, communications may be achieved through signal transmission (e.g., over appropriate circuits and buses that connect the hardware-implemented modules). In embodiments in which multiple hardware-implemented modules are configured or instantiated at different times, communications between such hardware-implemented modules may be achieved, for example, through the storage and retrieval of information in memory structures to which the multiple hardware-implemented modules have access. For example, one hardware-implemented module may perform an operation, and store the output of that operation in a memory device to which it is communicatively coupled. A further hardware-implemented module may then, at a later time, access the memory device to retrieve and process the stored output. Hardware-implemented modules may also initiate communications with input or output devices, and can operate on a resource (e.g., a collection of information).

Similarly, the methods described herein may be at least partially processor-implemented. In some example embodiments, the processor or processors may be located in a single location (e.g., within a home environment, an office environment, or a server farm), while in other embodiments the processors may be distributed across a number of locations.

The one or more processors may also operate to support performance of the relevant operations in a "cloud computing" environment or as a "software as a service" (SaaS). For example, at least some of the operations may be performed by a group of computers (as examples of machines including processors), these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., APIs).

Example embodiments may be implemented in digital electronic circuitry, or in computer hardware, firmware, or software, or in combinations of them. Example embodiments may be implemented using a computer program product, e.g., a computer program tangibly embodied in an information carrier, e.g., in a machine-readable medium for execution by, or to control the operation of, data processing apparatus, e.g., a programmable processor, a computer, or multiple computers.

A computer program can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a standalone program or as a module, subroutine, or other unit suitable for use in a computing environment.

In example embodiments, operations may be performed by one or more programmable processors executing a computer program to perform functions by operating on input data and generating output. Method operations can also be performed by, and apparatus of example embodiments may be implemented as, special purpose logic circuitry, e.g., an FPGA or an ASIC.

In embodiments deploying a programmable computing system, it will be appreciated that both hardware and software architectures merit consideration. Specifically, it will be appreciated that the choice of whether to implement certain functionality in permanently configured hardware (e.g., an ASIC), in temporarily configured hardware (e.g., a combination of software and a programmable processor), or in a combination of permanently and temporarily configured hardware may be a design choice. Below are set out hardware (e.g., machine) and software architectures that may be deployed, in various example embodiments.

<FIG> is a block diagram of a machine in the example form of a computer system <NUM> within which instructions <NUM> may be executed for causing the machine to perform any one or more of the methodologies discussed herein. In alternative embodiments, the machine operates as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine may operate in the capacity of a server or a client machine in server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular telephone, a web appliance, a network router, switch, or bridge, or any machine capable of executing instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term "machine" shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

The example computer system <NUM> includes a processor <NUM> (e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both), a main memory <NUM>, and a static memory <NUM>, which communicate with each other via a bus <NUM>. The computer system <NUM> may further include a video display unit <NUM> (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)). The computer system <NUM> also includes an alphanumeric input device <NUM> (e.g., a keyboard or a touch-sensitive display screen), a user interface (UI) navigation (or cursor control) device <NUM> (e.g., a mouse), a storage unit <NUM>, a signal generation device <NUM> (e.g., a speaker), and a network interface device <NUM>.

The storage unit <NUM> includes a machine-readable medium <NUM> on which is stored one or more sets of data structures and instructions <NUM> (e.g., software) embodying or utilized by any one or more of the methodologies or functions described herein. The instructions <NUM> may also reside, completely or at least partially, within the main memory <NUM> and/or within the processor <NUM> during execution thereof by the computer system <NUM>, with the main memory <NUM> and the processor <NUM> also constituting machine-readable media <NUM>.

While the machine-readable medium <NUM> is shown in an example embodiment to be a single medium, the term "machine-readable medium" may include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more instructions <NUM> or data structures. The term "machine-readable medium" shall also be taken to include any tangible medium that is capable of storing, encoding, or carrying instructions <NUM> for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present disclosure, or that is capable of storing, encoding, or carrying data structures utilized by or associated with such instructions <NUM>. The term "machine-readable medium" shall accordingly be taken to include, but not be limited to, solid-state memories, and optical and magnetic media. Specific examples of machine-readable media <NUM> include non-volatile memory, including by way of example semiconductor memory devices, e.g., erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory devices; magnetic disks such as internal hard disks and removable disks; magnetooptical disks; and compact disc read-only memory (CD-ROM) and digital versatile disc read-only memory (DVD-ROM )disks. A machine-readable medium is not a transmission medium.

Claim 1:
A method (<NUM>, <NUM>) to provide a trusted execution environment, TEE, the method comprising:
pre-initializing (<NUM>, <NUM>), by a processor, a pool of TEEs, the pre-initializing of each TEE of the pool of TEEs including allocating memory of a storage device for the TEE;
after the pre-initializing of the pool of TEEs, receiving, by the processor, a request (<NUM>, <NUM>); and
in response to the request:
selecting (<NUM>, <NUM>), by the processor, a TEE from the pre-initialized pool of TEEs; and
providing (<NUM>, <NUM>) access, by the processor, to the selected TEE;
further comprising:
prior to the providing of access to the selected TEE, modifying the selected TEE,
wherein the modifying selected TEE comprises:
modifying the selected TEE based on information in the request (<NUM>, <NUM>);
assigning an encryption key to the selected TEE; and
encrypting the memory allocated to the selected TEE using the encryption key;
wherein the encryption key is an ephemeral encryption key,
wherein the encryption key is generated within the processor on boot and never stored outside the processor.