Patent Description:
In a "bare metal" scenario, a tenant is provided dedicated access to the physical hardware of a given server, as opposed to a virtual server created on a hypervisor layer. Accordingly, the tenant is able to configure the physical processor, memory, and storage of the given server. Since the hardware is not virtualized and does not support multiple virtual machines, the tenant is able to consume all resources within the server. A tenant may run any desired operating system and applications on the server, including a hypervisor for creating virtual machines which may be used by the tenant.

Providing a tenant with dedicated access to a physical server raises several issues. First, the tenant should be assured that the owner of the server (i.e., a cloud provider) will be unable to execute code on the server while control of the server is with the tenant. Second, the owner should be able to revoke a tenant's access even if the tenant does not consent to the revocation. Third, the owner should be able to grant dedicated access to a next tenant after revoking access of a prior tenant, and to perform this grant-revoke-grant cycle one or more times over the lifetime of the server.

Current systems attempt to address the foregoing by: <NUM>) Provisioning a new owner's key into flash memory at each transfer; <NUM>) Providing an initialization phase in which a new owner's key is encrypted or signed with an on-device secret; or <NUM>) Providing an initialization phase in which a new owner activates their key and deactivates a manufacturer key using a one-time-programmable fuse array divided into key slots. Each of these systems presents limitations on transfer cycles, flash memory vulnerabilities and/or susceptibility to replay attacks. Systems are therefore desired to suitably and efficiently address security issues arising from transfers of a given physical server between an owner and one or more successive tenants.

<CIT> describes when terminating a process instantiated in a cloud, a cloud management system can provide and interact with an eraser agent on the computing systems supporting the process. The process can be any type of process that can exist in the cloud such as a virtual machine, software appliance, or software instance. The eraser agent can execute on the computing systems to erase information stored on physical storage devices of the computing systems and associated with the process. In particular, the eraser agent can utilize secure algorithms to alter and obscure the information stored on the physical storage devices of the computing systems and associated with the process.

The following description is provided to enable any person in the art to make and use the described embodiments. Various modifications, however, will be apparent to those in the art. A transfer of tenancy generally refers herein to a change in control over the bootloader image which is executed by the hardware platform. Tenancy may be transferred via any contractual arrangement, including but not limited to rental, lease or sale, and may be associated with a specified or open-ended duration. Tenancy transfer according to some embodiments utilizes a grant token which is cryptographically bound to the specific hardware platform to prevent its usage on other hardware platforms. After tenancy transfer, a tenant is able to sign firmware and software with its own keys without reliance on the keys of the owner or a third-party. Tenancy can be revoked by the tenant or the owner via a tenancy revocation process which utilizes a hardware platform-specific revocation token.

According to some embodiments, once an owner transfers tenancy of a hardware platform to a tenant, the owner will be unable to execute code on the hardware platform or revoke the tenancy without creating irrefutable and verifiable cryptographic evidence on the hardware platform. However, the owner maintains the ability to revoke tenancy (even without the tenant's consent) by altering the cryptographic state of the system in order to render previously-executable tenant code non-executable. The tenancy transfer and tenancy revocation processes therefore both generate irrefutable cryptographic evidence of a change in tenancy. The generation of this evidence according to some embodiments may advantageously serve to maintain trust and confidentiality between tenants and owners.

<FIG> illustrate a tenancy grant, a tenancy revocation and a second tenancy grant according to some embodiments. <FIG> will be discussed below to provide an overview of some embodiments, and additional implementation details will follow. Architecture <NUM> of <FIG> includes owner device <NUM> and hardware platform <NUM>. Generally, a hardware platform as described herein includes one or more Central Processing Units (CPUs), random access memory and compatible hardware (e.g., chipset, Graphics Processing Unit, Solid State Drives, cooling devices, I/O devices) which provide an environment for the execution of software applications. Hardware platform <NUM> may comprise a blade server or a rack server installed within a server rack located in a datacenter remote from owner device <NUM>, but embodiments are not limited thereto. For example, hardware platform <NUM> may comprise a desktop computer, a laptop computer, a tablet computer, a smartphone, or a wearable device. Owner device <NUM> is depicted as a laptop computer but may comprise any device suitable for communicating with hardware platform <NUM> and exchanging electronic files therewith. For the sake of simplicity, architecture <NUM> omits intermediate networks, devices and software components which may facilitate this communication and exchange. In some embodiments, device <NUM> accesses a cloud-based administration application (e.g., via a Web browser) to initiate communication with hardware platform <NUM>. The cloud-based administration application passes corresponding requests to a cloud-based orchestrator component which in turn communicates with a fabric controller of a server rack of a cloud datacenter in which hardware platform <NUM> is installed.

Hardware platform <NUM> includes a hardware-based Root of Trust (RoT). The RoT may comprise a microcontroller or may be embedded within a CPU of hardware platform <NUM>, for example. During a power-on sequence, the RoT verifies firmware as described herein prior to execution thereof by hardware platform <NUM>.

Briefly, hardware platform <NUM> includes a secure storage device (e.g., a one-time-programmable (OTP) fuse, a persistent monotonic counter, a hardware-protected and persistent non-volatile memory (NVM)) storing a representation (e.g., a hash) of an owner's public key. The hardware platform also includes a secure storage device for storing a counter. At reset, the RoT loads a boot manifest from a firmware storage device (e.g., a flash memory storage device) and confirms whether the representation of the owner's public key stored in the OTP fuse matches a representation within the boot manifest. If not, booting of hardware platform <NUM> is halted. If so, the boot manifest signature is verified using the owner's public key. If signature verification is successful, the boot manifest type (i.e., owner or tenant) is determined. The boot process then continues, as will be described below, based on the determined boot manifest type and on a parity of the value stored in the counter.

It will be assumed that the boot manifest type and parity of the value stored in the counter indicate execution of an owner secure boot flow. Accordingly, a bootloader image stored in the firmware storage device is verified using the owner's public key, loaded if the verification is successful, and executed. Booting of hardware platform <NUM> then continues based on the bootloader image and owner device <NUM> may subsequently interact therewith as is known in the art. For example, owner device <NUM> may install and configure applications on hardware platform <NUM> and execute those applications to utilize their functionality directly and/or to expose the functionality to public users for consumption over the Web.

<FIG> illustrates provisioning of a tenancy grant by owner device <NUM> according to some embodiments. In this regard, owner device <NUM> depicted in <FIG> may represent different devices under control of the owner of hardware platform <NUM> (i.e., all actions attributed to owner device <NUM> of <FIG> though <NUM> need not be performed by a same computing device).

Architecture <NUM> of <FIG> includes tenant device <NUM>, which may comprise any suitable computing device for communicating with owner device <NUM>. Tenant device <NUM> transmits a bootloader image (i.e., Tenant-Signed Bootloader<NUM>) signed by the tenant's bootloader signing certificate and the tenant's public key (i.e., Signing Public Key<NUM>), which corresponds to the signing certificate, to owner device <NUM>. The bootloader image and public key may be provided to owner device <NUM> via any suitable means, including but not limited to a portable storage device and a secure file repository.

As shown in <FIG>, owner device <NUM> requests a tenant grant from hardware platform <NUM> and receives a grant token (i.e., Grant Token<NUM>) and a revocation token (i.e., Revocation Token<NUM>) in response. According to some embodiments, owner device <NUM> may request the tenant grant via a tenancy administration agent provided by an administration application as described above. Hardware platform <NUM> may generate the grant token and the revocation token based on a device secret securely stored on hardware platform <NUM> and on a value equal to the current value of the tenancy counter + <NUM>.

Owner device <NUM> generates Tenant Grant Manifesti using Grant Token<NUM> and Signing Public Key<NUM>, and signs Tenant Grant Manifesti using a tenant grant manifest signing certificate. Tenant Grant Manifesti binds the tenant to hardware platform <NUM> by including the device-specific grant token and the tenant signing public key, and is bound to the owner via its signing by the tenant grant manifest signing certificate owned by the owner. Similarly, owner device <NUM> generates Tenant Revocation Manifest<NUM> using Revocation Token<NUM> and Signing Public Key<NUM>, and signs Tenant Revocation Manifesti using a tenant revocation manifest signing certificate. Tenant Boot Manifesti is also created to include the tenant grant manifest signing public key and the tenant revocation manifest signing public key, and is signed using the owner's boot manifest signing certificate.

Owner device <NUM> may then operate the tenancy administration agent to store Tenant Boot Manifest<NUM>, Tenant Grant Manifest<NUM>, Tenant Revocation Manifest<NUM> and Tenant-signed Bootloader<NUM> in a flash storage device of hardware platform <NUM>. Tenant Boot Manifesti and Tenant-signed Bootloader<NUM> may overwrite respective Owner Boot Manifest and Owner-signed Bootloader which were previously stored in the flash storage device.

Hardware platform <NUM> is rebooted thereafter to continue the tenancy transfer. Upon reset, the RoT verifies and loads the stored boot manifest (now Tenant Boot Manifest<NUM>) as described above. Tenant Boot Manifesti includes a flag indicating that it is a tenant boot manifest, causing the RoT to load the stored tenant grant manifest header and confirm whether it is signed with the tenant grant manifest signing certificate. The RoT also generates a grant token based on the device secret of hardware platform <NUM> and on a current value of the tenancy counter. If this newly-generated grant token is identical to the grant token of the tenant grant manifest, the tenancy counter is incremented by <NUM>.

Hardware platform <NUM> is rebooted once more. The stored boot manifest (i.e., Tenant Boot Manifesl<NUM>) is again verified and loaded, and the grant token generation and verification are executed as described above. The headers of the stored bootloader image (i.e., Tenant-signed Bootloader<NUM>) are loaded and signature-verified based on the bootloader signing public key (Signing Public Key<NUM>) stored in the tenant grant manifest (Tenant Grant Manifest<NUM>).

If all verifications are successful, the bootloader image (i.e., Tenant-signed Bootloader<NUM>) is fully loaded and booting continues based thereon as is known in the art. At this point, tenant device <NUM> may install, configure and execute applications on hardware platform <NUM> as illustrated at <FIG>.

<FIG> illustrates owner-initiated tenancy revocation provisioning according to some embodiments. As shown, owner device <NUM> copies an owner-signed bootloader, a signed owner boot manifest and the tenant revocation manifest to hardware platform <NUM>, which is thereafter rebooted.

Upon reboot, hardware platform <NUM> verifies and loads the stored boot manifest (now an owner-signed boot manifest) as described above. The parity of the tenancy counter is odd due to the prior incrementing of the value, and the owner-signed boot manifest includes a flag indicating that it is an owner tenant boot manifest. In response to these two factors, hardware platform <NUM> executes a tenancy revocation flow. In particular, the RoT generates a revocation token based on the device secret of hardware platform <NUM> and on a current value of the tenancy counter. If this newly-generated revocation token is identical to the revocation token of the stored tenant revocation manifest, the tenancy counter is incremented by <NUM> and hardware platform <NUM> is rebooted.

This reboot places hardware platform <NUM> in a similar state as shown in and described with respect to <FIG>, now illustrated in <FIG>. The flag of the current boot manifest (i.e., Owner) and the parity of the tenancy counter (i.e., Even) indicate that hardware platform <NUM> is configured to boot in an owner-controlled mode. The tenancy counter has been incremented from that illustrated in <FIG>, evidencing the transfer of tenancy from the tenant to the owner. <FIG> illustrates a tenancy transfer to a second tenant according to some embodiments. As shown, the second tenant operates tenant device <NUM> to provide a bootloader image (i.e., Tenant-Signed Bootloader<NUM>) signed by the second tenant's bootloader signing certificate and the second tenant's public key (i.e., Signing Public Key<NUM>), which corresponds to the signing certificate, to owner device <NUM>.

As described above, owner device <NUM> requests a tenant grant from hardware platform <NUM> and receives a grant token (i.e., Grant Token<NUM>) and a revocation token (i.e., Revocation Token<NUM>) in response. Hardware platform <NUM> may generate the grant token and the revocation token based on the aforementioned immutable device secret stored on hardware platform <NUM> and on a value equal to the current value of the tenancy counter + <NUM>. Notably, this value is different from (i.e., +<NUM>) the value used to generate the prior grant and revocation tokens. Accordingly, Grant Token<NUM> and Revocation Token<NUM> will differ from Grant Token<NUM> and Revocation Token<NUM>.

Owner device <NUM> generates Tenant Boot Manifest<NUM>, Tenant Grant Manifest<NUM> and Tenant Revocation Manifest<NUM> as described above with respect to Tenant Boot Manifest<NUM>, Tenant Grant Manifest<NUM> and Tenant Revocation Manifest<NUM>. Owner device <NUM> then stores Tenant Boot Manifest<NUM>, Tenant Grant Manifest<NUM> and Tenant Revocation Manifest<NUM> in a flash storage device of hardware platform <NUM>. Hardware platform <NUM> is then rebooted twice as described above to complete the tenancy transfer to the second tenant. As illustrated at <FIG>, tenant device <NUM> may thereafter install, configure and execute applications on hardware platform <NUM> as desired. <FIG> is a block diagram of a portion of a hardware platform according to some embodiments. RoT <NUM>, flash storage device <NUM>, secure storage device <NUM> and secure storage device <NUM> are mounted to motherboard <NUM>. According to some embodiments, motherboard <NUM> electrically connects RoT <NUM> to each of flash storage device <NUM>, storage device <NUM> and storage device <NUM> to provide communication therebetween as described herein. Motherboard <NUM> also provides power to RoT <NUM> and flash storage device <NUM> for operation as described herein. Motherboard <NUM> may support other discrete components and integrated circuit devices, including at least one CPU. Motherboard <NUM> and the components mounted thereon are also supported by power supply and cooling components as is known in the art.

RoT <NUM> may comprise a microcontroller or any other logic device capable of performing the functions attributed thereto herein. According to some embodiments, RoT <NUM> comprises an Advanced RISC Machine (ARM) controller. RoT <NUM> includes executable program code stored in Read-Only Memory (ROM) <NUM>. The program code stored in ROM <NUM> is considered immutable in that it cannot be changed or updated. The program code stored in ROM <NUM> is executed by RoT <NUM> at the start of a boot process in order to perform many of the tenancy transfer-related processes described herein.

Random Access Memory (RAM) <NUM> may store program code and data during operation of RoT <NUM>. As described herein, RoT <NUM> may load headers, manifests and bootloader images from flash storage device <NUM> into RAM <NUM> for evaluation or execution thereof. RAM <NUM> may be integrated with RoT <NUM> as depicted in <FIG> and/or a provided by a separate storage device. RoT <NUM> also includes security subsystem <NUM> according to some embodiments. Security subsystem <NUM> may provide asymmetric key-based decryption, cryptographic key derivation and signature verification. Security subsystem <NUM> may also store a unique device secret associated with the hardware platform and which is used to generate grant and revocation tokens associated with the hardware platform. The unique device secret may be stored in a separate secure storage device, such as a fuse bank accessible to RoT <NUM>, according to some embodiments.

Flash storage device <NUM> may comprise writable firmware storage as is known in the art. RoT <NUM> may load portions of files stored in flash storage device <NUM> directly into RAM <NUM>. As shown, prior to a transfer of control to a tenant, flash storage device <NUM> may store owner boot manifest <NUM> and owner-signed bootloader image <NUM>.

Secure storage device <NUM> may comprise an array (e.g., <NUM> to <NUM>) of fuse banks with each bank comprising <NUM> bits of OTP Error Correction Code (ECC) fuses. According to some embodiments, a <NUM>-bit value burned into a fuse bank of secure storage device <NUM> represents a Secure Hash Algorithm (SHA)-<NUM> hash of an Elliptic Curve Digital Signature Algorithm (ECDSA)-<NUM> public key which will be used to verify a signed owner boot manifest. Secure storage device <NUM> may be implemented by a persistent monotonic counter, a hardware-protected and persistent NVM, or any other suitable read-only device.

Secure storage device <NUM> may also comprise a fuse bank of OTP ECC fuses according to some embodiments. Secure storage device <NUM> is used to maintain a tenancy counter, or Tenant Security Version Number (TSVN). As described herein, the value of the counter is incremented each time a tenancy is granted or revoked. The size of secure storage device <NUM> determines a maximum number of allowed tenancy transfers. For example, in a case that secure storage device <NUM> includes <NUM> bits of fuses, <NUM> tenancy transfers will be supported.

To maintain cryptographic security, secure storage device <NUM> must only be updatable by execution of the immutable code stored ROM <NUM>. This code is initially executed upon platform boot, at which point RoT <NUM> may execute the code to increment the stored value as required by the execution logic. Once RoT <NUM> exits execution of the code of ROM <NUM>, storage device <NUM> is locked to prevent any write access.

<FIG> illustrates the structure of owner boot manifest <NUM> according to some embodiments. Owner boot manifest <NUM> may comprise an implementation of owner boot manifest <NUM> of <FIG>. In this regard, a boot manifest stored at a corresponding partition of flash storage device <NUM> is the first structure loaded by RoT <NUM> into RAM <NUM> during execution of the code of ROM <NUM>.

Owner boot manifest <NUM> includes unsigned manifest header <NUM>, signed manifest header <NUM> and manifest <NUM>. Unsigned manifest header <NUM> includes contains a boot manifest signature and a boot manifest signing public key. Moreover, signed manifest header <NUM> includes a manifest hash and a flag indicating a type (i.e., owner or tenant) of boot manifest <NUM>. Accordingly, RoT <NUM> may decrypt the signature using the boot manifest signing public key and compare the result to the manifest hash to verify whether manifest <NUM> was signed by the manifest signing certificate corresponding to the boot manifest signing public key. If so, RoT <NUM> may trust the contents of manifest <NUM>, which include a tenant revocation manifest signing public key and a bootloader signing public key.

Bootloader image <NUM> of <FIG> may comprise an implementation of owner-signed bootloader image <NUM> of <FIG>. Bootloader image <NUM> includes unsigned header <NUM>, signed header <NUM> and bootloader image <NUM>. Under certain conditions as will be described below, RoT <NUM> may use the bootloader signing public key of owner boot manifest <NUM> to decrypt unsigned header <NUM> and may then compare the result to the bootloader hash of signed header <NUM> to verify whether bootloader image <NUM> was signed by the owner's bootloader signing certificate. If so, RoT <NUM> may load bootloader image <NUM> into RAM <NUM> and execute bootloader image <NUM> therefrom.

<FIG> is a flow diagram of process <NUM> to verify a boot manifest and to select a boot flow according to some embodiments. Process <NUM> may be implemented in executable program code and/or in hardware. Such executable program code may be stored in an un-modifiable manner, such as within ROM <NUM> of RoT <NUM>. Process <NUM> may be executed by one or more processing units (e.g., processors, processor cores) executing program code, including but not limited to RoT <NUM>.

A subject hardware platform is rebooted at S510. The reboot may be software-initiated (e.g., in response to a command received from an orchestrator application) or hardware-initiated (e.g., manual operation of an on/off switch). Rebooting may comprise application of main power to the hardware platform. Prior to S510, the hardware platform may be receiving standby power and/or battery power (e.g., to power a Real Time Clock).

Conventionally, the applied power initially passes through in-rush circuity to a Complex Programmable Logic Device (CPLD), which ensures that the power rails (e.g., <NUM>. 3V, +5V, +12V) of the platform are energized in a time-sensitive sequence. The sequence ensures that a lower-voltage power rail is stable before a next-highest voltage power rail is energized. As the power-on sequence progresses towards final stages, an RoT, CPU, and other powered components of the hardware platform are held in reset. A power good signal is asserted once the highest-voltage power rail is stable, which causes the RoT to come out of reset and begin booting.

Booting starts at S520, in which it is determined whether a stored representation of the owner's public key matches a public key within a stored boot manifest. As described above, flash storage device <NUM> may include owner boot manifest <NUM>, of which owner boot manifest <NUM> is an example. At S520, RoT <NUM> may load unsigned header <NUM> into RAM <NUM>. RoT <NUM> compares the boot manifest signing public key of header <NUM> with the key stored in storage device <NUM>. Both boot manifest signing public key of header <NUM> and the key stored in storage device <NUM> may comprise key hashes. The boot process is halted at S520 if the keys (or key hashes) are not identical, and proceeds to S540 if they are identical.

At S540, it is determined whether the manifest header was signed with the owner's boot manifest signing certificate. S540 may comprise loading unsigned manifest header <NUM> and signed manifest header <NUM> into RAM <NUM>, decrypting the signature of unsigned manifest header <NUM> using the boot manifest signing public key of header <NUM>, and comparing the result to the manifest hash of signed manifest header <NUM>. The boot process fails at S520 if the result does not match the hash, indicating that the manifest header is not signed with the owner's boot manifest signing certificate. The boot manifest is loaded into RAM <NUM> at S550 if is determined at S540 that the manifest header was signed with the owner's boot manifest signing certificate. The RoT determines a next boot flow to execute based on the loaded boot manifest signed header and the parity of the tenancy counter value. First, at S560, a tenancy mode is determined based on the parity of the value stored in the tenancy counter. It will be assumed that no tenancy transfers of the hardware platform have yet occurred, and the tenancy counter is therefore set to <NUM>. Since even values of the tenancy counter are associated with an owner mode, flow proceeds to S565. The initial value of the tenancy counter may comprise any other owner-associated (e.g., even) value.

A type of the current boot manifest is determined at S565. As described above, signed manifest header <NUM> includes a flag indicating a type of boot manifest <NUM>. RoT <NUM> may therefore determine the type of the boot manifest stored in flash storage device <NUM> based on this flag. Continuing the present example, the type is "owner" and flow therefore proceeds to S570. Process <NUM> shows selection of one of four different boot flows depending on the boot manifest type and the parity of the tenancy counter value. According to some embodiments, a lifecycle of a hardware platform may progress successively through each of these flows as illustrated in <FIG>. For example, the upper-left quadrant of the <FIG> diagram indicates performance of an owner secure boot flow in a case that a signature-verified owner boot manifest is present in the flash storage device and the parity of the tenancy counter value indicates that the hardware platform is in owner mode.

Boot-up always proceeds in this manner until, at some subsequent reboot, it is determined that the signature-verified boot manifest in the flash storage device is a tenant boot manifest, for example by virtue of a tenant-associated flag within the signed header of the manifest. The hardware platform currently remains in owner mode, and therefore a tenancy transfer flow of the upper-right quadrant is executed instead of the owner secure boot flow.

At a next reboot, it is determined that the signature-verified boot manifest is a tenant boot manifest and that the parity of the tenancy counter has changed (as will be described below) to indicate a tenant mode. Accordingly, a tenant secure boot flow of the lower-right quadrant is executed.

The tenant secure boot flow is executed at boot-up until, during a given reboot, it is determined that the signature-verified boot manifest in the flash storage device is an owner boot manifest (e.g., because the owner or tenant has copied an owner boot manifest to the flash storage device). Therefore, instead of executing the tenant secure boot flow, a tenant revocation flow of the lower-left quadrant is executed. During the tenant revocation flow, the tenancy counter is incremented. Accordingly, at a next reboot, it is determined that the parity of the tenancy counter has changed to indicate an owner mode. If a signature-verified owner boot manifest is also present, then the owner secure boot flow is executed as described above.

<FIG> is a flow diagram of process <NUM> of an owner secure boot flow according to some embodiments. Process <NUM> may be executed at S570 of process <NUM> as described above.

At S710, the signed and unsigned headers of a flash-stored bootloader image are loaded into RAM. With reference to bootloader image <NUM>, RoT <NUM> may load unsigned header <NUM> and signed header <NUM> into RAM <NUM> at S710. Next, at S720, it is determined whether the bootloader header is signed with the owner's bootloader signing certificate. For example, RoT <NUM> may use the now-loaded bootloader signing public key of owner boot manifest <NUM> to decrypt unsigned header <NUM> and may compare the result to the bootloader hash of signed header <NUM>. If the result is not identical to the hash, the boot process fails at S730. If the bootloader header is signed with the owner's bootloader signing certificate, RoT <NUM> loads remaining bootloader image <NUM> into RAM <NUM> and begins execution at the entry point of bootloader image <NUM>.

An owner may continue to operate a hardware platform using a boot flow consisting of process <NUM> and process <NUM> over any desired period time, without any changes to the boot manifest or tenancy counter. Once the owner determines to transfer tenancy to a given tenant, the transfer may be initiated via a tenancy grant provisioning process. <FIG> is a flow diagram of process <NUM> to provision a tenancy grant according to some embodiments. Although process <NUM> will be described as being performed by an owner, process <NUM> may be performed by any party having use of the owner's manifest signing certificate.

Initially, a bootloader image signed by the tenant's bootloader signing certificate and a tenant public key corresponding to the tenant's bootloader signing certificate are received at S805. The bootloader image and public key may be provided to the owner via any suitable means.

A tenant grant is requested from the hardware platform by the owner at S810. In response, the hardware platform generates a grant token based on the tenant public key corresponding to the tenant's bootloader signing certificate, a platform secret and the current value of the tenancy counter. <FIG> illustrates system <NUM> for generation of grant token <NUM> according to some embodiments. System <NUM> may be implemented by security subsystem <NUM> of RoT <NUM>. As shown, key derivation function <NUM> receives tenant bootloader signing public key <NUM>, platform secret <NUM>, a value equal to the tenancy counter value + <NUM>, and a label "Tenancy Grant". Platform secret <NUM> is unique to the hardware platform and is stored in any suitable secure manner therein. The tenancy counter value is read, for example, from secure storage device <NUM>. According to some embodiments, the request for a grant token at S810 also causes the hardware platform to generate a revocation token. <FIG> illustrates system <NUM> for generation of revocation token <NUM> according to some embodiments. System <NUM> may be a same system as system <NUM> of <FIG>. Key derivation function <NUM> receives platform secret <NUM>, a value equal to the tenancy counter value + <NUM> (i.e., the same value used to generate grant token <NUM>), and a label "Tenancy Revocation", and generates revocation token <NUM> based thereon. The owner receives the grant token at S815 and the revocation token at S820.

The owner generates a tenant grant manifest at S825 based on the grant token and the tenant signing public key received at S805. <FIG> illustrates tenant grant manifest <NUM> which may be generated at S825 according to some embodiments. Tenant grant manifest <NUM> includes block header <NUM>, signed manifest header <NUM> and manifest <NUM>. The owner generates the manifest hash of signed manifest header <NUM> based on manifest <NUM> and, at S830, generates signature <NUM> from the manifest hash using the owner's tenant grant manifest signing certificate.

The owner generates a tenant revocation manifest at S835 based on the revocation token and the tenant signing public key. <FIG> illustrates tenant revocation manifest <NUM> which may be generated at S835 according to some embodiments. Tenant revocation manifest <NUM> includes block header <NUM>, signed manifest header <NUM> and manifest <NUM> (i.e., the revocation token). The manifest hash of signed manifest header <NUM> is generated based on manifest <NUM>. At S840, signature <NUM> is generated from the manifest hash using the owner's tenant revocation manifest signing certificate.

According to some embodiments, a revocation token is not generated and no revocation token is included in the tenant revocation manifest <NUM>. Rather, the tenant revocation manifest generated at S835 is global and may be used as described below to revoke tenancy from any hardware platform.

A tenant boot manifest is generated at S845. As illustrated in the example of <FIG>, manifest <NUM> of tenant boot manifest <NUM> may include the tenant grant manifest signing public key and the tenant revocation manifest signing public key. Tenant boot manifest <NUM> also includes signed manifest header <NUM> and unsigned manifest header <NUM>. Tenant boot manifest <NUM> is signed using the owner's boot manifest signing certificate at S845.

<FIG> and <FIG> illustrate chains of trust associated with the manifests and bootloader images described herein. <FIG> illustrates chain of trust <NUM> associated with owner-associated boot manifest <NUM>, owner-associated bootloader image <NUM> and owner-associated tenant revocation manifest <NUM>. In particular, owner-associated boot manifest <NUM> is signed with boot manifest signing certificate <NUM>, and a hash of the public key corresponding to boot manifest signing certificate <NUM> is a root of trust anchor <NUM> stored in secure form on the hardware platform. Boot manifest <NUM> becomes trusted upon comparison of root of trust anchor <NUM> with a public key hash of boot manifest <NUM>.

Bootloader image <NUM> is signed with bootloader signing certificate <NUM> and may be verified based on the bootloader signing public key contained in now-trusted boot manifest <NUM>.

Similarly, tenant revocation manifest <NUM> is signed with tenant revocation manifest signing certificate <NUM> and may be verified based on the tenant revocation manifest signing public key contained in now-trusted boot manifest <NUM>.

<FIG> illustrates chain of trust <NUM> associated with tenant-associated boot manifest <NUM>, tenant-associated tenant grant manifest <NUM>, tenant-associated bootloader image <NUM> and tenant-associated tenant revocation manifest <NUM>. As noted with respect to chain of trust <NUM>, a hash of the public key corresponding to boot manifest signing certificate <NUM> is stored in secure form on the hardware platform as a root of trust anchor <NUM>. Also similar to chain of trust <NUM>, tenant-associated boot manifest <NUM> is signed with boot manifest signing certificate <NUM> corresponding to root of trust anchor <NUM>. Accordingly, tenant boot manifest <NUM> also becomes trusted upon comparison of root of trust anchor <NUM> with a public key hash of tenant boot manifest <NUM>.

Tenant grant manifest <NUM> is signed with tenant grant manifest signing certificate <NUM> and may be verified based on the tenant grant manifest signing public key contained in now-trusted tenant boot manifest <NUM>. Bootloader image <NUM> is signed with tenant bootloader signing certificate <NUM> and may be verified based on the bootloader signing public key contained in now-trusted tenant grant manifest <NUM>. Tenant revocation manifest <NUM> is signed with tenant revocation manifest signing certificate <NUM> and may be verified based on the tenant revocation manifest signing public key also contained in tenant boot manifest <NUM>.

Returning to process <NUM>, the tenant boot manifest, the tenant grant manifest and the tenant revocation manifest are copied to the boot flash storage device of the hardware platform at S850. <FIG> illustrates motherboard <NUM> of <FIG> after tenancy grant provisioning according to process <NUM> according to some embodiments. Tenant boot manifest <NUM> has overwritten owner boot manifest <NUM> and tenant-signed bootloader image <NUM> has overwritten owner-signed bootloader image <NUM> within flash storage device <NUM>. Also shown are tenant grant manifest <NUM> and tenant revocation manifest <NUM> copies to their respective partitions of flash storage device <NUM>.

The hardware platform is then rebooted at S855 to continue the tenancy transfer. Flow therefore returns to S510 of process <NUM> and continues as described above. In particular, the unsigned header of the currently-stored boot manifest is loaded into RAM and the boot manifest signing public key is compared with the key stored on the hardware platform at S520. Assuming the comparison is successful, it is determined at S540 whether the boot manifest header was signed with the owner's boot manifest signing certificate. As described above, since the tenant boot manifest stored in the flash storage device during tenancy grant provisioning includes the boot manifest signing public key and was signed with the owner's boot manifest signing certificate, flow proceeds through S520 and S540 to S550 to load the unsigned manifest header and the signed manifest header of the tenant boot manifest into RAM.

A tenancy mode is determined at S560 based on the parity of the value stored in the tenancy counter. The counter has not yet been incremented in the present example, and therefore the parity indicates an owner mode. Flow therefore proceeds to S565, at which the flag of the stored boot manifest (i.e., the tenant boot manifest) is determined to indicate a "tenant" boot manifest type. Flow therefore proceeds to S575 to execute a tenancy transfer flow.

<FIG> illustrates process <NUM> of a tenancy transfer flow according to some embodiments. At S1710, it is determined whether the tenant grant manifest header is signed with the owner's tenant grant manifest signing certificate. S1710 may therefore include loading the tenant grant manifest signed header and block header into RAM, decrypting the signature of the block header using the owner's corresponding tenant grant manifest signing public key, and comparing the result to the manifest hash stored in the tenant grant manifest signed header. Upon verification of the signature, flow proceeds to S1730 to generate a grant token based on the tenant bootloader signing public key, the device secret of the hardware platform and on a current value of the tenancy counter. The hardware platform generates the grant token as illustrated in <FIG> and compares the generated token to the grant token of the stored tenant grant manifest at S1740. If the tokens match, the tenancy counter is incremented by <NUM> at S1750. In this regard, RoT <NUM> executes process <NUM> via execution of code stored in ROM <NUM> and, since RoT has not yet exited ROM <NUM>, RoT <NUM> is permitted to increment the tenancy counter value stored in storage device <NUM>.

The hardware platform is rebooted again at S1760. In response, S510 through S550 of process <NUM> are executed as described above in view of the stored tenant boot manifest. At S560, it is determined that the parity of the tenancy counter value is odd, by virtue of the incrementing performed at S1750 of process <NUM>. The odd value corresponds to a tenancy mode, causing flow to proceed from S560 to S580. At S580, it is determined (as above) that the stored boot manifest is of "tenant" type and flow proceeds accordingly to S585 to execute a tenant secure boot flow.

<FIG> illustrates process <NUM> of a tenant secure boot flow according to some embodiments. S1810 through S1840 proceed identically to that described above with respect to S1710 through S1740 in order to verify that the tenant grant manifest was signed by the owner and is associated with the present hardware platform. Next, at S1850, the signed and unsigned headers of the stored bootloader image are loaded. The signature of the unsigned header is then verified against the signed header at S1860 based on the bootloader signing public key which is stored in the now-verified tenant grant manifest.

If the verification at S1860 is successful, the stored bootloader image is loaded into RAM at S1870 and execution thereof begins at a corresponding entry point. The tenant may thereafter control operation of the hardware platform as desired.

Tenancy revocation may be performed by a current tenant or by the owner of the hardware platform. Tenancy revocation is initiated by copying an owner-signed owner-type boot manifest, an owner-signed bootloader, and the tenant revocation manifest to the flash storage device. The tenant revocation manifest may have been previously copied to the flash storage device during tenancy grant provisioning as described above, in which case it need not be re-copied. In a case that the tenant wishes to initiate tenancy revocation, the owner-signed owner boot manifest and owner-signed bootloader may be acquired from the owner. The hardware platform is then rebooted to complete tenancy revocation.

Upon reboot, process <NUM> executes as described above to verify the boot manifest signing public key of the stored boot manifest at S520 and the boot manifest signature at S540. A tenant tenancy mode is determined at S560 because the tenancy counter remains at the same value which existed after completion of the last tenancy grant. Then, at S580, it is determined that the stored boot manifest (which is the owner boot manifest copied during tenancy revocation provisioning) is an "owner"-type boot manifest. Flow therefore proceeds to S590 to execute a tenancy revocation flow.

<FIG> illustrates process <NUM> of a tenancy revocation flow according to some embodiments. At S1910, it is determined whether the tenant revocation manifest header is signed with the owner's tenant revocation manifest signing certificate. For example, the tenant revocation manifest signed header and block header are loaded into RAM, the signature of the block header is decrypted using the owner's corresponding tenant revocation manifest signing public key, and the result is compared to the manifest hash stored in the tenant revocation manifest signed header.

Assuming verification is successful, flow proceeds to S1930 to generate a revocation token based on the device secret of the hardware platform and on a current value of the tenancy counter. The hardware platform may generate the revocation token as illustrated in <FIG>, except the actual tenancy counter value, rather than tenancy counter+<NUM>, is used as an input to key derivation function <NUM>, because the tenancy counter has been incremented since the revocation token of the tenant revocation manifest was originally generated. The generated token is compared to the revocation token of the stored tenant revocation manifest at S1940. If the tokens match, the tenancy counter is incremented by <NUM> at S1950 and the hardware platform is rebooted at S1960.

Upon reboot, the hardware platform has returned to owner mode (i.e., the value of the tenancy counter is odd) and includes an owner-type boot manifest. Accordingly, flow proceeds through process <NUM> and process <NUM> as described above.

<FIG> illustrates datacenter <NUM> according to some embodiments. Datacenter <NUM> includes M server racks, each of which is associated with a dedicated fabric controller <NUM> - <NUM>. Orchestrator <NUM> communicates with fabric controllers <NUM> - <NUM>, and a given fabric controller communicates with each of N hardware platforms (e.g., server motherboards) mounted within a corresponding server rack. The number of hardware platforms per rack need not be identical.

Any or all of the hardware platforms of datacenter <NUM> may include an RoT having ROM-stored program code which is executable to provide the functions described herein to its associated hardware platform. Tenancy of such hardware platforms may therefore be securely transferred between an owner and successive tenants as described herein.

The foregoing diagrams represent logical architectures for describing processes according to some embodiments, and actual implementations may include more or different components arranged in other manners. Other topologies may be used in conjunction with other embodiments. Moreover, each component or device described herein may be implemented by any number of devices in communication via any number of other public and/or private networks. Two or more of such computing devices may be located remote from one another and may communicate with one another via any known manner of network(s) and/or a dedicated connection. Each component or device may comprise any number of hardware and/or software elements suitable to provide the functions described herein as well as any other functions.

All processes mentioned herein may be embodied in processor-executable program code read from one or more of non-transitory computer-readable media, such as ROM, RAM, a hard disk drive, a solid-state drive, and a flash drive, and then stored in a compressed, uncompiled and/or encrypted format. In some embodiments, hard-wired circuitry may be used in place of, or in combination with, program code for implementation of processes according to some embodiments. Embodiments are therefore not limited to any specific combination of hardware and software.

Claim 1:
A system (<NUM>) comprising:
a first storage device (<NUM>) storing a boot manifest (<NUM>), the boot manifest comprising a manifest type flag;
a second storage device (<NUM>) storing a counter value; and
a microcontroller (<NUM>) in communication with the first storage device and the second storage device, the microcontroller to perform a boot process, and configured to:
determine a(S565/S580) manifest type of the boot manifest based on the manifest type flag;
determine (S560) a tenancy mode based on a parity of the counter value;
execute an owner secure boot flow (S570) if the manifest type is an owner manifest type and the tenancy mode is an owner tenancy mode;
execute a tenancy revocation flow (S575) if the manifest type is the owner manifest type and the tenancy mode is a tenant tenancy mode;
execute a tenancy transfer flow (S585) if the manifest type is a tenant manifest type and the tenancy mode is the owner tenancy mode; and
execute a tenancy secure boot flow (S590) if the manifest type is the tenant manifest type and the tenancy mode is the tenant tenancy mode.