Patent Description:
Embodiments of the present disclosure include systems and methods for processor virtualization, and more particularly, to systems and methods to generate a chip-level security runtime environment by implementing a virtualization layer when the processor boots up.

Generally, platform virtualization may be used to separate software from underlying hardware resources. For example, using KVM (Kernel-based Virtual Machine) or XEN, multiple virtual machines can be executed on the same hardware concurrently, allowing the isolation of software and formation of security layers. In typical virtualization solutions, however, the different virtual machines may share hardware resources. Virtual machines may be operating with different operating systems, but the virtual machines may still share processor resources. This arrangement may expose users to attacks that exploit device layer processes, such as Spectre and Meltdown.

The disclosed systems and methods for processor virtualization and their hardware components are directed to improvements in the existing technology.

One aspect of the present disclosure is directed to a system for secure processor virtualization. The system may include a secure initialization memory including initialization instructions for launching a security runtime environment before operating systems and cryptographic keying for security handoffs. The system may also include one or more processors coupled to the secure initialization memory and configured to perform operations. The operations may include retrieving the initialization instructions from the secure initialization memory at startup, executing the initialization instructions to launch the security runtime environment and retrieve at least a portion of the cryptographic keying from the secure initialization memory, and generating specific keying for chip-level resources in the one or more processors by combining instruction sets of the chip-level resources and the cryptographic keying. The operations may also include initializing a plurality of isolated enclaves on the security runtime environment and pinning the chip-level resources to the plurality of enclaves according to the specific keying and by establishing exclusive or independent cryptographic links between the chip-level resources and the plurality of enclaves.

Another aspect of the present disclosure is directed to a computer-implemented method for processor virtualization. The method may include retrieving (from a secure initialization memory) initialization instructions for launching a security runtime environment at startup before initializing operating systems, executing the initialization instructions to launch the security runtime environment and retrieve at least a portion of cryptographic keying for security handoffs from the secure initialization memory, and generating specific keying for chip-level resources in one or more processors by combining instruction sets of the chip-level resources and the cryptographic keying. Further, the method may also include initializing a plurality of isolated enclaves on the security runtime environment and pinning the chip-level resources to the plurality of enclaves according to the specific keying and by establishing exclusive cryptographic links between the chip-level resources and the plurality of enclaves.

Yet another aspect of the present disclosure is directed to a non-transitory computer-readable medium storing instruction that, when executed by a processor, perform operations for processor virtualization. The operations may include retrieving initialization instructions for launching a security runtime environment at startup before initializing operating systems, executing the initialization instructions to launch the security runtime environment and retrieve at least a portion of cryptographic keying for security handoffs, and generating specific keying for chip-level resources in one or more processors by combining instruction sets of the chip-level resources and the cryptographic keying. Further, the operation may also include initializing a plurality of isolated enclaves on the security runtime environment and pinning the chip-level resources to the plurality of enclaves according to the specific keying and by establishing exclusive cryptographic links between the chip-level resources and the plurality of enclaves.

Reference will now be made in detail to the exemplary embodiments of the present disclosure described above and illustrated in the accompanying drawings.

The disclosure is directed to systems and methods that may create a chip-level SRE with a virtualization layer separating hardware from software. In some embodiments, the virtualization layer may be set up with a validated initialization process run from a dedicated memory (e.g., a secure initialization memory) with pre-stored virtualization instructions that get executed early during processor start up, before operating systems (OS). In some embodiments, the validated initialization process may include an enforced boot early in a customized BIOS boot process. The virtualization layer may be configured to pin chip resources to specific virtual cores, so users can create multiple workflows that run in isolated domains. With such configurations, the disclosed systems and methods may provide a full-stack security solution that protects from malicious events and minimize exposure to side attacks.

In some embodiments, the disclosed systems and methods may mitigate cyber vulnerabilities, such as speculative execution risk and zero-day attacks, by isolating on-chip hardware components. Moreover, the disclosed systems and methods may provide a platform for cryptographically bound signed code that may only execute the permitted user applications and may be resistant to unauthorized modifications. Further, the creation of a chip-level SRE to host different enclaves or tenants may mitigate information leakage outside of isolated domains and may allow a trusted compartmentalization for user applications. In some embodiments, the disclosed systems and methods may mitigate vulnerabilities in cache attacks, such as Translation Lookaside Buffer (TLB) and Branch Target Buffer (BTB), by isolating Last Level Cache (LLC) and/or timing behaviors between security domains. Moreover, the disclosed systems and methods may separate high-value software content from non-trusted or verified security workloads among shared processor resources.

The disclosed systems and methods may also mitigate risk and safety challenges within highly shared core, memory, interrupts, cache, and Direct Memory Access (DMA) resources. In some embodiments, the disclosed systems and methods may allow users to separate high-value software content from non-trusted or verified security workloads through the hardware firewalling of shared resources. Hardware isolation techniques may be used to create security domains within a protected operating environment. According to embodiments of the present disclosure, processor-shared resources may be firewalled, and runtime security domains may be equipped with cross domain protection against leakage, modification, and privilege escalation.

In some embodiments, methods of the present disclosure may include a mandatory protected boot in which hardware components may be partitioned for resource isolation. By performing the partition early on, the disclosed methods may reduce total code base (TCB) exposure. Other virtualization techniques may use bloated open sourced and insecure code. Methods of the present disclosure may solve issues present in these virtualization techniques because the disclosed methods may minimize exposure and potential attacks. Moreover, the disclosed methods and systems may minimize potential attacks based on hardware vulnerability by eliminating or disabling Option ROM, debug ports, foreign driver code, and/or System Management Modes. Further, the disclosed methods may incorporate cryptographical techniques to enhance security features. In some embodiments, processors may be configured to include a PCI Express manifest that may cryptographically confirm white listing and blacklisting of devices. Such confirmation may support attestation and encrypted communication, may enable trusted guest virtualization for user applications and software portability, and may improve integrity control of hardware and software for critical systems.

The disclosed systems and methods may improve processor security by providing boot and runtime protection of critical applications. The disclosed systems and methods may mitigate vulnerabilities by implementing a virtualization layer when the processor boots up, or close to boot up. Further, the disclosed systems and methods may facilitate an improved secure software stack by creating a secure environment that runs early from a secure initialization memory, such as the BIOS.

The disclosed systems and methods may also improve how the computer operates by creating isolated hardware resource domains that may separate the software from the hardware. The disclosed systems and methods may allow the computer to deploy a layer of security, so users can run multiple tenant workloads inside the same processor and may have one or multiple levels of separation from each other. This separation may improve the computer operation by enhancing security features and reducing exposure to potential attacks that seek to collect information or identify behaviors. Moreover, the disclosed systems and methods may allow the computer to perform new operations, such as hosting virtual machines using a hypervisor without a scheduler and with ring separation for security, pinning of the virtualization host kernel within cache memory of the processor cores, and passing unique cryptographic information to each individual virtual machine guest during the boot process.

For example, some embodiments of the disclosed systems and methods may allow the virtualization of multi-core processors that comply with safety and/or security certifications without Quality of Service (QoS) detriments. Certain industrial control systems require safety certifications that guarantee system integrity. In response, some security and/or safety solutions have opted to comply with certification conditions by preventing parallelization and, for example, disabling cores of multi-core processors. These shortcut solutions, however, result in QoS decrements and higher costs to meet certification requirements for safety-critical applications. The disclosed systems and methods provide tools for processor virtualization that facilitate meeting safety certifications without losing QoS capabilities because the disclosed systems and methods provide hardware isolation by exclusively mapping or pinning resources to specific workflows. Isolated workflows and processor components achieve the goals of safety and security without losing processing resources. Therefore, the disclosed systems and methods may facilitate separation of multi-core processors that meet safety certification requirements.

Moreover, the disclosed systems and methods may improve the technical field of processor security by providing initialization and/or boot protections. For instance, in accordance with an embodiment of the present disclosure, a system may create a chip-level security runtime environment (SRE) by implementing a virtualization layer before any OS starts interacting with chip-level resources. In one embodiment, the SRE may implement the visualization layer when the processor boots up. The system may include one or more processors and one or more storage devices storing instructions that, when executed, may configure the one or more processors to perform operations. The operations may include retrieving, from a secure initialization memory, a cryptographic keying for a security handoff and instructions for validated enforced boot, launching a protected SRE to operate the processor based on the initialization operations, pinning chip resources to specific virtual cores that work in isolated domains in the SRE, and binding user applications to the virtualized cores using hardware resource IDs to perform isolated workflows. In some embodiments, the operations may also include enforcing firewall policies to access processor resources and disabling processor features, such as option ROM, system management mode, and/or debug ports.

<FIG> is a block diagram of an exemplary system <NUM>, consistent with disclosed embodiments. As shown in <FIG>, system <NUM> may include a motherboard and peripherals.

The motherboard may include a processor <NUM>. In some embodiments, processor <NUM> may include any suitable processing device and/or a commercially available processor. In other embodiments, processor <NUM> may be a plurality of devices coupled together and configured to perform functions consistent with the present disclosure. For example, processor <NUM> may include a plurality of co-processors or graphical processing units, and each may be configured to run specific operations, such as floating-point arithmetic, graphics, signal processing, string processing, cryptography, or I/O interfacing. Processor <NUM> is further described in connection with <FIG>.

The motherboard may also include a co-processor <NUM>. Co-processor <NUM> may supplement the functions of processor <NUM>. In some embodiments, co-processor <NUM> may perform operations of floating-point arithmetic, graphics, signal processing, string processing, cryptography, or I/O interfacing with peripheral devices. Co-processor <NUM> may offload processor-intensive tasks from processor <NUM>, which may accelerate system <NUM> performance. In some embodiments, co-processor <NUM> may have customized operations. For example, co-processor <NUM> may run customized operations to initialize the SRE and/or enforce firewalls between isolated domains.

The motherboard may also include a clock generator <NUM> and a DRAM (dynamic random-access memory) <NUM>. In some embodiments, clock generator <NUM> may include an electronic oscillator configured to produce a timing signal. Clock generator <NUM> may produce a symmetrical wave and/or more complex arrangements. Clock generator <NUM> may also include a resonant circuit and an amplifier. In some embodiments, clock generator <NUM> may include one or more frequency dividers, clock multiplier sections, and programmable clocks. Clock generator <NUM> may be configured during a BIOS boot time to the selected value.

DRAM <NUM> may include a solid-state memory used for processor <NUM> operations. Persons of ordinary skill in the art would appreciate that DRAM <NUM> may include multiple types of memories and may not be limited to random-access memory. In some embodiments, for example, DRAM <NUM> may include read-only memory.

The motherboard may also include BIOS <NUM>. BIOS <NUM> may include non-volatile firmware used to perform hardware initialization during the booting process (power-on startup), and to provide runtime services for operating systems and programs. In some embodiments, BIOS <NUM> may include instructions to create an SRE and the isolated hardware domains. BIOS <NUM> may also include cryptographic keying for security handoffs when launching the protected SRE.

In some embodiments, BIOS <NUM> firmware may be pre-installed on a computer by an OEM (original equipment manufacturer) and may be configured to be the first software to run when the computer is powered on. In some embodiments, BIOS <NUM> may initialize and test the system hardware components. BIOS <NUM> may also load a boot loader from a mass memory device, which may then initialize a hypervisor with the virtualization layer that creates the isolated domains. Further, BIOS <NUM> may include cryptographical keys or instructions to connect virtual machines hosted in the SRE with isolated hardware resources in processor <NUM>.

The motherboard may also include a power reset <NUM>, which may be configured to reinitialize components of the motherboard and/or end current operations in system <NUM>. In some embodiments, power reset <NUM> may initialize components, and BIOS <NUM> may restart system <NUM> and deploy the virtualization layer and/or the SRE.

The motherboard may also include one or more bridge components to communicate with the peripherals. In some embodiments, the motherboard may include a USB (Universal Serial Bus) bridge <NUM>, a PCI (Peripheral Component Interconnect) bus bridge <NUM>, a SCSI (Small Computer System Interface) bus bridge <NUM>, and an EISA (Extended Industry Standard Architecture) bus bridge <NUM>.

The motherboard may include video RAM <NUM> and video processor <NUM>. Video RAM <NUM> may include a buffer between processor <NUM> and a display. In some embodiments, video RAM <NUM> may be implemented with a frame buffer so when images are to be sent to the display, they may be first read by the processor <NUM> as data from a form of main (non-video) RAM and then written to video RAM <NUM> in preparation for display. Video processor <NUM> may be implemented with an expansion card which may generate a feed of output images to a display device. In some embodiments, video processor <NUM> may include dedicated graphics cards and/or graphics processing unit (GPU).

In addition, the motherboard may include a network bus <NUM>. Network bus <NUM> may provide connectivity to any type of network configured to provide communications between the motherboard, system <NUM>, and other components that may communicate or be coupled with the motherboard. In some embodiments, network bus <NUM> may be a port for connection with any type of network that may provide communications, exchange information, and/or facilitate the exchange of information, such as the Internet, a Local Area Network, near field communication (NFC), optical code scanner, or other suitable connection(s) that facilitates the sending and receiving of information in system <NUM>.

As shown in <FIG>, the peripherals may include a monitor <NUM>, a keyboard <NUM>, and a mouse <NUM>. The peripherals may also include a hard drive <NUM>, external storage <NUM>, a modem <NUM>, and PCI slots <NUM>. Embodiments of the present disclosure also contemplate that system <NUM> may include any other suitable peripheral devices.

Persons of ordinary skill in the art would appreciate that the configuration and boundaries of the functional building blocks of system <NUM> have been illustrated herein for the convenience of the description. Embodiments of the present disclosure contemplate that alternative boundaries may be implemented so long as the specified functions and relationships thereof are appropriately performed. Alternatives (including equivalents, extensions, variations, deviations, etc., of those described herein) will be apparent to persons of ordinary skill in the art based on the teachings contained herein.

<FIG> shows a block diagram of an exemplary processor <NUM>, consistent with disclosed embodiments. Processor <NUM> may include one or more cores <NUM>, one or more icaches <NUM>, one or more dcaches <NUM>, and one or more L2 caches <NUM>. Further, processor <NUM> may include one or more memory controllers <NUM> and a platform cache <NUM>.

Cores <NUM> may be configured to perform parallel tasks to enhance efficiency of processor <NUM>. In some embodiments, cores <NUM> may be physically distinct cores. In other embodiments, cores <NUM> may be defined virtually with multithreading or hyper-threading that may split processing units into virtual cores.

As shown in <FIG>, a first level of cache memory may be divided in processor <NUM> having icache <NUM> and dcache <NUM>. icache <NUM> may include instruction cache that may contain pre-code to demarcate individual instructions for cores <NUM>. In some embodiments, icache <NUM> may include instructions to improve decode speed. dcache <NUM> may be a data cache. With this split between icache <NUM> and dcache <NUM>, two small caches may exist, one exclusively caching instruction code and the other exclusively caching data. Software may separate code from data (global and static variables, constants, etc.). This arrangement may create a spatial separation between the actual instruction code, the hard-coded data, and dynamically allocated data, to facilitate data processing.

L2 cache <NUM> may include level <NUM> cache with an increased capacity than icache <NUM> or dcache <NUM>. L2 cache <NUM> may serve as a bridge for the process and memory performance gap and may provide stored information to processor <NUM> without any interruptions or wait-states. L2 cache <NUM> may also be configured to reduce the access time of data. For example, L2 cache <NUM> may reduce the access time of data in events wherein the data may have already accessed before, so that the data may not need to be loaded again. In some embodiments, L2 cache <NUM> may perform buffering operations and may request data from the memory, serving as a closer waiting area compared to RAM.

Memory controllers <NUM> may include digital circuits that may manage the flow of data going to and from processor's <NUM> memories to processing units, such as cores <NUM>. In some embodiments, processor <NUM> may be configured to have a unique cache, or group of caches, associated with each one of cores <NUM>. Additionally, and/or alternatively, hardware components may be shared between different cores. In order to prevent leakage attacks, a virtualization layer may configure memory controllers <NUM> to isolate and segregate memory sections for only one, or a selected group, of cores <NUM>. Memory controllers <NUM> may be implemented as integrated memory controller (IMC) and/or a memory chip controller (MCC).

In some embodiments, processor <NUM> may include platform/L3 cache <NUM> to provide a higher level of cache that may facilitate distribution of information between the different cores <NUM>. In some embodiments, platform/L3 cache <NUM> may provide a higher-level cache that may store descriptors, keys, contexts, and other data needed for network packet processing. Such configurations may allow processor <NUM> to keep data-plane traffic out of external memories or peripherals. The virtualization layer of the SRE implemented from BIOS <NUM> may divide registers of platform/L3 cache <NUM> in specific isolated domains to mitigate vulnerability to certain attacks.

Processor <NUM> may also include one or more modules that may manage or control operations in processor <NUM>. In some embodiments, processor <NUM> may include a validated enforced boot <NUM>, an isolation manager <NUM>, a flash controller <NUM>, and a power management <NUM>.

Validated enforced boot <NUM> may be a module that forces a boot using only software that is trusted by the OEM. When the PC starts, validated enforced boot <NUM> firmware may check the signature of each piece of boot software, including, for example, BIOS firmware drivers (such as Option ROMs), EFI (Extensible Firmware Interface) applications, and the operating system. If the signatures are valid, processor <NUM> may boot, and the firmware may provide control to the operating system. If the signatures are invalid, the processor <NUM> may generate an alert. Isolation manager <NUM> may be configured to bind software application, tenants, and/or enclaves with specific hardware components isolated from each other. In some embodiments, isolation manager <NUM> may configure a multi-socket system creating a virtual trusted platform module (TPM) that may bind processor <NUM> resources to individual sockets and bridge chips. Because isolation manager <NUM> may pin hardware resources for specific applications or tenants, a virtualization layer configured to operate processor <NUM> may implement a hypervisor without a scheduler.

Flash controller <NUM> may be used to interface and operate a flash card. In some embodiments, flash controller <NUM> may operate in low duty-cycle environments, such as, for example, SD cards, CompactFlash cards, or other similar media. Power management <NUM> may be configured to control one or more components of processor <NUM> to enforce power management policies and/or disable certain elements. In some embodiments, power management <NUM> may perform demand-based switching (DBS) to minimize power consumption, activate turbo modes to enhance performance, and/or execute power mode managerial tasks. Moreover, and as further discussed in connection to <FIG>, power management <NUM> may disable certain elements of processor <NUM> as required by security levels of potential security concerns.

In some embodiments, processor <NUM> may include additional modules, such as external capacity module <NUM>, UART (Universal Asynchronous Receiver Transmitter) module <NUM>, SPI (Serial Peripheral Interface) module <NUM>, and USB module <NUM>.

Processor <NUM> may also include a cache coherency module <NUM> including one or more input-output memory management units (IOMMU). Cache coherency module <NUM> may configure cache memories in processor <NUM> to have shared resources. When clients in a system maintain caches of a common memory resource, problems may arise with incoherent data, particularly in a multiprocessing system. Cache coherence module <NUM> may manage such conflicts by maintaining a coherent view of the data values in multiple caches. The IOMMU in cache coherency module <NUM> may connect a direct-memory-access I/O bus to the main memory. IOMMU may translate CPU-visible virtual addresses to physical addresses and may map device-visible virtual addresses (also called device addresses or I/O addresses in this context) to physical addresses. Further, IOMMU may protect cache memories in processor <NUM> from faulty or malicious devices. In some embodiments, firmware stored in BIOS <NUM> (<FIG>) may manipulate IOMMU to segregate resources and create isolated domains with dedicated hardware components from processor <NUM>.

As shown in <FIG>, processor <NUM> may also include a management complex including a module PME (Power Management Event) <NUM>. In some embodiments, PME <NUM> may be configured to facilitate a BIOS setup utility and/or facilitate power for the network card when the system is shut down. BIOS <NUM> may be configured to run an SRE shutdown that may unpin resources from the isolated domains. Moreover, the management complex may include a DCE (Distributed Codec Engine) <NUM>, a security controller <NUM>, a buffer management <NUM>, and an I/O processor <NUM>.

Processor <NUM> may also include a buffer <NUM> configured to control data in cache coherency module <NUM>, a L2 switch <NUM> configured to control exchanges between, for example, L2 Cache <NUM> and dcache <NUM>, and an acceleration module <NUM>. Processor <NUM> may include a serializer/deserializer <NUM>, one or more PCI cards <NUM>, and one or more SATA (Serial Advanced Technology Attachment) modules <NUM>. Further, processor <NUM> may include a module for SR-IOV (Single Root I/O Virtualization) <NUM>.

<FIG> shows a block diagram of an exemplary enterprise security environment <NUM>, consistent with disclosed embodiments. Enterprise security environment <NUM> may include one or more processors <NUM>, one or more secure initialization memories <NUM>, and a protected environment <NUM>. In some embodiments, secure initialization memories <NUM> may include one or more BIOS memories that may store a customized boot (e.g., customized booting instructions) to launch an SRE. Additionally, and/or alternatively, secure initialization memories <NUM> may be memory sectors that are pre-configured to have processors <NUM> execute instructions before starting general operating systems or multilevel security systems. For example, in addition to BIOS memories, secure initialization memories <NUM> may include ROM memories, RAM memories, or cache memories with instructions for initialization before activating OSs.

As further discussed in connection with <FIG>, secure initialization memories <NUM> may include and/or store initialization instructions for launching an SRE as the first task, or close to the first task, that processor <NUM> perform at start up. For example, secure initialization memory <NUM> may include instructions to launch an SRE before any OS (including general purpose OS, multi-user OS, and/or multi-level security (MLS) system) gets executed. Moreover, secure initialization memories <NUM> may further include cryptographic keying for security handoffs and/or cryptographic information for establishing communication links with chip-level resources of processors <NUM>.

As shown in <FIG>, the aforementioned layers may support a secured enclave <NUM> and one or more hardened enclaves <NUM> and may communicate with I/O devices <NUM>. Moreover, in some embodiments, secured enclave <NUM> may include a secure OS <NUM>. In some embodiments, one or more of processors <NUM> (<FIG>) may be implemented as processors <NUM> in enterprise security environment <NUM>. Moreover, one or more of BIOS <NUM> (<FIG>) may implement secure initialization memories <NUM> in enterprise security environment <NUM>.

In enterprise security environment <NUM>, booting processors <NUM> may execute a virtualization initialization or a validated enforced boot, which allows for initializing and executing SRE virtualization early. Secure initialization memories <NUM> may include cryptographic information to verify cryptographic integrity of device drivers, option ROMS, and SRE virtualization. Secure initialization memories <NUM> may use key derivation functions, which uses unique hardware identification, which allow for the SRE virtualization to validate root of trust during execution. Secure initialization memories <NUM> may be programmed with a customized boot that may pin resources in processors <NUM> to certain virtualized environments. In some embodiments, secure initialization memories <NUM> may pin portions of cache memory, cores, and/or IOMMUs, in processors <NUM> to specific secured domains. Moreover, secure initialization memories <NUM> may provide cryptographic keying for a security handoff and can delay executing of specific phases until SRE virtualization execution, therefore virtualizing phases of secure initialization memories <NUM> into secured enclave <NUM> containers. This provides the benefit of lowering the total code base (TCB exposure) of secure initialization memories <NUM> where delayed execution can happen securely in a secured enclave <NUM>.

A validated initialization from secure initialization memories <NUM> may launch protected environment <NUM>, which may allow users to isolate processors <NUM> resources for enclaves, such as secured enclave <NUM> and/or hardened enclaves <NUM>. In some embodiments, protected environment <NUM> may create isolated domains that may protect processors <NUM> from cross-domain attacks. Protected environment <NUM> may protect processors <NUM> from leakage, modification, and privilege escalation attacks.

Protected environment <NUM> may create the isolated domains with specific processors <NUM> resources by binding user applications to virtualized guests using the PCH (Platform Controller Hub) and each processor's unique identification. Protected environment <NUM> may establish a multi-socket system that allocates resources from processors <NUM> to corresponding enclaves, virtual machines, kernels, and/or user applications. In addition, protected environment <NUM> may create and operate the isolated domains by manipulating interfacing elements that control data flow between enclaves and processors <NUM> resources. In some embodiments, protected environment <NUM> may manipulate addresses and/or registers of bridging chips or PCIe cards to isolate processors <NUM> resources. Furthermore, protected environment <NUM> may create the isolated domains by establishing chip-level cryptographic links that employ processors <NUM> entropy to define cryptographic functions. For instance, protected environment <NUM> may create isolated domains by assigning hardware resource IDs to specific enclaves and cryptographically securing the connections.

Moreover, to protect and create isolated domains, protected environment <NUM> may enforce on-chip firewall policies. In some embodiments, protected environment <NUM> may directly map a PCIe device cryptographically to a specific virtual machine guest or an application tenant. Protected environment <NUM> may implement rules or policies that may restrict access to certain resources based on the rules setup by the firewall. Further, protected environment <NUM> may implement a type <NUM> or type <NUM> hypervisor that has a built-in security policy to restrict access to processors <NUM> resources, protecting the individual, core, and memory resources of processors <NUM>. In some embodiments, protected environment <NUM> may implement firewall policies based on header processing and/or payload processing. Protected environment <NUM> may involve layered protocol wrappers or content addressable memory filters to associate processing requests from specific enclaves and determine whether they have access to specific resources or route them to a specific domain. Further, protected environment <NUM> may perform regular expression matching or implement a payload scanner to determine the access or route of the processing requests.

Protected environment <NUM> may include a flow buffering configured to control data flow between enclaves and processors <NUM> resources. In some embodiments, protected environment <NUM> may identify process requests and may use a queue manager based on enclave information. If a process requires use of SRAM, protected environment <NUM> may apply firewall and may consult a state table to determine, which (if any) SRAM registers or blocks can be accessed by the requesting enclave. Further, protected environment <NUM> may implement counters to track the number of requests and identify suspicious behavior.

In some embodiments, protected environment <NUM> may operate without a scheduler. Protected environment <NUM> may pin processors <NUM> resources to specific and/or unique applications or independent enclaves. Once a workflow is pinned to processors <NUM> resources, the workflow may stay with the resource to avoid generating side effects that may reveal data to attackers. By preventing sharing of resources, protected environment <NUM> may minimize exposure from cross-domain attacks. In some embodiments, protected environment <NUM> may be inserted into a scheduling queue for services. Further, protected environment <NUM> may include a combination of features, including having some resources of processors <NUM> being isolated and without access from other enclaves, and some other resources having some sharing ability depending on firewall policies.

Moreover, and as further described in connection with <FIG>, protected environment <NUM> may determine a security level for different workflows or enclaves and may disable or enable features of processors <NUM> according to the level of security or required execution flow. In some embodiments, protected environment <NUM> may disable or isolate the Option ROM, the System Management Mode, and the debug ports to reduce vulnerabilities that arise from sharing resources or typical execution flows.

In addition, and as further described in connection with <FIG>, protected environment <NUM> may also perform cryptographic methods to, for example, pass unique cryptographic information to each individual virtual machine or enclave. In some embodiments, pinning hardware resources from processors <NUM> to virtual machines may include a cryptographical link between processors <NUM> and each one of the tenants or guests in enterprise security environment <NUM>. Moreover, protected environment <NUM> may create cryptographic links between processor interfacing resources and tenants, such as virtual machines. In some embodiments, protected environment <NUM> may achieve the secure connection by directly mapping a PCI express device cryptographically to a particular Virtual Machine (VM) guest.

Secure initialization memories <NUM> may also perform cryptographical tasks when setting up enterprise security environment <NUM>. In some embodiments, and as further described in connection with <FIG>, during the initialization or boot process, secure initialization memories <NUM> may pass cryptographic information, such as keys or key generators, to resources of processors <NUM> or storage elements that would interact directly with virtual machines or enclaves.

Processors <NUM>, secure initialization memories <NUM>, and protected environment <NUM> may service hardened enclaves <NUM> and secured enclaves <NUM>. Hardened enclaves <NUM> may include virtual machines using proprietary or open-source operating systems. In some embodiments, hardened enclaves <NUM> may use Linux operating systems such as RedHat RHEL®, Canonical Ubuntu® and/or Windows® operating systems. Further, hardened enclaves <NUM> may operate software platform for cloud computing, including interrelated components, which may control diverse, multi-vendor hardware pools of processing, storage, and networking resources throughout data centers. Moreover, hardened enclaves <NUM> may run unified threat management (UTM), intrusion detection systems (IDS) and intrusion prevention systems (IPS), such as, for example, SNORT® and/or SURICATA®.

Secured enclaves <NUM> may include a secure OS <NUM>. In some embodiments, secure OS <NUM> may include a reduced OS that may be limited to necessary functions for specific tasks. To minimize the threat surface, secured enclaves <NUM> may operate with customized or isolated operating systems. In some embodiments, secured enclaves <NUM> may operate with a single-tenant physical server so that server resources may not be shared between customers. Further, secured enclave <NUM> may run workflows for the user with dedicated hardware entirely isolated for virtual machines or tenants. Additionally, and/or alternatively, secured enclave <NUM> may run a bare-metal OS configuration for the separation of tasks with custom OS implementations.

In some embodiments, enterprise security environment <NUM> may be implemented with a reduced number of lines of code to minimize exposed surface. For example, enterprise security environment <NUM> may be implemented with a minimalistic trusted code base which, in some embodiments may include no more than <NUM> million lines of code. When enterprise security environment <NUM> includes a secured enclave <NUM> with a secure OS <NUM> configured as a limited or bare metal OS, the instructions in secure initialization memories <NUM> may include a minimalistic trusted code base to launch protected environment <NUM>, which in some embodiments may be <NUM>,<NUM> lines of code or less. Enterprise security environment <NUM> may minimize the trusted computing base (TCB) threat surface. By reducing the code exposed to attacks, enterprise security environment <NUM> may mitigate risks of attacks, such as speculative execution risk and zero-day attacks.

The arrangement of isolated hardware resources in enterprise security environment <NUM>, where processors <NUM> resources may be pinned to specific domains, may enhance the processor security because it may mitigate hardware-related vulnerabilities. In some embodiments, enterprise security environment <NUM> may minimize physical, cyberspace, and supply chain vulnerabilities. By way of example, enterprise security environment <NUM> may mitigate attacks, such as: Storage removal from system; Probing of interface; Glitching power; Side Channel Analysis; Memory removal; Compromised firmware; Scan-based read-back attack; USB / peripheral trust subversion; Brute force crypto attack / replay attack; Known Common Vulnerability Enumeration; Authentication Abuse; Authentication Bypass; Excavation; Buffer Manipulation; Flooding; Integrity Attack; Pointer Attack; Excessive Allocation; Resource Leak Exposure; Parameter Injection; Content Spoofing; Resource Location Spoofing; Reversion Engineering; Functionality Misuse; Code Injection; Command Injection; Protocol Manipulation; Man in the middle (MITM) Attack / Interception; Modification During Manufacturing; Modification During Distribution; and/or Hardware Trojan Insertion.

<FIG> illustrates a flow chart of an exemplary chip-level SRE configuration process <NUM>, consistent with disclosed embodiments. Configuration process <NUM> may be carried out by processor <NUM> (<FIG>). In some embodiments, other elements of system <NUM>, such as BIOS <NUM> and/or co-processor <NUM> (<FIG>), may execute configuration process <NUM>. Additionally, and/or alternatively, configuration process <NUM>, or some steps of the process, may be executed by multiple elements of system <NUM>. For example, configuration process <NUM> may be executed by BIOS <NUM> in conjunction with processor <NUM> (<FIG>).

Configuration process <NUM> may start at step <NUM>, in which processor <NUM>, or any element or combination of elements performing the process, may run startup operations from an initialization memory, such as secure initialization memories <NUM>. In some embodiments, in which secure initialization memories <NUM> include a BIOS, at step <NUM> processor <NUM> may perform a validated enforced boot from a BIOS, such as BIOS <NUM>. In some embodiments, step <NUM> may be mandatory, and any time processor <NUM> reboots, processor <NUM> performs step <NUM>. The mandatory initialization from secure initialization memory <NUM> (or validated enforced boot) may check specific hardware devices according to pre-configured validations. In some embodiments, an OEM may load secure initialization memories <NUM>, such as BIOS <NUM>, with a pre-configured validated enforced initialization or boot that may attempt to initialize different elements of system <NUM> and/or resources in processor <NUM>.

In some embodiments, at step <NUM>, processor <NUM> may review and confirm certificates stored in a boot loader before launching the boot process. Processor <NUM> may correlate signatures during startup. If processor <NUM> or BIOS <NUM> identifies inconsistencies during the boot process, a firmware may prevent the boot and issue an alert. In some embodiments, using a unified extensible firmware interface, processor <NUM> or BIOS <NUM> may identify the boot process as not reliable and stop the booting process. The signature verification may prevent malware from hijacking the boot process and concealing itself from the operating system.

When the validated enforced initialization or boot is successful at step <NUM>, configuration process <NUM> may continue to step <NUM>. At step <NUM>, configuration process <NUM> may retrieve cryptographic keying for a security handoff. In some embodiments, processor <NUM> may retrieve cryptographic keying from secure initialization memory <NUM> for future operations in the SRE. Alternatively, and/or additionally, processor <NUM> may retrieve the cryptographic keying from BIOS <NUM>. The cryptographic keying may be used for cryptographical tasks and/or later creation of isolated domains, such as establishing links between chip-level resources and enclaves. Additionally, and/or alternatively, the cryptographic keying may provide a mechanism to guarantee reliability of different steps during configuration process <NUM>. In some embodiments, the cryptographic keying may be provided by OEM BIOS devices. For instance, BIOS <NUM> may be an OEM BIOS that is pre-coded with instructions to initialize configuration process <NUM> and cryptographic keying for security hand-off with processor <NUM>.

At step <NUM>, processor <NUM> may launch a protected SRE based on the validated enforced boot of step <NUM> and the security handoff of step <NUM>. In some embodiments, the validated enforced boot may launch a protected SRE at step <NUM> that may allow a user to isolate resources of processor <NUM> for different applications. The SRE may include a protected environment <NUM> (<FIG>) in which chip-level resources may be isolated in different domains by pinning processor resources (e.g., cores, cache memory, and/or IOMMU), to specific application enclaves.

At step <NUM>, processor <NUM> may determine and assign isolated resources for workflows or user applications. In some embodiments, processor <NUM> may determine portions of cache memory that would be assigned to one enclave. Based on instructions in BIOS <NUM>, processor <NUM> may determine that L2 Cache <NUM>(a) would be assigned to a first enclave, while L2 Cache <NUM>(b) (<FIG>) would be a second and distinct enclave. At step <NUM>, processor <NUM> may determine that different cores would be assigned to specific enclaves, having, for instance, core <NUM>(a) for the first enclave and core <NUM>(b) for a second enclave.

Different assignment combinations may also be possible during the assignment and isolation of step <NUM>. In some embodiments, icache <NUM>, dcache <NUM>, platform cache <NUM>, memory controllers <NUM>, and/or IOMMU in cache coherency module <NUM> may also be portioned in different combinations and permutations to isolate domains for virtual machines and/or SRE tenants. Additionally, and/or alternatively, once a resource is allocated for one workflow or enclave, it may not be allocated again. In such embodiments, elements may be isolated during step <NUM> and may not share any assignation to prevent cross-domain attacks. Additionally, and/or alternatively, based on validated enforced boot instructions, processor <NUM> may include some functional sharing between resources of processor <NUM>. In some embodiments, for low security tasks, configuration process <NUM> may allow some shared resources between enclaves.

At step <NUM>, processor <NUM> may cryptographically pin resources to applications on enclaves based on the assignments determined at step <NUM>. In some embodiments, using keying retrieved during step <NUM>, processor <NUM> may establish cryptographic links between resources of processor <NUM> and enclaves supported by the SRE. In some embodiments, the connection may be performed using the PCH and each processor unique ID to create a multi-socket system that may allocate resources for corresponding VM guests, kernel, and/or applications. Additionally, and/or alternatively, the connection may be performed by manipulating interfacing elements. In some embodiments, at step <NUM>, processor <NUM> may manipulate the configuration of PCIe <NUM> to isolate processor resources and/or enforce firewall policies.

Moreover, and as further described in connection to <FIG>, at step <NUM>, processor <NUM> may apply cryptography techniques to protect communication sockets communicating with resources of processor <NUM> and enclaves of the SRE. Processor <NUM> may apply symmetric cryptography (such as, for example, Serpent, DES, RC4, IDEA, and/or BLOWFISH) or asymmetric cryptography (such as, for example, Diffie Hellman, EES, and/or XTR).

Once processor resources and communication links have been established and secured between processor resources and applications, such as guest virtual machines, processor <NUM> may continue to step <NUM>. At step <NUM>, processor <NUM> may define the isolated processor domains, which may include on-chip resources that are isolated and uniquely assigned to specific applications. Once a workflow is pinned to an isolated domain, the workflow may stay with the isolated domain to mitigate potential behavior analysis attacks and cross-domain attacks.

In some embodiments, configuration process <NUM> may result in an SRE virtualization that minimizes possibility of attacks because it may reduce the threat surface. By initializing the SRE virtualization and cache/memory pinning from a validated enforced boot early in a customized BIOS boot process at step <NUM>, configuration process <NUM> may create a small but secure operating system that may allow secure division of processor resources. Moreover, the chip resource assignment and binding of steps <NUM>-<NUM> may result in a chip-level domain isolation that may pin virtual machines within a multi-socket system. Further, these steps may facilitate virtualization of cores pinning hardware resources cryptographical to each virtual guest machine.

Configuration process <NUM> may allow hosting of virtual machines using a hypervisor without a scheduler and with ring separation for security. Configuration process <NUM> may create an SRE that may mitigate hardware exploits because configuration process <NUM> may result in a hypervisor directly binding on-chip hardware resources with isolated applications, instead of creating a scheduler. Hosting and pinning of the virtualization host kernel within cache memory of the processor cores and passing of unique cryptographic information to each individual virtual machine guest during the boot process may improve the functioning of a computer by enhancing security features.

<FIG> illustrates a flow chart of an exemplary resource binding process <NUM>, consistent with disclosed embodiments. Binding process <NUM> may be carried out by processor <NUM> (<FIG>). In some embodiments, other elements of system <NUM>, such as BIOS <NUM> and/or co-processor <NUM> (<FIG>), may execute binding process <NUM>. Binding process <NUM> may be part of configuration process <NUM>. For example, binding process <NUM> may be executed during step <NUM> (<FIG>).

Binding process <NUM> may start at step <NUM>, in which processor <NUM> may have determined resources that may be allocated to virtualized guests and the distribution of the allocation. Binding process <NUM> may then continue to steps <NUM>-<NUM> to execute pin resources and bind isolated domains with applications. In some embodiments, as shown in <FIG>, steps <NUM>-<NUM> may be conducted in parallel, having processor <NUM> executing multiple tasks concurrently. In other embodiments, processor <NUM> may only perform one of the tasks in steps <NUM>-<NUM> (e.g., processor <NUM> may only perform step <NUM>). Embodiments of the present disclosure also contemplate that processor <NUM> may perform one or more steps <NUM>-<NUM> sequentially or in different orders.

At step <NUM>, processor <NUM> may create a multi-socket system between resources and guest enclaves. In some embodiments, processor <NUM> may establish a multi-socket system for cache memories in processor <NUM> that may be coherent between and/or across sockets. The multi-socket system may segregate the cache memories based on their assigned enclave or application, and a compiler may automatically generate the socket selection as instructed in the validated initialization or enforced boot.

At step <NUM>, processor <NUM> may configure interfacing elements for isolated operation. In some embodiments, processor <NUM> may configure PCIe <NUM> to specifically receive/transmit only processing requests associated with a unique enclave or unique ID of processor resources. Processor <NUM> may achieve this by updating relational tables in the interfacing elements of processor <NUM>.

At step <NUM>, processor <NUM> may deploy firewall policies and direct cryptographical connections. In some embodiments, processor <NUM> may create firewall policies for routing processing request from enclaves to specific isolated domains. Additionally, and/or alternatively, processor may establish cryptographical connections between specific resources of processor <NUM> and enclaves.

At step <NUM>, processor may also establish a hypervisor with ring separation to bind resources with different enclaves. In the hypervisor, applications may run isolated from other resources of processor <NUM>. The ring separation may create an environment in processor <NUM> that may prevent a rogue application to perform cross-domain attacks or spoof information between different enclaves. In some embodiments, the hypervisor created at step <NUM> may perform a particular task that may only be performed by specified resources in processor <NUM>. This arrangement may create an equivalent of a trusted computing base (TCB) that may be smaller and have a smaller surface of exposure. Ensuring that the hypervisor is small may reduce potential attack vectors. Moreover, because applications may run in isolated domains that do not share resources, the hypervisor performance may not depend on the availability of resources.

At step <NUM>, on-chip resources of processor <NUM> may be pinned or configured to work in isolation with specified workflows or enclaves. Processor <NUM> may execute workflows from SRE guests or tenants in the specific domains without any required scheduling. Because the resources have been carved out for the resource, the resources may be directly assigned to specific tasks and may not be shared between enclaves.

<FIG> illustrates a flow chart of an exemplary enclave feature disabling process <NUM>, consistent with disclosed embodiments. Disabling process <NUM> may be carried out by processor <NUM> (<FIG>). Additionally, and/or alternatively, other elements of system <NUM>, such as BIOS <NUM> and/or co-processor <NUM> (<FIG>), may execute disabling process <NUM>. Disabling process <NUM> may be part of configuration process <NUM>. In some embodiments, disabling process <NUM> may be executed during step <NUM> (<FIG>).

At step <NUM>, processor <NUM> may estimate enclave execution flows. Based on information in BIOS <NUM> and/or user inputted instructions, processor <NUM> may determine potential execution flows and a security level for the different enclaves. Based on the estimated execution flows and the required level of security, processor <NUM> may elect to disable certain features to enhance security or improve performance. In some embodiments, an execution workflow in processor <NUM> may include instructions to process data or perform a computational task. Embodiments of the present disclosure contemplate that execution flows may include operations of storing a sequence of instructions; locating memory address where the first instruction and copying to the program counter; addressing a program counter to memory via the address bus; responding to a memory by sending a copy of the state of the bits at that memory location on the data bus; automatically incrementing a pointer to contain the address of the next instruction in memory; and executing an instruction in the instruction register. Persons of ordinary skill in the art would appreciate that any other suitable operations during execution workflows may be possible. Based on the estimated execution workflows and security level, disabling process <NUM> may disable or isolate certain elements of processor <NUM> to reduce vulnerabilities that may arise from sharing resources in typical execution flows.

At step <NUM>, processor <NUM> may determine whether the requested or estimated workflow(s) require an option ROM. If processor <NUM> determines that a workflow does not require option ROM (step <NUM>: No), processor <NUM> may continue to step <NUM> and disable Option ROM features in processor <NUM>. In some embodiments, processor <NUM> may instruct power management <NUM> (<FIG>) to disable option ROM associated modules. If, however, processor <NUM> determines that a workflow does require option ROM (step <NUM>: yes), processor <NUM> may leave Option ROM working and continue to step <NUM>.

At step <NUM>, processor <NUM> may determine whether the requested or estimated workflow(s) require system management modules, such as elements in management complex in <FIG>. If processor <NUM> determines that a workflow does not require system management modules (step <NUM>: No), processor <NUM> may continue to step <NUM> and disable modules in the management complex. In some embodiments, processor <NUM> may disable security controller <NUM>, buffer management <NUM>, and/or PME <NUM>. However, if processor <NUM> determines that a workflow does require system management modules (step <NUM>: yes), processor <NUM> may leave active system management modules and continue to step <NUM>.

At step <NUM>, processor <NUM> may determine whether the estimated or requested workflow(s) or virtual machines supported by the SRE require debug ports. If processor <NUM> determines that a workflow or a virtual machine does not require debug ports (step <NUM>: No), processor <NUM> may continue to step <NUM> and disable debug ports. However, if processor <NUM> determines that a workflow or a virtual machine does require debug ports (step <NUM>: yes), processor <NUM> may leave debug ports active and continue to step <NUM>, where processor <NUM> may perform the workflows with the enabled/disabled elements.

Disabling process <NUM> may further improve security of processor <NUM> because it may minimize sharing of resources and may prevent leakage at the lower layers of the implementation stack caused by shared hardware resources and microarchitectural attacks.

<FIG> illustrates a flow chart of an exemplary cryptographical pinning process <NUM>, consistent with disclosed embodiments. Cryptographical pinning process <NUM> may be carried out by processor <NUM> (<FIG>). Additionally, and/or alternatively, other elements of system <NUM>, such as BIOS <NUM> and/or co-processor <NUM> (<FIG>), may execute cryptographical pinning process <NUM>. Cryptographical pinning process <NUM> may be part of configuration process <NUM>. Processor <NUM> may perform cryptographical pinning process <NUM> to pass unique cryptographic information to each individual virtual machine guest during the boot process (e.g., in step <NUM>) and/or for cryptographically pinning hardware resources (e.g., cores, cache, memory, and IOMMU) to each virtual guest machine (e.g., in step <NUM>).

At step <NUM>, processor <NUM> may obtain cryptographic information. In some embodiments, in configuring the SRE, processor <NUM> may obtain information directly form BIOS <NUM>, secure initialization memories <NUM>, and/or from one of the enclaves. At step <NUM>, processor <NUM> may read a pinning file for encryption or decryption. In some embodiments, for a specific resource and enclave connection, processor <NUM> may read the required protocol and security level. At step <NUM>, processor <NUM> may identify an algorithm and its key parameters.

At step <NUM>, processor <NUM> may determine if a key is available for an algorithm. If the key is not present (step <NUM>: No), processor <NUM> may continue to step <NUM> and use a Key Generator to generate the key for performing a cipher operation specified in the file of step <NUM>. Key Generator may be part of the validated enforced boot and may be provided with symmetric ciphers. At step <NUM>, the key may be stored in a specific path and for the specific enclave/resource pair. In some embodiments, the generated key may be stored in a cache memory that may be reserved for certain workflows or specific virtual machines. However, if the key is already present in step <NUM> (step <NUM>: yes), and the specified path is valid, that key may be used for cryptographically pinning the resource.

Using the key, the resources may be cryptographically pinned at step <NUM>. Processor <NUM> may perform the encryption. In some embodiments, at step <NUM>, processor <NUM> may set a timer for the encryption to prevent misuse of processor resources by leaving resources idle for too long.

At step <NUM>, processor <NUM> may perform exchanges between resources in isolated domains with their assigned enclaves using the cryptographic channels.

Cryptographical pinning process <NUM> is an exemplary process for establishing secured communications between chip-level resources and tenants of the SRE, such as virtual machines. Embodiments of the present disclosure also contemplate applying other cryptography methods to the SRE to secure communications. In some embodiments, processor <NUM> may implement public key-based methods by preprograming BIOS <NUM> and may use encoded plain text during the transmission of instructions. Additionally, and/or alternatively, processor <NUM> may employ a private key-based method for encoding/decoding ciphered instructions. In some embodiments, processor <NUM> may combine different strategies for pinning hardware resources cryptographically to each virtual guest machine. Depending on the level of security and ciphering strategies, processor <NUM> may apply faster or slower techniques. Additionally, and/or alternatively, certain elements of system <NUM> may be configured to specifically perform the cryptographical tasks. For example, co-processor <NUM> may be configured to perform any cryptographical tasks during configuration process <NUM> (<FIG>).

<FIG> illustrates a flow chart of an exemplary virtualization process <NUM>, consistent with disclosed embodiments. Virtualization process <NUM> may be carried out by processors <NUM> (<FIG>). Additionally, and/or alternatively, elements of system <NUM> (<FIG>), such as processor <NUM>, BIOS <NUM>, and/or co-processor <NUM> (<FIG>), may execute virtualization process <NUM>. Virtualization process <NUM> may be performed early during processor startup. For example, in some embodiments virtualization process <NUM> may be performed during boot operations. Alternatively, and/or additionally, virtualization process <NUM> may be performed before any OS is initiated or activated to create low level virtualization.

At step <NUM>, processors <NUM> may retrieve initialization instructions, early during startup, from a dedicated memory. For example, processors <NUM> may be configured to retrieve initialization instructions form secure initialization memory <NUM> and/or a BIOS memory, such as BIOS <NUM>. The initialization instructions may include routines and/or protocols to initialize an SRE, such as protected environment <NUM>. As further described in connection with <FIG>, the initialization instructions may also include booting instructions and validation routines.

At step <NUM>, processors <NUM> may launch an SRE, before loading any general purpose or multi-user OS. For example, before initialization of OSs, processors <NUM> may launch an SRE in step <NUM>. As previously discussed in connection with <FIG>, the SRE may host virtual machines using a hypervisor without a scheduler. In some embodiments, the SRE of step <NUM> may provide a virtualization layer that isolates hardware and/or chip-level resources.

At step <NUM>, processors <NUM> may retrieve cryptographic keying from a dedicated memory. For example, processors <NUM> may obtain cryptographic keying from secure initialization memory <NUM>. In some embodiments, depending on the launch of step <NUM> only a portion of the cryptographic keying may be retrieved in step <NUM>. For example, based on the scope of the SRE, processors <NUM> may select to retrieve only a portion of the cryptographic keying. In some embodiments, the cryptographic keying may be stored in secure initialization memory <NUM> and may include a key management system labeling the different type of keys that could be required during SRE operations.

At step <NUM>, processors <NUM> may collect and validate hardware instruction sets and/or processor drivers. For example, processors <NUM> may collect CPU instructions sets in step <NUM> to prepare for SRE-based hardware isolation. The instruction sets may include code that specify chip-level instructions, resource tables, and/or tasks that control of communications of chip-level resources. For example, instruction sets collected in step <NUM> may specify registers, protocols, and packet structures for communication between different chip-level resources. Further, in some embodiments processors <NUM> may validate the hardware instructions to evaluate their safety and security. For example, the initialization instructions of step <NUM> and/or the cryptographic keying of step <NUM> may include information for validating instruction sets.

In step <NUM>, processors <NUM> may determine whether the instruction sets pass validation tests. For example, processors <NUM> may perform verification tasks and determine that instruction sets comply with verification requirements and validation tests. If processors <NUM> determine that the instruction sets collected in step <NUM> do not pass the cryptographic tests (step <NUM>: No), processors <NUM> may generate an alert identifying invalid signatures or validations in step <NUM>. However, if in step <NUM> processors <NUM> determines instructions steps pass the cryptographic tests (step <NUM>: Yes), virtualization process <NUM> may continue to step <NUM>.

At step <NUM>, processors <NUM> may generate specific keying for chip-level resources using the instruction sets and cryptographic keying. For example, through the SRE, processors <NUM> may apply the cryptographic keying to the instructions sets and generate specific keying for the chip-level resources of processors <NUM>. As further described in connection with <FIG>, the SRE may use hardware entropy to generate hardware specific keys that can be used to link and/or communicate chip-level resources with work flows or enclaves.

At step <NUM>, processors <NUM> may initialize a plurality of isolated enclaves on the SRE. For example, processors <NUM> may initialize the hardened enclaves <NUM> and secure enclave <NUM> (<FIG>). In some embodiments, the enclaves may be initiated by allocating workflows or creating VMs for later assignation of hardware resources.

At step <NUM>, processors <NUM> may pin chip-level resources to enclaves according to specific keying. As further discussed in connection with <FIG>, processors <NUM> may establish cryptographic links communicating chip-level resources with the plurality of enclaves. In some embodiments, resources may be pinned to unique enclaves exclusively to minimize potential of cross-domain attacks. In other embodiments, certain resources may be shared between different enclaves and place firewall protections to prevent leakage or privilege escalation.

<FIG> illustrates a flow chart of an exemplary chip-level resource pinning process <NUM>, consistent with disclosed embodiments. Pinning process <NUM> may be carried out by processors <NUM> (<FIG>). Additionally, and/or alternatively, elements of system <NUM> (<FIG>), such as processor <NUM>, BIOS <NUM>, and/or co-processor <NUM> (<FIG>), may execute pinning process <NUM>. Pinning process <NUM> may be performed as part of configuration process <NUM> and/or virtualization process <NUM>.

At step <NUM>, processors <NUM> may collect and extract hardware entropy of chip-level and/or processor hardware resources. For example, processors <NUM> may collect and accumulate variances in the execution time of a given set of instructions by processors <NUM> resources. Processors <NUM> may accumulate the variances using, for example, a linear-feedback shift register (LFSR) with an irreducible primitive polynomial. The measurement of the execution time jitter may be performed over the logic of the LFSR and execution times may be used to generate an entropy pool.

At step <NUM>, processors <NUM> may implement a random number generator using the collected entropy. For example, processors <NUM> may perform operations of invoking memory accesses to induce timing variations, fetching a time stamp to calculate a delta to the time stamp of the previous loop iteration, injecting the time delta value into the entropy pool using an LFSR that operates bitwise on the time delta operation, and iterating the sequence to generate an entropy pool that allows generating a randomizing function and/or a random number generator.

At step <NUM>, processors <NUM> may collect instruction sets and/or resource tables associated with the on-chip level resources. For example, processors <NUM> may query instruction sets and/or hardware IDs from hardware resources of processors <NUM>. At step <NUM>, processors <NUM> may employ instructions sets, resource tables, randomizing functions, and/or random number generator, to establish chip-level cryptographic links. For example, as further described in connection with <FIG>, processors <NUM> may create connections through a multi-socket system between resources and enclaves and create cryptographic links or connections using the randomizing functions of step <NUM> to encrypt communications.

At step <NUM>, processors <NUM> may couple chip-level resources exclusively to an enclave through the cryptographic links. Further, at step <NUM> (and as further described in connection with <FIG>), processors <NUM> may deploy hardware-enforced firewalling that separate cryptographic links to isolate sensitive from untrusted workloads. For example, in some embodiments an SRE may include cross-domain protections. In such embodiments, processors <NUM> may deploy firewalls isolating VMs with policies rerouting requests from the plurality of enclaves to specific chip-level resources based on header or payload processing. In such embodiments, firewall policies may require processors <NUM> to employ a memory state table and determining SRAM registers or blocks that can be accessed by each of the plurality of enclaves. Thus, pinning process <NUM> may result in binding chip-level resources to exclusive enclaves using secure cryptographic links.

<FIG> illustrates a block diagram of an exemplary representation virtualized environment <NUM> with isolated resources, consistent with disclosed embodiments. Virtualized environment <NUM> may represent the virtual architecture of processor <NUM> or processors <NUM> after virtualization process <NUM> and pinning process <NUM>. Additionally, or alternatively, virtualized environment <NUM> may result from configuration process <NUM> and/or process <NUM>.

Virtualized environment <NUM> may include an SRE <NUM>. In some embodiments, SRE <NUM> may include protected environment <NUM>. Virtualized environment <NUM> may also include devices <NUM>, firmware <NUM>, cores <NUM>, cache <NUM>, and memory <NUM>. Each of these devices <NUM>, firmware <NUM>, cores <NUM>, cache <NUM>, and memory <NUM> may represent one or more devices of processor <NUM>. For example, cache <NUM> may represent icaches <NUM> and/or cores <NUM> may represent cores <NUM> (<FIG>).

As shown in <FIG>, as a result of virtualization process <NUM>, or configuration process <NUM>, devices <NUM>, firmware <NUM>, cores <NUM>, cache <NUM>, and memory <NUM> may be divided into isolated virtual machines <NUM>. Each of virtual machines <NUM>(a)-<NUM>(d) may have assigned and/or pinned a portion of devices <NUM>, firmware <NUM>, cores <NUM>, cache <NUM>, and memory <NUM>. In addition, in some embodiments each one of the virtual machines <NUM> may be separated by firewalls <NUM>. In some embodiments, as shown in <FIG>, the allocation of processor resources may be tailored to target application. In other embodiments, the allocation of processor resources may be uniform. Further, as also explained in connection with <FIG>, pinning resources may be complemented with hardware-enforce firewalling. In this way it is possible to have hardware enforced separation between different applications and workflows.

SRE <NUM> may communicate with each one of the virtual machines <NUM> using cryptographic links <NUM>. Each one of the cryptographic links <NUM> may be exclusive for each one of the virtual machines <NUM>. Further, as shown in <FIG>, SRE <NUM> may support applications (or enclaves) <NUM>. In such embodiments, SRE <NUM> may couple cryptographic links <NUM> with corresponding applications <NUM>. For example, SRE <NUM> may couple cryptographic links <NUM>, which communicate chip-level resources, with applications <NUM> through conduits <NUM> and connections <NUM>. As shown in <FIG>, to keep isolation between different machines, each application <NUM> may have a designated conduit <NUM> (a)-(d) and a designated connection <NUM> (a)-(d).

Virtualized environment <NUM> provides a visual representation of how the configuration process <NUM>, virtualization process <NUM>, and pinning process <NUM> may create virtualization and isolation completely down the stack of hardware and software layers. Such a configuration may reduce the attack surface by increasing workload isolation. The partitioning of shared resources shown in virtualized environment <NUM> may support availability, integrity, and confidentiality. Further, firewalls <NUM> may help separate sensitive data from untrusted workloads and provide a more deterministic workload performance. Moreover, the arrangement of SRE <NUM> may allow management of virtual machines <NUM> that are protected against leakage, modification, and privilege escalation attacks. In addition, employing exclusive cryptographic links <NUM> may result in greater data confidentiality and create a chain of trust that verifies and maintains system integrity at boot (e.g., from power on) though the launch of critical workloads.

Another aspect of the disclosure is directed to a non-transitory computer readable medium storing instructions that, when executed, may cause one or more processors to perform the methods, as discussed above. The computer-readable medium may include volatile or non-volatile, magnetic, semiconductor, tape, optical, removable, non-removable, or other types of computer-readable medium or computer readable storage devices. In some embodiments, the computer-readable medium may be the storage unit or the memory module having the computer instructions stored thereon, as disclosed. In some embodiments, the computer-readable medium may be a disc or a flash drive having the computer instructions stored thereon.

It will be apparent to those skilled in the art that various modifications and variations can be made to the disclosed system and related methods. Other embodiments will be apparent to those skilled in the art from consideration of the specification and practice of the disclosed system and related methods. It is intended that the specification and examples be considered as exemplary only, with a true scope being indicated by the claims and their equivalents.

Moreover, while illustrative embodiments have been described herein, the scope thereof includes any and all embodiments having equivalent elements, modifications, omissions, combinations (e.g., of aspects across various embodiments), adaptations and/or alterations as would be appreciated by those in the art based on the present disclosure. For example, the number and orientation of components shown in the exemplary systems may be modified. Further, with respect to the exemplary methods, the order and sequence of steps may be modified, and steps may be added or deleted. Further, the steps of the disclosed methods may be modified in any manner, including by reordering steps and/or inserting or deleting steps.

Claim 1:
A system for secure processor virtualization comprising:
a secure initialization memory comprising:
initialization instructions for launching a security runtime environment before operating systems; and
cryptographic keying for validated execution, the cryptographic keying comprising instructions for cryptographical tasks; and
one or more processors coupled to the secure initialization memory, wherein the one or more processors are configured to:
retrieve the initialization instructions from the secure initialization memory at startup;
execute the initialization instructions to launch the security runtime environment and retrieve at least a portion of the cryptographic keying from the secure initialization memory;
before loading an operating system, generate a plurality of specific keys for chip-level resources in the one or more processors by applying the cryptographic keying from the secure initialization memory to instruction sets of the chip- level resources, the specific keys being hardware specific keys for chip-level resources, the chip-level resources comprising CPU cores, cache memories, and input-output memory management units (IOMMUs);
initialize a plurality of virtual enclaves on the security runtime environment;
pin at least a portion of the CPU cores, cache memories, and IOMMUs to unique enclaves from the plurality of virtual enclaves by:
establishing exclusive cryptographic links between each resource in the portion and only one of the unique enclaves, the exclusive cryptographic links being established based on the specific keys associated with each resource ;
manipulating registers of bridging chips to assign hardware resource IDs of each of the CPU cores, cache memories, and IOMMUs in the portion to only one of the unique enclaves without sharing chip-level resource assignations with other of the plurality of virtual enclaves, and
after pinning the CPU cores, cache memories, and IOMMUs in the portion to the unique enclaves, executing guest workflows in specific ones of the unique enclaves, the specific ones of the unique enclaves being directly assigned without scheduling.