Patent Publication Number: US-11644980-B2

Title: Trusted memory sharing mechanism

Description:
BACKGROUND 
     Modern computing devices may include general-purpose processor cores as well as a variety of hardware accelerators for offloading compute-intensive workloads or performing specialized tasks. Hardware accelerators may include, for example, one or more field-programmable gate arrays (FPGAs) which may include programmable digital logic resources that may be configured by the end user or system integrator. Hardware accelerators may also include one or more application-specific integrated circuits (ASICs). Hardware accelerators may be embodied as I/O devices that communicate with the processor core over an I/O interconnect. Additionally, hardware accelerators may include one or more graphics processing units (GPUs) implemented to process graphics data. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The concepts described herein are illustrated by way of example and not by way of limitation in the accompanying figures. For simplicity and clarity of illustration, elements illustrated in the figures are not necessarily drawn to scale. Where considered appropriate, reference labels have been repeated among the figures to indicate corresponding or analogous elements. 
         FIG.  1    is a simplified block diagram of at least one embodiment of a computing device for secure I/O with an accelerator device; 
         FIG.  2    is a simplified block diagram of at least one embodiment of an accelerator device of the computing device of  FIG.  1   ; 
         FIG.  3    is a simplified block diagram of at least one embodiment of an environment of the computing device of  FIGS.  1  and  2   ; 
         FIG.  4    illustrates a computing device according to implementations of the disclosure; 
         FIG.  5    illustrates one embodiment of a computing platform; 
         FIG.  6 A  is a flow diagram illustrating one embodiment of a shared memory configuration process; 
         FIG.  6 B  is a sequence diagram illustrating one embodiment of a process for secure memory sharing; and 
         FIG.  7    illustrates one embodiment of a schematic diagram of an illustrative electronic computing device. 
     
    
    
     DETAILED DESCRIPTION OF THE DRAWINGS 
     While the concepts of the present disclosure are susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawings and will be described herein in detail. It should be understood, however, that there is no intent to limit the concepts of the present disclosure to the particular forms disclosed, but on the contrary, the intention is to cover all modifications, equivalents, and alternatives consistent with the present disclosure and the appended claims. 
     References in the specification to “one embodiment,” “an embodiment,” “an illustrative embodiment,” etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may or may not necessarily include that particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described. Additionally, it should be appreciated that items included in a list in the form of “at least one A, B, and C” can mean (A); (B); (C); (A and B); (A and C); (B and C); or (A, B, and C). Similarly, items listed in the form of “at least one of A, B, or C” can mean (A); (B); (C); (A and B); (A and C); (B and C); or (A, B, and C). 
     The disclosed embodiments may be implemented, in some cases, in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried by or stored on a transitory or non-transitory machine-readable (e.g., computer-readable) storage medium, which may be read and executed by one or more processors. A machine-readable storage medium may be embodied as any storage device, mechanism, or other physical structure for storing or transmitting information in a form readable by a machine (e.g., a volatile or non-volatile memory, a media disc, or other media device). 
     In the drawings, some structural or method features may be shown in specific arrangements and/or orderings. However, it should be appreciated that such specific arrangements and/or orderings may not be required. Rather, in some embodiments, such features may be arranged in a different manner and/or order than shown in the illustrative figures. Additionally, the inclusion of a structural or method feature in a particular figure is not meant to imply that such feature is required in all embodiments and, in some embodiments, may not be included or may be combined with other features. 
     Referring now to  FIG.  1   , a computing device  100  for secure I/O with an accelerator device includes a processor  120  and an accelerator device  136 , such as a field-programmable gate array (FPGA). In use, as described further below, a trusted execution environment (TEE) established by the processor  120  securely communicates data with the accelerator  136 . Data may be transferred using memory-mapped I/O (MMIO) transactions or direct memory access (DMA) transactions. For example, the TEE may perform an MMIO write transaction that includes encrypted data, and the accelerator  136  decrypts the data and performs the write. As another example, the TEE may perform an MMIO read request transaction, and the accelerator  136  may read the requested data, encrypt the data, and perform an MMIO read response transaction that includes the encrypted data. As yet another example, the TEE may configure the accelerator  136  to perform a DMA operation, and the accelerator  136  performs a memory transfer, performs a cryptographic operation (i.e., encryption or decryption), and forwards the result. As described further below, the TEE and the accelerator  136  generate authentication tags (ATs) for the transferred data and may use those ATs to validate the transactions. The computing device  100  may thus keep untrusted software of the computing device  100 , such as the operating system or virtual machine monitor, outside of the trusted code base (TCB) of the TEE and the accelerator  136 . Thus, the computing device  100  may secure data exchanged or otherwise processed by a TEE and an accelerator  136  from an owner of the computing device  100  (e.g., a cloud service provider) or other tenants of the computing device  100 . Accordingly, the computing device  100  may improve security and performance for multi-tenant environments by allowing secure use of accelerator devices. 
     The computing device  100  may be embodied as any type of device capable of performing the functions described herein. For example, the computing device  100  may be embodied as, without limitation, a computer, a laptop computer, a tablet computer, a notebook computer, a mobile computing device, a smartphone, a wearable computing device, a multiprocessor system, a server, a workstation, and/or a consumer electronic device. As shown in  FIG.  1   , the illustrative computing device  100  includes a processor  120 , an I/O subsystem  124 , a memory  130 , and a data storage device  132 . Additionally, in some embodiments, one or more of the illustrative components may be incorporated in, or otherwise form a portion of, another component. For example, the memory  130 , or portions thereof, may be incorporated in the processor  120  in some embodiments. 
     The processor  120  may be embodied as any type of processor capable of performing the functions described herein. For example, the processor  120  may be embodied as a single or multi-core processor(s), digital signal processor, microcontroller, or other processor or processing/controlling circuit. As shown, the processor  120  illustratively includes secure enclave support  122 , which allows the processor  120  to establish a trusted execution environment known as a secure enclave, in which executing code may be measured, verified, and/or otherwise determined to be authentic. Additionally, code and data included in the secure enclave may be encrypted or otherwise protected from being accessed by code executing outside of the secure enclave. For example, code and data included in the secure enclave may be protected by hardware protection mechanisms of the processor  120  while being executed or while being stored in certain protected cache memory of the processor  120 . The code and data included in the secure enclave may be encrypted when stored in a shared cache or the main memory  130 . The secure enclave support  122  may be embodied as a set of processor instruction extensions that allows the processor  120  to establish one or more secure enclaves in the memory  130 . For example, the secure enclave support  122  may be embodied as Intel® Software Guard Extensions (SGX) technology. 
     The memory  130  may be embodied as any type of volatile or non-volatile memory or data storage capable of performing the functions described herein. In operation, the memory  130  may store various data and software used during operation of the computing device  100  such as operating systems, applications, programs, libraries, and drivers. As shown, the memory  130  may be communicatively coupled to the processor  120  via the I/O subsystem  124 , which may be embodied as circuitry and/or components to facilitate input/output operations with the processor  120 , the memory  130 , and other components of the computing device  100 . For example, the I/O subsystem  124  may be embodied as, or otherwise include, memory controller hubs, input/output control hubs, sensor hubs, host controllers, firmware devices, communication links (i.e., point-to-point links, bus links, wires, cables, light guides, printed circuit board traces, etc.) and/or other components and subsystems to facilitate the input/output operations. In some embodiments, the memory  130  may be directly coupled to the processor  120 , for example via an integrated memory controller hub. Additionally, in some embodiments, the I/O subsystem  124  may form a portion of a system-on-a-chip (SoC) and be incorporated, along with the processor  120 , the memory  130 , the accelerator device  136 , and/or other components of the computing device  100 , on a single integrated circuit chip. Additionally, or alternatively, in some embodiments the processor  120  may include an integrated memory controller and a system agent, which may be embodied as a logic block in which data traffic from processor cores and I/O devices converges before being sent to the memory  130 . 
     As shown, the I/O subsystem  124  includes a direct memory access (DMA) engine  126  and a memory-mapped I/O (MMIO) engine  128 . The processor  120 , including secure enclaves established with the secure enclave support  122 , may communicate with the accelerator device  136  with one or more DMA transactions using the DMA engine  126  and/or with one or more MMIO transactions using the MMIO engine  128 . The computing device  100  may include multiple DMA engines  126  and/or MMIO engines  128  for handling DMA and MMIO read/write transactions based on bandwidth between the processor  120  and the accelerator  136 . Although illustrated as being included in the I/O subsystem  124 , it should be understood that in some embodiments the DMA engine  126  and/or the MMIO engine  128  may be included in other components of the computing device  100  (e.g., the processor  120 , memory controller, or system agent), or in some embodiments may be embodied as separate components. 
     The data storage device  132  may be embodied as any type of device or devices configured for short-term or long-term storage of data such as, for example, memory devices and circuits, memory cards, hard disk drives, solid-state drives, non-volatile flash memory, or other data storage devices. The computing device  100  may also include a communications subsystem  134 , which may be embodied as any communication circuit, device, or collection thereof, capable of enabling communications between the computing device  100  and other remote devices over a computer network (not shown). The communications subsystem  134  may be configured to use any one or more communication technology (e.g., wired or wireless communications) and associated protocols (e.g., Ethernet, Bluetooth®, Wi-Fi®, WiMAX, 3G, 4G LTE, etc.) to effect such communication. 
     The accelerator device  136  may be embodied as a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a coprocessor, or other digital logic device capable of performing accelerated functions (e.g., accelerated application functions, accelerated network functions, or other accelerated functions), GPUs, etc. Illustratively, the accelerator device  136  is an FPGA, which may be embodied as an integrated circuit including programmable digital logic resources that may be configured after manufacture. The FPGA may include, for example, a configurable array of logic blocks in communication over a configurable data interchange. The accelerator device  136  may be coupled to the processor  120  via a high-speed connection interface such as a peripheral bus (e.g., a PCI Express bus) or an inter-processor interconnect (e.g., an in-die interconnect (IDI) or QuickPath Interconnect (QPI)), or via any other appropriate interconnect. The accelerator device  136  may receive data and/or commands for processing from the processor  120  and return results data to the processor  120  via DMA, MMIO, or other data transfer transactions. 
     As shown, the computing device  100  may further include one or more peripheral devices  138 . The peripheral devices  138  may include any number of additional input/output devices, interface devices, hardware accelerators, and/or other peripheral devices. For example, in some embodiments, the peripheral devices  138  may include a touch screen, graphics circuitry, a graphical processing unit (GPU) and/or processor graphics, an audio device, a microphone, a camera, a keyboard, a mouse, a network interface, and/or other input/output devices, interface devices, and/or peripheral devices. 
     Referring now to  FIG.  2   , an illustrative embodiment of a field-programmable gate array (FPGA)  200  is shown. As shown, the FPGA  200  is one potential embodiment of an accelerator device  136 . The illustratively FPGA  200  includes a secure MMIO engine  202 , a secure DMA engine  204 , one or more accelerator functional units (AFUs)  206 , and memory/registers  208 . As described further below, the secure MMIO engine  202  and the secure DMA engine  204  perform in-line authenticated cryptographic operations on data transferred between the processor  120  (e.g., a secure enclave established by the processor) and the FPGA  200  (e.g., one or more AFUs  206 ). In some embodiments, the secure MMIO engine  202  and/or the secure DMA engine  204  may intercept, filter, or otherwise process data traffic on one or more cache-coherent interconnects, internal buses, or other interconnects of the FPGA  200 . 
     Each AFU  206  may be embodied as logic resources of the FPGA  200  that are configured to perform an acceleration task. Each AFU  206  may be associated with an application executed by the computing device  100  in a secure enclave or other trusted execution environment. Each AFU  206  may be configured or otherwise supplied by a tenant or other user of the computing device  100 . For example, each AFU  206  may correspond to a bitstream image programmed to the FPGA  200 . As described further below, data processed by each AFU  206 , including data exchanged with the trusted execution environment, may be cryptographically protected from untrusted components of the computing device  100  (e.g., protected from software outside of the trusted code base of the tenant enclave). Each AFU  206  may access or otherwise process stored in the memory/registers  208 , which may be embodied as internal registers, cache, SRAM, storage, or other memory of the FPGA  200 . In some embodiments, the memory  208  may also include external DRAM or other dedicated memory coupled to the FPGA  200 . 
     Referring now to  FIG.  3   , in an illustrative embodiment, the computing device  100  establishes an environment  300  during operation. The illustrative environment  300  includes a trusted execution environment (TEE)  302  and the accelerator  136 . The TEE  302  further includes a trusted agent  303 , host cryptographic engine  304 , a transaction dispatcher  306 , a host validator  308 , and a direct memory access (DMA) manager  310 . The accelerator  136  includes an accelerator cryptographic engine  312 , a memory range selection engine  313 , an accelerator validator  314 , a memory mapper  316 , an authentication tag (AT) controller  318 , and a DMA engine  320 . The various components of the environment  300  may be embodied as hardware, firmware, software, or a combination thereof. As such, in some embodiments, one or more of the components of the environment  300  may be embodied as circuitry or collection of electrical devices (e.g., host cryptographic engine circuitry  304 , transaction dispatcher circuitry  306 , host validator circuitry  308 , DMA manager circuitry  310 , accelerator cryptographic engine circuitry  312 , accelerator validator circuitry  314 , memory mapper circuitry  316 , AT controller circuitry  318 , and/or DMA engine circuitry  320 ). It should be appreciated that, in such embodiments, one or more of the host cryptographic engine circuitry  304 , the transaction dispatcher circuitry  306 , the host validator circuitry  308 , the DMA manager circuitry  310 , the accelerator cryptographic engine circuitry  312 , the accelerator validator circuitry  314 , the memory mapper circuitry  316 , the AT controller circuitry  318 , and/or the DMA engine circuitry  320  may form a portion of the processor  120 , the I/O subsystem  124 , the accelerator  136 , and/or other components of the computing device  100 . Additionally, in some embodiments, one or more of the illustrative components may form a portion of another component and/or one or more of the illustrative components may be independent of one another. 
     The TEE  302  may be embodied as a trusted execution environment of the computing device  100  that is authenticated and protected from unauthorized access using hardware support of the computing device  100 , such as the secure enclave support  122  of the processor  120 . Illustratively, the TEE  302  may be embodied as one or more secure enclaves established using Intel SGX technology. The TEE  302  may also include or otherwise interface with one or more drivers, libraries, or other components of the computing device  100  to interface with the accelerator  136 . 
     The host cryptographic engine  304  is configured to generate an authentication tag (AT) based on a memory-mapped I/O (MMIO) transaction and to write that AT to an AT register of the accelerator  136 . For an MMIO write request, the host cryptographic engine  304  is further configured to encrypt a data item to generate an encrypted data item, and the AT is generated in response to encrypting the data item. For an MMIO read request, the AT is generated based on an address associated with MMIO read request. 
     The transaction dispatcher  306  is configured to dispatch the memory-mapped I/O transaction (e.g., an MMIO write request or an MMIO read request) to the accelerator  136  after writing the calculated AT to the AT register. An MMIO write request may be dispatched with the encrypted data item. 
     The host validator  308  may be configured to verify that an MMIO write request succeeded in response dispatching the MMIO write request. Verifying that the MMIO write request succeeded may include securely reading a status register of the accelerator  136 , securely reading a value at the address of the MMIO write from the accelerator  136 , or reading an AT register of the accelerator  136  that returns an AT value calculated by the accelerator  136 , as described below. For MMIO read requests, the host validator  308  may be further configured to generate an AT based on an encrypted data item included in a MMIO read response dispatched from the accelerator  136 ; read a reported AT from a register of the accelerator  136 ; and determine whether the AT generated by the TEE  302  matches the AT reported by the accelerator  136 . The host validator  308  may be further configured to indicate an error if those ATs do not match, which provides assurance that data was not modified on the way from the TEE  302  to the accelerator  136 . 
     The accelerator cryptographic engine  312  is configured to perform a cryptographic operation associated with the MMIO transaction and to generate an AT based on the MMIO transaction in response to the MMIO transaction being dispatched. For an MMIO write request, the cryptographic operation includes decrypting an encrypted data item received from the TEE  302  to generate a data item, and the AT is generated based on the encrypted data item. For an MMIO read request, the cryptographic operation includes encrypting a data item from a memory of the accelerator  136  to generate an encrypted data item, and the AT is generated based on that encrypted data item. 
     The accelerator validator  314  is configured to determine whether the AT written by the TEE  302  matches the AT determined by the accelerator  136 . The accelerator validator  314  is further configured to drop the MMIO transaction if those ATs do not match. For MMIO read requests, the accelerator validator  314  may be configured to generate a poisoned AT in response to dropping the MMIO read request, and may be further configured to dispatch a MMIO read response with a poisoned data item to the TEE  302  in response to dropping the MMIO read request. 
     The memory mapper  316  is configured to commit the MMIO transaction in response to determining that the AT written by the TEE  302  matches the AT generated by the accelerator  136 . For an MMIO write request, committing the transaction may include storing the data item in a memory of the accelerator  136 . The memory mapper  316  may be further configured to set a status register to indicate success in response to storing the data item. For an MMIO read request, committing the transaction may include reading the data item at the address in the memory of the accelerator  136  and dispatching an MMIO read response with the encrypted data item to the TEE  302 . 
     The DMA manager  310  is configured to securely write an initialization command to the accelerator  136  to initialize a secure DMA transfer. The DMA manager  310  is further configured to securely configure a descriptor indicative of a host memory buffer, an accelerator  136  buffer, and a transfer direction. The transfer direction may be host to accelerator  136  or accelerator  136  to host. The DMA manager  310  is further configured to securely write a finalization command to the accelerator  136  to finalize an authentication tag (AT) for the secure DMA transfer. The initialization command, the descriptor, and the finalization command may each be securely written and/or configured with an MMIO write request. The DMA manager  310  may be further configured to determine whether to transfer additional data in response to securely configuring the descriptor, the finalization command may be securely written in response to determining that no additional data remains for transfer. 
     The AT controller  318  is configured to initialize an AT in response to the initialization command from the TEE  302 . The AT controller  318  is further configured to finalize the AT in response to the finalization command from the TEE  302 . 
     The DMA engine  320  is configured to transfer data between the host memory buffer and the accelerator  136  buffer in response to the descriptor from the TEE  302 . For a transfer from host to accelerator  136 , transferring the data includes copying encrypted data from the host memory buffer and forwarding the plaintext data to the accelerator  136  buffer in response to decrypting the encrypted data. For a transfer from accelerator  136  to host, transferring the data includes copying plaintext data from the accelerator  136  buffer and forwarding encrypted data to the host memory buffer in response encrypting the plaintext data. 
     The accelerator cryptographic engine  312  is configured to perform a cryptographic operation with the data in response to transferring the data and to update the AT in response to transferring the data. For a transfer from host to accelerator  136 , performing the cryptographic operation includes decrypting encrypted data to generate plaintext data. For a transfer from accelerator  136  to host, performing the cryptographic operation includes encrypting plaintext data to generate encrypted data. 
     The host validator  308  is configured to determine an expected AT based on the secure DMA transfer, to read the AT from the accelerator  136  in response to securely writing the finalization command, and to determine whether the AT from the accelerator  136  matches the expected AT. The host validator  308  may be further configured to indicate success if the ATs match and to indicate failure if the ATs do not match. 
       FIG.  4    illustrates another embodiment of a computing device  400 . Computing device  400  represents a communication and data processing device including or representing (without limitations) smart voice command devices, intelligent personal assistants, home/office automation system, home appliances (e.g., washing machines, television sets, etc.), mobile devices (e.g., smartphones, tablet computers, etc.), gaming devices, handheld devices, wearable devices (e.g., smartwatches, smart bracelets, etc.), virtual reality (VR) devices, head-mounted display (HMDs), Internet of Things (IoT) devices, laptop computers, desktop computers, server computers, set-top boxes (e.g., Internet based cable television set-top boxes, etc.), global positioning system (GPS)-based devices, automotive infotainment devices, etc. 
     In some embodiments, computing device  400  includes or works with or is embedded in or facilitates any number and type of other smart devices, such as (without limitation) autonomous machines or artificially intelligent agents, such as a mechanical agents or machines, electronics agents or machines, virtual agents or machines, electromechanical agents or machines, etc. Examples of autonomous machines or artificially intelligent agents may include (without limitation) robots, autonomous vehicles (e.g., self-driving cars, self-flying planes, self-sailing boats, etc.), autonomous equipment self-operating construction vehicles, self-operating medical equipment, etc.), and/or the like. Further, “autonomous vehicles” are not limed to automobiles but that they may include any number and type of autonomous machines, such as robots, autonomous equipment, household autonomous devices, and/or the like, and any one or more tasks or operations relating to such autonomous machines may be interchangeably referenced with autonomous driving. 
     Further, for example, computing device  400  may include a computer platform hosting an integrated circuit (“IC”), such as a system on a chip (“SOC” or “SOC”), integrating various hardware and/or software components of computing device  400  on a single chip. 
     As illustrated, in one embodiment, computing device  400  may include any number and type of hardware and/or software components, such as (without limitation) graphics processing unit (“GPU” or simply “graphics processor”)  416 , graphics driver (also referred to as “GPU driver”, “graphics driver logic”, “driver logic”, user-mode driver (UMD), user-mode driver framework (UMDF), or simply “driver”)  415 , central processing unit (“CPU” or simply “application processor”)  412 , hardware accelerator  414  (such as an FPGA, ASIC, a re-purposed CPU, or a re-purposed GPU, for example), memory  408 , network devices, drivers, or the like, as well as input/output (I/O) sources  404 , such as touchscreens, touch panels, touch pads, virtual or regular keyboards, virtual or regular mice, ports, connectors, etc. Computing device  400  may include operating system (OS)  406  serving as an interface between hardware and/or physical resources of the computing device  400  and a user. 
     It is to be appreciated that a lesser or more equipped system than the example described above may be utilized for certain implementations. Therefore, the configuration of computing device  400  may vary from implementation to implementation depending upon numerous factors, such as price constraints, performance requirements, technological improvements, or other circumstances. 
     Embodiments may be implemented as any or a combination of: one or more microchips or integrated circuits interconnected using a parent board, hardwired logic, software stored by a memory device and executed by a microprocessor, firmware, an application specific integrated circuit (ASIC), and/or a field programmable gate array (FPGA). The terms “logic”, “module”, “component”, “engine”, “circuitry”, “element”, and “mechanism” may include, by way of example, software, hardware and/or a combination thereof, such as firmware. 
     Computing device  400  may host network interface device(s) to provide access to a network, such as a LAN, a wide area network (WAN), a metropolitan area network (MAN), a personal area network (PAN), Bluetooth, a cloud network, a mobile network (e.g., 3rd Generation (3G), 4th Generation (4G), etc.), an intranet, the Internet, etc. Network interface(s) may include, for example, a wireless network interface having antenna, which may represent one or more antenna(s). Network interface(s) may also include, for example, a wired network interface to communicate with remote devices via network cable, which may be, for example, an Ethernet cable, a coaxial cable, a fiber optic cable, a serial cable, or a parallel cable. 
     Embodiments may be provided, for example, as a computer program product which may include one or more machine-readable media having stored thereon machine executable instructions that, when executed by one or more machines such as a computer, network of computers, or other electronic devices, may result in the one or more machines carrying out operations in accordance with embodiments described herein. A machine-readable medium may include, but is not limited to, floppy diskettes, optical disks, CD-ROMs (Compact Disc-Read Only Memories), and magneto-optical disks, ROMs, RAMS, EPROMs (Erasable Programmable Read Only Memories), EEPROMs (Electrically Erasable Programmable Read Only Memories), magnetic or optical cards, flash memory, or other type of media/machine-readable medium suitable for storing machine-executable instructions. 
     Moreover, embodiments may be downloaded as a computer program product, wherein the program may be transferred from a remote computer (e.g., a server) to a requesting computer (e.g., a client) by way of one or more data signals embodied in and/or modulated by a carrier wave or other propagation medium via a communication link (e.g., a modem and/or network connection). 
     Throughout the document, term “user” may be interchangeably referred to as “viewer”, “observer”, “speaker”, “person”, “individual”, “end-user”, and/or the like. It is to be noted that throughout this document, terms like “graphics domain” may be referenced interchangeably with “graphics processing unit”, “graphics processor”, or simply “GPU” and similarly, “CPU domain” or “host domain” may be referenced interchangeably with “computer processing unit”, “application processor”, or simply “CPU”. 
     It is to be noted that terms like “node”, “computing node”, “server”, “server device”, “cloud computer”, “cloud server”, “cloud server computer”, “machine”, “host machine”, “device”, “computing device”, “computer”, “computing system”, and the like, may be used interchangeably throughout this document. It is to be further noted that terms like “application”, “software application”, “program”, “software program”, “package”, “software package”, and the like, may be used interchangeably throughout this document. Also, terms like “job”, “input”, “request”, “message”, and the like, may be used interchangeably throughout this document. 
       FIG.  5    illustrates one embodiment of a distributed computing platform  500 , according to one implementation of the disclosure. As shown in  FIG.  5   , platform  500  includes computer systems A and B that are coupled (e.g., via a network). In one embodiment, each system includes host  510  (e.g.,  510 A and  510 B) and an accelerator  520  (e.g.,  520 A and  520 B). Each host operates a trusted execution environment (TEE)  505  (e.g.,  505 A and  505 B) that secures an application  501  and a user-mode driver (UMD)  502  (e.g.,  502 A and  502 B). In such an embodiment, TEE  505  provides for a secure interface (or channel) between each host  510  and its respective accelerator  520 . 
     According to one embodiment, a UMD  502  may issue instructions instructs received at a kernel mode driver (KMD) to perform an operation at an accelerator  520 A. For example, UMD  502 A may issue an instruction to perform an operation to store a result at a virtual address (e.g., result @ va1) at accelerator  520 A, while UMD  502 B may issue an instruction to perform an operation to read data from a first virtual address at accelerator  520 B and store the data at a second virtual address at accelerator  520 B (read from va1′ (=va1 @acc1) into va2′). 
     Each KMD  504  programs and controls a page table  524  within a respective accelerator, and facilitates execution of instructions from a host  510  at an accelerator  520  via page tables  524 . Page tables  524  include a mapping between virtual addresses (VAs) and physical addresses (PAs). For example, page table  524 A at accelerator  520 A maps va1 to pa1. Each accelerator  520  also includes local memory  522 , which includes a kernel (or program code) that, when executed by one or more processing units  540 , computes the results indicated by the UMD  502  operation and stores the result in a PA in memory  522  mapped to the VA indicated in the operation. For example, the kernel within memory  522 A, when executed by the one or more processing units  540 A, computes the result and writes the results to va1. As shown in page table  524 A va1 maps to pa1, which is where the result is stored within memory  522 A. 
     The operation within accelerator  520 B operates similarly. For example, the kernel within memory  522 B, when executed by the one or more processing units  540 B, reads from va1′ and computes a result that is stored into va2. However, page table  524 B indicates that va1′ maps to pa1 located in memory  522 A of accelerator  520 A. Thus, accelerator  520 B needs to read the memory  522 A. 
     However, a security issue exists with the sharing of memory  522 A accelerator  520 B. The problem is that local memory  522  is conventionally managed by the host KMD  504 , which is outside the trusted computing base (TCB) of the host TEE  505 . Accordingly, workloads in local memory  522  may be vulnerable to multiple different attacks, such as attacks via page table address translation manipulations. For example, data belonging to a victim on accelerator  520 A may be provided to an attacker running on accelerator  520 B, causing loss of confidentiality due to unauthorized access. Additionally, data may be read from an incorrect address on accelerator  520 A into accelerator  520 B (or incorrect number of bytes can be read), compromising the integrity of the result computed on accelerator  520 B. 
     According to one embodiment, each accelerator  520  incudes a trusted agent  530  (e.g.,  530 A and  530 B) that is implemented to validate address translations submitted by a KMD  504 . In such an embodiment, trusted agent  530  receives a randomly generated key (or token) from host  510 A that is used to validate an accelerator  520  (e.g., accelerator  520 B) requesting access to local memory  522  (e.g.,  522 A) access prior to sharing data. In a further embodiment, the token is shared with both trusted agents  530 A and  530 B prior to memory sharing, and is used to confirm that an external accelerator  520  has been authorized to access the local memory  522  of another accelerator  520 . Accordingly, trusted agents  530  use a token to validate address translations to ensure that memory sharing between accelerator  520 A and accelerator  520 B may be performed in a trusted manner. 
     In one embodiment, the token is shared during a shared memory protocol.  FIG.  6 A  is a flow diagram illustrating one embodiment of the memory sharing protocol. At processing block  610 , UMD  502 A (or UMD1) transmits a message to UMD  502 B (or UMD2) indicating an exported buffer at a VA (e.g., address va) of a location at memory  522 A that is to be shared. Additionally, the message includes a token (T) to be used authorize memory sharing. At processing block  620 , UMD1 transmits an authorization message to trusted agent  530 A (or trusted agent 1) indicating that trusted agent 1 is to export a particular byte length (e.g., len bytes) at address va to accelerator  520 B (or accelerator  2 ) upon receiving the token. 
     At processing block  630 , UMD2 transmits a message to trusted agent  530 B (or trusted agent 2) indicating that buffer of byte length is to be imported from virtual address va′ into memory  522 B at accelerator  2  using the token. In one embodiment, KMD  504 A and  504 B ensures that va maps to pa on  520 A and va′ maps to pa′ on  520 B, respectively, and that pa=pa′. As a result, the physical address of the exported buffer on  520 A matches the source address for the buffer imported on  520 B. At processing block  640 , confirms whether the translation entry (e.g., va′→pa′) is validated (e.g., whether the entry is present in page table  524 B. In one embodiment, the confirmation is performed by determining whether a present bit associated with the translation entry in the page table is enabled. Trusted agent 2 flags an unvalidated translation entry. 
     At processing block  650 , trusted agent 2 transmits a message to trusted agent 1 including the token and a request to import a byte length into physical address pa′ (e.g., import len, pa′, T). At processing block  660 , trusted agent 1 validates the request using the previous export authorization from UMD1. As a result, trusted agent 1 validates the requestor using the token (e.g., accelerator  2 , len, T). For example, trusted agent 1 determines whether the token received from UMD1 matches the token received from trusted agent 2. Additionally, trusted agent 1 validates whether the accelerator  1  and accelerator  2  physical addresses match (e.g., pa (from va→pa mapping)=pa′). 
     At processing block  670 , trusted agent 1 transmits a message to trusted agent 2 indicating that the results of the validation. In one embodiment, accelerator  520  is granted access to the memory  522 A upon a determination that the validation passes, and is not granted access to the memory  522 A upon a determination that the validation fails. At processing block  680 , trusted agent 2 flags its page table entry is valid upon receiving an indication that the physical addresses match. Subsequently, accelerator  2  has shared access to memory at accelerator  1 .  FIG.  6 B  is a sequence diagram illustrating another embodiment of the process. 
     Although described above with reference to memory reads, the above-described mechanism similarly processes memory writes. Thus, the above-described mechanism prevents untrusted software from bypassing sharing protocols since a trusted agent requires a protocol to be followed prior to permitting external memory access. Validation of token prevents a malicious application from requesting an import of memory that was not authorized, while physical address validation prevents copying from incorrect memory addresses or an incorrect number of bytes. Additionally, embodiments may be implemented in a network having more than accelerators  520 A and  520 B. In such embodiments, the above-described process is implemented to enable an external accelerator to access the local memory of another accelerator. 
       FIG.  7    is a schematic diagram of an illustrative electronic computing device to enable enhanced protection against adversarial attacks according to some embodiments. In some embodiments, the computing device  700  includes one or more processors  710  including one or more processors cores  718  and a TEE  764 , the TEE including a machine learning service enclave (MLSE)  780 . In some embodiments, the computing device  700  includes a hardware accelerator  768 , the hardware accelerator including a cryptographic engine  782  and a machine learning model  784 . In some embodiments, the computing device is to provide enhanced protections against ML adversarial attacks, as provided in  FIGS.  1 - 6   . 
     The computing device  700  may additionally include one or more of the following: cache  762 , a graphical processing unit (GPU)  712  (which may be the hardware accelerator in some implementations), a wireless input/output (I/O) interface  720 , a wired I/O interface  730 , memory circuitry  740 , power management circuitry  750 , non-transitory storage device  760 , and a network interface  770  for connection to a network  772 . The following discussion provides a brief, general description of the components forming the illustrative computing device  700 . Example, non-limiting computing devices  700  may include a desktop computing device, blade server device, workstation, or similar device or system. 
     In embodiments, the processor cores  718  are capable of executing machine-readable instruction sets  714 , reading data and/or instruction sets  714  from one or more storage devices  760  and writing data to the one or more storage devices  760 . Those skilled in the relevant art will appreciate that the illustrated embodiments as well as other embodiments may be practiced with other processor-based device configurations, including portable electronic or handheld electronic devices, for instance smartphones, portable computers, wearable computers, consumer electronics, personal computers (“PCs”), network PCs, minicomputers, server blades, mainframe computers, and the like. 
     The processor cores  718  may include any number of hardwired or configurable circuits, some or all of which may include programmable and/or configurable combinations of electronic components, semiconductor devices, and/or logic elements that are disposed partially or wholly in a PC, server, or other computing system capable of executing processor-readable instructions. 
     The computing device  700  includes a bus or similar communications link  716  that communicably couples and facilitates the exchange of information and/or data between various system components including the processor cores  718 , the cache  762 , the graphics processor circuitry  712 , one or more wireless I/O interfaces  720 , one or more wired I/O interfaces  730 , one or more storage devices  760 , and/or one or more network interfaces  770 . The computing device  700  may be referred to in the singular herein, but this is not intended to limit the embodiments to a single computing device  700 , since in certain embodiments, there may be more than one computing device  700  that incorporates, includes, or contains any number of communicably coupled, collocated, or remote networked circuits or devices. 
     The processor cores  718  may include any number, type, or combination of currently available or future developed devices capable of executing machine-readable instruction sets. 
     The processor cores  718  may include (or be coupled to) but are not limited to any current or future developed single- or multi-core processor or microprocessor, such as: on or more systems on a chip (SOCs); central processing units (CPUs); digital signal processors (DSPs); graphics processing units (GPUs); application-specific integrated circuits (ASICs), programmable logic units, field programmable gate arrays (FPGAs), and the like. Unless described otherwise, the construction and operation of the various blocks shown in  FIG.  7    are of conventional design. Consequently, such blocks need not be described in further detail herein, as they will be understood by those skilled in the relevant art. The bus  716  that interconnects at least some of the components of the computing device  700  may employ any currently available or future developed serial or parallel bus structures or architectures. 
     The system memory  740  may include read-only memory (“ROM”)  742  and random access memory (“RAM”)  746 . A portion of the ROM  742  may be used to store or otherwise retain a basic input/output system (“BIOS”)  744 . The BIOS  744  provides basic functionality to the computing device  700 , for example by causing the processor cores  718  to load and/or execute one or more machine-readable instruction sets  714 . In embodiments, at least some of the one or more machine-readable instruction sets  714  cause at least a portion of the processor cores  718  to provide, create, produce, transition, and/or function as a dedicated, specific, and particular machine, for example a word processing machine, a digital image acquisition machine, a media playing machine, a gaming system, a communications device, a smartphone, or similar. 
     The computing device  700  may include at least one wireless input/output (I/O) interface  720 . The at least one wireless I/O interface  720  may be communicably coupled to one or more physical output devices  722  (tactile devices, video displays, audio output devices, hardcopy output devices, etc.). The at least one wireless I/O interface  720  may communicably couple to one or more physical input devices  724  (pointing devices, touchscreens, keyboards, tactile devices, etc.). The at least one wireless I/O interface  720  may include any currently available or future developed wireless I/O interface. Example wireless I/O interfaces include, but are not limited to: BLUETOOTH®, near field communication (NFC), and similar. 
     The computing device  700  may include one or more wired input/output (I/O) interfaces  730 . The at least one wired I/O interface  730  may be communicably coupled to one or more physical output devices  722  (tactile devices, video displays, audio output devices, hardcopy output devices, etc.). The at least one wired I/O interface  730  may be communicably coupled to one or more physical input devices  724  (pointing devices, touchscreens, keyboards, tactile devices, etc.). The wired I/O interface  730  may include any currently available or future developed I/O interface. Example wired I/O interfaces include, but are not limited to: universal serial bus (USB), IEEE 1394 (“FireWire”), and similar. 
     The computing device  700  may include one or more communicably coupled, non-transitory, data storage devices  760 . The data storage devices  760  may include one or more hard disk drives (HDDs) and/or one or more solid-state storage devices (SSDs). The one or more data storage devices  760  may include any current or future developed storage appliances, network storage devices, and/or systems. Non-limiting examples of such data storage devices  760  may include, but are not limited to, any current or future developed non-transitory storage appliances or devices, such as one or more magnetic storage devices, one or more optical storage devices, one or more electro-resistive storage devices, one or more molecular storage devices, one or more quantum storage devices, or various combinations thereof. In some implementations, the one or more data storage devices  760  may include one or more removable storage devices, such as one or more flash drives, flash memories, flash storage units, or similar appliances or devices capable of communicable coupling to and decoupling from the computing device  700 . 
     The one or more data storage devices  760  may include interfaces or controllers (not shown) communicatively coupling the respective storage device or system to the bus  716 . The one or more data storage devices  760  may store, retain, or otherwise contain machine-readable instruction sets, data structures, program modules, data stores, databases, logical structures, and/or other data useful to the processor cores  718  and/or graphics processor circuitry  712  and/or one or more applications executed on or by the processor cores  718  and/or graphics processor circuitry  712 . In some instances, one or more data storage devices  760  may be communicably coupled to the processor cores  718 , for example via the bus  716  or via one or more wired communications interfaces  730  (e.g., Universal Serial Bus or USB); one or more wireless communications interfaces  720  (e.g., Bluetooth®, Near Field Communication or NFC); and/or one or more network interfaces  770  (IEEE 802.3 or Ethernet, IEEE 802.11, or Wi-Fi®, etc.). 
     Processor-readable instruction sets  714  and other programs, applications, logic sets, and/or modules may be stored in whole or in part in the system memory  740 . Such instruction sets  714  may be transferred, in whole or in part, from the one or more data storage devices  760 . The instruction sets  714  may be loaded, stored, or otherwise retained in system memory  740 , in whole or in part, during execution by the processor cores  718  and/or graphics processor circuitry  712 . 
     The computing device  700  may include power management circuitry  750  that controls one or more operational aspects of the energy storage device  752 . In embodiments, the energy storage device  752  may include one or more primary (i.e., non-rechargeable) or secondary (i.e., rechargeable) batteries or similar energy storage devices. In embodiments, the energy storage device  752  may include one or more supercapacitors or ultracapacitors. In embodiments, the power management circuitry  750  may alter, adjust, or control the flow of energy from an external power source  754  to the energy storage device  752  and/or to the computing device  700 . The power source  754  may include, but is not limited to, a solar power system, a commercial electric grid, a portable generator, an external energy storage device, or any combination thereof. 
     For convenience, the processor cores  718 , the graphics processor circuitry  712 , the wireless I/O interface  720 , the wired I/O interface  730 , the storage device  760 , and the network interface  770  are illustrated as communicatively coupled to each other via the bus  716 , thereby providing connectivity between the above-described components. In alternative embodiments, the above-described components may be communicatively coupled in a different manner than illustrated in  FIG.  7   . For example, one or more of the above-described components may be directly coupled to other components, or may be coupled to each other, via one or more intermediary components (not shown). In another example, one or more of the above-described components may be integrated into the processor cores  718  and/or the graphics processor circuitry  712 . In some embodiments, all or a portion of the bus  716  may be omitted and the components are coupled directly to each other using suitable wired or wireless connections. 
     Illustrative examples of the technologies disclosed herein are provided below. An embodiment of the technologies may include any one or more, and any combination of, the examples described below. 
     Example 1 includes a platform comprising an apparatus including a first host and a first accelerator communicatively coupled to the first host, including a first memory, a first page table to perform a translation of virtual addresses to physical addresses in the first memory and a first trusted agent to validate the address translations. 
     Example 2 includes the subject matter of Example 1, wherein the first trusted agent receives a first message from the first host including a token that is used to validate the address translations. 
     Example 3 includes the subject matter of any of Examples 1-2, wherein the first message further comprises a virtual address at the first memory that is to be shared and a length associated with the virtual address indicating a length of memory authorized to be exported. 
     Example 4 includes the subject matter of any of Examples 1-3, wherein the first host transmits a second message including the token to a second host to be shared with a second trusted agent at a second accelerator, wherein the token indicates that the second accelerator has been authorized to access the first memory. 
     Example 5 includes the subject matter of any of Examples 1-4, wherein the second message includes the first virtual address and the length associated with the virtual address. 
     Example 6 includes the subject matter of any of Examples 1-5, wherein the first trusted agent receives a third message from the second trusted agent including the token and a request to access the length at a physical address in the first memory. 
     Example 7 includes the subject matter of any of Examples 1-6, wherein the first trusted agent validates the request by determining whether the token received from the first host matches the token received from the second trusted agent. 
     Example 8 includes the subject matter of any of Examples 1-7, wherein the first trusted agent further validates the request by determining whether the physical address received from the second trusted agent matches a physical address associated with virtual address received from the host. 
     Example 9 includes the subject matter of any of Examples 1-8, wherein the first trusted agent transmits a fourth message to the second trusted agent indicating that the second accelerator has access to the first memory upon determining that the request has been validated. 
     Example 10 includes the subject matter of any of Examples 1-9, wherein the fourth message indicates that the second accelerator does not have access to the first memory upon determining that the request has not been validated. 
     Example 11 includes a method to facilitate secure memory sharing, comprising a first trusted agent at a first accelerator receiving a first message from a first host including a first token, the first trusted agent receiving a second message from a second trusted agent at a second accelerator requesting to access memory at the first accelerator, wherein the second message includes a second token and the trusted agent validating the request to determine whether the second accelerator is to be granted access to the memory. 
     Example 12 includes the subject matter of Example 11, wherein the second accelerator is granted the access to the memory upon a determination that the first token matches the second token. 
     Example 13 includes the subject matter of any of Examples 11-12, wherein the first message further comprises a virtual address at the first memory that is to be shared. 
     Example 14 includes the subject matter of any of Examples 11-13, wherein the request further comprises a physical address in the memory that is requested to be accessed. 
     Example 15 includes the subject matter of any of Examples 11-14, wherein the first trusted agent further validates the request by determining whether the physical address received from the second trusted agent matches a physical address associated with the virtual address received from the host. 
     Example 16 includes at least one computer readable medium having instructions stored thereon, which when executed by one or more processors, cause the processors to receive a first message from a first host including a first token, receive a second message from a trusted agent at an external accelerator requesting to access local memory, wherein the second message includes a second token and validate the request to determine whether the second accelerator is to be granted access to the memory. 
     Example 17 includes the subject matter of Example 16, wherein the second accelerator is granted the access to the memory upon a determination that the first token matches the second token. 
     Example 18 includes the subject matter of any of Examples 16-17, wherein the first message further comprises a virtual address at the first memory that is to be shared. 
     Example 19 includes the subject matter of any of Examples 16-18, wherein the request further comprises a physical address in the memory that is requested to be accessed. 
     Example 20 includes the subject matter of any of Examples 16-19, which when executed by one or more processors, further cause the processors to validate the request by determining whether the physical address received from the second trusted agent matches a physical address associated with the virtual address received from the host. 
     The above Detailed Description includes references to the accompanying drawings, which form a part of the Detailed Description. The drawings show, by way of illustration, specific embodiments that may be practiced. These embodiments are also referred to herein as “examples.” Such examples may include elements in addition to those shown or described. However, also contemplated are examples that include the elements shown or described. Moreover, also contemplated are examples using any combination or permutation of those elements shown or described (or one or more aspects thereof), either with respect to a particular example (or one or more aspects thereof), or with respect to other examples (or one or more aspects thereof) shown or described herein. 
     Publications, patents, and patent documents referred to in this document are incorporated by reference herein in their entirety, as though individually incorporated by reference. In the event of inconsistent usages between this document and those documents so incorporated by reference, the usage in the incorporated reference(s) are supplementary to that of this document; for irreconcilable inconsistencies, the usage in this document controls. 
     In this document, the terms “a” or “an” are used, as is common in patent documents, to include one or more than one, independent of any other instances or usages of “at least one” or “one or more.” In addition, “a set of” includes one or more elements. In this document, the term “or” is used to refer to a nonexclusive or, such that “A or B” includes “A but not B,” “B but not A,” and “A and B,” unless otherwise indicated. In the appended claims, the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and “wherein.” Also, in the following claims, the terms “including” and “comprising” are open-ended; that is, a system, device, article, or process that includes elements in addition to those listed after such a term in a claim are still deemed to fall within the scope of that claim. Moreover, in the following claims, the terms “first,” “second,” “third,” etc. are used merely as labels, and are not intended to suggest a numerical order for their objects. 
     The terms “logic instructions” as referred to herein relates to expressions which may be understood by one or more machines for performing one or more logical operations. For example, logic instructions may comprise instructions which are interpretable by a processor compiler for executing one or more operations on one or more data objects. However, this is merely an example of machine-readable instructions and examples are not limited in this respect. 
     The terms “computer readable medium” as referred to herein relates to media capable of maintaining expressions which are perceivable by one or more machines. For example, a computer readable medium may comprise one or more storage devices for storing computer readable instructions or data. Such storage devices may comprise storage media such as, for example, optical, magnetic or semiconductor storage media. However, this is merely an example of a computer readable medium and examples are not limited in this respect. 
     The term “logic” as referred to herein relates to structure for performing one or more logical operations. For example, logic may comprise circuitry which provides one or more output signals based upon one or more input signals. Such circuitry may comprise a finite state machine which receives a digital input and provides a digital output, or circuitry which provides one or more analog output signals in response to one or more analog input signals. Such circuitry may be provided in an application specific integrated circuit (ASIC) or field programmable gate array (FPGA). Also, logic may comprise machine-readable instructions stored in a memory in combination with processing circuitry to execute such machine-readable instructions. However, these are merely examples of structures which may provide logic and examples are not limited in this respect. 
     Some of the methods described herein may be embodied as logic instructions on a computer-readable medium. When executed on a processor, the logic instructions cause a processor to be programmed as a special-purpose machine that implements the described methods. The processor, when configured by the logic instructions to execute the methods described herein, constitutes structure for performing the described methods. Alternatively, the methods described herein may be reduced to logic on, e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC) or the like. 
     In the description and claims, the terms coupled and connected, along with their derivatives, may be used. In particular examples, connected may be used to indicate that two or more elements are in direct physical or electrical contact with each other. Coupled may mean that two or more elements are in direct physical or electrical contact. However, coupled may also mean that two or more elements may not be in direct contact with each other, but yet may still cooperate or interact with each other. 
     Reference in the specification to “one example” or “some examples” means that a particular feature, structure, or characteristic described in connection with the example is included in at least an implementation. The appearances of the phrase “in one example” in various places in the specification may or may not be all referring to the same example. 
     The above description is intended to be illustrative, and not restrictive. For example, the above-described examples (or one or more aspects thereof) may be used in combination with others. Other embodiments may be used, such as by one of ordinary skill in the art upon reviewing the above description. The Abstract is to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Also, in the above Detailed Description, various features may be grouped together to streamline the disclosure. However, the claims may not set forth every feature disclosed herein as embodiments may feature a subset of said features. Further, embodiments may include fewer features than those disclosed in a particular example. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment. The scope of the embodiments disclosed herein is to be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. 
     Although examples have been described in language specific to structural features and/or methodological acts, it is to be understood that claimed subject matter may not be limited to the specific features or acts described. Rather, the specific features and acts are disclosed as sample forms of implementing the claimed subject matter.