Method for key sharing between accelerators

A host processing device instructs a plurality of data processing (DP) accelerators to configure themselves for secure communications. The host device generates an adjacency table of each of the plurality of DP accelerators. Then the host device then establishes a session key communication with each DP accelerator and sends the DP accelerator a list of other DP accelerators that the DP accelerator is to establish a session key with, for secure communications between the DP accelerators. The DP accelerator establishes a different session key for each pair of the plurality of DP accelerators. When all DP accelerators have established a session key for communication with other DP accelerators, according to the respective list of other DP accelerators sent by the host device, then the host device can assign work tasks for performance by a plurality of DP accelerators, each communicating over a separately secured communication channel.

TECHNICAL FIELD

Embodiments of the present disclosure relate generally to artificial intelligence model training and inference. More particularly, embodiments of the disclosure relate to sharing keys between data processing accelerators configured to communicate with each other.

BACKGROUND

Data processing accelerators (DPAs) that are configured to communicate with a host computing device generally cannot communicate securely with each other. Communication, e.g. peer-to-peer communication, between DPAs is useful so that two or more DPAs can cooperate and coordinate to perform a data processing task on behalf of a host computing device. However, it is important that DPAs communicate securely so that the processing task performed by cooperating DPAs is performed securely such that another computing entity may not alter a result produced by the communicating DPAs, and may not steal code or data from any of the communicating DPAs.

DETAILED DESCRIPTION

The following embodiments relate to usage of a data processing (DP) accelerator to increase processing throughput of certain types of operations that may be offloaded (or delegated) from a host device to one or more DP accelerators. A DP accelerator can be a graphics processing unit (GPU), an artificial intelligence (AI) accelerator, math coprocessor, digital signal processor (DSP), or other type of processor. A DP accelerator can be a proprietary design, such as a Baidu® AI accelerator, or another GPU, and the like. While embodiments are illustrated and described with host device securely coupled to one or more DP accelerators, the concepts described herein can be implemented more generally as a distributed processing system.

The host device and the DP accelerator can be interconnected via a high-speed bus, such as a peripheral component interconnect express (PCIe), or other high-speed bus. The host device and DP accelerator can exchange keys and initiate a secure channel over the PCIe bus before performing operations of the aspects of the invention described below. Embodiments are described herein for generating one or more keys for securing communications between a host and a DP accelerator, and for securing communications between any two DP accelerators in a plurality of DP accelerators. In an embodiment, communications between any two DP accelerators use one or more keys that are unique with respect to any other two DP accelerators. Some of the operations include the DP accelerator using an artificial intelligence (AI) model to perform inferences using data provided by the host device. Before the AI model inferences are delegated to a DP accelerator, secure communication channels are established between the host and DP accelerator and between the DP accelerator and any other DP accelerator that may participate in the AI model inference.

In a first aspect, a computer-implemented method of a first data processing (DP) accelerator securely communicating with one or more second DP accelerators includes receiving, by the first DP accelerator, from a host device, a list of the one or more second DP accelerators. For each of the one or more second DP accelerators, a session key is determined between the first DP accelerator and the second DP accelerator. The session key is stored by the first DP accelerator in association with the second DP accelerator, and an indication is transmitted by the first DP accelerator, to the host device, that the first DP accelerator is configured for secure communication between with each of the one or more second DP accelerators. In an embodiment, the method further includes generating, by the first DP accelerator, an adjacency table with an entry for each of the one or more second DP accelerators. Each entry includes a reference to a communication port of the first DP accelerator for communicating with one of the one of more second DP accelerators. The entry further includes the session key stored in association with the one of the one or more second DP accelerators. In an embodiment, the session key generated for the first DP accelerator and the one of the one or more second DP accelerators is unique with respect to the first DP accelerator and any other of the one or more second DP accelerators. In an embodiment, each entry for one of the one or more second DP accelerators further includes a unique identifier of the one of the one or more second DP accelerators. The can method further include receiving, by the first DP accelerator, from the host device, a processing task to perform and an indication of one of the one or more second DP accelerators to perform at least a portion of the processing task. The method can further include instructing, by the first DP accelerator, the one of the one or more second DP accelerators to perform the at least a portion of the processing task, using a message that is secured with the session key associated with the first DP accelerator and the one of the one or more second DP accelerators. In an embodiment, the method can further include receiving, by the first DP accelerator, from the one of the one or more second DP accelerators, a message secured using the session key associated with the first DP accelerator and the one of the one or more second DP accelerators, the message comprising a result from the at least a portion of the task, and returning, by the first DP accelerator, a message containing a result of the processing task, the message secured using a session key that is associated with the first DP accelerator and the host device.

In some embodiments, the host device can transmit a kernel to the DP processing device to use in performing one or more operations. In this context, a kernel is a small piece of code, provided to the DP accelerator, to be executed by the DP accelerator to perform the intended function of the kernel. In an embodiment, a kernel is provided to the DP accelerator by the host device as a part of performing proof-of-trust operations by the DP accelerator that will be validated by the host device. In some embodiments, the DP accelerator is not aware of the purpose of the kernel it executes on behalf of the host device.

In some embodiments, the kernel can be a “watermark-enabled kernel.” A watermark-enabled kernel is a kernel that, when executed, is capable of extracting a watermark from an artificial intelligence (AI) model. An AI watermark is associated with a specific AI model and can be embedded or “implanted,” within the AI model using several different methods. The watermark may be implanted into one or more weight variables of the one or more nodes of the AI model. In an embodiment, the watermark is stored in one or more bias variables of the one or more nodes of the AI modes, or by creating one or more additional nodes of the AI model during the training to store the watermark.

In second aspect, a host device includes a processing system includes one or more processors and a memory coupled to the processor to store instructions. The instructions, when executed by the processor, cause the processor to perform operations for the host device that configure a plurality of data processing (DP) accelerators for secure communication. The operations include, for each DP accelerator in the plurality of DP accelerators: establishing a session key between the host device and the DP accelerator, and storing, by the host device, the session key in an adjacency table for the host, wherein each entry in the adjacency table includes: the session key, a reference to a communication port of the host device for communicating with the DP accelerator, and a unique identifier of the DP accelerator. The adjacency table enables the host device to communicate securely with each DP accelerator in the plurality of DP accelerators, using a session key that is unique to the particular DP accelerator that the host device is communicating with. The operations further include the host device transmitting, to the DP accelerator, a list of one or more additional DP accelerators, and an instruction that the DP accelerator generate a unique session key for each additional DP accelerator for securing communication between the DP accelerator and the additional DP accelerator in the list of one or more additional DP accelerators. The operations further include receiving, from the DP accelerator, an indication that the DP accelerator has finished generating the unique session keys for communicating between the DP accelerator and the one or more additional DP accelerators. Once the DP accelerator acknowledges that the DP accelerator has finished establishing a unique session key for secure communications for each of the additional DP accelerators, the host device can begin assigning processing tasks to DP accelerators. In an embodiment, the operations further include transmitting, by the host device, to the DP accelerator, a processing task for the DP accelerator and at least one of the one or more additional DP accelerators to perform, the transmitting secured using the session key established for the host device and the DP accelerator. The host device can receive, from the DP accelerator, a result from the processing task. The result can be secured using the session key established for the host device and the DP accelerator. In an embodiment, transmitting the processing task can include transmitting instructions from the host device to the DP accelerator that a sub-task of the processing task is to be performed by a specific one of the one or more additional DP accelerators. In an embodiment, the processing task can be one or more of: an artificial intelligence inference, encrypting or decrypting data, digitally signing data, extracting a watermark from an AI model, implanting a watermark into an AI model, implanting a watermark into an AI model inference result, or combinations of these.

Any of the above functionality can be programmed as executable instructions onto one or more non-transitory computer-readable media. When the executable instructions are executed by a processing system having at least one hardware processor, the processing systems causes the functionality to be implemented.

Any of the above functionality can be implemented by a processing system having at least one hardware processor, coupled to a memory programmed with executable instructions that, when executed, cause the processing system to implement the functionality.

FIG.1is a block diagram illustrating an example of system configuration for securing communication between a host104and data processing (DP) accelerators105-107according to some embodiments. Referring toFIG.1, system configuration100includes, but is not limited to, one or more client devices101-102communicatively coupled to DP server104(e.g. host) over network103. Client devices101-102may be any type of client devices such as a personal computer (e.g., desktops, laptops, and tablets), a “thin” client, a personal digital assistant (PDA), a Web enabled appliance, a Smart watch, or a mobile phone (e.g., Smartphone), etc. Alternatively, client devices101-102may be other servers. Network103may be any type of networks such as a local area network (LAN), a wide area network (WAN) such as the Internet, or a combination thereof, wired or wireless.

Server (e.g., host)104may be any kind of servers or a cluster of servers, such as Web or cloud servers, application servers, backend servers, or a combination thereof. Server104further includes an interface (not shown) to allow a client such as client devices101-102to access resources or services (such as resources and services provided by DP accelerators via server104) provided by server104. For example, server104may be a cloud server or a server of a data center that provides a variety of cloud services to clients, such as, for example, cloud storage, cloud computing services, artificial intelligence training services, data mining services, etc. Server104may be configured as a part of software-as-a-service (SaaS) or platform-as-a-service (PaaS) system over the cloud, which may be a private cloud, public cloud, or a hybrid cloud. The interface may include a Web interface, an application programming interface (API), and/or a command line interface (CLI).

For example, a client, in this example, a user application of client device101(e.g., Web browser, application), may send or transmit an instruction (e.g., artificial intelligence (AI) training, inference instruction, etc.) for execution to server104and the instruction is received by server104via the interface over network103. In response to the instruction, server104communicates with DP accelerators105-107to fulfill the execution of the instruction. In some embodiments, the instruction is a machine learning type of instruction where DP accelerators, as dedicated machines or processors, can execute the instruction many times faster than execution by server104. Server104thus can control/manage an execution job for the one or more DP accelerators in a distributed fashion. Server104then returns an execution result to client devices101-102. A DP accelerator or AI accelerator may include one or more dedicated processors such as a Baidu® artificial intelligence (AI) chipset available from Baidu, Inc.® or alternatively, the DP accelerator may be an AI chipset from another AI chipset provider.

According to one embodiment, each of the applications accessing any of DP accelerators105-107hosted by data processing server104(also referred to as a host) may verify that the application is provided by a trusted source or vendor. Each of the applications may be launched and executed within a trusted execution environment (TEE) specifically configured and executed by a central processing unit (CPU) of host104. When an application is configured to access any one of the DP accelerators105-107, an obscured connection can be established between host104and the corresponding one of the DP accelerator105-107, such that the data exchanged between host104and DP accelerators105-107is protected against attacks from malware/intrusions.

FIG.2Ais a block diagram illustrating an example of a multi-layer protection solution for obscured communications between a host system104and data process (DP) accelerators105-107according to some embodiments. In one embodiment, system200provides a protection scheme for obscured communications between host104and DP accelerators105-107with or without hardware modifications to the DP accelerators. Referring toFIG.2A, host machine or server104can be depicted as a system with one or more layers to be protected from intrusion such as user application(s)205, runtime libraries206, driver209, operating system211, and hardware213(e.g., security module (trusted platform module (TPM))/central processing unit (CPU)). Memory safe applications207can run in a sandboxed memory. Below the applications205and run-time libraries206, one or more drivers209can be installed to interface to hardware213and/or to DP accelerators105-107.

Hardware213can include one or more processor(s)201and storage device(s)204. Storage device(s)204can include one or more artificial intelligence (AI) models202, and one or more kernels203. Kernels203can include signature kernels, watermark-enabled kernels, encryption and/or decryption kernels, and the like. A signature kernel, when executed, can digitally sign any input in accordance with the programming of the kernel. A watermark-enabled kernel can extract a watermark from a data object (e.g. an AI model or other data object). A watermark-enabled kernel can also implant a watermark into an AI model, an inference output, or other data object. A watermark kernel (e.g. a watermark inherited kernel) can inherit a watermark from another data object and implant that watermark into a different object, such as an inference output or an AI model. A watermark, as used herein, is an identifier associated with, and can be implanted into, an AI model or an inference generated by an AI model. For example, a watermark may be implanted in one or more weight variables or bias variables. Alternatively, one or more nodes (e.g., fake nodes that are not used or unlikely used by the artificial intelligence model) may be created to implant or store the watermark.

Host machine104is typically a CPU system which can control and manage execution of jobs on the host machine104and/or DP accelerators105-107. In order to secure/obscure a communication channel215between DP accelerators105-107and host machine104, different components may be required to protect different layers of the host system that are prone to data intrusions or attacks. For example, a trusted execution environment (TEE) can protect the user application205layer and the runtime library206layer from data intrusions.

System200includes host system104and DP accelerators105-107according to some embodiments. DP accelerators can include Baidu® AI chipsets or another AI chipset such as a graphical processing units (GPUs) that can perform artificial intelligence (AI)-intensive computing tasks. In one embodiment, host system104includes a hardware that has one or more CPU(s)213equipped with a security module (such as a trusted platform module (TPM)) within host machine104. A TPM is a specialized chip on an endpoint device that stores cryptographic keys (e.g., RSA cryptographic keys) specific to the host system for hardware authentication. Each TPM chip can contain one or more RSA key pairs (e.g., public and private key pairs) called endorsement keys (EK) or endorsement credentials (EC), i.e., root keys. The key pairs are maintained inside the TPM chip and cannot be accessed by software. Critical sections of firmware and software can then be hashed by the EK or EC before they are executed to protect the system against unauthorized firmware and software modifications. The TPM chip on the host machine can thus be used as a root of trust for secure boot. The TPM chip can include a secure memory for storing keys that are rooted, e.g. in hardware, and keys that are derived from the rooted keys. In an embodiment, secure storage can include a rooted asymmetric key pair (RK): a public key (PK_RK) and a private key (SK_RK) of the asymmetric rooted key (RK) pair.

The TPM chip also secure driver(s)209and operating system (OS)211in a working kernel space to communicate with the DP accelerators105-107. Here, driver209is provided by a DP accelerator vendor and can serve as a driver for the user application to control a communication channel(s)215between host and DP accelerators. Because the TPM chip and secure boot processor protects the OS211and drivers209in their kernel space, TPM also effectively protects the driver209and OS211.

Since communication channels215for DP accelerators105-107may be exclusively occupied by the OS211and driver209, thus, communication channels215can be secured through the TPM chip. In one embodiment, communication channels215include a peripheral component interconnect or peripheral component interconnect express (PCIE) channel. In one embodiment, communication channels215are obscured communication channels. Communication channels may be connected to one or more hardware communication ports, accessible by drivers209, for communicating over communication channels215with DP accelerators105-107. Communication channels215may be secured using a session key as described herein. Each communication channel215may be secured using a different session key than other communication channels215. Drivers209may include an adjacency table that maps DP accelerators105-107each to a hardware communication port, and a session key associated with each hardware communication port.

Host machine104can include trusted execution environment (TEE)210which is enforced to be secure by TPM/CPU213. A TEE is a secure environment. TEE can guarantee code and data which are loaded inside the TEE to be protected with respect to confidentiality and integrity. Examples of a TEE may be Intel® software guard extensions (SGX), or AMD® secure encrypted virtualization (SEV). Intel® SGX and/or AMD® SEV can include a set of central processing unit (CPU) instruction codes that allows user-level code to allocate private regions of memory of a CPU that are protected from processes running at higher privilege levels. Here, TEE210can protect user applications205and runtime libraries206, where user application205and runtime libraries206may be provided by end users and DP accelerator vendors, respectively. Here, runtime libraries206can convert application programming interface (API) calls to commands for execution, configuration, and/or control of the DP accelerators. In one embodiment, runtime libraries206provides a predetermined set of (e.g., predefined) kernels for execution by the user applications. In an embodiment, the kernels may be stored in storage device(s)204as kernels203.

Host machine104can include memory safe applications207which are implemented using memory safe languages such as Rust, and GoLang, etc. These memory safe applications running on memory safe Linux® releases, such as MesaLock Linux®, can further protect system200from data confidentiality and integrity attacks. However, the operating systems may be any Linux® distributions, UNIX®, Windows® OS, or Mac® OS.

The host machine104can be set up as follows: A memory safe Linux® distribution is installed onto a system equipped with TPM secure boot. The installation can be performed offline during a manufacturing or preparation stage. The installation can also ensure that applications of a user space of the host system are programmed using memory safe programming languages. Ensuring other applications running on host system104to be memory safe applications can further mitigate potential confidentiality and integrity attacks on host system104.

After installation, the system can then boot up through a TPM-based secure boot. The TPM secure boot ensures only a signed/certified operating system and accelerator driver are launched in a kernel space that provides the accelerator services. In one embodiment, the operating211system can be loaded through a hypervisor (not shown). A hypervisor or a virtual machine manager is a computer software, firmware, or hardware that creates and runs virtual machines. A kernel space is a declarative region or scope where kernels (i.e., a predetermined set of (e.g., predefined) functions for execution) are identified to provide functionalities and services to user applications. In the event that integrity of the system is compromised, TPM secure boot may fail to boot up and instead shuts down the system.

After secure boot, runtime libraries206runs and creates TEE210, which places runtime libraries206in a trusted memory space associated with CPU213. Next, user application205is launched in TEE210. In one embodiment, user application205and runtime libraries206are statically linked and launched together. In another embodiment, runtime library206is launched in TEE210first and then user application205is dynamically loaded in TEE210. In another embodiment, user application205is launched in TEE first, and then runtime206is dynamically loaded in TEE210. Statically linked libraries are libraries linked to an application at compile time. Dynamic loading can be performed by a dynamic linker. Dynamic linker loads and links shared libraries for running user applications at runtime. Here, user applications205and runtime libraries206within TEE210are visible to each other at runtime, e.g., all process data are visible to each other. However, external access to the TEE is denied.

In one embodiment, the user application205can only call a kernel from a set of kernels as predetermined by runtime libraries206. In another embodiment, user application205and runtime libraries206are hardened with side channel free algorithm to defend against side channel attacks such as cache-based side channel attacks. A side channel attack is any attack based on information gained from the implementation of a computer system, rather than weaknesses in the implemented algorithm itself (e.g. cryptanalysis and software bugs). Examples of side channel attacks include cache attacks which are attacks based on an attacker's ability to monitor a cache of a shared physical system in a virtualized environment or a cloud environment. Hardening can include masking of the cache, outputs generated by the algorithms to be placed on the cache. Next, when the user application finishes execution, the user application terminates its execution and exits from the TEE.

In one embodiment, TEE210and/or memory safe applications207are not necessary, e.g., user application205and/or runtime libraries206are hosted in an operating system environment of host104.

In one embodiment, the set of kernels include obfuscation kernel algorithms. In one embodiment, the obfuscation kernel algorithms can be symmetric or asymmetric algorithms. A symmetric obfuscation algorithm can obfuscate and de-obfuscate data communications using a same algorithm. An asymmetric obfuscation algorithm requires a pair of algorithms, where a first of the pair is used to obfuscate and the second of the pair is used to de-obfuscate, or vice versa. In another embodiment, an asymmetric obfuscation algorithm includes a single obfuscation algorithm used to obfuscate a data set but the data set is not intended to be de-obfuscated, e.g., there is absent a counterpart de-obfuscation algorithm. Obfuscation refers to obscuring of an intended meaning of a communication by making the communication message difficult to understand, usually with confusing and ambiguous language. Obscured data is harder and more complex to reverse engineering. An obfuscation algorithm can be applied before data is communicated to obscure (cipher/decipher) the data communication reducing a chance of eavesdrop. In one embodiment, the obfuscation algorithm can further include an encryption scheme to further encrypt the obfuscated data for an additional layer of protection. Unlike encryption, which may be computationally intensive, obfuscation algorithms may simplify the computations. Some obfuscation techniques can include but are not limited to, letter obfuscation, name obfuscation, data obfuscation, control flow obfuscation, etc. Letter obfuscation is a process to replace one or more letters in a data with a specific alternate letter, rendering the data meaningless. Examples of letter obfuscation include a letter rotate function, where each letter is shifted along, or rotated, a predetermine number of places along the alphabet. Another example is to reorder or jumble up the letters based on a specific pattern. Name obfuscation is a process to replace specific targeted strings with meaningless strings. Control flow obfuscation can change the order of control flow in a program with additive code (insertion of dead code, inserting uncontrolled jump, inserting alternative structures) to hide a true control flow of an algorithm/AI model. Systems and methods for sharing keys used for obfuscation are described herein, below.

In summary, system200provides multiple layers of protection for DP accelerators (for data transmissions including machine learning models, training data, and inference outputs) from loss of data confidential and integrity. System200can include a TPM-based secure boot protection layer, a TEE protection layer, and a kernel validation/verification layer. Furthermore, system200can provide a memory safe user space by ensuring other applications on the host machine are implemented with memory safe programming languages, which can further eliminate attacks by eliminating potential memory corruptions/vulnerabilities. Moreover, system200can include applications that use side-channel free algorithms so to defend against side channel attacks, such as cache based side channel attacks.

Runtime206can provide obfuscation kernel algorithms to obfuscate data communication between a host104and DP accelerators105-107. In one embodiment, the obfuscation can be pair with a cryptography scheme. In another embodiment, the obfuscation is the sole protection scheme and cryptography-based hardware is rendered unnecessary for the DP accelerators.

FIG.2Bis a block diagram illustrating an example of a host channel manager (HCM)259communicatively coupled to one or more accelerator channel managers (ACMs)270that interface to DP accelerators105-107, according to some embodiments. Referring toFIG.2B, in one embodiment, HCM259includes authentication module251, termination module252, key manager253, key(s) store254, and cryptography engine255. Authentication module251can authenticate a user application running on host server104for permission to access or use a resource of a DP accelerator105. Termination module252can terminate a connection (e.g., channels associated with the connection would be terminated). Key manager253can manage (e.g., create or destroy) asymmetric key pairs or symmetric keys for encryption/decryption of one or more data packets for different secure data exchange channels. Here, each user application (as part of user applications205ofFIG.2A) can correspond or map to different secure data exchange channels, on a one-to-many relationship, and each data exchange channel can correspond to a DP accelerator105. Each application can utilize a plurality of session keys, where each session key is for a secure channel corresponding to a DP accelerator (e.g., accelerators105-107). Key(s) store254can store encryption asymmetric key pairs or symmetric keys. Cryptography engine255can encrypt or decrypt a data packet for the data exchanged through any of the secure channels. Note that some of these modules can be integrated into fewer modules.

In one embodiment, DP accelerator105includes ACM270and security unit (SU)275. Security unit275can include key manager271, key(s) store272, true random number generator273, and cryptography engine274. Key manager271can manage (e.g., generate, safe keep, and/or destroy) asymmetric key pairs or symmetric keys. Key(s) store272can store the cryptography asymmetric key pairs or symmetric keys in secure storage within the security unit275. True random number generator273can generate seeds for key generation and cryptographic engine274uses. Cryptography engine274can encrypt or decrypt key information or data packets for data exchanges. In some embodiments, ACM270and SU275is an integrated module.

FIG.3is a block diagrams illustrating a method300of a host and data processing accelerator, or two data processing accelerators, generating a session key for securing communications, according to an embodiment. Method300can be used between a first data processing (DP) accelerator “Accelerator1” and a second node, “Node2.” Node2can be either a host device or second DP accelerator. Accelerator1has a rooted key pair PK_RK1and SK_RK1. PK_RK1is a public key of a rooted asymmetric key pair of Accelerator1(RK1). SK_RK1is a private (secret) key (SK) of rooted asymmetric key pair of Accelerator1(RK1). Rooted key pair RK1is stored in a secured storage of Accelerator1. Similarly, Node2(either a host or another DP accelerator) has a rooted key pair PK_RK2and SK_RK2. RK2can be stored in a secure storage of Node2.

In operation301, Accelerator1generates a derived asymmetric key pair, PK_D1and SK_D1, from rooted key pair PK_RK1and SK_RK1. Deriving an asymmetric key pair is known in the art and will not be described herein.

In operation302, Accelerator1sends to Node2, a “Get Public Key” command (GET_PUB_KEY) to request a public key of Node2. The GET_PUB_KEY includes encrypting two of Accelerator1's public keys: PK_RK1and PK_D1. In an embodiment, PK_RK1and PK_D1can be encrypting using Accelerator1's private rooted key SK_RK1. The GET_PUB_KEY command further includes Accelerator1's public rooted key, PK_RK1in clear-text form. Node2can decrypt Accelerator1's encrypted keys using PK_RK1and verify that the GET_PUB_KEY request did, in fact, come from Accelerator1.

In operation303, Node2generates a derived asymmetric key pair PK_D2and SK_D2from Node2's rooted key pair PK_RK2and SK_RK2. Derived keys PK_D2and SK_D2can be stored in secure storage at Node2.

In operation304, Node2can decrypt the received “GET_PUB_KEY” command from Accelerator1, using the clear-text public rooted key of Accelerator1: PK_RK1. Once decrypted, Node2obtains Accelerator1's derived public key: PK_D1.

In operation305, Node2sends to Accelerator1a “Return Public Key” (RET_PUB_KEY) message. The message includes Node2's PK_RK2and PK_D2, encrypted using Node2's private rooted key, SK_RK2. Node2's public rooted key PK_RK2is packaged with the encrypted keys PK_RK2and PK_D2, and packaged keys are then encrypted using Accelerator1's derived public key PK_D1.

In operation306, Accelerator1decrypts the RET_PUB_KEY message using Accelerator1's private derived key SK_D1. After decryption, Accelerator1can obtain Node2's public rooted key, PK_RK2. Accelerator1then decrypts the encrypted keys PK_RK2and PK_D2using Node2's newly-obtained public rooted key, PK_RK2. Accelerator1can then obtain Node2's derived public key, PK_D2. In an embodiment, Accelerator1can verify PK_RK2either, or both, the decrypted PK_RK2and clear-text PK_RK2by checking with the host device or a history copy of PK_RK2.

In operation308, Accelerator1can send a command “Generate Session Key” (CMD_SESS_KEY) to Node2. The command includes nonce nc1, encrypted using Node2's public derived key PK_D2. CMD_SESS_KEY instructs Node2to generate a session key from Accelerator1's nonce nc1and a nonce nc2that is generated by Node2.

In operation309, Node2can decrypt nonce nc1in the received CMD_SESS_KEY using Node2's private derived key SK_D2.

In operation310, Node2can generate a nonce, nc2. Node2can then generate a session key, based on nonces nc1and nc2. Node2stores the session key in an adjacency table of Node2. The session key is stored in association with Accelerator1and a unique identifier of Accelerator1.

In operation311, Node2can send nonce nc2to Accelerator1. Node2packages nc1, nc2, and PK_D1in a first package and encrypts the first package using Node2's private derived key, SR_D2. Node2then adds PK_D2to the encrypted first package, and generates a second encrypted package that is encrypted using Accelerator1's public derived key, PK_D1. The encrypted second package is then transmitted to Accelerator1.

In operation312, Accelerator1receives the encrypted second package from Node2and decrypts the second package using Accelerator1's derived private key, SK_D1. Accelerator1can then remove PK_D2from the decrypted second package, leaving just the encrypted first package. In an embodiment, Accelerator1can verify that PK_D2removed from the decrypted second package matches the PK_D2previously received in operation305and decrypted in operation306, above. Accelerator1can also verify that the nc1obtained from the decrypted first package, and previously sent to Node2in operation308, has not expired (aka, “verify freshness”). Accelerator1can then generate a session key based upon nonces nc1and nc2. Accelerator1can store the generated session key in Accelerator1's adjacency table, in association with a unique identifier of the Node2and the session key.

At this point, both Accelerator1and Node2have a same session key that was derived from nonces nc1and nc2Both Accelerator1and Node2have stored the session key in their respective adjacency tables. Adjacency tables are described in detail, below, with reference toFIG.5.

FIG.4is a block diagram illustrating a hardware configuration400of a host computing device104and a plurality of data processing accelerators105-107that securely communicate with one another, according to an embodiment.

Host104is communicatively coupled to each of DP accelerator105,106, and107. Host104includes a communication interface having, e.g., ports0,1, and2. InFIG.4, DP accelerator105's communication port0is communicatively coupled to host104's communication port0. DP accelerator106's communication port0is communicatively coupled to host104's communication port1. DP accelerator107's communication port0is communicatively coupled to host104's communication port2. DP accelerator's105-107are also communicatively coupled to each other. DP accelerator105's communication port1is communicatively coupled to DP accelerator106's communication port1. DP accelerator105's communication port2is communicatively coupled to DP accelerator107's communication port2. DP accelerator106's communication port2is communicatively coupled to DP accelerator107's communication port1. Each of the foregoing communication channels is secured by a different session key for the other communication channels. Thus, is any one of the communication channels is compromised, the other communication channels are still secure. Further, there is redundancy in communication with respect to the host device104. Each DP accelerator105-107can monitor their own communication ports to ensure that the each communication channel is operable. If a channel fails, one or both of the DP accelerators at either end of the channel can notify the host104of the failed communication channel.

Each of the host104, and DP accelerator105,106, and107, can have an adjacency table that stores a list of nodes (DP accelerators or host) that the host104or DP accelerator105-107is communicatively coupled to. Adjacency tables are described below, with reference toFIG.5.

FIG.5is a block diagram illustrating secure communications adjacency tables500,510,520, and530between a host device104and a plurality of data processing (DP) accelerators105-107, according to an embodiment.

As shown inFIG.4, above, host104and DP accelerators105-107are communicatively coupled via communication ports on each of the host and DP accelerators. Host104, e.g., can have an adjacency table500that lists the DP accelerators (DPA) that are communicatively coupled to host104. DP accelerators, e.g.105-107, can have a unique ID501, e.g. DP_105_ID, etc., so that the DP accelerator can be referred to by name. In an embodiment, when a host wants to send a message to a DP accelerator, the message can have the format [source, message payload, destination]. Host can refer to itself, as sender, by its own ID501e.g. HOST_104_ID. Host can refer to a destination DP accelerator by its unique ID501, by a port502to which the DP accelerator is connected at the host, or by the address503in memory to which DP accelerator port is connected at the host. Thus, if host having ID501of HOST_104_ID sends a message to DP accelerator106, the host can look up the ID501of DP accelerator106, or the port502, or address of the port503to use as the destination address for the message. The message can be encrypted using the session key504for the host and DP accelerator106. Similarly, DP accelerator105can have an adjacency table510, stored in memory of DP accelerator105, indicating an ID, port512, address513, and session key514for communicating with each of host104, DP accelerator106, or DP accelerator107. DP accelerator106can have an adjacency table520, stored in memory of DP accelerator106, indicating an ID521, port522, address523, and session key524for communicating with each of host104, DP accelerator105, and DP accelerator107. DP accelerator107can have an adjacency table530, stored in memory of DP accelerator107, indicating an ID531, port532, address533, and session key534of host104, DP accelerator105, and DP accelerator107.

Determining and generating session keys for each channel between two devices (host to DP accelerator, or DP accelerator to DP accelerator) are described above with reference toFIG.3, and a method is described below with reference toFIG.6. A session key of NULL indicates that the session key has not yet been determined between the two nodes (host or DP accelerator) referenced in the line item of the adjacency table having the NULL session key. For example, DP accelerator106adjacency table520indicates a line item for DP accelerator105, having unique ID DPA_105_ID, and a null session identifier. The null session identifier indicates that DP accelerator106and DP accelerator105have not yet determined a session key for communication between DP accelerator106and DP accelerator105.

FIG.6is block diagrams illustrating a method of a host device instructing a plurality of data processing accelerators to configure themselves for secure communications, according to an embodiment.

In operation601, a host, e.g. host104, generates and stores an adjacency table that lists each DP accelerator that is configured for communication with the host. In an embodiment, one or more DP accelerators can be configured by a system administrator using a configuration file. The configuration file can indicate which DP accelerators can communicate with which other DP accelerators. The configuration file can specify the unique identifier for the host and DP accelerators, the specific communication port number to which each DP accelerator is assigned, and/or the memory address corresponding to the host communication port number associated with the DP accelerator. There can be any number of DP accelerators. For simplicity, one host,104, and three DP accelerators, e.g.105-107, are described. The generated adjacency table for the host can be similar to host table500, described above with reference toFIG.5.

In operation602, logic in the host can iterate through the list of DP accelerators configured for the host. For each DP accelerator, operations603through605can be performed. In there are no more DP accelerators in the list, then method600ends.

In operation603, host selects a DP accelerator from the list and generates a session key with the selected DP accelerator. Generating a session key between an accelerator and a host (Node) is described above with reference toFIG.3. Host stores the generated session key in an entry in the adjacency table corresponding to the selected DP accelerator. Host uses the configuration file complete the entry in the adjacency table, including the unique identifier of the DP accelerator, the port number of the host for communicating with the DP accelerator, and the memory address of the port. In an embodiment, the memory address can be calculated from a base address of the communication ports, and an offset in memory for each port number.

In operation604, host transmits instructions to the selected DP accelerator for the DP accelerator to create its own adjacency table. The information in the host-transmitted instructions can be obtained from the configuration file. The instructions include a list of other DP accelerators that the selected DP accelerator is to include when the selected DP accelerator generates its own adjacency table. The instructions can further include a unique identifier of each of the other DP accelerators, a port number of the selected DP accelerator to assign to each of the other DP accelerators, a memory address to assign to each of the other DP accelerators, and a NULL value for the session key associated with each of the other DP accelerators. The instructions further include an instruction that the selected DP accelerator is to generate its own adjacency table, and to generate and store a session key with each of the other DP accelerators in the adjacency table of the selected DP accelerator. A method for a selected DP accelerator to generate its own adjacency table is described below with reference toFIG.7.

In operation605, host receives a signal from the selected DP accelerator that the selected DP accelerator has generated its own adjacency table, populated the adjacency table with the information provided in operation604, above, and has generated and stored a session key for each of the other DP accelerators in the selected DP accelerator's adjacency table. Method600continues at operation602.

FIG.7is a block diagram illustrating a method700of a data processing accelerator configuring itself for secure communication with one or more other data processing accelerators, according to an embodiment.

In operation701, a DP accelerator (“this” DP accelerator) receives instructions from a host device to generate an adjacency table for this DP accelerator. The information in the host-transmitted instructions can be obtained by the host from an administrator-created configuration file. In an embodiment, the instructions can be default instructions. The instructions include a list of other DP accelerators that the DP accelerator is to include when this DP accelerator generates its own adjacency table. The instructions can further include a unique identifier of each of the other DP accelerators, a port number of this DP accelerator to assign to each of the other DP accelerators in the instructions, a memory address to assign to each of the other DP accelerators, and a NULL value for the session key associated with each of the other DP accelerators. The instructions further include an instruction that the DP accelerator is to generate its own adjacency table, and to generate and store a session key with each of the other DP accelerators in the adjacency table of this DP accelerator.

In operation702, the DP accelerator stores the adjacency table that lists each of the other DP accelerators that this DP accelerator is to generate and store a session key for.

In operation703, logic of the DP accelerator iterates through the list of other DP accelerators. If there are more DP accelerators in the list of other DP accelerators, then the logic selects a next DP accelerator from the list.

In operation704, the (“this”) DP accelerator and the selected DP accelerator generate a session key for use in communicating between this DP accelerator and the selected DP accelerator. Generating a session key between a DP accelerator and a node (host or DP accelerator) is described above with reference toFIG.3. DP accelerator logic stores the session key in its adjacency table for this DP accelerator, in association with the selected DP accelerator.

In operation705, if there are no more DP accelerators in the list, and thus no more session keys to generate, then this DP accelerator transmits a message or signal to the host that this DP accelerator has finished generating its adjacency table and has generated a session key for secure communication with each of the other DP accelerators in the adjacency table. In an embodiment, each session key in the adjacency table is different than other session keys in the adjacency table.

FIG.8is block diagram illustrating a method800of a data processing accelerator receiving a processing task from a host and performing one or more sub-tasks of the tasks by one or more additional data processing accelerators, according to an embodiment.

In operation801, a DP accelerator receives a processing task from a host device. In an embodiment, the processing task includes instructions on dividing the processing task into sub-tasks that are to be processed on at least on additional DP accelerator, and the DP accelerator has an entry in the adjacency table of the DP accelerator for securely communicating with the at least one additional DP accelerator. In this embodiment, it is assumed that host determined that the at least one additional DP accelerator is, or soon will be, idle such that the at least one additional DP accelerator can perform one or more sub-tasks on behalf of the DP accelerator.

In operation802, the DP accelerator transmits one or more sub-tasks to the at least one additional DP accelerator with instructions to perform the sub-task(s). The at least one additional DP accelerator performs the one or more sub-tasks.

In operation803, the DP accelerator also performs one or more sub-tasks of the received processing task.

In operation804, the DP accelerator receives one or more results from the at least one additional DP accelerator. The DP accelerator completes its one or more sub-tasks of the processing tasks, and returns, to the host, one or more results from the one or more sub-tasks perform by the DP accelerator and the one or more sub-tasks performed by the at least one additional DP accelerator. Method800ends.

With respect to any of the above aspects, a host processor may be a central processing unit (CPU) and a DP accelerator may be a general-purpose processing unit (GPU) coupled to the CPU over a bus or interconnect. A DP accelerator may be implemented in a form of an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) device, or other forms of integrated circuits (ICs). Alternatively, the host processor may be a part of a primary data processing system while a DP accelerator may be one of many distributed systems as secondary systems that the primary system can offload its data processing tasks remotely over a network (e.g., cloud computing systems such as a software as a service or SaaS system, or a platform as a service or Paas system). A link between a host processor and a DP accelerator may be a peripheral component interconnect express (PCIe) link or a network connection such as Ethernet connection.