Provisioning federated computation on distributed private data

A method comprises receiving in a governor device, from a plurality of data owner devices, metadata for one or more datasets maintained by the plurality of data owner devices, registering the metadata for the one or more datasets with the governor device, in response to a request from an aggregator, providing at least a portion of the metadata for the one or more datasets to the aggregator, receiving, from the aggregator, a compute plan to be implemented by the plurality of data owner devices, distributing at least a portion of the compute plan to the plurality of data owner devices, in response to receiving, from the plurality of data owner devices, a verification report and a certification for an enclave, binding the enclave to a host device, and providing the compute plan to the plurality of data owner devices.

BACKGROUND

In the field of federated computation involving distributed private data, a central institution (called aggregator) performs a compute on data that is distributed among a set of non-co-located data-owner institutions. The aggregator pushes the compute to the data-owner sites, where compute happens on the respective local data and whose results get pushed back to the aggregator. The aggregator may then aggregate the local results and repeat the compute again by providing the aggregated result as an extra input to the next compute. From a security and privacy perspective, it is of interest to use Trusted Execution Environments (TEEs) to carry out the local compute at the data-owner institutions and the aggregator, to facilitate preserving the integrity of the local computes on the different data owner sites, protecting the confidentiality of the aggregated results from the data owner sites, and protecting the confidentiality of local results from the aggregator before computing the aggregated result.

DETAILED DESCRIPTION OF THE DRAWINGS

As described above, in the field of federated computation involving distributed private data, a central institution (called aggregator) performs a compute on data that is distributed among a set of non-co-located data-owner institutions. The aggregator pushes the compute to the data-owner sites, where compute happens on the respective local data and whose results get pushed back to the aggregator. The aggregator may then aggregate the local results and repeat the compute again by providing the aggregated result as an extra input to the next compute. From a security and privacy perspective, it is of interest to use Trusted Execution Environments (TEEs) to carry out the local compute at the data-owner institutions and the aggregator, to facilitate preserving the integrity of the local computes on the different data owner sites, protecting the confidentiality of the aggregated results from the data owner sites, and protecting the confidentiality of local results from the aggregator (before computing the aggregated result).

To address these and other issues, described herein is an orchestration framework to carry out federated compute on distributed private data where the computes are protected by TEEs. In some examples the framework facilitates the following key requirements typically associated with standing-up a secure federated compute: (1) Maintaining consensus among data-owner and aggregator institutions about the compute plan including the data sets that will be used in the compute. (2) Preserving immutability of the compute plan once consensus is reached and delivering such compute plan directly to the TEEs that carry out the compute operations. (3) Standing up remote attested TEEs (at the participating data owner sites) that faithfully execute the expected compute plan. Further, the framework provides attestation assurance for TEEs that permits any data owner institution to verify that every other data owner institution indeed executes the expected compute plan. (4) Providing a mechanism for protecting the confidentiality of the “core compute software” from the admins that deploy the TEEs at the various data owner sites. This “core compute software” is typically treated as an IP belonging to the aggregator institution. (5) Providing a mechanism for the data owner administrators to poll the status of the overall execution yet keep the meta-data about the execution secure from entities that are not privy to such information.

In some examples, subject matter described herein comprises: 1) construction of a secure device that performs the dual role of infrastructure as well as application orchestrator, and 2) a sequence of carefully crafted distributed systems protocols employed among the data-owners, the aggregator and the device in order to achieve the five requirements stated above. In some examples the governor device may be designed and deployed as an Intel SGX-protected blockchain application. SGX for governor device guarantees privacy of confidential information handled by the governor device, while the blockchain guarantees robustness of the secure governor device—both confidentiality as well as robustness properties of the governor device are useful to deliver the 5 requirements described above.

FIG.1is a simplified schematic diagram of a distributed computing environment in which federated computation on distributed private data may be implemented, in accordance with an embodiment. Referring toFIG.1, the environment100comprises one or more data owner devices100, which may be embodied as a computing system comprising a trusted execution environment (TEE)112and one or more data files114. Environment100further comprises one or more aggregators120, which may also be embodied as a computing system comprising a trusted execution environment (TEE)122and one or more data files124. The TEE112may be communicatively coupled to the TEE122via a direct, secure communication link. Environment100further comprises one or more compute software certification authorities130and an attestation verification service140. In accordance with aspects described herein, environment100further comprises a governor device150, which, as described above, may be embodied as an Intel SGX-protected blockchain application.

Having described components of the environment100, a description of distributed systems protocols employed among the data-owner(s)110, aggregator,120and the governor device150will be provided with reference to the schematic drawings inFIGS.2-5and the flow chart depicted inFIG.6.

FIG.2depicts a first phase of an execution flow in a method to provision federated computation on distributed private data, according to embodiments. Referring toFIG.2, in some examples an asset (e.g., dataset) registration and discovery phase may be facilitated by the governor device. At operation610one or more data owner devices110register meta-data info regarding datasets with the governor device150. At operation615the aggregator120launches an inquiry to the governor device150to discover datasets, and at operation620the aggregator120authors the compute plan for federated computation. In some examples the compute plan may comprise the configuration of the execution including which data owner devices110are allowed to be in the federation, which software is allowed to be executed, and which data files114,124are allowed to be used in the federated computation.

FIG.3depicts a second phase an execution flow in a method to provision federated computation on distributed private data, according to embodiments. Referring toFIG.3, in some examples the second phase serves to address the competing requirements of 1) ensuring the compute software code meets the requirements of the data owner devices, (e.g., does not exfiltrate data or install malware), and 2) maintaining the confidentiality of any core intellectual property (IP) in the compute software code. Referring toFIG.6, at operation625, the aggregator's compute software code is reviewed and signed by one or more compute software certification authorities, which comprises a group of entities trusted by both the data owner devices110and the aggregator120capable of vetting the compute plan software. At operation630, the aggregator120generates a docker image containing the graphene binary file that the data owner devices110are to instantiate. At operation635the aggregator encrypts the compute plan software code, which is added as a protected file in the graphene manifest. The identities of the certification authorities that signed the software certification are listed in the compute plan. The compute plan further contains a hash of the docker image as well as the expected MREnclave of the Graphene enclave. Once the aggregator packs everything, it registers the compute plan with the governor device150.

FIG.4depicts a third phase an execution flow in a method to provision federated computation on distributed private data, according to embodiments. Referring toFIG.4, in some examples, in the third phase the data owner device(s)110administrator initiates a request to discover (operation645) the compute plan via the governor device150. In some examples, the data owner device(s)110may obtain the docker image directly from the aggregator120and review the compute plan. In some examples the data owner device(s) administrator does not see the encrypted code. Instead, they rely on the signature obtained from the trusted compute software certification authority.

At operation650the data owner device(s)110administrator approve the plan and launches the enclave. In some examples the enclave generates a self-signed X509 certificate and attaches a certificate hash to the quote. The data owner device(s)110administrator gets the quote attested to verify the hardware properties of the enclave by the attestation verification service and, at operation655, submits the verification report along with the enclave certificate to the governor device150. At operation660, the governor device150checks software properties of the enclave, (e.g., the MREnclave), then binds the enclave to the host. This binding property enables the aggregator120to identify the specific data owner device110from which a request comes when the enclave in the data owner device makes a task request to an enclave in the aggregator120, and to verify the integrity of the enclave.

After the enclaves are registered, at operation665the enclaves obtain the compute plan information directly from the governor device and at operation670the enclaves verify the authenticity of the compute plan software. In some examples, to verify the software the enclaves, obtain compute software decryption keys from the compute plan author (e.g., the aggregator), decrypt the software inside the enclave, and verify the signature of the software (that the SW was unencrypted when signed by the certification authorities). If the signature check is successful, the enclave makes a remote procedure call (RPC) to the governor device150and informs the governor device150of the successful software verification. The governor device150activates the compute plan after all enclaves successfully verify compute software validity.

At this point the federated compute plan is provisioned, and execution of the plan can begin. By following the protocol described herein, all participant enclaves execute the expected compute plan.FIG.5depicts the execution phase an execution flow in a federated computation on distributed private data, according to embodiments. Referring toFIG.5, in some examples, in the execution phase the data owner device(s)110execute the compute plan code and pass results to the aggregator120. The governor device150stores execution checkpoints for the compute plan.

FIG.7is a schematic diagram of a state machine in a system to implement federated computation on distributed private data may be implemented, in accordance with an embodiment. Referring toFIG.7, the first state710is a participant registration state in which the various entities involved in the federated computing register with the governor device150. The second state715is a data set registration in which the data sets are registered with the governor device150. The third state720is a compute plan registration state in which the compute plan is registered with the governor device150. The fourth state725is a GSGX registration in which the enclaves are registered with the governor device150. The fifth state is a GSGX activation state in which the enclaves are activated. The sixth state735is a compute plan activation state in which the compute plan is activated by the governor device150. The seventh state740is a federated compute execution state in which the compute plan is executed by the various entities in the federation.

FIG.8is a block diagram illustrating a computing architecture which may be adapted to implement a secure address translation service using a permission table (e.g., HPT135or HPT260) and based on a context of a requesting device in accordance with some examples. The embodiments may include a computing architecture supporting one or more of (i) verification of access permissions for a translated request prior to allowing a memory operation to proceed; (ii) prefetching of page permission entries of an HPT responsive to a translation request; and (iii) facilitating dynamic building of the HPT page permissions by system software as described above.

In various embodiments, the computing architecture800may comprise or be implemented as part of an electronic device. In some embodiments, the computing architecture800may be representative, for example, of a computer system that implements one or more components of the operating environments described above. In some embodiments, computing architecture800may be representative of one or more portions or components in support of a secure address translation service that implements one or more techniques described herein.

As shown inFIG.8, the computing architecture800includes one or more processors802and one or more graphics processors808, and may be a single processor desktop system, a multiprocessor workstation system, or a server system having a large number of processors802or processor cores807. In on embodiment, the system800is a processing platform incorporated within a system-on-a-chip (SoC or SOC) integrated circuit for use in mobile, handheld, or embedded devices.

An embodiment of system800can include, or be incorporated within, a server-based gaming platform, a game console, including a game and media console, a mobile gaming console, a handheld game console, or an online game console. In some embodiments system800is a mobile phone, smart phone, tablet computing device or mobile Internet device. Data processing system800can also include, couple with, or be integrated within a wearable device, such as a smart watch wearable device, smart eyewear device, augmented reality device, or virtual reality device. In some embodiments, data processing system800is a television or set top box device having one or more processors802and a graphical interface generated by one or more graphics processors808.

In some embodiments, the one or more processors802each include one or more processor cores807to process instructions which, when executed, perform operations for system and user software. In some embodiments, each of the one or more processor cores807is configured to process a specific instruction set814. In some embodiments, instruction set809may facilitate Complex Instruction Set Computing (CISC), Reduced Instruction Set Computing (RISC), or computing via a Very Long Instruction Word (VLIW). Multiple processor cores807may each process a different instruction set809, which may include instructions to facilitate the emulation of other instruction sets. Processor core807may also include other processing devices, such a Digital Signal Processor (DSP).

In some embodiments, the processor802includes cache memory804. Depending on the architecture, the processor802can have a single internal cache or multiple levels of internal cache. In some embodiments, the cache memory is shared among various components of the processor802. In some embodiments, the processor802also uses an external cache (e.g., a Level-3 (L3) cache or Last Level Cache (LLC)) (not shown), which may be shared among processor cores807using known cache coherency techniques. A register file806is additionally included in processor802which may include different types of registers for storing different types of data (e.g., integer registers, floating point registers, status registers, and an instruction pointer register). Some registers may be general-purpose registers, while other registers may be specific to the design of the processor802.

In some embodiments, one or more processor(s)802are coupled with one or more interface bus(es)810to transmit communication signals such as address, data, or control signals between processor802and other components in the system. The interface bus810, in one embodiment, can be a processor bus, such as a version of the Direct Media Interface (DMI) bus. However, processor buses are not limited to the DMI bus, and may include one or more Peripheral Component Interconnect buses (e.g., PCI, PCI Express), memory buses, or other types of interface buses. In one embodiment the processor(s)802include an integrated memory controller816and a platform controller hub830. The memory controller816facilitates communication between a memory device and other components of the system800, while the platform controller hub (PCH)830provides connections to I/O devices via a local I/O bus.

Memory device820can be a dynamic random-access memory (DRAM) device, a static random-access memory (SRAM) device, flash memory device, phase-change memory device, or some other memory device having suitable performance to serve as process memory. In one embodiment the memory device820can operate as system memory for the system800, to store data822and instructions821for use when the one or more processors802execute an application or process. Memory controller hub816also couples with an optional external graphics processor812, which may communicate with the one or more graphics processors808in processors802to perform graphics and media operations. In some embodiments a display device811can connect to the processor(s)802. The display device811can be one or more of an internal display device, as in a mobile electronic device or a laptop device or an external display device attached via a display interface (e.g., DisplayPort, etc.). In one embodiment the display device811can be a head mounted display (HMD) such as a stereoscopic display device for use in virtual reality (VR) applications or augmented reality (AR) applications.

In some embodiments the platform controller hub830enables peripherals to connect to memory device820and processor802via a high-speed I/O bus. The I/O peripherals include, but are not limited to, an audio controller846, a network controller834, a firmware interface828, a wireless transceiver826, touch sensors825, a data storage device824(e.g., hard disk drive, flash memory, etc.). The data storage device824can connect via a storage interface (e.g., SATA) or via a peripheral bus, such as a Peripheral Component Interconnect bus (e.g., PCI, PCI Express). The touch sensors825can include touch screen sensors, pressure sensors, or fingerprint sensors. The wireless transceiver826can be a WI-FI transceiver, a BLUETOOTH transceiver, or a mobile network transceiver such as a 3G, 4G, Long Term Evolution (LTE), or 5G transceiver. The firmware interface828enables communication with system firmware, and can be, for example, a unified extensible firmware interface (UEFI). The network controller834can enable a network connection to a wired network. In some embodiments, a high-performance network controller (not shown) couples with the interface bus810. The audio controller846, in one embodiment, is a multi-channel high definition audio controller. In one embodiment the system800includes an optional legacy I/O controller840for coupling legacy (e.g., Personal System 2 (PS/2)) devices to the system. The platform controller hub830can also connect to one or more Universal Serial Bus (USB) controllers842connect input devices, such as keyboard and mouse843combinations, a camera844, or other USB input devices.

The following clauses and/or examples pertain to further embodiments or examples. Specifics in the examples may be used anywhere in one or more embodiments. The various features of the different embodiments or examples may be variously combined with some features included and others excluded to suit a variety of different applications. Examples may include subject matter such as a method, means for performing acts of the method, at least one machine-readable medium including instructions that, when performed by a machine cause the machine to perform acts of the method, or of an apparatus or system for facilitating hybrid communication according to embodiments and examples described herein.Example 1 is method comprising receiving in a governor device, from a plurality of data owner devices, metadata for one or more datasets maintained by the plurality of data owner devices, registering the metadata for the one or more datasets with the governor device, in response to a request from an aggregator, providing at least a portion of the metadata for the one or more datasets to the aggregator; receiving, from the aggregator, a compute plan to be implemented by the plurality of data owner devices; distributing at least a portion of the compute plan to the plurality of data owner devices; in response to receiving, from the plurality of data owner devices, a verification report and a certification for an enclave, binding the enclave to a host device; and providing the compute plan to the plurality of data owner devices.Example 2 includes the subject matter of Example 1, wherein the compute plan specifies a set of data owner devices from the plurality of data owner devices that are to execute the compute plan; one or more pieces of software to be executed in the compute plan; and one or more data files to be used in the compute plan.Example 3 includes the subject matter of Examples 1-2, wherein the compute plan comprises an image containing a binary file for execution; a hash of the image; a certification from a certification authority; and an enclave identifier.Example 4 includes the subject matter of Examples 1-3, wherein the compute plan comprises an encrypted file comprising the one or more data files to be used in the compute plan.Example 5 includes the subject matter of Examples 1-4, wherein the plurality of data owner devices approve the compute plan; and launch an enclave to execute the compute plan.Example 6 includes the subject matter of Examples 1-5, wherein the enclave generates a certificate; and returns the certificate to a data owner device in the plurality of data owner devices.Example 7 includes the subject matter of Examples 1-6, further comprising monitoring one or more checkpoints in an execution phase of the compute plan.Example 8 is an apparatus, comprising a processor; and a computer readable memory comprising instructions which, when executed by the processor, cause the processor to receive in a governor device, from a plurality of data owner devices, metadata for one or more datasets maintained by the plurality of data owner devices; register the one or more datasets with the governor device; in response to a request from an aggregator, provide at least a portion of the metadata for the one or more datasets to the aggregator; receive, from the aggregator, a compute plan to be implemented by the plurality of data owner devices; distribute at least a portion of the compute plan to the plurality of data owner devices; in response to a receipt, from the plurality of data owner devices, of a verification report and a certification for an enclave, bind the enclave to a host device; and provide the compute plan to the plurality of data owner devices.Example 9 includes the subject matter of Example 8, wherein the compute plan specifies a set of data owner devices from the plurality of data owner devices that are to execute the compute plan; one or more pieces of software to be executed in the compute plan; and one or more data files to be used in the compute plan.Example 10 includes the subject matter of Examples 8-9 wherein the compute plan comprises an image containing a binary file for execution; a hash of the image; a certification from a certification authority; and an enclave identifier.Example 11 includes the subject matter of Examples 8-10, wherein the compute plan comprise an encrypted file comprising the one or more data files to be used in the compute plan.Example 12 includes the subject matter of Examples 8-11, wherein the plurality of data owner devices approve the compute plan; and launch an enclave to execute the compute plan.Example 13 includes the subject matter of Examples 8-12, wherein the enclave generates a certificate; and returns the certificate to a data owner device in the plurality of data owner devices.Example 14 includes the subject matter of Examples 8-13, further comprising monitoring one or more checkpoints in an execution phase of the compute plan.Example 15 is one or more computer-readable storage media comprising instructions stored thereon that, in response to being executed, cause a computing device to receive in a governor device, from a plurality of data owner devices, metadata for one or more datasets maintained by the plurality of data owner devices; register the one or more datasets with the governor device; in response to a request from an aggregator, provide at least a portion of the metadata for the one or more datasets to the aggregator; receive, from the aggregator, a compute plan to be implemented by the plurality of data owner devices; distribute at least a portion of the compute plan to the plurality of data owner devices; in response to a receipt, from the plurality of data owner devices, of a verification report and a certification for an enclave, bind the enclave to a host device; and provide the compute plan to the plurality of data owner devices.Example 16 includes the subject matter of Example 15, wherein the compute plan specifies a set of data owner devices from the plurality of data owner devices that are to execute the compute plan; one or more pieces of software to be executed in the compute plan; and one or more data files to be used in the compute plan.Example 17 includes the subject matter of Examples 15-16, wherein the compute plan comprises an image containing a binary file for execution; a hash of the image; a certification from a certification authority; and an enclave identifier.Example 18 includes the subject matter of Examples 15-17, wherein the compute plan comprise an encrypted file comprising the one or more data files to be used in the compute plan.Example 19 includes the subject matter of Examples 15-18, wherein the plurality of data owner devices approve the compute plan; and launch an enclave to execute the compute plan.Example 20 includes the subject matter of Examples 15-19, wherein the enclave generates a certificate; and returns the certificate to a data owner device in the plurality of data owner devices.Example 21 includes the subject matter of Examples 15-20, further comprising monitoring one or more checkpoints in an execution phase of the compute plan.

In the description above, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the described embodiments. It will be apparent, however, to one skilled in the art that embodiments may be practiced without some of these specific details. In other instances, well-known structures and devices are shown in block diagram form. There may be intermediate structure between illustrated components. The components described or illustrated herein may have additional inputs or outputs that are not illustrated or described.

Portions of various embodiments may be provided as a computer program product, which may include a computer-readable medium having stored thereon computer program instructions, which may be used to program a computer (or other electronic devices) for execution by one or more processors to perform a process according to certain embodiments. The computer-readable medium may include, but is not limited to, magnetic disks, optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically-erasable programmable read-only memory (EEPROM), magnetic or optical cards, flash memory, or other type of computer-readable medium suitable for storing electronic instructions. Moreover, embodiments may also be downloaded as a computer program product, wherein the program may be transferred from a remote computer to a requesting computer.

An embodiment is an implementation or example. Reference in the specification to “an embodiment,” “one embodiment,” “some embodiments,” or “other embodiments” means that a particular feature, structure, or characteristic described in connection with the embodiments is included in at least some embodiments, but not necessarily all embodiments. The various appearances of “an embodiment,” “one embodiment,” or “some embodiments” are not necessarily all referring to the same embodiments. It should be appreciated that in the foregoing description of exemplary embodiments, various features are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various novel aspects. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, novel aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims are hereby expressly incorporated into this description, with each claim standing on its own as a separate embodiment.