White Paper

The Enclave Device Blueprint for
Confidential Computing at the Edge:
A community approach to bring confidential
computing at the edge and IoT into mainstream
for privacy and safety

A White Paper by Arm, Microsoft and Scalys
Authors:
Eustace Asanghanwa, Principal Program Manager, Microsoft
Femi Idowu, Cloud Solutions Architect, Microsoft
Evert Pap, Architect and Technical Lead Engineering,
Scalys (Subsidiary of Sintecs BV)
Dmitry Lavnikevich, Software Architect,
Scalys (Subsidiary of Sintecs BV)Paul Howard, Principal System Solutions Architect, Arm
Marc Meunier, Sr Manager Software Ecosystem
Development, Arm and Governing Board Member on
Confidential Compute Consortium

1

To help protect data in the Internetof Things (IoT), developers are using
a relatively new technique from thecloud, called confidential computing,to isolate sensitive operations in a
Trusted Execution Environments during
processing. IoT data remains protected
when in use, so it’s easier to meet strict
regulatory requirements for data privacy
and introduce new use cases involving
sensitive data. This paper summarizes
confidential computing and its potential
for edge and highlights how Arm andMicrosoft are working with otherindustry leaders to make it easier toenable secure and isolated environments
for confidential computing.

Securing the Cloud was the first step
Cloud computing is an essential infrastructure for most enterprises and developers today.
Gartner expects global end-user spending on Public Cloud Services to exceed $480 Billion
in 2022. Today, majority of the data produced by enterprises and endpoint devices is processedin the cloud, and with more and more information being transmitted to, stored in, and processedin hybrid, public, and multi-cloud environments. With so much data at risk, the issue of cloudbased data security becomes increasingly important. Regulatory bodies have also taken notice of
information leaks and are now more active in defining requirements around data privacy.
When considering how best to protect data in the cloud, it helps to begin by thinking about datastates, since the state of cloud data – that is, what data is doing at any given moment – can influence
the security requirements.
In general, there are three data states to consider: at rest, in transit, and in use. When data is
at rest, it is not being accessed and is stored somewhere, such as a file server or a database.
When data is in transit, it is using a network connection, such as Wi-Fi, cellular, or Ethernet,
to travel from one point to another. And when data is in use, it is actively being processed byapplication code, as part of a workload during runtime.

Firewalls and encryption have been used for decades to protect data while it’s at rest andin transit. Firewalls, for example, are a first line of defense, preventing unauthorized accessto the network and isolating one network from another. Encryption, which scrambles
data and renders it unreadable to anyone without the correct decryption key, is anotherpowerful security mechanism, since it ensures that data is safe even if attackers get past the
firewall. Any encrypted data obtained by an unauthorized person or entity is unreadable andessentially worthless.
Protecting data in use, however, is something that has been harder to do. That’s because,with standard technology, data must first be decrypted before it can be processed.
The moments before, during, and after processing, when data is unencrypted, are vulnerable

Confidential computing fills
a security gap by protecting
data while it’s in use

times that offer opportunities to perform a memory dump, compromise the root user,
or carry out other malicious acts.
A recently developed technique for cloud operations, called “confidential computing,”
helps fill this security gap by protecting cloud data while it’s in use.

Protected Processing in the CloudConfidential computing secures data in use by isolating it in a hardware-based Trusted
Execution Environment (TEE). This protects the code and data within the environment
and prevents unauthorized access.
While data is being processed, it’s invisible and unknowable to the operating system,
the hypervisor of a virtual machine, and other compute-stack resources. It also can’t
be seen by the cloud provider or their employees.

2Extensive Cloud Benefits

Confidential computing
makes it safer to run
sensitive applications
on public clouds

By protecting data in use, confidential computing makes cloud-based operation safer,more attractive, and more flexible. Confidential computing gives greater assurance that cloud
operations are protected and private, and that makes it safer to move more applications –including those that involve sensitive data, such as Personally Identifiable Information (PII),
financial details, and medical records – onto hybrid and public clouds. Sharing multi-tenant
environments become less risky, too, because there’s less chance that workloads from one
tenant will interfere with workloads of another, or that a security breach will allow a tenant
to gain access to another tenant’s data or resources.
Confidential computing isolates code, along with data, so it’s easier to protect Intellectual
Property (IP), such as proprietary business logic, analytics functions, algorithms that use
machine language or artificial intelligence, and even entire applications. At the same time,
having the ability to retain control of private information makes it easier to collaborate withthird parties, without risking proprietary data or algorithms. Each party shares only what itneeds to for the project, and is able to gain insights from the collaboration, while sensitive
information remains beyond the reach of those who don’t need to see it.
Another benefit to cloud customers is that confidential computing makes it possible to avoid
vendor lock-ins, which prevent users from switching services. Businesses can use a mix ofproviders, choosing the best one for each technology or service, and don’t need to worry if
the competition uses the same provider.
A Long-term Trend
Confidential computing is still a comparatively new feature of cloud services but,due to its many business benefits, has caught on quickly. It promises to be a lasting trend.
Major cloud service providers like Microsoft Azure now offer it as an option, and rapid
expansion is expected across all geographic regions. The analysts at Market Digits Forecast
predict the global market for confidential computing will have a Compound Annual Growth
Rate (CAGR) of 52.21 percent between 2022 and 2030, for a Total Available Market (TAM)of USD 37 billion.
Another indication that confidential computing is an important technique with a promising
future is the fact that it now has its own industry group. The Confidential Computing
Consortium (CCC) brings together hardware vendors, cloud providers, and softwaredevelopers to accelerate the adoption of TEE technologies and standards. The CCC fosters
cross-industry collaboration in an open framework, and includes a project community at theLinux Foundation. Arm and Microsoft are premier members of the CCC and actively engaged
in the CCC’s efforts to define and accelerate the adoption of confidential computing.

3

Edge Migration
The proliferation of IoT devices has introduced new challenges around data processing.
In many cases, there is so much data being generated at the Edge that it is no longer feasible
to send all this data up to the cloud for processing. In many cases today, data needs to beprocessed at the Edge and only a subset of the data makes it to the cloud. Intelligent edge
devices often process highly sensitive workloads to generate results that are also highly
sensitive, making it necessary to use strong protection to preserve integrity, privacy,
and confidentiality. IoT devices may not always be in trusted custody, and have becomefrequent targets for tampering, forgery, and other types of hacking. To add to the challenge,
devices at the edge may be connected to the cloud, but might also need to operate
independently from the cloud connection.
The increasing use of intelligent edge devices, equipped with more processing power than
the typical sensor or actuator, makes it all the more important to protect IoT applications.
Public clouds are also a prominent part of the IoT picture, since today’s IoT companies
tend to use a cloud-only approach that employs someone else’s IT infrastructure. The IoT
devices in these deployments are, by and large, using cloud-based solutions to handle
security-sensitive data and software, so it’s important for IoT companies to understand how
confidential computing can help secure backend systems, comply with data-residency rules,
and protect privacy.
But IoT deployments can also benefit from confidential computing in another way. Many of
today’s edge and IoT devices can be thought of as cloud computers in miniature, with thesame need to protect data at rest, in transit, and in use. This is especially true for edge devices
that process and analyze data before sending it to a gateway or cloud, and for IoT devices
equipped with storage for housing collected data. By migrating confidential computing fromthe cloud to the edge and into IoT devices, it’s possible to create end-to-end protections thatsafeguard data at every point, from silicon to the cloud.

Edge devices are popular targets for hacking

4Benefits of Confidential Computing

Protecting Edge Data and Outcomesat the Edge

When deployed in an edge device, a Trusted Execution Environment (TEE) provides the same	
Secure execution of code – run only
sanctioned, intact code and control
access to resources
Trusted I/O for command and control– protect the command centers ofcritical infrastructures
Secure reporting – compose secure 		
reports, for use in billing and other areas
Secure metering – deploy consumptionbased monetization or warranty 		
management
Secure activity logging – enable highintegrity recording of activity for
non-repudiation
Protected data confidentiality
and integrity – defend against
hackers and malware that exploit bugsin the operating systemkind of protected environment for processing. Sensitive data and operations used in the IoT
device can’t be observed by other pieces of the system, making it extremely difficult to modify
data, subvert code, or influence outcomes.
Creating a secured TEE in device hardware provides a higher level of security than using
software alone to protect data. At present, the edge device places a high level of trust insupervisor software, in the form of kernels and hypervisors that manage applications andvirtual machines. The drawback, from a security standpoint, is that supervisors can accessthe data and code used by applications, and hackers can manipulate supervisors to leakconfidential data or algorithms held in applications. With confidential computing implemented
on the IoT device with hardware-based TEEs, the supervisor’s access status changes from“will not” to “cannot,” thereby minimizing a critical vulnerability and essentially eliminating
one of the ways that hackers exploit deployments for exfiltration and malicious tampering.
What’s more, adding confidential computing to an IoT device makes it possible to protect
offline availability, when device isn’t connected to the cloud but still need to process data
locally, such as when devices are deployed in hard-to-reach areas, such as an agricultural field,
a utility grid, a pipeline, or on an offshore ship. Offline availability also covers situations where
privacy is so important that data should not leave the system, such as in medical equipment
and high-security facilities.
Deploying edge devices that use confidential computing – by use of a hardware-based
TEE– opens up possibilities for new use cases, especially those that involve sensitive data or
generate insights that need to be protected. For example, medical researchers can use TEE
devices to collect health data without risking personal identities, and manufacturing facilities
can use machine-learning algorithms to control factory equipment while safeguarding
proprietary techniques. TEE devices also make it easier to work with proprietary algorithms,
used to derive unique insights, while minimizing the risk of revealing proprietary methods andlosing a competitive advantage. TEE devices can also make it easier to support the zero-trustmodel of computing, where software runs on someone else’s hardware, and the model ofsoftware offloading, where devices locate and use nearby compute resources.
Developing with Trusted Execution Environments at the Edge
In the cloud, confidential computing is a feature that cloud providers build and operate
on behalf of their customers. But developers of edge devices, looking to add confidential
computing to their design, need to build their own solutions, and that can be challenging
and expensive. Bringing confidential computing to the edge involves a new approach,both in how developers think about edge security and how they create edge devices.

5

That being said, the idea of using hardware based security creating a protected Trusted
Execution Environment in hardware, to safeguard sensitive data and operations at the edge,
isn’t actually new. Banking, government, and other niche industries have used dedicatedhardware chips like HSMs and TPMs to protect financial transactions, personal identities,
security clearances, and other sensitive information for many years. For a long time, though,the development of applications that use HSMs was something that only a few experts, withextensive experience in tamper proofing and specific HSM hardware offerings, were equipped
to do. Delivering a secured application with a HSM at its center meant having direct accessto the HSM hardware, and involved a time-consuming, rigorous process to verify protection.
These devices are also quite limited in what they can process and are used more to protect
secrets then to protect operations.

For many years, only experts in hardware security
had the skills needed to work with HSMs

In recent years, TEE functionality has expanded to support flexible use cases and there’s beena concerted effort to collaboratively expand the ecosystem for secure applications in the IoT.
The Confidential Computing Consortium is just one example of how the developer community
is coming together to make confidential computing more accessible.
The result is that developers now have access to a growing number of building blocks that
support the emulation of TEEs and the development of confidential applications. That meansthat developing and debugging trusted applications is no longer a specialty, reserved only for
TEE hardware engineers, but something any developer can take on.
A Modular Approach
Through a collaboration with the IoT team at the cloud provider Microsoft Azure,the developers at Scalys, a frontrunner in the development of edge security, and Arm,
we have created a modular approach for developing confidential compute capable devices.
The approach uses Trust Zone paired with Arm Cassini initiatives to provide a base of security
in the Azure IoT Edge development machine, the Open Enclave Software Development Kit(SDK), and the Azure-certified TrustBox Edge from Scalys.

We want to make it easier for everyone in the IoT
to take advantage of confidential computing

6

Arm Project Cassini
Arm Project Cassini is an open collaborative framework that pulls together standards andbest practice to ensure a secure and cloud native software experience that scales across
Arm devices.

Azure IoT Edge
Azure IoT Edge is a fully managed service from Microsoft that delivers cloud intelligence
locally by deploying Azure AI, Azure services, and custom logic directly onto edge devices,
including gateways. This lets organizations bring AI and cloud analytics to the edge,
especially when connectivity is poor or not available, or when high latency or high costs
prevent connection to the cloud.

In a single development environment, developers can create confidential applications
for deployment in cloud Trusted Execution Environments and in TEE-enabled IoT edge
devices. The Azure IoT Edge enables safe and secured execution inside of Trusted Execution
Environments. The workloads that execute in Trusted Execution Environments, referred to asconfidential applications, are encrypted from the time they leave the developer build machine
to when they land inside the device’s TEE, where they are decrypted for safe execution.
As shown in the diagram, once a confidential application is created in the development
machine, it is packaged as an IoT Edge Module. The application is encrypted before being
pushed to the container registry, and remains encrypted, throughout the IoT deployment
process, until the module is launched on the IoT Edge device.
Not until the confidential application is within the confines of the device TEE is it decrypted
and available for execution. An additional layer of asymmetric encryption is used to transfer
data confidentially into the TEE of the target device without it ever being exposed outside of
the TEE. The private key is stored within the TEE and never revealed.

7Confidential Data and Code Remains Encrypted Until Executed in the TEE(Source: Microsoft, 2020)

Azure Samples
Azure Samples is an official Microsoft repository that provides a centralized experience forcode-sample discovery, with the following goals in mind:
A centralized location for both developers and customers to discover samples for any
Microsoft tool, product or service.
A flexible means to access source-control on GitHub, keeping pace with ongoing
community-contributions and empowering the developer experience.
An accessible web frontend that promotes code discovery and distribution, as well as 		collaboration between partners.

Azure Samples is organized as a repository with three basic building-blocks:
The backend service: Leverages index-driven searches across code-sample repository hosted
on Github to ensure that customers have a direct access to the latest version of code-samples.
The Samples GitHub Code-Repository: Individual repositories that contain code-samples
hosted on GitHub.
Sample Browser: The customer-facing front-end experience situated on the Azure Samples
project home page. This is the central page where customers can find relevant code samples
through tailored searches.

Code-base
It is important to note that the Code-base located on Azure-Samples, is code that is meant toprovide examples, programs, and libraries, with the purpose of educating developers on how
to build basic functionality, that enables them to eventually create more complex applications.

8

Open Enclave SDK
The Open Enclave SDK is, as its name implies, an open-source library for developing
applications that make use of hardware-based TEEs, or Trusted Execution Environments.
Launched by Microsoft in 2018 and now governed by the Confidential Computing
Consortium, the Open Enclave SDK is a hardware-agnostic tool for building confidential
applications for multiple platforms and environments in C and C++. It provides a single
abstraction layer over any hardware-based TEE.
The resulting application partitions itself into two components: an untrusted component,
called the host, and a trusted component, called the Enclaves. The host component runs
unmodified on the untrusted operating system, while the trusted component runs within
the TEE.

A single TEE can support multiple Confidential applications
Source: OpenEnclave.io, 2021

The SDK generalizes the development of confidential applications across TEEs from different
hardware vendors, and is created with multiple software platforms, including Windows andLinux, in mind. In its present implementation, it provides support for Intel SGX as well as
preview support for OP-TEE on Arm TrustZone.
As an open-source project, Open Enclave SDK is agnostic to specific vendors, service
providers, and choice of operating systems. The project encourages community engagement
and provides links, from its GitHub page, to information about how to contribute.

9Scalys TrustBox Edge
The Scalys TrustBox Edge is an industrial-grade, tamper-resistant, secure router and IoT
gateway that is certified for Azure IoT Edge and features a collaborative integration of Arm
TrustZone security technology, NXP Layerscape processing, and the Open Enclave SDK.
Winner of the 2019 CES Best of Innovations award in the category of Cybersecurity andPersonal Privacy, the TrustBox Edge is optimized for confidential computing usingTEE andlets developers focus immediately on the creation of confidential workloads.
In its latest iteration, the TrustBox Edge design is based on the NXP Layerscape LS1028A
dual-core processor, a low-power communications processor with a 64-bit Arm Cortex-A72
core. The LS1028A features an Arm TrustZone, a hybrid hardware and software mechanism
for protecting sensitive assets that supports TEE implementations. The design comes
pre-loaded with the Open Enclave SDK and is Azure IoT Edge Certified as well as in
the process to meet Arm SystemReady certification.
Using the TrustBox Edge, development teams can save time and effort by evaluating on
the same hardware as goes to production. The design already meets industry standards forsecurity and compliance, so development can proceed directly from evaluation to pilot
and production.
Migrating to the Silicon Level
Arm is working to expand the confidential compute capabilities by making it a more integral
part of the compute infrastructure with Arm compute architecture. The Armv9 architecture,
which is Arm’s first major chip architecture upgrade in a decade, includes a Confidential
Compute Architecture (CCA). The CCA protects in-use portions of code and data fromaccess or modification, even from authorized software, by putting computation into
hardware-based security.
The CCA uses a concept called Realms, which are similar to containers and serve a purpose
similar to a TEE. Realms work in a region separate from secure and non-secure environments.
They protect sensitive data and code, whether the data is at rest, in transit, or in use. The Arm
CCA builds on the strong security foundations of Arm TrustZone and will extend the broad
adoption of confidential computing to every industry sector where microprocessors are in use.

With Armv9, confidential
computing becomes anintegral part of thecompute architecture
Confidential Computing Concepts are Central to the new Armv9 Architecture (Source: Arm 2021)
10

Enclave Device Blueprint
The Enclave Device Blueprint is a collection of projects, technologies, services, and motions to
help realize a scalable end-to-end secure IoT edge deployment with confidential computing.
The Blueprint aims to solve specific challenges:
Simplify enclave device engineering: abstract and simplify beyond Open Enclave SDK
Enable enclave devices to scale across different technologies and OS
by enabling OEMs to engage at a higher level of abstraction.
Provide tooling resources to enable building end-to-end solutions. eg. resources needed ina developer build machine to create the confidential application workloads and workflows
Enable maintenance and support scale by contributing and building from a common
OSS project. Solution builders are continually asking for 10 years or greater device 		
maintenance support.

Essential Blueprint components:
There are new components required to create the secured end to end framework.
These include:

Solution Architecture

11

The overall architecture covers the end-to-end solution that spans across the build pipeline,
across the cloud infrastructure and the end devices with their confidential enclaves. The goalof the solution is to build a confidential application in the build pipeline and ensure that it isnever exposed outside the secure environment.
Key management framework

The key management architecture is central to ensuring end to end security. The architecture
leverages the sharing of a common symmetrical encryption key between the build pipeline
and the target device. This way, the build pipeline can produce an encrypted application that
only the target device can decrypt an execute. This key is protected at all times by storing
it securely in a key Vault or managed HSM in the cloud. Whenever the key is shared, it isprotected with additional encryption, eg. Edge Device public key. The concept of a one-time
use key for transport is referred to as a class key.

Build Pipeline

The application development is simplified by leveraging OpenEnclave SDK and Visual Studio
IDE to create a uniform and familiar development environment. The application can be
debugged and tested locally using QEMU. It can then be encrypted with the symmetrical
key and embedded in a confidential package to prepare it for secure storage and transport.
Finally, the application is distributed in a container where it sits securely in the confidential
package.

12Edge Devices

The device needs to have both the Trusted Execution environment: OPTEE and theConfidential Package Manager running on it. We also need the IOT Edge runtime and acontainer runtime. The Host OS is less critical. The OS used for real deployment can differ,
but a hardened Yocto build is recommended, or the use of a binary Linux release tailored tolong time support.
Real world realization of the Enclave Device Blueprint
This real-world realization of the Enclave Device blueprint is available to help you start yourjourney in confidential computing at the edge.
The Scalys TrustBox Edge 201 comes with hardware backed security and software
attestation. The box is in the process of meeting the Arm SystemReady certification
requirements and is ready to support the confidential packaging system from Azure Cloudinterface. This provides a great platform to take the blueprint and create a customer
ready solution.
With a Scalys TrustBox Edge 201 and the opensource software, you have everything youneed to get started. At the Project home page, you will find information on the enclave device,
key components, solution, and demo.
Just The Beginning
We view the IoT as a community that benefits from collaboration, and we think IoT
deployments succeed when trust spans from the device manufacturer to the solution
integration and the solution backend. Confidential computing has an important role to playin that chain of trust, by protecting data when it’s in use, either in the cloud or at the edge.
The model we present here for developing a secure enclave IoT device is just one exampleof how the migration of confidential computing to the IoT can be made easier. As the IoT
industry works together to expand the ecosystem for secure applications, developers will
be able to access a wider variety of building blocks.
Our work in trusted devices, including our support for the approach described here, is anatural extension of our efforts to simplify development on a broader scale, from device
to cloud.
13

Whether it’s core projects, such as the SystemReady compliance certification program for

Learn more about the
Enclave Device Blueprint
from the project page.

hardware and firmware standards, or the Microsoft Azure IoT cloud services for cloud-native
development, we are committed to make it easier to develop and deploy secure solutions.
The ultimate goal of all these efforts is to abstract the complexities away from solution
builds, so developers can spend less time developing and debugging baseline security
features, and spend more time focused on new ideas that add value.
Join Us
The effort to migrate confidential computing to the IoT benefits from your involvement.
We invite you to learn more about confidential computing and how it relates to the IoT,
and encourage you to join us as part of the broader community that’s working to build theecosystem for confidential IoT development.
The CCC website (https://confidentialcomputing.io), which includes links to white papers
and open-source projects, is a good place to start.
Also join the projects that provide the building blocks to facilitate delivery, management andmonitoring of applications in confidential compute environments at the edge and in IoT:

Arm Project Cassini

An initiative to help deliver a cloud-native
software experience across a secure Armedge ecosystem

ParsecTo help abstract RoT and support key handling
functions.

OpenEnclave SDK

To help with the development of applications
that leverage TEEs

Arm SystemReadyTo ensure standard based hardwareand firmware.

Trustbox Edge

Connect with Scalys to accelerate yourdevelopment and deployments of secure
IoT devices.

All brand names or product names are the property of their respective holders. Neither the whole nor any part of the information contained in,or the product described in, this document may be adapted or reproduced in any material form except with the prior written permission of the
copyright holder. The product described in this document is subject to continuous developments and improvements. All particulars of the productand its use contained in this document are given in good faith. All warranties implied or expressed, including but not limited to implied warranties
of satisfactory quality or fitness for purpose are excluded. This document is intended only to provide information to the reader about the product.
To the extent permitted by local laws Arm shall not be liable for any loss or damage arising from the use of any information in this document or
any error or omission in such information.
© Arm Ltd. 2021

2021 Microsoft. All rights reserved. This white paper is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR
IMPLIED, IN THIS DOCUMENT. This document is provided “as is.” Information and views expressed in this document, including URL and other
Internet website references, may change without notice. You bear the risk of using it. This document does not provide you with any legal rights to
any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes.

14