TS-Perf: Performance measuring method in the TEE(Trusted Execution Environment) of various CPUs
RISC-V Day Tokyo Summer 2024
August/1
IISEC: Institute of Information Security （情報セキュリティ大学院大学）
Kuniyasu Suzaki （須崎 有康）

1

Who am I ? (Kuniyasu Suzaki) Professor, IISEC: Institute of Information Security (Graduate School) from 2022/9/1
 Former : AIST (National Institute of Advanced Industrial Science and Technology)
Joined a national project (2018-2023) for RISC-V based TEE research at TRASIO (Technology
Research Association of Secure IoT Edge application based on RISC-V Open architecture）
TRASIO members were Hitachi, NSITEXE (Current Denso), SECOM, Keio University, and AIST.

• セキュアオープンアーキテクチャ・エッジ基盤技術研究組合(TRASIO)• NEDOプロジェクト「セキュアオープンアーキテクチャ基盤技術とそのＡＩエッジ応用研究開発 FY2018-2022」

2

Contents
 What is TEE (Trusted Execution Environment) ?
 RISC-V TEE “Keystone” (based on PMP: Physical Memory Protection)

 Why is measuring performance difficult on TEE?
 TS-Perf Used Techniques
Portable Library which offers same APIs.
Separate Compilation
Report after TEE process termination because of communication overhead between TEE and REE.

 Conclusion
This presentation is based on TS-Perf: General Performance Measurement of Trusted Execution Environment and Rich Execution
Environment on Intel SGX, Arm TrustZone, and RISC-V Keystone [IEEE Access 2021]
 Library Implementation and Performance Analysis of GlobalPlatform TEE Internal API for Intel SGX and RISCV Keystone [TrustCom 2020]3

What is TEE? (1/2)
 TEE (Trusted Execution Environment) is one ofHIEE(Hardware-assisted Isolated ExecutionEnvironments)
 HIEE includes
X86 SMM (System Management Mode) used by BIOS/UEFI
Intel ME (Management Engine) and TPM which are anther chip.

 TEE is featured to be programable by third party.

 TEE separates CPU into two worlds.
 Normal World (i.e., REE: Rich Execution Environment)
Normal OS(Linux, Windows) runs

 Secure World (i.e., TEE: Trusted Execution Environment)It is independent form vulnerabilities of OS and Hypervisor.
Critical apps runs.

Typical Figure4What is TEE? (2/2)
 Features： (To speak radically) TEE offers a temporal isolation execution only.
 Long term key managements require another method.
Hardware Root of Trust is an anti-tamper hardware to store keys.
Remote Attestation (method to confirm the soundness of hardware and software) is based on the keys
saved in Hardware Root of Trust.

 Available CPUs ARM TrustZone (Smart phone) Intel SGX (PC, Server), Intel TDX(Server)
 AMD SEV (Server)
 RISC-V has many (later slide)

TEE’s Image(This image resembles to Arm TrustZone)

 Some TEEs are based on virtualization technology, called Confidential Computing
and used on Cloud mainly.

5Apps on TEE and CC Critical Processing
 Key Management

Killer Application on TEEof Smartphone.

Android KeyMaster DRM
Smartphone’s Widevine(Google)
Windows Ultra HD Blu-ray Viewer
uses Intel SGX

 Personal Info

Cannot be Killer App.

• Small Memory• Suitable for Smartphone• Arm TrustZone

Fingerprint authentication
Candidate for Killer App?
FIDO AuthenticatorHardware Wallet for Crypto Currency• RISC-V?

 Data and Code Hiding Machine Learning Privacy Protection
 Gene Data Processing••••

Large Memory
Required Server & CloudIntel SGX、AMD SEV
Target of Confidential
Computing?

Candidate for Killer App on Cloud?
6

RISC-V TEE
 TEE based on RISC-V
 Sanctum [MIT,USENIX Sec’16]
 TIMBER-V [Graz University of Technology, NDSS’19]
 MI6 [MIT,MICRO’19]
 Keystone [UC Berkeley, EuroSys’20]
Academia HECTOR-V [Graz University of Technology, arXiv’21]
 uTango [University of Minho, arXiv’21]
 Cure [Darmstadt University of Technology,USENIX Sec’21]
 CHERI-TrEE [University of Cambridge, IEEE S&P’23]
 HPMP (Hybrid Physical Memory Protection) [Shanghai Jiao Tong University, MICRO’23]
 MultiZone [HexFive] SiFive Shield / World Guard [SiFive]

Industry AP-TEE （Application Processor –TEE) [RISC-V International TEE WG]
 CoVE (Confidential Virtual Machine for RISC-V) [Rivos Inc., arXiv’23]

Many implementations require hardware extension Keystone and MultiZone use PMP (PhysicalMemory Protection) which is default function of RISC-V.

7

RISC-V TEE ”Keystone”
 TEE developed by UC Berkeley
 The developments are lead by RISC-V core members (Prof. Krste Asanović)
 PMP(Physical Memory Protection), which is defined by Privileged Architecture Specification,is utilized.
 A Project of CCC(Confidential Computing Consortium)

8Normal 64bit RISC-V This figure assumes Linux on
64bit Rocket (4 cores)

Rocket (4 cores) Apps uses User Mode
NormalApp1

NormalApp2

U modeLinuxOSS modeM mode Linux Kernel uses Supervisor Mode Interrupts go to M mode and thehandlers runs in Supervisor mode.
(This figure omits the parts)

RV64GC
Memory9Keystone on RISC-V

 No Hardware modification
 Memory Protection by PMP

Rocket (4 cores)
TEE

REE
NormalApp1

Enclave
Normal
App2LinuxOS

Enclave

Enclave

Trusted App1

Trusted App2

U mode

Eyrie
Runtime

Eyrie
RuntimeS modeEnclave
SM (Security Monitor)

RV64GC
MemoryM mode PMP can separate memory regions
and execution environments.
 PMP has priority (high – low).

 Each PMP offers “Enclave”
 The highest PMP is used by“Secure Monitor” (M mode) The lowest PMP is used by “Linux”as REE
 In this figure 2 TEEs (Enclaves) usePMP.
10

Why is measuring performance difficult on TEE?
 There are 3 problems.
1. Most TEE assumes original SDK (Software Development Kit), which makes
difficult to compare the performance.
RISC-V Keystone assumes “Eyrie” as an OS environment, which is not POSIX.

2. Different binaries between TEE and REEApplication’s performance on TEE and REE can not be easily compared because mostbinaries are build for TEE and REE.

3. The communication overhead is heavy between TEE and REE.
TEE has no display and needs to communicate to the REE, but the overhead is heavy.
The measurement results cannot report immediately because it affects the TEEperformance.

TS-Perf solved these problems.
11Problem 1: different TEE SDKs
 This problem is solved by offering the same API library.
 We developed a portable library of GlobalPlatform‘s TEE Internal API
because the specification is opened and widely used on most TEEs on
smartphones (Arm TrustZone).
 The portable library is developed for RISC-V Keystone and Intel SGX.
 Using the portable library, we can compare the same software on 3 TEE architectures(RISC-V Keystone, Intel SGX, and Arm TrustZone).

• Library Implementation and Performance Analysis of GlobalPlatform TEE Internal API for Intel SGX andRISC-V Keystone [TrustCom 2020]12

Problem 2: Different binaries
between TEE and REE
 This problem is solved by “separate compilation” to compare the same binary
performance between TEE and REE.
For TEE
sourcecode

compiler(TS-perffor TEE)

objectfile

TS-perffor TEE(record&report)

TS-perffor TEE(record&report)

For REE

Measurement
TargetTS-perffor REE(record&report)

Measurement
TargetTS-perffor REE(record&report)same source
code

Measurement
Targetsame object

Measurement
Targetlinkerexec
binaryREE
Measurement

TEE
Measurement
same binary13

Problem 3: Heavy communication overhead between TEE and REE
 This problem is solved by the performance reporting just before the termination ofthe process.
 We utilize GCC; “profile option”, “constructor” and “destructor”.
 “constructor” keeps memory region to log the measurement of profile option.
 “destructor” reports the measurement results from TEE to REE at the end of TEE process.
REE process
Create TEE process

TEE process“constructor” keeps memory regionmain() {measurement start
Measurement target binary
measurement end and keep the data
to reserved memory}

Get the TEE performanceand show it on the display.

“destructor” reports the measurement to REE
and terminate the process.
14Performance measuring
between different TEEs and REEs
 Three TEE architectures (Arm Trust Zone, Intel SGX, and RISC-V Keystone)

 CPU time counter

15Performance measuring
between different TEEs and REEs
 Performance Comparison between TEE and REE on Arm Cortex-A, Intel
X86-64, and RISC-V U540

16

Measuring the affect of memory encryption
on Intel SGX
 Measuring wide memory sequential access and random access This result shows the performance difference appears over the cache size (4MB L2).

17

Conclusions
 Measuring TEE performance had 3 problems.
1. Different programming style.
2. Different binary between TEE and REE
3. Heavy communication overhead between TEE and REE

 How to Solve by TS-Perf
1. Provide a portable GlobalPlatform API by a library.
2. Separate Compilation allows to compare the true binary performance between TEE and REE.
3. GCC customization to report the performance results after process termination

 TS-Pers measures performance between TEE and REE.
 Measuring true difference of TEE and REE.
This presentation is based on TS-Perf: General Performance Measurement of Trusted Execution Environment and Rich
Execution Environment on Intel SGX, Arm TrustZone, and RISC-V Keystone [IEEE Access 2021]
 Library Implementation and Performance Analysis of GlobalPlatform TEE Internal API for Intel SGX
18and RISC-V Keystone [TrustCom 2020]