  Trusted Execution Environments (TEEs), such as Intel Software Guard
eXtensions (SGX), are considered as a promising approach to resolve security
challenges in clouds. TEEs protect the confidentiality and integrity of
application code and data even against privileged attackers with root and
physical access by providing an isolated secure memory area, i.e., enclaves.
The security guarantees are provided by the CPU, thus even if system software
is compromised, the attacker can never access the enclave's content. While this
approach ensures strong security guarantees for applications, it also
introduces a considerable runtime overhead in part by the limited availability
of protected memory (enclave page cache). Currently, only a limited number of
performance measurement tools for TEE-based applications exist and none offer
performance monitoring and analysis during runtime.
  This paper presents TEEMon, the first continuous performance monitoring and
analysis tool for TEE-based applications. TEEMon provides not only fine-grained
performance metrics during runtime, but also assists the analysis of
identifying causes of performance bottlenecks, e.g., excessive system calls.
Our approach smoothly integrates with existing open-source tools (e.g.,
Prometheus or Grafana) towards a holistic monitoring solution, particularly
optimized for systems deployed through Docker containers or Kubernetes and
offers several dedicated metrics and visualizations. Our evaluation shows that
TEEMon's overhead ranges from 5% to 17%.
