  Following the increasing interest and adoption of FaaS systems, benchmarking
frameworks for determining non-functional properties have also emerged. While
existing (microbenchmark) frameworks only evaluate single aspects of FaaS
platforms, a more holistic, application-driven approach is still missing.
  In this paper, we design and present BeFaaS, an application-centric
benchmarking framework for FaaS environments that focuses on the evaluation
with realistic and typical use cases for FaaS applications. BeFaaS comes with
two built-in benchmarks (an e-commerce and an IoT application), is extensible
for new workload profiles and new platforms, supports federated benchmark runs
in which the benchmark application is distributed over multiple providers, and
supports a fine-grained result analysis.
  Our evaluation compares three major FaaS providers in single cloud provider
setups and analyzes the traces of a federated fog setup. It shows that BeFaaS
is capable of running each benchmark automatically with minimal configuration
effort and providing detailed insights for each interaction.
