Package: edgerun-galileo-experiments
Galileo ExperimentsThis repository contains code that invokes the galileo shell to start experiments.
It also includes deployment files for necessary components and describes in detail which other services are
required to start an experiment.
Further, this project aims to unify different sub-projects ofedgerunand make deployment and experiment setup easy.
The main goals and functionalities are:A framework for distributed load testing experimentsFine-grained telemetry data collectionHTTP Trace recording for any serviceA container orchestration adaption for ease of useWho is this project for?For everyone that wants to see resource usage and application performance with easy configurable workload creation.
All components are tailored and suited to run on low performance devices (i.e., Raspberry Pi) but can run on default server VMs too.Common questions that can be answered by performing and analyzing Galileo experiments:How much CPU usage does my application use?What is the average execution time of my application across the cluster?What are the differences in terms of resource usage between two nodes hosting the same application?What is the impact of having multiple applications running on one node?All these questions can be easily answered and have a simple flow in common:Deploy base infrastructureDeploy my applicationStart requestsAnalyze in Jupyter NotebooksIn summary: simple profiling tasks.
But, this framework also targets full end-to-end tests to evaluate important cluster components (i.e, load balancer, scheduler and scaling).Therefore, experiments can be done to evaluate new implementations for  the aforementioned components.High level overviewKubernetes cluster setupOverviewMain componentsThe cluster setup consists of the following main components:KubernetesGalileo (for clients and experiment shell)TelemdController (i.e., the load balancer)MySQL (i.e., MariaDB)InfluxDB v2Etcd (Kubernetes requires an instance to run!)Kubernetes is used to host the clients (galileo running in a Pod), telemetry agents (telemd), load balancer and the applications to test.
Redis is used as a pub/sub system through which all data is sent (i.e., telemetry) and recorded by the Galileo Shell (i.e., the program that prepares and executes an experiment).
The Galileo Shell persists data in MariaDB and InfluxDB.
The provided load balancer implementation uses etcd to watch for weights for the round-robin algorithm and galileo uses redis to provide the clients with routing rules (rtbl).Main interactionsThe figure above depicts all components and also highlights important interactions.
Those interactions are in short:Client nodes send HTTP requests to the Controller (load balancer), which forwards requests to the worker nodes which host the application pods.Client nodes report the results of each request (i.e., trace) via Redis.The Go-based load balancer implementation fetches weights and ip addresses from the etcd instance.The clients get the routing rules from the Redis instance (set viartblfrom Galileo)Worker nodes report resource usage (i.e., telemd) via Redis, which is saved in InfluxDBThe Galileo Shell starts the experiments and saves metadata (i.e., the cluster hosts, misc. data) in the MariaDBDeploymentThis project provides deployment files for the following components:Galileo (for clients and experiment shell)TelemdController (i.e., the load balancer)Which leaves the following components to be additionally deployed:KubernetesMySQL (i.e., MariaDB)InfluxDB v2Etcd (Kubernetes requires an instance to run!)Deployment files can be found indeployment/kubernetes.
Note, that we use Kubernetes node labels to schedule the workers (i.e., clients).
On nodes that should act as clients execute the following command:kubectl label node <node> node-role.kubernetes.io/client=trueThe following label is used to identify nodes with hosting capabilities (i.e., workers):kubectl label node <node> node-role.kubernetes.io/worker=trueIf you have multiple zones (i.e., clusters) in which you want to have seperate clients, adapt thezonearguments (default ismain).
You can easily group your nodes by labelling it with the following command:kubectl label node <node> ether.edgerun.io/zone=mainGalileo WorkersThe galileo workers run on each client node and is connected via Redis to receive commands and also the routing rules.
Routing rules are simple key-value pairs, whereas the key represents your service name and the value is a list of hosts
with the respective weight.You can easily set these in your program viartbl.set('service', ['127.0.0.1:8080], [1]).Data storageGalileo requires the following data components that are either deployed in the cluster or externally:Redis (pub/sub for telemetry and traces)MySQL (i.e., MariaDB) (persistent storage for experiment metadata)InfluxDB v2 (stores runtime data - telemetry and traces)All connection parameters are set via environment variables.ExtensionsTheextension repositoryis meant to provide examples  on how to implement and use the project to run experiments.
It will be continually updated and include new services.Environment variablesThe following table shows all environment variables to be set.
For ease of use a.envis included which includes all variables (underbin/.env).VariableDefaultDescriptiongalileo_expdb_drivermixedUses a SQL database to store experiment metadata and InfluxDB to store runtime data (i.e., traces)galileo_logging_levelDEBUGLogger level (DEBUG, INFO, WARN, ERROR)galileo_expdb_mysql_hostlocalhostMySQL hostgalileo_expdb_mysql_port3307MySQL portgalileo_expdb_mysql_dbdbMySQL databasegalileo_expdb_mysql_useruserMySQL usergalileo_expdb_mysql_passwordpasswordMySQL passwordgalileo_expdb_influxdb_urlhttp://localhost:8086InfluxDB urlgalileo_expdb_influxdb_tokenauth-tokenInfluxDB authentication tokengalileo_expdb_influxdb_timeout10000InfluxDB timeout in msgalileo_expdb_influxdb_orgorgInfluxDB organization namegalileo_expdb_influxdb_org_idorg-idInfluxDB organization IDgalileo_redis_hostlocalhostRedis hostgalileo_redis_passwordoptionalRedis portKUBECONFIGnot setPath to the kubeconfig