Package: edgerun-edge-chaos
Edge ChaosThis project's aim is to cause chaos in edge-cloud environments.Users can start and stop programs that should disrupt co-located applications.
Currently the following features are implemented:CPU stress (usingstress-ng)Network traffic shaping (usingtc)InstallRun the following steps to install all dependencies:python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txtBuildWe offer scripts to containerize the application for all common architectures (i.e.,amd64,arm32v7andarm64v8):./scripts/docker-build.sh $archRelease./scripts/docker-release.sh $repository $versionTheversionargument is optional and defaults to$(git rev-parse --short HEAD).RunWhile it's main intended use case is to run in a container, you can also start it natively:python3 -u -m edgechaos.daemon.runTo start as a container run:docker run --network=host edgerun/edge-chaos:latestUsageEdgeChaos runs as a daemon (native, in a container or in a Kubernetes Pod) and waits to receive commands.
Currently, commands are expected to arrive via Redis Pub/Sub or via AMQP (i.e., RabbitMQ).Supportedinteraction:Redis: the daemon waits for messages published via the channeledgechaos/$edgechaos_host.RabbitMq: the daemon watis for messages published on the exchangeedgechaoswith the routing key `$edgechaos_host.Whereas$edgechaos_hostis set as environment variable and defaults to theHOSTNAME.The expected body is the same across the different interaction methods.
The daemon expects the message to be a JSON object, that has aname,parametersandkindkey.
Thenameindicates the type of attack (i.e.,cpu) and theparametersspecify further information necessary for the
attack.
Thekindspecifies whether it's astartorstopevent.
You can get a more detailed glimpse into the format by taking a look at the corresponding
dataclassChaosCommand.Important:The body must be always the same. Which means if you want to stop an attack, you have to send the same
body as before, exceptkindis set tostop.To give an example, the following two JSON objects show how to start a CPU attack (using 1 core) and stop it.Start the attack:{"name":"stress-ng","parameters":{"cpu":1},"kind":"start"}And stop it:{"name":"stress-ng","parameters":{"cpu":1},"kind":"stop"}Available chaos attacksIn the following we list all available attacks and specify their respective JSON objects for invocation.stress-ngstress-ngis a powerful stress test program that
hasover 280 different types of attacks (stressors).
Therefore, users can specify any arbitrary combination of arguments that will be passed on tostress-ng.
Which means that any key-value pair in theparametersobject is passed on tostress-ng.Stress-ng attacks can be executed in two ways:Astartandstopevent is sent, in both cases the remaining content of the message must be identical.Stress-ng offers parameters to stop the stress test after a certain amount of operations or time (i.e.,timeout).
In this case notstopevent is required.The request should look like this.
The content of theparameterwill be passed onto stress-ng, though it is not necessary to prefix the arguments (i.e,.
JSON object keys) with--:{"name":"stress-ng","parameters":{"cpu":0},"kind":"start"}Note that in the example attack,0indicates that stress-ng should use all available cores.tctcis a Linux traffic shaping tool that can modify the traffic on
network interfaces.This wiki entryoffers a quick look into the capabilities oftc.
As before withstress-ng, we do not want to limit users in their chaos attack configuration and thus just pass on
any parameter totc.In contrast tostress-ngattacks, each attack needs to be manually stopped.
That means that the edge-chaos agent does not modify the parameters and just passes on parameters.
To stop the modification, it is necessary to send the correcttccommand (see down below for an example)
and that thekindkey is set tostop.Important:Manually stoppingtccommands means that the edge-chaos agent does not stop executed commands on
shutdown.
Every settcrule has to be manually deleted.Further, becausetcexpects a list of parameters rather than flags, we expect theparametersobject to have
a single key (tc) which value is a list of strings that is passed on, without modification, to thetccommand.For example, to add a 100ms delay on the egress of theeth0network interface, send:{"name":"tc","parameters":{"tc":["qdisc","add","dev","eth0","root","netem","delay","100ms"]},"kind":"start"}And to remove thetcrule, send:{"name":"tc","parameters":{"tc":["qdisc","del","dev","eth0","root","netem","delay","100ms"]},"kind":"stop"}Notethat the value ofkindhasnoinfluence on the command.
However, it is recommended to set it appropriately for post-attack analysis.Environment variablesNameDefaultDescriptionedgechaos_logging_levelINFOSetslogger leveledgechaos_redis_hostlocalhostRedis hostedgechaos_redis_port6379Redis portedgechaos_redis_passwordN/ARedis passwordedgechaos_listener_typeredisListener type (currently supported:redis,rabbitmq)edgechaos_client_typeredisClient type (currently supported:redis,rabbitmq)edgechaos_host$HOSTNAMEHostname, determines the channel the daemon listens toedgechaos_rabbitmq_urlN/ARabbitMq connection urledgechaos_rabbitmq_exchangeedgechaosUsed as name for the exchange to use for attacks