Package: mite
MiteLoad Testing FrameworkMite is a load testing framework with distributed components written in Python.
Requests are executed asynchronously, allowing large throughput from relatively small infrastructure.InstallationpipinstallmiteThis requires that you have libcurl installed on your system (including C header files for development, which are often distributed separately from the shared libraries).  On Ubuntu, this can be accomplished with the command:sudo apt install libcurl4 libcurl4-openssl-dev(NB we recommend using a version of libcurl linked against openssl
rather than gnutls, since the latter has memory leak problems)You can also use the dockerfile included in this repository to run
mite.  In order to get a shell in a container with mite installed, run
these commands (assuming you have docker installed on your machine):docker build -t mite .
docker run --rm -it mite shRunmite --helpfor a full list of commandsYour first scenarioScenarios are a combination of 3 things, a set of journeys to run, a
datapool to provide test data for the journey (if applicable), and a
volume model.JourneysJourneys are async python functions, and are where you put the logic of
what you're trying to achieve.  Below is a simple example:importasyncioasyncdefjourney(ctx,arg1,arg2):asyncwithctx.transaction('test1'):ctx.send('test_message',content=ctx.config.get('test_msg','Not set'),sum=arg1+arg2)awaitasyncio.sleep(0.5)This journey just sends a log message to be collected by the framework
and waits for half a second.  This journey takes 3 arguments, a context
and two numbers (which will be supplied by the datapool, see below).ContextThe context is important for every journey.  It provides a number of
useful methods for timing/naming of transactions, sending messages and
defaults to including http functionality from acurl.  Functionality can
be included in the journey by attaching it to the context.In the above example we see an example of naming a transactionasync with ctx.transaction('test1'):.  This will capture timing for the
indented block.We also see that the config is attached to the context withctx.config.get('test_msg', 'Not set')and in this case, the value of
that config value is sent to the log collector withctx.send('test_message', content=ctx.config.get('test_msg', 'Not set'))DatapoolsTo feed data into your test journey, you'll need a datapool. Several of
these are already provided in mite and usually read data in from an
iterable. To specify a custom datapool implementation, you simply need a
class that implements acheckinmethod which adds data to the pool and
acheckoutmethod which removes data from the pool to be used by
journeys.For the above journey, which expects two arguments, we will show an
example of the RecyclableIterableDataPool.frommite.datapoolsimportRecyclableIterableDataPooldatapool=RecyclableIterableDataPool([(i,i+2)foriinrange(5000)])This pool will share data out to runners and check it back in when the
runners are done with that block of data.  In this case it cycles
through a list of tuples which each contain two integers.Volume ModelA volume model defines how many instances of the journey should be ran
within a window of time.  The window is definied as a start and an end
time (measured in seconds since the beginning of the test), which will be
fed to the model by the framework.  This allows complex calculations to
specify the current volume based on the current time.  The volume model
can also raiseStopVolumeModelto inform mite that the load injection
should stop for a given journey.  For this example we'll use a simple
volume model which merely runs ten journeys simultaneously, forever.volumemodel=lambdastart,end:10ScenarioWe now need to package the journey, datapool and volume model into a
scenario.  This is a simple as defining a function which returns
a list of triplets of (journey name, datapool, volumemodel).defscenario():return[['mite.example:journey',datapool,volumemodel],]The journey name should be a string with two parts separated by a
colon.  The first part is the name of a python module that is
importable; the second is the name of a journey (an async function)
within that module.  It is necessary to specify this as a string, rather
than as a python object, because the journey will be executed in a
different python process than the scenario function is.  Thus, we need a
name for the journey that allows any python process to find it.The volume model and data pool, on the other hand, are only used in the
python process where the scenario function runs.  They are both python
objects.Testing the journeyBefore running the scenario, we should test the journey in isolation as
a scenario can be made up of multiple journeys.  This can be done with
themite journey testcommand.  We just need to pass the name of the
journey and the datapool it requires:MITE_CONF_test_msg="Hello from mite"mitejourneytestmite.example:journeymite.example:datapoolIf something goes wrong, adding the--debuggingflag to this command
will drop excution into a debug session. The choice of debugger used can be
managed by setting thePYTHONBREAKPOINTenvironment variablebefore running mite. Python's built-inpdb)
debugger is invoked by default, but this can be changed to use, say, theipdb debugger:pip install ipdb
export PYTHONBREAKPOINT=ipdb.set_trace
export PYTHONPOSTMORTEM=ipdb.post_mortemPYTHONPOSTMORTEMis a mite-specific extension toPEP
553which defines thePYTHONBREAKPOINTfunctionality.Run the scenarioIn order to run the finished scenario locally, which will include all
the necessary fixtures, run the following command:MITE_CONF_test_msg="Hello from mite"mitescenariotestmite.example:scenarioDistributed deploymentsIn order to scale up miteʼs capability to inject load, you will need to
run it as a distributed suite of components.  You can learn more about
how to accomplish this in thedocumentation.Deploy distributed mite with docker composeBuild mite image:docker build -t mite .Run mite deployments:Usemakefrommite/localdir:make up # start mite containers
make status # check status of mite containers
make clean # remove all mite containersordocker-compose -f docker_compose.yml upFor more information on distributed mite usage,infoMaintainersIf you run into any trouble or need support getting to grips with Mite,
reach out onSlackif you work at Sky,
or contact one of the maintainers if you're an external contributer:Jordan Brennan💻Aaron Ecay💻Davide Annunziata💻Ryan Linnit💻Cosmin Purcherea💻Special thanks to the following contributors:Tony Simpson