Package: cyanide
Version:1.3.0Web:https://cyanide.readthedocs.io/Download:http://pypi.python.org/pypi/cyanide/Source:http://github.com/celery/cyanide/Keywords:celery, stress, integration, functional, testingIntroductionTipsConfiguration TemplatesExamplesTest Suite OptionsVagrantStartingSSHStoppingEnvironment VariablesCYANIDE_TRANSCYANIDE_BROKERCYANIDE_BACKENDCYANIDE_QUEUECYANIDE_PREFETCHAWS_REGIONCustom SuitesInstallationDownloading and installing from sourceUsing the development versionIntroductionThis stress test suite will attempt to break the Celery worker in different
ways, and can also be used to write new stress test suites for projects
depending on Celery infrastructure.The worker must currently be started separately, and it’s encouraged
to repeat the suite using workers started with different configuration values.Ideas include:Default, single process:$ celery -A cyanide worker -c 1Default, multiple processes:$ celery -A cyanide worker -c 8Frequentmaxtasksperchildrecycling, single child process:$ celery -A cyanide worker -c 1 --maxtasksperchild=1Frequent autoscale scale down &maxtasksperchild, single child process:$ AUTOSCALE_KEEPALIVE=0.01 celery -A cyanide worker \
>   --autoscale=1,0 --maxtasksperchild=1Frequentmaxtasksperchild, multiple child processes:$ celery -A cyanide worker -c 8 --maxtasksperchild=1Processes terminated by time limits:$ celery -A cyanide worker --time-limit=1Frequentmaxtasksperchild, single child process with late ack:$ celery -A cyanide worker -c1 --maxtasksperchild=1 -Z acks_lateWorker using theeventletpool:Start the worker, here having a thousand green-threads:$ celery -A cyanide worker -c1000 -P eventletYou must activate thegreentest group when starting the test suite:$ celery cyanide -g greenWorker using thegeventpool:Start the worker, here having a thousand green-threads:$ celery -A cyanide worker -c1000 -P geventYou must activate thegreentest group when starting the test suite:$ celery cyanide -g greenTipsIt’s a good idea to include the--purge<celery worker--purge>argument to clear out tasks from previous runs.Note that the stress client will probably hang if the test fails, so this
test suite is currently not suited for automatic runs.Configuration TemplatesYou can select a configuration template using the-Zcommand-line argument
to anycelery-Acyanidecommand or thecelery cyanidecommand used to execute the test suite.The templates available are:defaultUsing AMQP as a broker, RPC as a result backend,
and using JSON serialization for task and result messages.Both broker and result store is expected to run at localhost.vagrant1Use the VM started bycelery vagrant upas the broker
and result backend (RabbitMQ).vagrant1_redisUse the VM started bycelery vagrant upas the broker
and result backend (Redis).redisUsing Redis as a broker and result backend.redistoreUsing Redis as a result backend only.acks_lateEnables late ack globally.pickleUsing pickle as the serializer for tasks and results
(also allowing the worker to receive and process pickled messages)confirmsEnables RabbitMQ publisher confirmations.eventsConfigure workers to send task events.proto1Use version 1 of the task message protocol (pre 4.0)You can see the resulting configuration from any template by running
the command:$ celery -A cyanide report -Z redisExamplesExample running the stress test using theredisconfiguration template:$ cyanide -Z redisExample running the worker using theredisconfiguration template:$ celery -A cyanide worker -Z redisYou can also mix several templates by providing a comma-separated list:$ celery -A cyanide worker -Z redis,acks_lateIn this example (redis,acks_late) theredistemplate will be used
as main configuration, and then the additional keys from theacks_latetemplate
will be merged as changes.Test Suite OptionsAfter one or more worker instances are running, you can start executing the
tests.By default the complete test suite will be executed:$ celery cyanideYou can also specify what test cases to run by providing one or more names
as arguments:$ celery cyanide revoketermfast revoketermslowA full list of test case names can be retrieved with the-l<celery cyanide-l>switch:$ celery cyanide -l
.> 1) chain,
.> 2) chaincomplex,
.> 3) parentids_chain,
.> 4) parentids_group,
.> 5) manyshort,
.> 6) unicodetask,
.> 7) always_timeout,
.> 8) termbysig,
.> 9) timelimits,
.> 10) timelimits_soft,
.> 11) alwayskilled,
.> 12) alwaysexits,
.> 13) bigtasksbigvalue,
.> 14) bigtasks,
.> 15) smalltasks,
.> 16) revoketermfast,
.> 17) revoketermslowYou can also start from an offset within this list, e.g. to skip the first two
tests use--offset=2<celery cyanide--offset>:$ celery cyanide --offset=2Seecelery cyanide--helpfor a list of all available
command-line options.VagrantStartingCyanide ships with a complete virtual machine solution to run your tests.
The image ships with Celery, Cyanide, RabbitMQ and Redis and can be deployed
simply by running thecelery vagrantcommand:$ celery vagrant upThe IP address of the new virtual machine will be 192.168.33.123,
and you can easily tell both the worker and cyanide test suite to use
it by specifying thevagrant1(RabbitMQ) orvagrant1_redistemplates:$ celery -A worker -Z vagrant1
$ celery cyanide -Z vagrant1SSHTo open an SSH session with the virtual machine after starting
withcelery vagrant updo:$ ssh $(celery vagrant sshargs)StoppingTo shutdown the virtual machine run the command:$ celery vagrant haltTo destroy the instance run the command:$ celery vagrant destroyNoteTo completely wipe your instance you need to remove the.vagrantdirectory.The location of this directory can be retrieved by executing
the following:$ celery vagrant statedir
/opt/devel/cyanide/cyanide/vagrant/.vagrantYou can combine this withrmto force removal of this
directory:$ rm -rf $(celery vagrant statedir)Environment VariablesCYANIDE_TRANSIf theCYANIDE_TRANSenvironment variable is set
the stress test suite will use transient task messages instead of persisting
messages to disk.To avoid declaration collision thecstress.transqueue name will be used
when this option is enabled.CYANIDE_BROKERYou can set theCYANIDE_BROKERenvironment variable
to change the default broker used:$ CYANIDE_BROKER='amqp://' celery -A cyanide worker # ...
$ CYANIDE_BROKER='amqp://' celery cyanideCYANIDE_BACKENDYou can set theCYANIDE_BACKENDenvironment variable to change
the result backend used:$ CYANIDE_BACKEND='amqp://' celery -A cyanide worker # ...
$ CYANIDE_BACKEND='amqp://' celery cyanideCYANIDE_QUEUEA queue namedc.stressis created and used by default for all task
communication.You can change the name of this queue using theCYANIDE_QUEUEenvironment variable:$ CYANIDE_QUEUE='cyanide' celery -A cyanide worker # ...
$ CYANIDE_QUEUE='cyanide' celery cyanideCYANIDE_PREFETCHTheCYANIDE_PREFETCHenvironment variable sets the default prefetch
multiplier (default value is 10).AWS_REGIONTheAWS_REGIONenvironment variable changes the Amazon AWS region
to something other than the defaultus-east-1, to be used with thesqstemplate.Custom SuitesYou can define custom suites (look at source code ofcyanide.suites.defaultfor inspiration), and tell cyanide to use that
suite by specifying thecelery cyanide-Soption:$ celery cyanide -S proj.funtests:MySuiteInstallationYou can install cyanide either via the Python Package Index (PyPI)
or from source.To install usingpip:$ pip install -U cyanideDownloading and installing from sourceDownload the latest version of cyanide fromhttp://pypi.python.org/pypi/cyanideYou can install it by doing the following:$ tar xvfz cyanide-0.0.0.tar.gz
$ cd cyanide-0.0.0
$ python setup.py build
# python setup.py installThe last command must be executed as a privileged user if
you are not currently using a virtualenv.Using the development versionWith pipYou can install the latest snapshot of cyanide using the following
pip command:$ pip install https://github.com/celery/cyanide/zipball/master#egg=cyanide