Package: collective.recipe.celery
ContentsIntroductionSupported optionsGeneral optionsCelery optionsChangelogExample usageIntroductionThis recipe installs Celery and creates aceleryconfig.pymodule with
the specified configuration options. It helps managing multiple configurations
(e.g. development and production) using buildout.You can use it in a part like this:[celery]
recipe = collective.recipe.celery
broker-transport = sqlakombu.transport.Transport
broker-host = sqlite:///celery_broker.db
result-backend = database
result-dburi = sqlite:///celery_results.db
imports = myapp.tasks
eggs =
    kombu-sqlalchemy
    myappSupported optionsGeneral optionseggsA list of additional eggs you want to make available to Celery. Use this to
add additional dependencies such askombu-sqlalchemyor the module(s)
containing your task definitions.scriptsControls which scripts are generated. If the option is omitted, then all
scripts will be generated. If no value is given, then script generation is
disabled.config-pathThe location of the directory containing theceleryconfig.pymodule. By
default the config module is created in the part directory.
You can use this in other parts to include the config module:[celery]
recipe = collective.recipe.celery

[myapp]
recipe = zc.recipe.egg
eggs = myapp
extra-paths = ${celery:config-path}Celery optionsThe following configuration options are supported. See Celery documentation for
more details.broker-transportThe Kombu transport to use. You can use a custom transport class name, or
select one of the built-in transports:amqplib,pika,redis,beanstalk,sqlalchemy,django,mongodb,couchdb.broker-hostThe hostname of the broker.broker-portThe port number of the broker.broker-userThe username to connect as.broker-passwordThe password to connect with.broker-vhostThe virtual host.result-backendThe backend used to store task results. Can be one ofdatabase,cache,mongodb,redis,tyrantoramqp.result-dburiConnection string for the database result backend.importsA list of modules to import when the celery daemon starts. Specify one
module per line.celeryd-log-fileThe filename where the celery daemon logs messages to.celeryd-log-levelThe log level, can be one ofDEBUG,INFO,WARNING,ERRORorCRITICAL.celeryd-concurrencyThe number of concurrent worker processes/threads/green threads, executing
tasks.additional-configAny additional configuration directives can be added using theadditional-configoption.Example:additional-config =
    CELERY_TASK_PUBLISH_RETRY=True
    CELERY_TASK_PUBLISH_RETRY_POLICY={"max_retries": 2,
                                      "interval_start": 10,
                                      "interval_step": 0,
                                      "interval_max": 10}Changelog1.0 (2011-08-15)Initial release.
[buchi]Example usageWe’ll start by creating a buildout that uses the recipe:>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = celery
... index = %(server)s/index
... find-links = %(server)s
...
... [celery]
... recipe = collective.recipe.celery
... broker-transport = sqlakombu.transport.Transport
... broker-host = sqlite:///celery_broker.db
... result-backend = database
... result-dburi = sqlite:///celery_results.db
... imports = myapp.tasks
... """% dict(server=link_server))Running the buildout gives us:>>> print system(buildout)
Installing celery.
celery: Creating directory /sample-buildout/parts/celery.
celery: Generated config file /sample-buildout/parts/celery/celeryconfig.py.
Getting distribution for 'celery'.
Got celery 2.3.1.
Generated script '/sample-buildout/bin/celeryctl'.
Generated script '/sample-buildout/bin/celeryd'.
<BLANKLINE>Check that we have the celery scripts:>>> ls(sample_buildout, 'bin')
-  buildout
-  celeryctl
-  celerydCheck that we got a celery config file:>>> ls(sample_buildout, 'parts', 'celery')
- celeryconfig.pyIf we run the celeryd script, it prints out the config data:>>> print(system(join(sample_buildout, 'bin', 'celeryd')))
BROKER_HOST='sqlite:///celery_broker.db'
BROKER_TRANSPORT='sqlakombu.transport.Transport'
CELERY_IMPORTS=('myapp.tasks',)
CELERY_RESULT_BACKEND='database'
CELERY_RESULT_DBURI='sqlite:///celery_results.db'
<BLANKLINE>We can include additional eggs using the eggs option:>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = celery
... index = %(server)s/index
... find-links = %(server)s
...
... [celery]
... recipe = collective.recipe.celery
... eggs =
...     other
... """% dict(server=link_server))

>>> print system(buildout),
Uninstalling celery.
Installing celery.
celery: Generated config file /sample-buildout/parts/celery/celeryconfig.py.
Getting distribution for 'other'.
Got other 1.0.
Generated script '/sample-buildout/bin/celeryctl'.
Generated script '/sample-buildout/bin/celeryd'.We can control which scripts are generated using the scripts option.
If no value is given, then script generation is disabled:>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = celery
... index = %(server)s/index
... find-links = %(server)s
...
... [celery]
... recipe = collective.recipe.celery
... scripts =
... """% dict(server=link_server))

>>> print system(buildout),
Uninstalling celery.
Installing celery.
celery: Generated config file /sample-buildout/parts/celery/celeryconfig.py.

>>> ls(sample_buildout, 'bin')
-  buildoutLet’s create the celeryd script only:>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = celery
... index = %(server)s/index
... find-links = %(server)s
...
... [celery]
... recipe = collective.recipe.celery
... scripts =
...     celeryd
... """% dict(server=link_server))

>>> print system(buildout),
Uninstalling celery.
Installing celery.
celery: Generated config file /sample-buildout/parts/celery/celeryconfig.py.
Generated script '/sample-buildout/bin/celeryd'.

>>> ls(sample_buildout, 'bin')
-  buildout
-  celerydThe supported configuration directives may be of various types including
strings, integers and tuples:>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = celery
... index = %(server)s/index
... find-links = %(server)s
...
... [celery]
... recipe = collective.recipe.celery
... broker-port = 8080
... broker-user = guest
... imports =
...     myapp.tasks
...     other.tasks
... """% dict(server=link_server))

>>> print system(buildout),
Uninstalling celery.
Installing celery.
celery: Generated config file /sample-buildout/parts/celery/celeryconfig.py.
Generated script '/sample-buildout/bin/celeryctl'.
Generated script '/sample-buildout/bin/celeryd'.Let’s verify the generated config data:>>> cat(sample_buildout, 'parts', 'celery', 'celeryconfig.py')
BROKER_PORT = 8080
BROKER_USER = 'guest'
CELERY_IMPORTS = ('myapp.tasks', 'other.tasks')
<BLANKLINE>The recipe supports a limited set of celery’s configuration directives. Any
additional directives can be added using the additional-config option:>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = celery
... index = %(server)s/index
... find-links = %(server)s
...
... [celery]
... recipe = collective.recipe.celery
... additional-config =
...     CELERY_TASK_PUBLISH_RETRY = True
...     CELERY_TASK_PUBLISH_RETRY_POLICY = {"max_retries": 2,
...                                         "interval_start": 10,
...                                         "interval_step": 0,
...                                         "interval_max": 10}
... """% dict(server=link_server))

>>> print system(buildout),
Uninstalling celery.
Installing celery.
celery: Generated config file /sample-buildout/parts/celery/celeryconfig.py.
Generated script '/sample-buildout/bin/celeryctl'.
Generated script '/sample-buildout/bin/celeryd'.Let’s verify the generated config data:>>> cat(sample_buildout, 'parts', 'celery', 'celeryconfig.py')
CELERY_TASK_PUBLISH_RETRY = True
CELERY_TASK_PUBLISH_RETRY_POLICY = {"max_retries": 2,
"interval_start": 10,
"interval_step": 0,
"interval_max": 10}
<BLANKLINE>