Package: baseplate-celery
baseplate-celeryBaseplate Celery allows us to integrate Celery in Baseplate.py services.Celeryis an open source asynchronous task queue. Integrating your Baseplate.py service with Celery can enable you to offload some tasks to run in the background, while leveraging some of Baseplate.py's configurations and telemetry.Baseplate Celery will allow you to:Instantiate an instance of Celery.Identify background task functions within your application with a simple decorator.Push background tasks to a message queue (or broker), currently supported broker is Redis.Instantiate Celery workers to pick up tasks from the queue and execute in the background.UsageInstall the library:$pipinstallbaseplate-celery1. Instantiate BaseplateCeleryConfigureBaseplateCeleryfrom the main application's configuration file.[app:main]...# required: name of applicationcelery.service_name=helloworld# required: the Redis instance to connect to as the message brokercelery.broker_url=redis://localhost:6379/1...It is recommended to create a dedicated module forBaseplateCeleryin your application. For example, your project structure could look like:helloworld/jobs/__init__.py
               /celery.py
               /tasks.pyCreate theBaseplateCeleryinstance. To useBaseplateCelerywithin your application, import this instance.helloworld/jobs/celery.pyfrombaseplate-celeryimportBaseplateCelerycelery_app=BaseplateCelery("helloworld")2. Identify background tasksCreate a file calledtasks.pyin this module which contains all the functions you would like to register as Celery tasks. One option is to define all tasks directly in this file. Another option if the tasks are defined in various parts of your application is to import all of them into thetasks.pyfile.helloworld/jobs/tasks.pyfrom.celeryimportcelery_app@celery_app.taskdefadd(ctx,x,y):returnx+yAll task functions pass the BaseplateRequestContextas a parameter to avoid issues of concurrency.3. Configure BaseplateCelery workersIn your application's configuration.inifile, add a section for configuring thecelery-worker.[app:celery-worker]# required: factory specifies the entry point for setting up Baseplate on the Celery workerfactory=helloworld:make_celery_worker# required: the module which contains the `tasks.py` filetasks=helloworld.jobs# optional: additional command line arguments for starting the Celery workercommand=--loglevel INFO --pool gevent --time-limit 180 --concurrency 1To start the celery worker, use thebaseplate-servecommand:baseplate-serve example.ini --app-name celery-workerYour application might already have amake_baseplatefunction doing all the Baseplate setup. The celery worker's entrypoint function should do the exact same Baseplate setup and then start the workers. For example:defmake_baseplate(app_config):baseplate=Baseplate(app_config)baseplate.configure_observers()celery_app.set_baseplate(baseplate)...defmake_celery_worker(app_config):make_baseplate(app_config)celery_app.run_workers(app_config)4. Push background tasks to brokerNow that your application is set configured with a BaseplateCelery instance and the Celery workers are running, you can push a task to the broker by calling:add.delay(4, 4)Documentation:See [the documentation] for more information.[the documentation]:DevelopmentA Dockerfile is provided to get a development environment running. To use it,
build the base Docker image:$dockerbuild-tbaseplate_celery.And then fire up the environment and use the provided Makefile targets to do
common tasks:$dockerrun-it-v$PWD:/src-w/srcbaseplate_celery$makefmtThe following make targets are provided:fmt: Apply automatic formatting to the source code.lint: Run linters on the code.test: Run the test suite.docs: Build the docs. Output can be found inbuild/html/.Note: some tests are skipped by default locally because they are quite slow.
Enable these by setting CI=true in the environment:CI=true make test.