Package: django-beanstalk-worker
django-beanstalk-workerAn app for handling deferred and periodic tasks on beanstalk worker machinesOverviewInstallation and Setup1: Installed Apps2: Environment Variables3: Django Settings4: URLs's5: Beanstalk Worker ConfigurationUseDeclare a task functionCall the task from anywhere in your codeCall the task from CRONCall the task from the command lineDevelopment, Testing and the FakeTaskServersettings.DEBUGTest supportsettings.BEANSTALK_WORKEROverviewThe core of this app is the task decorator.When a decorated task function is called instead of running immediately an SQS message is queued. When unqueued by a worker that message will cause the original function to be called.The arguments may be any combination of JSON serializable types, datetimes and decimals, including keyword arguments. Any other argument types will cause an error.Queuing of SQS messages happens in transaction on_commit so if you exception out of a transaction any tasks called in that transaction will not be run.Task functions can also be invoked by cron or a management command as needed.Installation and Setup1: Installed AppsAddbeanstalk_workertoinstalled_apps.2: Environment VariablesIn theSoftwaresection of the Beanstalk configuration of your worker environment add the environment variableWORKERand set it to13: Django SettingsAdd the following settings to your Django settings:BEANSTALK_WORKER=bool(os.environ.get("WORKER",False))BEANSTALK_TASK_SERVICE="beanstalk_worker.services.TaskService"BEANSTALK_SQS_URL=<SQSqueueURL>BEANSTALK_SQS_REGION=<Amazonregion>You can find the SQS queue URL in theWorkersection of the Beanstalk environment configuration.For test and development you can omit the SQS settings and should setBEANSTALK_TASK_SERVICE = "beanstalk_worker.services.FakeTaskService"4: URLs'sIn your top level URL's addifsettings.BEANSTALK_WORKER:urlpatterns.append(url(r"^tasks/",include("beanstalk_worker.urls")))This will add the URLS/tasks/task/and/tasks/cron. You can move the base of these URL's if you do so other instructions on this page will need updating appropriately.DO NOT include these URL's in a production web server, only the worker.For test and development you won't have seperate web and worker machines so always include the URL's.5: Beanstalk Worker ConfigurationIn theWorkersection of the Beanstalk configuration of your worker environment setHTTP pathto/tasks/task/andMIME typetoapplication/jsonUseDeclare a task functionfrombeanstalk_workerimporttask@taskdefmy_task(message="hi"):print(message)Call the task from anywhere in your codemy_task("hello world")This will queue an SQS message instructing a worker to run the actual function.The arguments may be any combination of JSON serializable types, datetimes and decimals, including keyword argumentsCall the task from CRONIncron.yamladd-name:"my_project.my_app.tasks.my_task"url:"/tasks/cron/"schedule:"00***"my_project.my_app.tasks.my_taskshould be replaced with the fully qualified name of your task function.Arguments are not currently supported for cron.Call the task from the command line./manage.py run_task my_project.my_app.tasks task "['hello world']"my_project.my_app.tasks my_taskshould be replaced with the fully qualified name of your task function, also note the space between module name and function name.Development, Testing and the FakeTaskServerThe FakeTaskServer will internally queue tasks but will not run them unless instructed to.settings.DEBUGWhenDEBUG = Trueis set in the Django settings an additional URL is exposed at/tasks/run_all/if you poke this URL theFakeTaskServicewill run all queued tasks. This can be handy for local development.Test supportIn tests you can acquire the running task service instance withfrom beanstalk_worker import task_service. This class has two helpersclearwhcih will discard all queued tasks andrun_allwhich will immediately run all queued tasks.settings.BEANSTALK_WORKERWhile running a task the FakeTaskServer will patchsettings.BEANSTALK_WORKERtoTrue. This lets youassert settings.BEANSTALK_WORKERin code that should only ever be run on a worker.