Issue 1:
Hi people,

I have 2 Django projects (at different branchs/versions) on the same machine and both use celery for asynchronous tasks.

I figure thanks to sentry that tasks from app A are sent to workers of app B. Each application runs under a different unix user and workers/wsgi run through supervisor.

Here the different configurations:

Application A

Supervisor

[program:demo-a-worker]
command=/home/demo_a/start_worker.sh
numprocs=1
directory=/home/demo_a
autostart=true
autorestart=true
user=demo_a
redirect_stderr=true
stdout_logfile=/var/log/demo_a/worker.log
stdout_logfile_maxbytes=1MB
stdout_logfile_backups=10
stdout_capture_maxbytes=1MB

Script

exec $PYENV/celery worker -A mydjangoproject -B -Q demo_a --loglevel=INFO --hostname demo_a.%h

Application B

Supervisord

[program:demo-b-worker]
command=/home/demo_b/start_worker.sh
numprocs=1
directory=/home/demo_b
autostart=true
autorestart=true
user=demo_b
redirect_stderr=true
stdout_logfile=/var/log/demo_b/worker.log
stdout_logfile_maxbytes=1MB
stdout_logfile_backups=10
stdout_capture_maxbytes=1MB

Script

exec $PYENV/celery worker -A mydjangoproject -B -Q demo_b --loglevel=INFO --hostname demo_b.%h

Celery configuration in mydjangoproject

# Set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mydjangoproject.settings')
os.environ.setdefault("DJANGO_CONFIGURATION", "Dev")

# Load environment configuration
from configurations import importer
importer.install()

class Celery(celery.Celery):
"""
Celery configuration class
"""
def on_configure(self):
# Get Raven configuration from Django settings
if not hasattr(settings, 'RAVEN_CONFIG'):
return
# Initialize Raven client
client = raven.Client(**settings.RAVEN_CONFIG)
# Register a custom filter to filter out duplicate logs
register_logger_signal(client)
# Hook into the Celery error handler
register_signal(client)

# Initialize Celery application
app = Celery('mydjangoproject')

# Using a string here means the worker will not have to
# pickle the object when using Windows.
app.config_from_object('django.conf:settings')

# Auto-discover celery tasks inside Django applications
from django.conf import settings
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

Settings

# Celery configuration
BROKER_URL = values.Value('redis://localhost:6379/0')
BROKER_TRANSPORT_OPTIONS = {
'visibility_timeout': 3600,
'fanout_prefix': True,
'fanout_patterns': True,
}
CELERY_TASK_SERIALIZER = 'pickle'
CELERY_RESULT_SERIALIZER = 'pickle'
CELERY_RESULT_BACKEND = values.Value('redis')
CELERY_TASK_RESULT_EXPIRES = 3600
CELERY_ACCEPT_CONTENT = ['pickle', 'json', 'msgpack', 'yaml']
CELERY_DISABLE_RATE_LIMITS = True
CELERY_CHORD_PROPAGATES = True
CELERY_ALWAYS_EAGER = values.BooleanValue(True)
CELERY_DEFAULT_QUEUE = values.Value('celery', environ_name='QUEUE_NAME')

Values are environment configuration, in production CELERY_ALWAYS_EAGER=False and queue name is the same as in the command line.

I take care that each worker has its own name and queue but it seems not to solve the issue.

Thanks!

Issue 2:
Checklist

[ x] I have checked the issues list

for similar or identical feature requests.

[ x] I have checked the pull requests list

for existing proposed implementations of this feature.

[ x] I have checked the commit log

to find out if the same feature was already implemented in the

main branch.

[x ] I have included all related issues and possible duplicate issues

in this issue (If there are none, check this box anyway).

Related Issues and Possible Duplicates

Related Issues

None

Possible Duplicates

None

Brief Summary

This is related to another issue I filed: ovh/celery-director#195: In a workflow, a step might need some info/value generated in a grand parent step(not a direct parent step), one solution is enable a step/task to save the info into payload, and then later step could access this payload. I am not good at celery, not sure whether this is possible. But the use case is very usefully.

Design

Architectural Considerations

None

Proposed Behavior

Proposed UI/UX

Diagrams

N/A

Alternatives

None