Issue 1:
Related Issues

#6017 Celery Multi creates pid and log files in the wrong directory

Environment & Settings

Celery version:

celery report Output:

# celery report

software -> celery:4.4.3 (cliffs) kombu:4.6.9 py:3.7.7
billiard:3.6.3.0 py-amqp:2.6.0
platform -> system:Linux arch:64bit, ELF
kernel version:4.19.0-8-amd64 imp:CPython
loader -> celery.loaders.default.Loader
settings -> transport:amqp results:disabled

Steps to Reproduce

Required Dependencies

Minimal Celery Version: 4.4.3

Minimally Reproducible Test Case

celery multi start ... --pidfile=/var/run/demo/celeryd-%n.pid

Expected Behavior

celery runs

Actual Behavior

start failed

celery multi v4.4.3 (cliffs)
_annotate_with_default_opts: print options
OrderedDict([('--app', 'service.celery:app'),
('--pidfile', '/var/run/demo/celeryd-%n.pid'),
('--logfile', '/var/log/demo/celeryd-%n%I.log'),
('--loglevel', 'INFO'),
('--workdir', '/var/lib/demo-celery'),
('--events', None),
('--heartbeat-interval', '5'),
('--without-gossip', None),
('--queues', 'high'),
('--concurrency', '1'),
('-n', 'high@celeryd.worker')])
Traceback (most recent call last):
File "/var/www/misc/ve-2006011156/bin/celery", line 8, in <module>
sys.exit(main())
File "/var/www/misc/ve-2006011156/lib/python3.7/site-packages/celery/__main__.py", line 16, in main
_main()
File "/var/www/misc/ve-2006011156/lib/python3.7/site-packages/celery/bin/celery.py", line 322, in main
cmd.execute_from_commandline(argv)
File "/var/www/misc/ve-2006011156/lib/python3.7/site-packages/celery/bin/celery.py", line 495, in execute_from_commandline
super(CeleryCommand, self).execute_from_commandline(argv)))
File "/var/www/misc/ve-2006011156/lib/python3.7/site-packages/celery/bin/base.py", line 305, in execute_from_commandline
return self.handle_argv(self.prog_name, argv[1:])
File "/var/www/misc/ve-2006011156/lib/python3.7/site-packages/celery/bin/celery.py", line 487, in handle_argv
return self.execute(command, argv)
File "/var/www/misc/ve-2006011156/lib/python3.7/site-packages/celery/bin/celery.py", line 419, in execute
).run_from_argv(self.prog_name, argv[1:], command=argv[0])
File "/var/www/misc/ve-2006011156/lib/python3.7/site-packages/celery/bin/celery.py", line 335, in run_from_argv
return cmd.execute_from_commandline([command] + argv)
File "/var/www/misc/ve-2006011156/lib/python3.7/site-packages/celery/bin/multi.py", line 266, in execute_from_commandline
return self.call_command(argv[0], argv[1:])
File "/var/www/misc/ve-2006011156/lib/python3.7/site-packages/celery/bin/multi.py", line 273, in call_command
return self.commands[command](*argv) or EX_OK
File "/var/www/misc/ve-2006011156/lib/python3.7/site-packages/celery/bin/multi.py", line 143, in _inner
return fun(self, *args, **kwargs)
File "/var/www/misc/ve-2006011156/lib/python3.7/site-packages/celery/bin/multi.py", line 151, in _inner
return fun(self, self.cluster_from_argv(argv), **kwargs)
File "/var/www/misc/ve-2006011156/lib/python3.7/site-packages/celery/bin/multi.py", line 361, in cluster_from_argv
_, cluster = self._cluster_from_argv(argv, cmd=cmd)
File "/var/www/misc/ve-2006011156/lib/python3.7/site-packages/celery/bin/multi.py", line 366, in _cluster_from_argv
return p, self.Cluster(list(nodes), cmd=cmd)
File "/var/www/misc/ve-2006011156/lib/python3.7/site-packages/celery/apps/multi.py", line 306, in <genexpr>
for name in names
File "/var/www/misc/ve-2006011156/lib/python3.7/site-packages/celery/apps/multi.py", line 314, in _node_from_options
p.optmerge(namespace, options), p.passthrough)
File "/var/www/misc/ve-2006011156/lib/python3.7/site-packages/celery/apps/multi.py", line 136, in __init__
options or OrderedDict())
File "/var/www/misc/ve-2006011156/lib/python3.7/site-packages/celery/apps/multi.py", line 145, in _annotate_with_default_opts
self._setdefaultopt(options, ['--pidfile', '-p'], '/var/run/celery/%n.pid')
File "/var/www/misc/ve-2006011156/lib/python3.7/site-packages/celery/apps/multi.py", line 159, in _setdefaultopt
os.makedirs(dir_path)
File "/var/www/misc/ve-2006011156/lib/python3.7/os.py", line 223, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/var/run/celery'

systemd[1]: demo@celeryd.service: Control process exited, code=exited, status=1/FAILURE

Issue 2:
Checklist

I have verified that the issue exists against the master branch of Celery.

This has already been asked to the discussion group first.

I have read the relevant section in the

contribution guide

on reporting bugs.

I have checked the issues list

for similar or identical bug reports.

I have checked the pull requests list

for existing proposed fixes.

I have checked the commit log

to find out if the bug was already fixed in the master branch.

I have included all related issues and possible duplicate issues

in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

I have verified that the issue exists against the master branch of Celery.

I have included the contents of pip freeze in the issue.

I have included all the versions of all the external dependencies required

to reproduce this bug.

Optional Debugging Information

I have tried reproducing the issue on more than one Python version

and/or implementation.

I have tried reproducing the issue on more than one message broker and/or

result backend.

I have tried reproducing the issue on more than one version of the message

broker and/or result backend.

I have tried reproducing the issue on more than one operating system.

I have tried reproducing the issue on more than one workers pool.

I have tried reproducing the issue with autoscaling, retries,

ETA/Countdown & rate limits disabled.

I have tried reproducing the issue after downgrading

and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

None

Possible Duplicates

None

Environment & Settings

Celery version:

celery report Output:

Steps to Reproduce

Required Dependencies

Minimal Python Version: N/A or Unknown

Minimal Celery Version: N/A or Unknown

Minimal Kombu Version: N/A or Unknown

Minimal Broker Version: N/A or Unknown

Minimal Result Backend Version: N/A or Unknown

Minimal OS and/or Kernel Version: N/A or Unknown

Minimal Broker Client Version: N/A or Unknown

Minimal Result Backend Client Version: N/A or Unknown

Python Packages

pip freeze Output:

Other Dependencies

N/A

Minimally Reproducible Test Case

Expected Behavior

Actual Behavior