Issue 1:
Checklist

I have verified that the issue exists against the master branch of Celery.

This has already been asked to the discussion group first.

I have read the relevant section in the

contribution guide

on reporting bugs.

I have checked the issues list

for similar or identical bug reports.

I have checked the pull requests list

for existing proposed fixes.

I have checked the commit log

to find out if the bug was already fixed in the master branch.

I have included all related issues and possible duplicate issues

in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

I have verified that the issue exists against the master branch of Celery.

I have included the contents of pip freeze in the issue.

I have included all the versions of all the external dependencies required

to reproduce this bug.

Optional Debugging Information

I have tried reproducing the issue on more than one Python version

and/or implementation.

I have tried reproducing the issue on more than one message broker and/or

result backend.

I have tried reproducing the issue on more than one version of the message

broker and/or result backend.

I have tried reproducing the issue on more than one operating system.

I have tried reproducing the issue on more than one workers pool.

I have tried reproducing the issue with autoscaling, retries,

ETA/Countdown & rate limits disabled.

I have tried reproducing the issue after downgrading

and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

None

Possible Duplicates

#6307

Environment & Settings

Celery version:

celery report Output:

software -> celery:5.1.2 (sun-harmonics) kombu:5.1.0 py:3.9.1
billiard:3.6.4.0 py-amqp:5.0.6
platform -> system:Darwin arch:64bit
kernel version:20.5.0 imp:CPython
loader -> celery.loaders.default.Loader
settings -> transport:amqp results:disabled

deprecated_settings: None

Steps to Reproduce

Required Dependencies

Minimal Python Version: N/A or Unknown

Minimal Celery Version: N/A or Unknown

Minimal Kombu Version: N/A or Unknown

Minimal Broker Version: N/A or Unknown

Minimal Result Backend Version: N/A or Unknown

Minimal OS and/or Kernel Version: N/A or Unknown

Minimal Broker Client Version: N/A or Unknown

Minimal Result Backend Client Version: N/A or Unknown

Python Packages

pip freeze Output:

amqp==5.0.6
appdirs==1.4.4
billiard==3.6.4.0
black==20.8b1
celery==5.1.2
click==7.1.2
click-didyoumean==0.0.3
click-plugins==1.1.1
click-repl==0.2.0
flake8==3.9.2
Flask==2.0.1
isort==5.9.1
itsdangerous==2.0.1
Jinja2==3.0.1
kombu==5.1.0
MarkupSafe==2.0.1
mccabe==0.6.1
motor==2.4.0
mypy-extensions==0.4.3
numpy==1.21.0
pathspec==0.8.1
prompt-toolkit==3.0.19
pycodestyle==2.7.0
pyflakes==2.3.1
pymongo==3.11.4
pytz==2021.1
PyYAML==5.4.1
regex==2021.4.4
six==1.16.0
toml==0.10.2
typed-ast==1.4.3
typing-extensions==3.10.0.0
vine==5.0.0
wcwidth==0.2.5
Werkzeug==2.0.1

Other Dependencies

N/A

Minimally Reproducible Test Case

celery.py

from celery import Celery, signals
from click import Option
import codecs
import yaml

def setup(yml_path: str):
global CONF
with codecs.open(yml_path, "r", "utf-8") as config_file:
CONF = yaml.load(config_file, Loader=yaml.FullLoader)

def load_config(yml_path: str):
conf =setup(yml_path) # Load conf file from custom yaml
celery_config = conf.get("celery", {})
app_config = celery_config.get("app", {})

app.conf.update(app_config)

return conf
def _create_celery():
app = Celery(include=["ais.tasks"])
app.user_options["preload"] = [
Option(["--config", "-C"], default="config/config.yml")
]

return app

"""
signal handlers
Warning: signal handlers only executed on server side
"""

@signals.user_preload_options.connect
def handle_preload_options(options, **kwargs):
load_config(options["config"])

app = _create_celery()

config.yml

celery:
app:
app_name: parser-worker
worker_concurrency: 4
broker_url: redis://localhost:6379/0

Expected Behavior

Concurrency worker prefork should be 4 process by app.conf.update from config.yml

Actual Behavior

Concurrency worker still prefork 8 process

Trace Code

The worker init flow with click confused

Module path .venv/lib/python3.9/site-packages/celery/bin/worker.py

I found use click lib instead of argparse in bin/worker

It use value or ctx.obj.app.conf.worker_concurrency for lambda value

we hope ctx.obj.app.conf.worker_concurrency be any int but 0.

Unfortunately, it seems not in this case.

ctx.obj.app.conf.worker_concurrency was setup after @handle_preload_options according to call stack.

The concurrency will also be 0 in worker/worker.py, though it use either func to either('worker_concurrency', concurrency).

Issue 2:
Checklist

I have read the relevant section in the

contribution guide

on reporting bugs.

I have checked the issues list

for similar or identical bug reports.

I have checked the pull requests list

for existing proposed fixes.

I have checked the commit log

to find out if the bug was already fixed in the master branch.

I have included all related issues and possible duplicate issues

in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

I have verified that the issue exists against the master branch of Celery.

I have included the contents of pip freeze in the issue.

Forbidden by company policy

I have included all the versions of all the external dependencies required

to reproduce this bug.

Optional Debugging Information

I have tried reproducing the issue on more than one Python version

and/or implementation.

I have tried reproducing the issue on more than one message broker and/or

result backend.

I have tried reproducing the issue on more than one version of the message

broker and/or result backend.

I have tried reproducing the issue on more than one operating system.

I have tried reproducing the issue on more than one workers pool.

I have tried reproducing the issue with autoscaling, retries,

ETA/Countdown & rate limits disabled.

I have tried reproducing the issue after downgrading

and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

None

Possible Duplicates

None

Environment & Settings

Celery version: 4.2.1

celery report Output:

Cannot include this due to company security policy

Steps to Reproduce

Required Dependencies

Minimal Python Version: N/A or Unknown

Minimal Celery Version: N/A or Unknown

Minimal Kombu Version: N/A or Unknown

Minimal Broker Version: N/A or Unknown

Minimal Result Backend Version: N/A or Unknown

Minimal OS and/or Kernel Version: N/A or Unknown

Minimal Broker Client Version: N/A or Unknown

Minimal Result Backend Client Version: N/A or Unknown

Python Packages

pip freeze Output:

Cannot include this due to company security policy

Other Dependencies

N/A

Minimally Reproducible Test Case

Set broker_url to an string without a protocol

Start Celery worker with multi start, with a logfile specified with --logfile=

multi start claims the worker started successfully, when in fact it died immediately. No error message is written to STDOUT or STDERR and nothing at all is written to the logfile

Expected Behavior

At least one of the following:

multi start reports the worker did not start successfully

An error message or traceback is written to STDOUT or STDERR

The error is recorded in the logfile

Actual Behavior

[REDACTED_PROMPT]$ ~/tools/python/bin/celery multi start worker -A [REDACTED_PACKAGE].celery:APP --pidfile=celery-worker.pid --logfile=celery-worker.log --detach -O fair -l debug
celery multi v4.2.1 (windowlicker)
> Starting nodes...
> worker@[REDACTED_HOST]: OK
[REDACTED_PROMPT]$ # multi start reports the worker started successfully, and no error message was printed, yet...
[REDACTED_PROMPT]$ ps -aux | grep celery
foo_user 17792 0.0 0.0 9092 676 pts/0 S+ 19:29 0:00 grep --color=auto celery
[REDACTED_PROMPT]$ # The worker is not running
[REDACTED_PROMPT]$ cat celery-worker.log
[REDACTED_PROMPT]$ # And nothing is in the log, either