Issue 1:
It always use pickle instead.

Issue 2:
Checklist

I have verified that the issue exists against the master branch of Celery.

This has already been asked to the discussion group first.

I have read the relevant section in the

contribution guide

on reporting bugs.

I have checked the issues list

for similar or identical bug reports.

I have checked the pull requests list

for existing proposed fixes.

I have checked the commit log

to find out if the bug was already fixed in the master branch.

I have included all related issues and possible duplicate issues

in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

I have verified that the issue exists against the master branch of Celery.

I have included the contents of pip freeze in the issue.

I have included all the versions of all the external dependencies required

to reproduce this bug.

Optional Debugging Information

I have tried reproducing the issue on more than one Python version

and/or implementation.

I have tried reproducing the issue on more than one message broker and/or

result backend.

I have tried reproducing the issue on more than one version of the message

broker and/or result backend.

I have tried reproducing the issue on more than one operating system.

I have tried reproducing the issue on more than one workers pool.

I have tried reproducing the issue with autoscaling, retries,

ETA/Countdown & rate limits disabled.

I have tried reproducing the issue after downgrading

and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

None

Possible Duplicates

None

Environment & Settings

Celery Version: 5.0.0 (singularity)

celery report Output:

root@ad36e1cf0d81:/app# celery -A app.worker report

software -> celery:5.0.0 (singularity) kombu:5.0.2 py:3.7.5
billiard:3.6.3.0 py-amqp:5.0.1
platform -> system:Linux arch:64bit
kernel version:5.4.0-1025-aws imp:CPython
loader -> celery.loaders.app.AppLoader
settings -> transport:amqp results:disabled

broker_url: 'amqp://prediction_celery:********@broker:5672//'
deprecated_settings: None
task_routes: {
'app.worker.*': {'queue': 'high_memory_usage'}}
task_serializer: 'pickle'
accept_content: ['json', 'pickle']
broker_transport_options: {
'interval_max': 0.5,
'interval_start': 0,
'interval_step': 0.2,
'max_retries': 3}
worker_prefetch_multiplier: 1

software -> celery:5.0.0 (singularity) kombu:5.0.2 py:3.7.5
billiard:3.6.3.0 py-amqp:5.0.1
platform -> system:Linux arch:64bit
kernel version:5.4.0-1025-aws imp:CPython
loader -> celery.loaders.default.Loader
settings -> transport:amqp results:disabled

deprecated_settings: None

Steps to Reproduce

Required Dependencies

Minimal Python Version: 3.7.5

Minimal Celery Version: 5.0.0

Minimal Kombu Version: 5.0.2

Minimal Broker Version: RabbitMQ version: 3.8.9

Minimal Result Backend Version: N/A or Unknown

Minimal OS and/or Kernel Version: N/A or Unknown

Minimal Broker Client Version: N/A or Unknown

Minimal Result Backend Client Version: N/A or Unknown

Python Packages

pip freeze Output:

amqp==5.0.1
bidalgo-common==0.0
billiard==3.6.3.0
boto3==1.10.45
botocore==1.13.45
celery==5.0.0
certifi==2020.6.20
chardet==3.0.4
click==7.1.2
click-didyoumean==0.0.3
click-repl==0.1.6
colormath==3.0.0
common==0.1
coverage==5.0.3
creative-analysis===-module.-creative-analysis.version-.from.-tmp-pip-req-build-v4szy4xr-creative-analysis-version.py-
custom-stats==0.1
cycler==0.10.0
dataclasses==0.6
dataclasses-json==0.3.7
decorator==4.4.2
docutils==0.15.2
idna==2.8
importlib-metadata==2.0.0
jmespath==0.10.0
joblib==0.16.0
kiwisolver==1.2.0
kombu==5.0.2
marshmallow==3.8.0
marshmallow-enum==1.5.1
matplotlib==3.1.1
mypy-extensions==0.4.3
mysqlclient==2.0.1
networkx==2.5
nlp-tools==0.0
nltk==3.4.5
numpy==1.16.5
pandas==0.25.1
patsy==0.5.1
Pillow==5.4.1
prompt-toolkit==3.0.7
pydantic==1.6.1
PyMySQL==0.9.3
pyparsing==2.4.7
python-dateutil==2.8.1
python-dotenv==0.10.3
pytz==2020.1
redis==3.3.11
requests==2.22.0
s3transfer==0.2.1
scikit-learn==0.21.3
scipy==1.3.1
six==1.15.0
SQLAlchemy==1.3.19
statsmodels==0.11.0
stringcase==1.2.0
text2digits==0.0.9
typing-extensions==3.7.4.3
typing-inspect==0.6.0
urllib3==1.25.10
vine==5.0.0
wcwidth==0.2.5
Werkzeug==1.0.1
zipp==3.2.0

Other Dependencies

Status of node rabbit@f1b3c114b8d6 ...
Runtime

OS PID: 275

OS: Linux

Uptime (seconds): 759

Is under maintenance?: false

RabbitMQ version: 3.8.9

Node name: rabbit@f1b3c114b8d6

Erlang configuration: Erlang/OTP 23 [erts-11.1] [source] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:64]

Erlang processes: 669 used, 1048576 limit

Scheduler run queue: 1

Cluster heartbeat timeout (net_ticktime): 60

Minimally Reproducible Test Case

task.py:

import os
from celery import Celery
from time import sleep

celery_app = Celery("worker", broker=os.environ['CELERY_BROKER_URL'])

@celery_app.task()
def run():
sleep(60)
return

run.py:

from task import run

if __name__ == '__main__':
run.apply_async()
run.apply_async()

IMPORTANT - Start broker - bug happens only when first tasks run after broker starts.

Run celery:

$ celery -A task worker --pool=prefork --concurrency=2

Run tasks:

$ python run.py

inspect active tasks:

$ celery -A task inspect active

inspect reserved tasks:

$ celery -A task inspect reserved

Expected Behavior

Worker is running 2 active tasks

Actual Behavior

The worker is running only 1 active task and you can see 1 reserved task.

Same test performed on 4.4.7 (cliffs) worked as expected

Even in 5.0.0 when you run again

Run tasks:

$ python run.py

inspect active tasks:

$ celery -A task inspect active

inspect reserved tasks:

$ celery -A task inspect reserved

It will work as expected - you will see 2 active tasks and 0 reserved