Issue 1:
Checklist

I have verified that the issue exists against the master branch of Celery.

This has already been asked to the discussion group first.

I have read the relevant section in the

contribution guide

on reporting bugs.

I have checked the issues list

for similar or identical bug reports.

I have checked the pull requests list

for existing proposed fixes.

I have checked the commit log

to find out if the bug was already fixed in the master branch.

I have included all related issues and possible duplicate issues

in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

I have verified that the issue exists against the master branch of Celery.

I have included the contents of pip freeze in the issue.

I have included all the versions of all the external dependencies required

to reproduce this bug.

Optional Debugging Information

I have tried reproducing the issue on more than one Python version

and/or implementation.

I have tried reproducing the issue on more than one message broker and/or

result backend.

I have tried reproducing the issue on more than one version of the message

broker and/or result backend.

I have tried reproducing the issue on more than one operating system.

I have tried reproducing the issue on more than one workers pool.

I have tried reproducing the issue with autoscaling, retries,

ETA/Countdown & rate limits disabled.

I have tried reproducing the issue after downgrading

and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

None

Possible Duplicates

None

Environment & Settings

Celery version:

celery report Output:

Steps to Reproduce

Required Dependencies

Minimal Python Version: 3.6

Minimal Celery Version: 4.3.0

Minimal Kombu Version: N/A or Unknown

Minimal Broker Version: Redis 5.x

Minimal Result Backend Version: N/A or Unknown

Minimal OS and/or Kernel Version: N/A or Unknown

Minimal Broker Client Version: N/A or Unknown

Minimal Result Backend Client Version: N/A or Unknown

Minimal example:

Python Packages

pip freeze Output:

alembic==0.9.10
amqp==2.5.2
ansible==2.8.4
arrow==0.15.5
asn1crypto==0.24.0
Authlib==0.14.1
awscli==1.16.289
backcall==0.1.0
backoff==1.10.0
bcrypt==3.1.7
beautifulsoup4==4.8.2
billiard==3.5.0.5
blist==1.3.6
boto3==1.12.14
botocore==1.15.14
cachetools==3.1.1
celery==4.2.2
certifi==2019.11.28
cffi==1.14.0
chardet==3.0.4
cloudpickle==1.2.1
colorama==0.3.9
crc16==0.1.1
cryptography==2.8
cyordereddict==1.0.0
dask==2.10.1
decorator==4.4.0
deepdiff==3.3.0
docopt==0.6.2
docutils==0.15.2
dpkt==1.9.2
fsspec==0.6.2
greenlet==0.4.15
idna==2.9
importlib-metadata==1.5.0
ipython==7.8.0
ipython-genutils==0.2.0
jedi==0.15.1
Jinja2==2.11.1
jmespath==0.9.5
jsonpickle==1.3
kombu==4.6.0
kvdr==1.0.4
locket==0.2.0
lxml==4.5.0
lz4==3.0.2
Mako==1.1.2
MarkupSafe==1.1.1
msgpack==1.0.0
msgpack-python==0.5.6
numexpr==2.7.1
numpy==1.18.1
pandas==0.22.0
pandas-datareader==0.8.1
paramiko==2.7.1
parso==0.5.1
partd==1.1.0
pexpect==4.7.0
pickleshare==0.7.5
prompt-toolkit==2.0.9
psutil==5.7.0
psycopg2-binary==2.8.4
ptyprocess==0.6.0
pudb==2019.1
pyasn1==0.4.7
pycparser==2.20
pycrypto==2.6.1
pydocstyle==1.1.1
Pygments==2.4.2
Pympler==0.8
PyNaCl==1.3.0
pyOpenSSL==19.1.0
pysftp==0.2.9
python-dateutil==2.8.1
python-editor==1.0.4
python-gnupg==0.4.5
python-redis==0.2.2
python-snappy==0.5.4
pytz==2019.3
PyYAML==5.3
rarfile==3.1
redis==3.4.1
requests==2.23.0
requests-oauth2==0.3.0
rsa==3.4.2
s3fs==0.4.0
s3transfer==0.3.3
six==1.14.0
sortedcontainers==2.1.0
soupsieve==2.0
SQLAlchemy==1.3.13
tables==3.6.1
toolz==0.10.0
traitlets==4.3.2
urllib3==1.25.8
urwid==2.0.1
vine==1.3.0
wcwidth==0.1.7
wrapt==1.11.2
xlrd==1.2.0
zipp==3.1.0

Other Dependencies

N/A

Minimally Reproducible Test Case

from myapp.app import app
from time import sleep

def print_stats():
insp = app.control.inspect()
active_lst = insp.active()
cluster_stats = insp.stats()
active_queues = insp.active_queues()
all_stats = {
"active": active_lst,
"stats": cluster_stats,
"queues": active_queues
}
print(all_stats)

def main():
while True:
print_stats()
sleep(10)

if __name__ == '__main__':
main()

Expected Behavior

No memory leaks

Actual Behavior

The memory consumption by the tiny example constantly grows.

I left the script running over night and it always gets killed after trying to allocate more memory than the system has, so Linux automatically kills it.

Issue 2:
Checklist

I have verified that the issue exists against the master branch of Celery.

This has already been asked to the discussion group first.

I have read the relevant section in the

contribution guide

on reporting bugs.

I have checked the issues list

for similar or identical bug reports.

I have checked the pull requests list

for existing proposed fixes.

I have checked the commit log

to find out if the bug was already fixed in the master branch.

I have included all related issues and possible duplicate issues

in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

I have verified that the issue exists against the master branch of Celery.

I have included the contents of pip freeze in the issue.

I have included all the versions of all the external dependencies required

to reproduce this bug.

Optional Debugging Information

I have tried reproducing the issue on more than one Python version

and/or implementation.

I have tried reproducing the issue on more than one message broker and/or

result backend.

I have tried reproducing the issue on more than one version of the message

broker and/or result backend.

I have tried reproducing the issue on more than one operating system.

I have tried reproducing the issue on more than one workers pool.

I have tried reproducing the issue with autoscaling, retries,

ETA/Countdown & rate limits disabled.

I have tried reproducing the issue after downgrading

and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

None

Possible Duplicates

None

Environment & Settings

Celery version:

celery report Output:

Steps to Reproduce

Required Dependencies

Minimal Python Version: 3.8.5

Minimal Celery Version: 5.1.2

Minimal Kombu Version: N/A or Unknown

Minimal Broker Version: N/A or Unknown

Minimal Result Backend Version: N/A or Unknown

Minimal OS and/or Kernel Version: Pop!_OS 21.04

Minimal Broker Client Version: pyamqp

Minimal Result Backend Client Version: redis

Python Packages

pip freeze Output:

Other Dependencies

N/A

Minimally Reproducible Test Case

Expected Behavior

Actual Behavior

When using .signature(..., queue=Queue(...),...) in a chord or chain, it tries to serialize the Queue() object, which fails. group execution does work.

Exception

Traceback (most recent call last):
File "/home/darren/PycharmProjects/pyfi/venv/lib/python3.8/site-packages/kombu-5.1.0-py3.8.egg/kombu/serialization.py", line 42, in _reraise_errors
yield
File "/home/darren/PycharmProjects/pyfi/venv/lib/python3.8/site-packages/kombu-5.1.0-py3.8.egg/kombu/serialization.py", line 213, in dumps
payload = encoder(data)
File "/home/darren/PycharmProjects/pyfi/venv/lib/python3.8/site-packages/kombu-5.1.0-py3.8.egg/kombu/utils/json.py", line 68, in dumps
return _dumps(s, cls=cls or _default_encoder,
File "/home/darren/condas/miniconda3/lib/python3.8/json/__init__.py", line 234, in dumps
return cls(
File "/home/darren/condas/miniconda3/lib/python3.8/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/home/darren/condas/miniconda3/lib/python3.8/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/home/darren/PycharmProjects/pyfi/venv/lib/python3.8/site-packages/kombu-5.1.0-py3.8.egg/kombu/utils/json.py", line 58, in default
return super().default(o)
File "/home/darren/condas/miniconda3/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type Queue is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "pyfi/client/example/example2.py", line 8, in <module>
result = chain([
File "/home/darren/PycharmProjects/pyfi/celery/canvas.py", line 612, in __call__
return self.apply_async(args, kwargs)
File "/home/darren/PycharmProjects/pyfi/celery/canvas.py", line 640, in apply_async
return self.run(args, kwargs, app=app, **(
File "/home/darren/PycharmProjects/pyfi/celery/canvas.py", line 669, in run
result_from_apply = first_task.apply_async(**options)
File "/home/darren/PycharmProjects/pyfi/celery/canvas.py", line 219, in apply_async
return _apply(args, kwargs, **options)
File "/home/darren/PycharmProjects/pyfi/celery/app/base.py", line 767, in send_task
amqp.send_task_message(P, name, message, **options)
File "/home/darren/PycharmProjects/pyfi/celery/app/amqp.py", line 510, in send_task_message
ret = producer.publish(
File "/home/darren/PycharmProjects/pyfi/venv/lib/python3.8/site-packages/kombu-5.1.0-py3.8.egg/kombu/messaging.py", line 166, in publish
body, content_type, content_encoding = self._prepare(
File "/home/darren/PycharmProjects/pyfi/venv/lib/python3.8/site-packages/kombu-5.1.0-py3.8.egg/kombu/messaging.py", line 254, in _prepare
body) = dumps(body, serializer=serializer)
File "/home/darren/PycharmProjects/pyfi/venv/lib/python3.8/site-packages/kombu-5.1.0-py3.8.egg/kombu/serialization.py", line 213, in dumps
payload = encoder(data)
File "/home/darren/condas/miniconda3/lib/python3.8/contextlib.py", line 131, in __exit__
self.gen.throw(type, value, traceback)
File "/home/darren/PycharmProjects/pyfi/venv/lib/python3.8/site-packages/kombu-5.1.0-py3.8.egg/kombu/serialization.py", line 46, in _reraise_errors
reraise(wrapper, wrapper(exc), sys.exc_info()[2])
File "/home/darren/PycharmProjects/pyfi/venv/lib/python3.8/site-packages/kombu-5.1.0-py3.8.egg/kombu/exceptions.py", line 21, in reraise
raise value.with_traceback(tb)
File "/home/darren/PycharmProjects/pyfi/venv/lib/python3.8/site-packages/kombu-5.1.0-py3.8.egg/kombu/serialization.py", line 42, in _reraise_errors
yield
File "/home/darren/PycharmProjects/pyfi/venv/lib/python3.8/site-packages/kombu-5.1.0-py3.8.egg/kombu/serialization.py", line 213, in dumps
payload = encoder(data)
File "/home/darren/PycharmProjects/pyfi/venv/lib/python3.8/site-packages/kombu-5.1.0-py3.8.egg/kombu/utils/json.py", line 68, in dumps
return _dumps(s, cls=cls or _default_encoder,
File "/home/darren/condas/miniconda3/lib/python3.8/json/__init__.py", line 234, in dumps
return cls(
File "/home/darren/condas/miniconda3/lib/python3.8/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/home/darren/condas/miniconda3/lib/python3.8/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/home/darren/PycharmProjects/pyfi/venv/lib/python3.8/site-packages/kombu-5.1.0-py3.8.egg/kombu/utils/json.py", line 58, in default
return super().default(o)
File "/home/darren/condas/miniconda3/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
kombu.exceptions.EncodeError: Object of type Queue is not JSON serializable