Issue 1:
Checklist

I have verified that the issue exists against the master branch of Celery.

This has already been asked to the discussion group first.

I have read the relevant section in the

contribution guide

on reporting bugs.

I have checked the issues list

for similar or identical bug reports.

I have checked the pull requests list

for existing proposed fixes.

I have checked the commit log

to find out if the bug was already fixed in the master branch.

I have included all related issues and possible duplicate issues

in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

I have verified that the issue exists against the master branch of Celery.

I have included the contents of pip freeze in the issue.

I have included all the versions of all the external dependencies required

to reproduce this bug.

Optional Debugging Information

[] I have tried reproducing the issue on more than one Python version

and/or implementation.

[] I have tried reproducing the issue on more than one message broker and/or

result backend.

I have tried reproducing the issue on more than one version of the message

broker and/or result backend.

[] I have tried reproducing the issue on more than one operating system.

[] I have tried reproducing the issue on more than one workers pool.

[] I have tried reproducing the issue with autoscaling, retries,

ETA/Countdown & rate limits disabled.

I have tried reproducing the issue after downgrading

and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

#3667

#5890

#6567

Possible Duplicates

-None

Environment & Settings

Celery version: f091bab

celery report Output:

software -> celery:5.0.5 (singularity) kombu:5.0.2 py:3.7.9
billiard:3.6.3.0 redis:3.5.0
platform -> system:Linux arch:64bit, ELF
kernel version:3.10.0-1160.11.1.el7.x86_64 imp:CPython
loader -> celery.loaders.app.AppLoader
settings -> transport:redis results:redis://localhost:6379/0

accept_content: ['pickle']
result_serializer: 'pickle'
task_serializer: 'pickle'
worker_concurrency: 1
broker_url: 'redis://localhost:6379/0'
result_backend: 'redis://localhost:6379/0'
deprecated_settings: None

Calling app.control.inspect().active() can cause kombu.exceptions.EncodeError: Object of type <CLASSNAME> is not JSON serializable even when app is configured to not use JSON serialization (e.g. when pickle is configured).

The issue was likely introduced by this commit:

9f0a554

It can be observed that the Control class does not pass through any of the app object's serialization configuration when constructing a Mailbox in celery/app/control.py.

Steps to Reproduce

Required Dependencies

Minimal Python Version: Unknown

Minimal Celery Version: Unknown

Minimal Kombu Version: Unknown

Minimal Broker Version: Unknown

Minimal Result Backend Version: Unknown

Minimal OS and/or Kernel Version: Unknown

Minimal Broker Client Version: Unknown

Minimal Result Backend Client Version: Unknown

Python Packages

pip freeze Output:

Other Dependencies

N/A

Minimally Reproducible Test Case

Before running the code below, ensure a redis server is running for Celery to connect to.

from pathlib import Path
from tempfile import NamedTemporaryFile
import threading
import time

from celery import Celery
from celery.utils.log import get_task_logger

logger = get_task_logger(__name__)

class CeleryConfig:
task_serializer = 'pickle'
result_serializer = 'pickle'
accept_content = ['pickle']
worker_concurrency = 1

app = Celery('query_active', broker='redis://localhost:6379/0', backend='redis://localhost:6379/0', strict_typing=False)
app.config_from_object(CeleryConfig)

class SomeClass:
def __init__(self, some_value):
self.sleep_time = some_value

@app.task
def sleep_task(some_object: SomeClass, filepath):
Path(filepath).touch()
logger.info(f'Sleeping for {some_object.sleep_time}')
time.sleep(some_object.sleep_time)
logger.info(f'Done sleeping {some_object.sleep_time}')

if __name__ == '__main__':
# Start a Celery Worker in a thread
argv = ['--app=query_active', 'worker', '--loglevel=DEBUG']
threading.Thread(target=app.worker_main, args=(argv,)).start()
time.sleep(5)
print('Celery should be up now')

with NamedTemporaryFile() as f:
# supply SomeClass, an argument that will not be JSON serializable.
r = sleep_task.s(SomeClass(120), f.name).apply_async()
while not Path(f.name).exists(): # wait till task is running.
time.sleep(1)

# Querying active causes a serialization error.
a = app.control.inspect().active()
print(f'Active: {a}')

Expected Behavior

The example runs a trivial task, then attempts to query for that task via app.control.inspect().active().

Actual Behavior

Instead of receiving the task data, app.control.inspect().active() returns None and the following trace can be seen on the console logs:

Control command error: EncodeError(TypeError('Object of type SomeClass is not JSON serializable'))
Traceback (most recent call last):
File "/ws/djungic-sjc/firex_ws5/lib/python3.7/site-packages/kombu/serialization.py", line 49, in _reraise_errors
yield
File "/ws/djungic-sjc/firex_ws5/lib/python3.7/site-packages/kombu/serialization.py", line 220, in dumps
payload = encoder(data)
File "/ws/djungic-sjc/firex_ws5/lib/python3.7/site-packages/kombu/utils/json.py", line 66, in dumps
**dict(default_kwargs, **kwargs))
File "/auto/firex/sw/python/3.7.9/lib/python3.7/json/__init__.py", line 238, in dumps
**kw).encode(obj)
File "/auto/firex/sw/python/3.7.9/lib/python3.7/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/auto/firex/sw/python/3.7.9/lib/python3.7/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/ws/djungic-sjc/firex_ws5/lib/python3.7/site-packages/kombu/utils/json.py", line 55, in default
return super().default(o)
File "/auto/firex/sw/python/3.7.9/lib/python3.7/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type SomeClass is not JSON serializable

Issue 2:
Checklist

I have checked the issues list

for similar or identical bug reports.

I have checked the pull requests list

for existing proposed fixes.

I have checked the commit log

to find out if the bug was already fixed in the master branch.

I have included all related issues and possible duplicate issues in this issue

(If there are none, check this box anyway).

Related Issues and Possible Duplicates

Related Issues

None

Possible Duplicates

None

Description

I think we need to have some sort of information on what features the different backends/brokers can provide.

For example, it seems that with Redis, if there is a task call, and no workers, that specific task call will never get executed. I'm guessing RabbitMQ knows to wait for workers (subscriptions) to connect, once a worker connects it will start sending previously queued task information? I'd also imagine this would be the case for AWS SQS. I haven't gotten to test SQS or RabbitMQ brokers so I can't say with certainty that this is what would happen though.

Suggestions

Maybe a few listed differences or use cases.