Issue 1:
Checklist

I have verified that the issue exists against the master branch of Celery.

This has already been asked to the discussions forum first.

I have read the relevant section in the

contribution guide

on reporting bugs.

I have checked the issues list

for similar or identical bug reports.

I have checked the pull requests list

for existing proposed fixes.

I have checked the commit log

to find out if the bug was already fixed in the master branch.

I have included all related issues and possible duplicate issues

in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

I have verified that the issue exists against the master branch of Celery.

I have included the contents of pip freeze in the issue.

I have included all the versions of all the external dependencies required

to reproduce this bug.

Optional Debugging Information

I have tried reproducing the issue on more than one Python version

and/or implementation.

I have tried reproducing the issue on more than one message broker and/or

result backend.

I have tried reproducing the issue on more than one version of the message

broker and/or result backend.

I have tried reproducing the issue on more than one operating system.

I have tried reproducing the issue on more than one workers pool.

I have tried reproducing the issue with autoscaling, retries,

ETA/Countdown & rate limits disabled.

I have tried reproducing the issue after downgrading

and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

Little bit related -

#6672

#5890

Possible Duplicates

None

Environment & Settings

Celery version: 5.2.7 (dawn-chorus)

celery report Output:

$ celery report

software -> celery:5.2.7 (dawn-chorus) kombu:5.2.4 py:3.9.9
billiard:3.6.4.0 py-amqp:5.1.1
platform -> system:Darwin arch:64bit
kernel version:21.4.0 imp:CPython
loader -> celery.loaders.default.Loader
settings -> transport:amqp results:disabled

deprecated_settings: None

Steps to Reproduce

Here the global serializer is json which is the default setting in celery. Now we will declare a task with serialization set to pickle.

# In tasks.py
from celery import shared_task

@serial_task(serializer='pickle')
def pickle_seriliazable_args_task(nice_set):
print('Starting Task')
# This delay would indicate a long running task
# So that we have enough time to request the task stats from shell
sleep(200)
print('Task Finished')

Open a django-manage shell and run

In [1]: from myapp.apps.niceapp.tasks import pickle_serialized_task

In [2]: task = pickle_serialized_task.delay(set([1, 2, 3]))

In [3]: celery_hostname = 'celery@work' # update hostname as per your config

In [4]: celery_inspect = task.app.control.inspect([celery_hostname])

In [5]: celery_inspect.active()

On running celery_inspect.active() following error is thrown.

2022-07-01 15:07:50,791: INFO/MainProcess] Task myapp.apps.niceapp.tasks.pickle_serialized_task[e61f2103-1e19-4593-b2bd-6d69bd273a3c] received
[2022-07-01 15:07:50,815: WARNING/ForkPoolWorker-6] Starting Task
[2022-07-01 15:09:09,906: ERROR/MainProcess] Control command error: EncodeError(TypeError('Object of type set is not JSON serializable'))
Traceback (most recent call last):
File "/Users/amitphulera/.pyenv/versions/3.9.9/envs/hq_celery/lib/python3.9/site-packages/kombu/serialization.py", line 39, in _reraise_errors
yield
File "/Users/amitphulera/.pyenv/versions/3.9.9/envs/hq_celery/lib/python3.9/site-packages/kombu/serialization.py", line 210, in dumps
payload = encoder(data)
File "/Users/amitphulera/.pyenv/versions/3.9.9/envs/hq_celery/lib/python3.9/site-packages/kombu/utils/json.py", line 68, in dumps
return _dumps(s, cls=cls or _default_encoder,
File "/Users/amitphulera/.pyenv/versions/3.9.9/lib/python3.9/json/__init__.py", line 234, in dumps
return cls(
File "/Users/amitphulera/.pyenv/versions/3.9.9/lib/python3.9/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/Users/amitphulera/.pyenv/versions/3.9.9/lib/python3.9/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/Users/amitphulera/.pyenv/versions/3.9.9/envs/hq_celery/lib/python3.9/site-packages/kombu/utils/json.py", line 58, in default
return super().default(o)
File "/Users/amitphulera/.pyenv/versions/3.9.9/lib/python3.9/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type set is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/amitphulera/.pyenv/versions/3.9.9/envs/hq_celery/lib/python3.9/site-packages/celery/worker/pidbox.py", line 44, in on_message
self.node.handle_message(body, message)
File "/Users/amitphulera/.pyenv/versions/3.9.9/envs/hq_celery/lib/python3.9/site-packages/kombu/pidbox.py", line 141, in handle_message
return self.dispatch(**body)
File "/Users/amitphulera/.pyenv/versions/3.9.9/envs/hq_celery/lib/python3.9/site-packages/kombu/pidbox.py", line 108, in dispatch
self.reply({self.hostname: reply},
File "/Users/amitphulera/.pyenv/versions/3.9.9/envs/hq_celery/lib/python3.9/site-packages/kombu/pidbox.py", line 145, in reply
self.mailbox._publish_reply(data, exchange, routing_key, ticket,
File "/Users/amitphulera/.pyenv/versions/3.9.9/envs/hq_celery/lib/python3.9/site-packages/kombu/pidbox.py", line 275, in _publish_reply
producer.publish(
File "/Users/amitphulera/.pyenv/versions/3.9.9/envs/hq_celery/lib/python3.9/site-packages/kombu/messaging.py", line 166, in publish
body, content_type, content_encoding = self._prepare(
File "/Users/amitphulera/.pyenv/versions/3.9.9/envs/hq_celery/lib/python3.9/site-packages/kombu/messaging.py", line 254, in _prepare
body) = dumps(body, serializer=serializer)
File "/Users/amitphulera/.pyenv/versions/3.9.9/envs/hq_celery/lib/python3.9/site-packages/kombu/serialization.py", line 210, in dumps
payload = encoder(data)
File "/Users/amitphulera/.pyenv/versions/3.9.9/lib/python3.9/contextlib.py", line 137, in __exit__
self.gen.throw(typ, value, traceback)
File "/Users/amitphulera/.pyenv/versions/3.9.9/envs/hq_celery/lib/python3.9/site-packages/kombu/serialization.py", line 43, in _reraise_errors
reraise(wrapper, wrapper(exc), sys.exc_info()[2])
File "/Users/amitphulera/.pyenv/versions/3.9.9/envs/hq_celery/lib/python3.9/site-packages/kombu/exceptions.py", line 21, in reraise
raise value.with_traceback(tb)
File "/Users/amitphulera/.pyenv/versions/3.9.9/envs/hq_celery/lib/python3.9/site-packages/kombu/serialization.py", line 39, in _reraise_errors
yield
File "/Users/amitphulera/.pyenv/versions/3.9.9/envs/hq_celery/lib/python3.9/site-packages/kombu/serialization.py", line 210, in dumps
payload = encoder(data)
File "/Users/amitphulera/.pyenv/versions/3.9.9/envs/hq_celery/lib/python3.9/site-packages/kombu/utils/json.py", line 68, in dumps
return _dumps(s, cls=cls or _default_encoder,
File "/Users/amitphulera/.pyenv/versions/3.9.9/lib/python3.9/json/__init__.py", line 234, in dumps
return cls(
File "/Users/amitphulera/.pyenv/versions/3.9.9/lib/python3.9/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/Users/amitphulera/.pyenv/versions/3.9.9/lib/python3.9/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/Users/amitphulera/.pyenv/versions/3.9.9/envs/hq_celery/lib/python3.9/site-packages/kombu/utils/json.py", line 58, in default
return super().default(o)
File "/Users/amitphulera/.pyenv/versions/3.9.9/lib/python3.9/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
kombu.exceptions.EncodeError: Object of type set is not JSON serializable

Required Dependencies

Minimal Python Version: N/A or Unknown

Minimal Celery Version: N/A or Unknown

Minimal Kombu Version: N/A or Unknown

Minimal Broker Version: N/A or Unknown

Minimal Result Backend Version: N/A or Unknown

Minimal OS and/or Kernel Version: N/A or Unknown

Minimal Broker Client Version: N/A or Unknown

Minimal Result Backend Client Version: N/A or Unknown

Python Packages

pip freeze Output:

alabaster==0.7.12
alembic==1.7.7
amqp==5.1.1
appnope==0.1.3
architect==0.6.0
asgiref==3.5.0
asttokens==2.0.5
attrs==21.4.0
Babel==2.9.1
backcall==0.2.0
Beaker==1.11.0
beautifulsoup4==4.10.0
billiard==3.6.4.0
black==22.1.0
boto3==1.17.85
botocore==1.20.85
cachetools==5.0.0
case==1.5.3
celery==5.2.7
certifi==2020.6.20
cffi==1.14.3
chardet==3.0.4
click==8.1.3
click-didyoumean==0.3.0
click-plugins==1.1.1
click-repl==0.2.0
cloudant==2.14.0
colorama==0.4.3
contextlib2==21.6.0
coverage==5.5
cryptography==3.4.8
csiphash==0.0.5
datadog==0.39.0
ddtrace==0.44.0
debugpy==1.6.0
decorator==4.0.11
defusedxml==0.7.1
Deprecated==1.2.10
diff-match-patch==20200713
dimagi-memoized==1.1.3
Django==3.2.13
django-appconf==1.0.5
django-autoslug==1.9.8
django-braces==1.14.0
django-bulk-update==2.2.0
django-celery-results==2.4.0
django-compressor==2.4
django-countries==7.3.2
django-crispy-forms==1.10.0
django-cte==1.2.0
django-extensions==3.1.3
django-formtools==2.3
django-logentry-admin==1.0.6
django-oauth-toolkit==1.5.0
django-otp==0.9.4
django-phonenumber-field==5.2.0
django-prbac==1.0.1
django-recaptcha==2.0.6
django-redis==4.12.1
django-redis-sessions==0.6.2
django-statici18n==1.9.0
django-tastypie==0.14.4
django-transfer==0.4
django-two-factor-auth==1.13.2
django-user-agents==0.4.0
djangorestframework==3.12.2
dnspython==1.15.0
docutils==0.16
dropbox==9.3.0
elasticsearch2==2.5.1
elasticsearch5==5.5.6
email-validator==1.1.3
et-xmlfile==1.0.1
ethiopian-date-converter==0.1.5
eulxml==1.1.3
executing==0.8.3
fakecouch==0.0.15
Faker==5.0.2
fixture==1.5.11
flake8==3.9.2
flaky==3.7.0
flower==1.0.0
freezegun==1.1.0
future==0.18.2
gevent==21.8.0
ghdiff==0.4
git-build-branch==0.1.13
gnureadline==8.0.0
google-api-core==2.5.0
google-api-python-client==2.32.0
google-auth==2.6.0
google-auth-httplib2==0.1.0
google-auth-oauthlib==0.4.6
googleapis-common-protos==1.54.0
greenlet==1.1.2
gunicorn==20.0.4
hiredis==2.0.0
httpagentparser==1.9.0
httplib2==0.20.4
humanize==4.1.0
idna==2.10
imagesize==1.2.0
importlib-metadata==4.11.3
iniconfig==1.1.1
intervaltree==3.1.0
ipython==8.2.0
iso8601==0.1.13
isodate==0.6.1
jedi==0.18.1
Jinja2==2.11.3
jmespath==0.10.0
json-delta==2.0
jsonfield==2.1.1
jsonobject==2.0.0
jsonobject-couchdbkit==1.0.1
jsonschema==3.2.0
jwcrypto==0.8
kafka-python==1.4.7
kombu==5.2.4
laboratory==0.2.0
linecache2==1.0.0
lxml==4.7.1
Mako==1.1.3
mando==0.6.4
Markdown==3.3.6
MarkupSafe==1.1.1
matplotlib-inline==0.1.3
mccabe==0.6.1
mock==4.0.3
mypy-extensions==0.4.3
ndg-httpsclient==0.5.1
nose==1.3.7
nose-exclude==0.5.0
oauthlib==3.1.0
oic==1.3.0
openpyxl==3.0.9
packaging==20.4
parso==0.8.3
pathspec==0.9.0
pep517==0.10.0
pexpect==4.8.0
phonenumberslite==8.12.48
pickle5==0.0.11
pickleshare==0.7.5
Pillow==9.0.1
pip-tools==6.6.0
platformdirs==2.4.1
pluggy==1.0.0
ply==3.11
polib==1.1.1
prometheus-client==0.14.1
prompt-toolkit==3.0.26
protobuf==3.15.0
psutil==5.8.0
psycogreen==1.0.2
psycopg2==2.8.6
psycopg2cffi==2.9.0
ptyprocess==0.7.0
pure-eval==0.2.2
py==1.11.0
py-cpuinfo==8.0.0
py-KISSmetrics==1.1.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycodestyle==2.7.0
pycparser==2.20
pycryptodome==3.10.1
pycryptodomex==3.14.1
pyflakes==2.3.1
PyGithub==1.54.1
Pygments==2.11.2
pygooglechart==0.4.0
pyjwkest==1.4.2
PyJWT==1.7.1
pyOpenSSL==20.0.1
pyparsing==3.0.7
pyphonetics==0.5.3
pyrsistent==0.17.3
PySocks==1.7.1
pytest==7.1.2
pytest-benchmark==3.4.1
pytest-django==4.5.2
python-dateutil==2.8.2
python-editor==1.0.4
python-imap==1.0.0
python-magic==0.4.22
python-mimeparse==1.6.0
python-termstyle==0.1.10
python3-saml==1.12.0
pytz==2022.1
PyYAML==5.4.1
pyzxcvbn==0.8.0
qrcode==4.0.4
quickcache==0.5.4
radon==5.1.0
rcssmin==1.0.6
redis==3.5.3
reportlab==3.6.9
requests==2.25.1
requests-mock==1.9.3
requests-oauthlib==1.3.1
requests-toolbelt==0.9.1
rjsmin==1.1.0
rsa==4.8
s3transfer==0.4.2
schema==0.7.5
sentry-sdk==0.19.5
setproctitle==1.2.2
sh==1.14.2
simpleeval==0.9.10
simplejson==3.17.2
six==1.16.0
sniffer==0.4.1
snowballstemmer==2.0.0
socketpool==0.5.3
sortedcontainers==2.3.0
soupsieve==2.0.1
Sphinx==4.1.2
sphinx-rtd-theme==0.5.2
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-django==0.5.1
sphinxcontrib-htmlhelp==2.0.0
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
sqlagg==0.17.2
SQLAlchemy==1.3.19
sqlalchemy-postgres-copy==0.5.0
sqlparse==0.3.1
stack-data==0.1.4
stripe==2.54.0
suds-py3==1.4.5.0
tenacity==6.2.0
testil==1.1
text-unidecode==1.3
tinys3==0.1.12
toml==0.10.2
tomli==2.0.1
toposort==1.7
tornado==6.1
traceback2==1.4.0
traitlets==5.1.1
tropo-webapi-python==0.1.3
turn-python==0.0.1
twilio==6.5.1
typing_extensions==4.1.1
ua-parser==0.10.0
Unidecode==1.2.0
unittest2==1.1.0
uritemplate==4.1.1
urllib3==1.26.5
user-agents==2.2.0
uWSGI==2.0.19.1
vine==5.0.0
wcwidth==0.2.5
Werkzeug==1.0.1
wrapt==1.12.1
xlrd==2.0.1
xlwt==1.3.0
xmlsec==1.3.12
yapf==0.31.0
zipp==3.7.0
zope.event==4.5.0
zope.interface==5.4.0

Other Dependencies

N/A

Minimally Reproducible Test Case

# run celery with `pickle_serialized_task` as defined in reproduction steps and run the snippet below

from myapp.apps.niceapp.tasks import pickle_serialized_task
task = pickle_serialized_task.delay(set([1, 2, 3]))
celery_hostname = 'celery@work'
celery_inspect = task.app.control.inspect([celery_hostname])
celery_inspect.active()

Expected Behavior

This should list the details about the task.

Actual Behavior

It is erroring out with the stack trace shared above.

Issue 2:
Related Issues and Possible Duplicates

Related Issues

#6672, #5890

Related PRs

#6757

Steps to Reproduce

Here is the script that reproduces the issue:

import time

from celery import Celery

app = Celery('example')
app.conf.update(
backend_url='redis://localhost:6379',
broker_url='redis://localhost:6379',
result_backend='redis://localhost:6379',
task_serializer='json',
accept_content=['pickle', 'json'],
)

@app.task(name='task1', serializer='pickle')
def task1(*args, **kwargs):
print('Start', args, kwargs)
time.sleep(30)
print('Finish', args, kwargs)

def main():
task1.delay({1, 2, 3}) # set is not JSON serializable.
# inspect queue items
inspected = app.control.inspect()
active_tasks = inspected.active()
print(active_tasks)

if __name__ == '__main__':
main()

See also following discussion: #6757 (comment)

Expected Behavior

Control/Inspect serialization should support custom per-task serializer

Actual Behavior

Control/Inspect serialization is driven only by task_serializer configuration.