Repository: celery/celery

Issue:
app.control.inspect().active() requires deserialization
<!--
Please fill this template entirely and do not erase parts of it.
We reserve the right to close without a response
bug reports which are incomplete.
-->
# Checklist
<!--
To check an item on the list replace [ ] with [x].
-->
- [x] I have verified that the issue exists against the `master` branch of Celery.
- [x] This has already been asked to the [discussion group](https://groups.google.com/forum/#!forum/celery-users) first.
- [x] I have read the relevant section in the
  [contribution guide](http://docs.celeryproject.org/en/latest/contributing.html#other-bugs)
  on reporting bugs.
- [x] I have checked the [issues list](https://github.com/celery/celery/issues?q=is%3Aissue+label%3A%22Issue+Type%3A+Bug+Report%22+-label%3A%22Category%3A+Documentation%22)
  for similar or identical bug reports.
- [x] I have checked the [pull requests list](https://github.com/celery/celery/pulls?q=is%3Apr+label%3A%22PR+Type%3A+Bugfix%22+-label%3A%22Category%3A+Documentation%22)
  for existing proposed fixes.
- [x] I have checked the [commit log](https://github.com/celery/celery/commits/master)
  to find out if the bug was already fixed in the master branch.
- [x] I have included all related issues and possible duplicate issues
  in this issue (If there are none, check this box anyway).

## Mandatory Debugging Information

- [x] I have included the output of ``celery -A proj report`` in the issue.
    (if you are not able to do this, then at least specify the Celery
     version affected).
- [x] I have verified that the issue exists against the `master` branch of Celery.
- [x] I have included the contents of ``pip freeze`` in the issue.
- [x] I have included all the versions of all the external dependencies required
  to reproduce this bug.

## Optional Debugging Information
<!--
Try some of the below if you think they are relevant.
It will help us figure out the scope of the bug and how many users it affects.
-->
- [ ] I have tried reproducing the issue on more than one Python version
  and/or implementation.
- [ ] I have tried reproducing the issue on more than one message broker and/or
  result backend.
- [ ] I have tried reproducing the issue on more than one version of the message
  broker and/or result backend.
- [ ] I have tried reproducing the issue on more than one operating system.
- [ ] I have tried reproducing the issue on more than one workers pool.
- [ ] I have tried reproducing the issue with autoscaling, retries,
  ETA/Countdown & rate limits disabled.
- [ ] I have tried reproducing the issue after downgrading
  and/or upgrading Celery and its dependencies.

## Related Issues and Possible Duplicates
<!--
Please make sure to search and mention any related issues
or possible duplicates to this issue as requested by the checklist above.

This may or may not include issues in other repositories that the Celery project
maintains or other repositories that are dependencies of Celery.

If you don't know how to mention issues, please refer to Github's documentation
on the subject: https://help.github.com/en/articles/autolinked-references-and-urls#issues-and-pull-requests
-->

#### Related Issues

- #3667
- #6672

#### Possible Duplicates

- None

## Environment & Settings
<!-- Include the contents of celery --version below -->
**Celery version**:
<!-- Include the output of celery -A proj report below -->
<details>
<summary><b><code>celery report</code> Output:</b></summary>
<p>

```
software -> celery:5.1.2 (sun-harmonics) kombu:5.1.0 py:3.8.10
            billiard:3.6.4.0 redis:3.5.3
platform -> system:Linux arch:64bit, ELF
            kernel version:3.10.0-1160.11.1.el7.x86_64 imp:CPython
loader   -> celery.loaders.app.AppLoader
settings -> transport:redis results:redis://localhost:6379/0

broker_url: 'redis://localhost:6379/0'
result_backend: 'redis://localhost:6379/0'
task_serializer: 'pickle'
accept_content: ['pickle']
deprecated_settings: None
worker_concurrency: 1
result_serializer: 'pickle'
event_serializer: 'pickle'

```

</p>
</details>

# Steps to Reproduce

The breakage was introduced by 40e2212208eafab819216 first released in Celery v4.4.0 (42a5befd07a94e35 works). 

Prior to 40e221220, the result of active() had `kwargs` and `args` as strings and therefore never required deserialization:

```
{'celery@worker1': [{'id': 'f12b987c-a711-4a8f-8d3c-532e922ebeea', 'name': 'celery_revoke_retry.sleep', 'args': '()', 'kwargs': "{'sleep_arg': <__main__.SleepArg object at 0x7fec3c059e80>}", 'type': 'celery_revoke_retry.sleep', 'hostname': 'celery@worker1', 'time_start': 1625674850.914238, 'acknowledged': True, 'delivery_info': {'exchange': '', 'routing_key': 'celery', 'priority': 0, 'redelivered': None}, 'worker_pid': 29697}]}
```

40e2212208eafab819216 requires clients to be able to deserialize task arguments. This re-introduces bug #3667 and makes the
comment of app.control.active no longer accurate:

```
	no objects will need serialization now that we have argsrepr/kwargsrepr.
```

The fix for #3667 specifically states it's required for the payload to be JSON serializable (i.e. not require deserializing pickled task arguments): 

https://github.com/smartlane/celery/commit/cbe89437fda38ee1a98376d694d9d3028e683b50.

A reasonable fix would be to add back support for the `safe` argument to `active()` to make the v4.3.1 behaviour possible again. 

Currently (~v5.1.0), querying active tasks when some argument can't be deserialized causes `active()` to throw kombu.exceptions.DecodeError.	


## Required Dependencies
<!-- Please fill the required dependencies to reproduce this issue -->

* **Minimal Celery Version**: 4.4.0



## Minimally Reproducible Test Case
<!--
Please provide a reproducible test case.
Refer to the Reporting Bugs section in our contribution guide.

We prefer submitting test cases in the form of a PR to our integration test suite.
If you can provide one, please mention the PR number below.
If not, please attach the most minimal code example required to reproduce the issue below.
If the test case is too large, please include a link to a gist or a repository below.
-->

<details>
<p>

Query app.control.inspect().active() from a context that cannot deserialize an active task's argument (SleepArg in the example below).

This program creates the task and sleeps:

```python
import threading
import time
from celery import Celery

app = Celery('celery_example', broker='redis://localhost:6379/0', backend='redis://localhost:6379/0')
app.conf.update(
    worker_concurrency=1,
    task_serializer = 'pickle',
    accept_content = ['pickle'],
    result_serializer = 'pickle',
)

class SleepArg:
    def __init__(self, sleep_time) -> None:
        self.sleep_time = sleep_time

@app.task
def sleep(sleep_arg: SleepArg):
    print("running sleep")
    time.sleep(sleep_arg.sleep_time)

def main():
    argv1 = ['--app=celery_example', 'worker', '--loglevel=DEBUG', '--hostname', 'worker1']
    threading.Thread(target=app.worker_main, args=(argv1,)).start()
    time.sleep(5)
    print('Celery Worker1 should be up now')

    sleep.s(sleep_arg=SleepArg(60)).apply_async().get(disable_sync_subtasks=False)

if __name__ == '__main__':
    main()
```

This program attempts to query active tasks, which fails because it can't deserialize `SleepArg`:

```python
from celery import Celery

if __name__ == '__main__':
    app = app = Celery(broker='redis://localhost:6379/0', backend='redis://localhost:6379/0')
    app.conf.update(
        worker_concurrency=1,
        task_serializer = 'pickle',
        accept_content = ['pickle', 'json'],
    )
    print(app.control.inspect(timeout=4).active())
```

</p>
</details>

# Expected Behavior
It should be possible to query active tasks from contexts that can't necessarily de-serialize task arguments. There is lots of useful information in the active query beyond the task arguments.

Historically, Celery offered the `safe` argument to `active()`, but this was dropped in favour of preventing data from requiring
deserialization from being in the response. 40e2212208ea breaks this.

# Actual Behavior
The program that queries active tasks hits the following exception.

```
Traceback (most recent call last):
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/serialization.py", line 42, in _reraise_errors
    yield
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/serialization.py", line 255, in loads
    return decode(data)
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/serialization.py", line 327, in unpickle
    return pickle_loads(str_to_bytes(s))
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/serialization.py", line 51, in pickle_loads
    return load(BytesIO(s))
AttributeError: Can't get attribute 'SleepArg' on <module '__main__' from '../firex_ws/plugins/celery_list_active.py'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "../firex_ws/plugins/celery_list_active.py", line 10, in <module>
    print(app.control.inspect(timeout=4).active())
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/celery/app/control.py", line 149, in active
    return self._request('active')
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/celery/app/control.py", line 106, in _request
    return self._prepare(self.app.control.broadcast(
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/celery/app/control.py", line 741, in broadcast
    return self.mailbox(conn)._broadcast(
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/pidbox.py", line 347, in _broadcast
    return self._collect(reply_ticket, limit=limit,
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/pidbox.py", line 389, in _collect
    self.connection.drain_events(timeout=timeout)
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/connection.py", line 318, in drain_events
    return self.transport.drain_events(self.connection, **kwargs)
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/transport/virtual/base.py", line 960, in drain_events
    get(self._deliver, timeout=timeout)
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/transport/redis.py", line 420, in get
    ret = self.handle_event(fileno, event)
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/transport/redis.py", line 402, in handle_event
    return self.on_readable(fileno), self
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/transport/redis.py", line 398, in on_readable
    chan.handlers[type]()
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/transport/redis.py", line 789, in _brpop_read
    self.connection._deliver(loads(bytes_to_str(item)), dest)
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/transport/virtual/base.py", line 980, in _deliver
    callback(message)
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/transport/virtual/base.py", line 630, in _callback
    return callback(message)
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/messaging.py", line 620, in _receive_callback
    decoded = None if on_m else message.decode()
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/message.py", line 190, in decode
    self._decoded_cache = self._decode()
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/message.py", line 194, in _decode
    return loads(self.body, self.content_type,
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/serialization.py", line 255, in loads
    return decode(data)
  File "/auto/firex/sw/python/3.8.10/lib/python3.8/contextlib.py", line 131, in __exit__
    self.gen.throw(type, value, traceback)
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/serialization.py", line 46, in _reraise_errors
    reraise(wrapper, wrapper(exc), sys.exc_info()[2])
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/exceptions.py", line 21, in reraise
    raise value.with_traceback(tb)
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/serialization.py", line 42, in _reraise_errors
    yield
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/serialization.py", line 255, in loads
    return decode(data)
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/serialization.py", line 327, in unpickle
    return pickle_loads(str_to_bytes(s))
  File "/ws/djungic-sjc/celery_venv/venv/lib/python3.8/site-packages/kombu/serialization.py", line 51, in pickle_loads
    return load(BytesIO(s))
kombu.exceptions.DecodeError: Can't get attribute 'SleepArg' on <module '__main__' from '../firex_ws/plugins/celery_list_active.py'>

```


Discussion:
Hey @djungic :wave:,
Thank you for opening an issue. We will get back to you as soon as we can.
Also, check out our [Open Collective](https://opencollective.com/celery) and consider backing us - every little helps!

We also offer priority support for our sponsors.
If you require immediate assistance please consider sponsoring us.


Resolving patch:
diff --git a/celery/app/control.py b/celery/app/control.py
--- a/celery/app/control.py
+++ b/celery/app/control.py
@@ -135,6 +135,8 @@ def clock(self):
 
     def active(self, safe=None):
         """Return list of tasks currently executed by workers.
+        Arguments:
+            safe (Boolean): Set to True to disable deserialization.
 
         Returns:
             Dict: Dictionary ``{HOSTNAME: [TASK_INFO,...]}``.
@@ -142,11 +144,8 @@ def active(self, safe=None):
         See Also:
             For ``TASK_INFO`` details see :func:`query_task` return value.
 
-        Note:
-            ``safe`` is ignored since 4.0 as no objects will need
-            serialization now that we have argsrepr/kwargsrepr.
         """
-        return self._request('active')
+        return self._request('active', safe=safe)
 
     def scheduled(self, safe=None):
         """Return list of scheduled tasks with details.
diff --git a/celery/worker/control.py b/celery/worker/control.py
--- a/celery/worker/control.py
+++ b/celery/worker/control.py
@@ -362,9 +362,9 @@ def reserved(state, **kwargs):
 
 
 @inspect_command(alias='dump_active')
-def active(state, **kwargs):
+def active(state, safe=False, **kwargs):
     """List of tasks currently being executed."""
-    return [request.info()
+    return [request.info(safe=safe)
             for request in state.tset(worker_state.active_requests)]
 
 
diff --git a/celery/worker/request.py b/celery/worker/request.py
--- a/celery/worker/request.py
+++ b/celery/worker/request.py
@@ -600,8 +600,8 @@ def info(self, safe=False):
         return {
             'id': self.id,
             'name': self.name,
-            'args': self._args,
-            'kwargs': self._kwargs,
+            'args': self._args if not safe else self._argsrepr,
+            'kwargs': self._kwargs if not safe else self._kwargsrepr,
             'type': self._type,
             'hostname': self._hostname,
             'time_start': self.time_start,


Test patch:
diff --git a/t/unit/app/test_control.py b/t/unit/app/test_control.py
--- a/t/unit/app/test_control.py
+++ b/t/unit/app/test_control.py
@@ -95,7 +95,11 @@ def assert_broadcast_called(self, command,
 
     def test_active(self):
         self.inspect.active()
-        self.assert_broadcast_called('active')
+        self.assert_broadcast_called('active', safe=None)
+
+    def test_active_safe(self):
+        self.inspect.active(safe=True)
+        self.assert_broadcast_called('active', safe=True)
 
     def test_clock(self):
         self.inspect.clock()
diff --git a/t/unit/worker/test_control.py b/t/unit/worker/test_control.py
--- a/t/unit/worker/test_control.py
+++ b/t/unit/worker/test_control.py
@@ -298,6 +298,20 @@ def test_active(self):
         finally:
             worker_state.active_requests.discard(r)
 
+    def test_active_safe(self):
+        kwargsrepr = '<anything>'
+        r = Request(
+            self.TaskMessage(self.mytask.name, id='do re mi',
+                             kwargsrepr=kwargsrepr),
+            app=self.app,
+        )
+        worker_state.active_requests.add(r)
+        try:
+            active_resp = self.panel.handle('dump_active', {'safe': True})
+            assert active_resp[0]['kwargs'] == kwargsrepr
+        finally:
+            worker_state.active_requests.discard(r)
+
     def test_pool_grow(self):
 
         class MockPool:
diff --git a/t/unit/worker/test_request.py b/t/unit/worker/test_request.py
--- a/t/unit/worker/test_request.py
+++ b/t/unit/worker/test_request.py
@@ -232,7 +232,7 @@ def test_info_function(self):
             kwargs[str(i)] = ''.join(
                 random.choice(string.ascii_lowercase) for i in range(1000))
         assert self.get_request(
-            self.add.s(**kwargs)).info(safe=True).get('kwargs') == kwargs
+            self.add.s(**kwargs)).info(safe=True).get('kwargs') == '' # mock message doesn't populate kwargsrepr
         assert self.get_request(
             self.add.s(**kwargs)).info(safe=False).get('kwargs') == kwargs
         args = []
@@ -240,7 +240,7 @@ def test_info_function(self):
             args.append(''.join(
                 random.choice(string.ascii_lowercase) for i in range(1000)))
         assert list(self.get_request(
-            self.add.s(*args)).info(safe=True).get('args')) == args
+            self.add.s(*args)).info(safe=True).get('args')) == [] # mock message doesn't populate argsrepr
         assert list(self.get_request(
             self.add.s(*args)).info(safe=False).get('args')) == args