Repository: celery/celery

Issue:
Celery does not respect exceptions types when using a serializer different than pickle.
## Checklist
```
~ : celery -A analystick report
software -> celery:4.0.0 (latentcall) kombu:4.0.0 py:3.5.2
            billiard:3.5.0.2 redis:2.10.5
platform -> system:Darwin arch:64bit imp:CPython
loader   -> celery.loaders.app.AppLoader
settings -> transport:redis results:redis://127.0.0.1:6379/1
```

## Steps to reproduce
(See example code below)

## Expected behavior
**When using a result serializer different than pickle**, exceptions types should be the same as the raised exception.

## Actual behavior
Celery does not respect the exception type but create _a new type_ instead.

The main problem is that instead of using the actual type of the exception, celery will [reconstruct a type](https://github.com/celery/celery/blob/0f87321df385c5f3dca717ec2a4a9c0d25f88054/celery/utils/serialization.py#L43-L45) on the fly, but without respecting the original exception module.

For example, using the `yaml` result serializer (I believe it will be the same for `json`): 


* if a task raises a `ValueError`, the caller will receive a `celery.backends.base.ValueError`
* if a task raises a `custom.module.CustomError`, the caller will receive a `celery.backends.base.CustomError`

This ends with wrongs behaviour when raising a exception from a task and trying to catch it from the caller.

### Minimal reproductible test
As an example, I've setup a minimal reproductible test, using a redis backend :

celery config (I can provide a full config if needed):


```python
CELERY_TASK_SERIALIZER = 'yaml'
CELERY_RESULT_SERIALIZER='yaml'
```

Tasks : 

```python
# module myapp.tasks
from myapp import celery_app
@celery_app.task
def raises_valueerror():
    raise ValueError('Builtin exception')

class CustomError(Exception):
    pass

@celery_app.task
def raises_customerror():
    raise CustomError('Custom exception', {'a':1})
```

Unittest : 


```python
from myapp import tasks
from myapp.tasks import CustomError

def test_builtin_exception():
    t = tasks.raises_valueerror.s()
    r = t.apply_async()

    exc = None
    try:
        r.get(propagate=True)
    except Exception as e:
        exc = e

    # with celery bug, the actual class of exc will be  `celery.backends.base.ValueError` instead of builtin ValueError
    assert isinstance(exc, ValueError), "Actual class %s}" % (exc.__class__)

def test_custom_exception():
    t = tasks.raises_customerror.s()
    r = t.apply_async()

    exc = None
    try:
        r.get(propagate=True)
    except Exception as e:
        exc = e

    # with celery bug, the actual class of exc will be  `celery.backends.base.CustomError` instead of builtin CustomError
    assert isinstance(exc, CustomError), "1/2 Actual class is %s" % (exc.__class__)
    assert isinstance(exc, tasks.CustomError), "2/2 Actual class is %s" % (exc.__class__)

```

Theses tests will fail with the following errors : 

```
# ...
AssertionError: Actual class <class 'celery.backends.base.ValueError'>
# ...
AssertionError: 1/2 Actual class is <class 'celery.backends.base.CustomError'>
```

Another side effect for this problem will be that a code like the one below won't work if a subtask raise a `ValueError`, as the propagated exception won't be of the builtin type `ValueError` but `celery.backends.base.ValueError`:

```python
try:
    r.get(propagate=True)
except ValueError as e:
    # do something        
```

This problem will be the same also for any custom exceptions.

While I'm not sure about the possible side-effects, [I have a fix for this](https://github.com/jcsaaddupuy/celery/commit/8d4e613e24f6561fdaafd4e6ede582ceac882804) and I will gladly create a PR for this problem as it seems pretty critical.

What do you think ?




Discussion:
This is actually deliberate, as none of json, yaml or msgpack are able to reconstruct exceptions.

The alternative is to load exceptions of any type, which opens up for similar security issues as using pickle.
If this is deliberate, perhaps this issue should be closed? Or are there any plans to try to make it work, despite the security implications?
Is this situation documented? I was not able to find anything regarding this behaviour in the documentation.
@vladcalin @estan please check https://github.com/celery/celery/pull/3592 . As you can see this is still on-going. Any feedback is appreciated!
Hey here, will this issue be fixed one day ? Should we stop using custom exception ? Is there any workaround for now ?

Resolving patch:
diff --git a/celery/backends/base.py b/celery/backends/base.py
--- a/celery/backends/base.py
+++ b/celery/backends/base.py
@@ -9,6 +9,7 @@
 from __future__ import absolute_import, unicode_literals
 
 import datetime
+import inspect
 import sys
 import time
 from collections import namedtuple
@@ -34,7 +35,6 @@
 from celery.utils.functional import LRUCache, arity_greater
 from celery.utils.log import get_logger
 from celery.utils.serialization import (create_exception_cls,
-                                        ensure_serializable,
                                         get_pickleable_exception,
                                         get_pickled_exception)
 
@@ -236,9 +236,14 @@ def prepare_exception(self, exc, serializer=None):
         serializer = self.serializer if serializer is None else serializer
         if serializer in EXCEPTION_ABLE_CODECS:
             return get_pickleable_exception(exc)
+        # retrieve exception original module
+        exc_module = inspect.getmodule(type(exc))
+        if exc_module:
+            exc_module = exc_module.__name__
+
         return {'exc_type': type(exc).__name__,
-                'exc_message': ensure_serializable(exc.args, self.encode),
-                'exc_module': type(exc).__module__}
+                'exc_args': exc.args,
+                'exc_module': exc_module}
 
     def exception_to_python(self, exc):
         """Convert serialized exception to Python exception."""
diff --git a/celery/utils/serialization.py b/celery/utils/serialization.py
--- a/celery/utils/serialization.py
+++ b/celery/utils/serialization.py
@@ -8,11 +8,11 @@
 from base64 import b64decode as base64decode
 from base64 import b64encode as base64encode
 from functools import partial
+from importlib import import_module
 from inspect import getmro
 from itertools import takewhile
 
 from kombu.utils.encoding import bytes_to_str, str_to_bytes
-
 from celery.five import (bytes_if_py2, items, python_2_unicode_compatible,
                          reraise, string_t)
 
@@ -81,6 +81,14 @@ def itermro(cls, stop):
 
 def create_exception_cls(name, module, parent=None):
     """Dynamically create an exception class."""
+    try:
+        mod = import_module(module)
+        exc_cls = getattr(mod, name, None)
+        if exc_cls and isinstance(exc_cls, type(BaseException)):
+            return exc_cls
+    except ImportError:
+        pass
+    # we could not find the exception, fallback and create a type.
     if not parent:
         parent = Exception
     return subclass_exception(name, parent, module)


Test patch:
diff --git a/t/unit/backends/test_base.py b/t/unit/backends/test_base.py
--- a/t/unit/backends/test_base.py
+++ b/t/unit/backends/test_base.py
@@ -225,6 +225,10 @@ def _delete_group(self, group_id):
         self._data.pop(group_id, None)
 
 
+class CustomTestError(Exception):
+    pass
+
+
 class test_BaseBackend_dict:
 
     def setup(self):
@@ -245,13 +249,26 @@ def test_delete_group(self):
         self.b.delete_group('can-delete')
         assert 'can-delete' not in self.b._data
 
-    def test_prepare_exception_json(self):
-        x = DictBackend(self.app, serializer='json')
-        e = x.prepare_exception(KeyError('foo'))
-        assert 'exc_type' in e
+    @pytest.mark.parametrize(("serializer"), (("pickle", "json")))
+    def test_prepare_builtin_exception(self, serializer):
+        x = DictBackend(self.app, serializer=serializer)
+        e = x.prepare_exception(ValueError('foo'))
+        if not isinstance(e, BaseException):
+            # not using pickle
+            assert 'exc_type' in e
+        e = x.exception_to_python(e)
+        assert e.__class__ is ValueError
+        assert e.args == ("foo", )
+
+    @pytest.mark.parametrize(("serializer"), (("pickle", "json")))
+    def test_prepare_custom_exception(self, serializer):
+        x = DictBackend(self.app, serializer=serializer)
+        e = x.prepare_exception(CustomTestError('foo'))
+        if not isinstance(e, BaseException):
+            assert 'exc_type' in e
         e = x.exception_to_python(e)
-        assert e.__class__.__name__ == 'KeyError'
-        assert str(e).strip('u') == "'foo'"
+        assert e.__class__ is CustomTestError
+        assert e.args == ("foo", )
 
     def test_save_group(self):
         b = BaseBackend(self.app)