Repository: Parsl/parsl

Issue:
RemoteExceptionWrapper should not serialise unnecessarily.
**Is your feature request related to a problem? Please describe.**

Exceptions from tasks are wrapped in parsl's `RemoteExceptionWrapper` which is intended to be a helper that can ensure exceptions can be represented as values and moved between python runtimes.

As part of this, it always serializes exception information.

Serialization is not always necessary, but always happens: at least in the case of the local thread provider, the exception object never moves between python runtimes.

In some cases, exceptions cannot be serialised/deserialised properly: for example, the globus SDK does strange things with exception initializers (#785, #1997) which interact with the strange things RemoteExceptionWrapper is doing, but usually globus calls are made on the submit side (because the globus file staging providers run tasks locally).

**Describe the solution you'd like**
I would like RemoteExceptionWrapper to only perform the serialization/deserialization loop when necessary: when the RemoteExceptionWrapper itself is serialized/deserialized, rather than always.

**Describe alternatives you've considered**
Persuade Globus to do their exceptions differently.

**Additional context**
The only practical case that I have seen this affect things is the error handling path for globus file staging, although it is a broader theoretical problem.


Discussion:

Resolving patch:
diff --git a/parsl/app/errors.py b/parsl/app/errors.py
--- a/parsl/app/errors.py
+++ b/parsl/app/errors.py
@@ -2,8 +2,6 @@
 from functools import wraps
 from typing import Callable, List, Union, Any, TypeVar, Optional
 from types import TracebackType
-
-import dill
 import logging
 from tblib import Traceback
 
@@ -113,8 +111,8 @@ def __str__(self) -> str:
 class RemoteExceptionWrapper:
     def __init__(self, e_type: type, e_value: BaseException, traceback: Optional[TracebackType]) -> None:
 
-        self.e_type = dill.dumps(e_type)
-        self.e_value = dill.dumps(e_value)
+        self.e_type = e_type
+        self.e_value = e_value
         self.e_traceback = None if traceback is None else Traceback(traceback)
         if e_value.__cause__ is None:
             self.cause = None
@@ -124,20 +122,20 @@ def __init__(self, e_type: type, e_value: BaseException, traceback: Optional[Tra
 
     def reraise(self) -> None:
 
-        t = dill.loads(self.e_type)
+        t = self.e_type
 
         # the type is logged here before deserialising v and tb
         # because occasionally there are problems deserialising the
         # value (see #785, #548) and the fix is related to the
         # specific exception type.
-        logger.debug("Reraising exception of type {}".format(t))
+        logger.debug("Reraising exception of type {}".format(self.e_type))
 
         v = self.get_exception()
 
         reraise(t, v, v.__traceback__)
 
-    def get_exception(self) -> Exception:
-        v = dill.loads(self.e_value)
+    def get_exception(self) -> BaseException:
+        v = self.e_value
         if self.cause is not None:
             v.__cause__ = self.cause.get_exception()
         if self.e_traceback is not None:


Test patch: