Repository: PrefectHQ/prefect

Issue:
Serialization fails with no attribute __qualname__
## Description
Serialization fails when a task uses an instance of a callable class as a cache_validator.

```
AttributeError                            Traceback (most recent call last)
~/.virtualenvs/iguazu-venv/lib/python3.7/site-packages/prefect/utilities/serialization.py in _serialize(self, value, attr, obj, **kwargs)
    379         try:
--> 380             qual_name = to_qualified_name(value)
    381         except:

~/.virtualenvs/iguazu-venv/lib/python3.7/site-packages/prefect/utilities/serialization.py in to_qualified_name(obj)
     36     """
---> 37     return obj.__module__ + "." + obj.__qualname__
     38

AttributeError: 'Validator' object has no attribute '__qualname__'

During handling of the above exception, another exception occurred:

... 

~/.virtualenvs/iguazu-venv/lib/python3.7/site-packages/prefect/utilities/serialization.py in _serialize(self, value, attr, obj, **kwargs)
    383                 raise ValidationError("Invalid function reference: {}".format(value))
    384             else:
--> 385                 return qual_name
    386
    387         # sort matches such that the longest / most specific match comes first

UnboundLocalError: local variable 'qual_name' referenced before assignment
```

More details:

I found a small bug on the serialization code. In `prefect.utilities.serialization.StatefulFunctionReference`  :
```python
    def _serialize(self, value, attr, obj, **kwargs):  # type: ignore
        if value is None and self.allow_none:
            return None

        try:
            qual_name = to_qualified_name(value)
        except:
            if self.reject_invalid:
                raise ValidationError("Invalid function reference: {}".format(value))
            else:
                return qual_name
        ... more code ...
```

There is a return with an undefined variable (because when this fails, there is no qual_name variable.

I am not sure if this is a problem of that particular function, or is from `to_qualified_name`, which fails with an input instance instead of a class.
The culprit from my side is that I have a task with `cache_validator=SomeClass()`  … where SomeClass is a class with a `__call__`  member function, instead of using a cache_validator that is a function. I am attaching an example on the reproduction section.


## Expected Behavior
Since `cache_validator` in a task is type-annotated to a `Callable`, I expected to be able to use an instance of a callable class without issues when serializing.



## Reproduction
```python
import datetime

from prefect import Flow, Task
from prefect.environments.storage import Docker


class Validator:
    def __init__(self):
        print('I am a validator, ready to say false al the time')
    def __call__(self, state, inputs, parameters):
        return False

my_task = Task(cache_validator=Validator(), cache_for=datetime.timedelta(days=1))
with Flow('example') as flow:
    my_task()

flow.storage = Docker()
flow.register()
```


## Environment
```
$ prefect diagnostics
{
  "config_overrides": {},
  "env_vars": [],
  "system_information": {
    "platform": "Darwin-19.3.0-x86_64-i386-64bit",
    "prefect_version": "0.10.4",
    "python_version": "3.7.5"
  }
}
```


Discussion:
Using the following small fix:
```diff
diff --git a/src/prefect/utilities/serialization.py b/src/prefect/utilities/serialization.py
index bba9a6ac..7a17e186 100644
--- a/src/prefect/utilities/serialization.py
+++ b/src/prefect/utilities/serialization.py
@@ -34,7 +34,7 @@ def to_qualified_name(obj: Any) -> str:
     Returns:
         - str: the qualified name
     """
-    return obj.__module__ + "." + obj.__qualname__
+    return obj.__module__ + "." + obj.__class__.__qualname__


 def from_qualified_name(obj_str: str) -> Any:
```

can workaround this problem, but only for a couple of instructions; it will later fail with:
```
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/inspect.py in getclosurevars(func)
   1400
   1401     if not isfunction(func):
-> 1402         raise TypeError("{!r} is not a Python function".format(func))
   1403
   1404     code = func.__code__

TypeError: <__main__.Validator object at 0x7fe1d983d6d0> is not a Python function
```

For this reason, the question may be: is it allowed to have callable classes as cache validators?
Hi @dojeda!

Fundamentally, cache validators are not designed to be represented as callable classes. Is there more you can share about what makes a callable class as a cache validator useful as opposed to writing it a function? And would it be a feasible workaround for you to define a function that wraps your class call?
Hi @lauralorenz!

It was the type annotation that led me to believe that it could be a class. Our use-case was that we wanted to have different behaviors on caching for some classes, using a combination of the `all_inputs`, `all_parameters` and a task-specific member variable `force` that we set manually.

Here is a copy of our cache validator:
```python
from typing import Any, Dict
import logging

from prefect.engine.cache_validators import all_inputs, all_parameters
from prefect.engine.state import Cached


logger = logging.getLogger(__name__)


class ParametrizedValidator:

    def __init__(self, use_inputs=True, use_parameters=True, force=False):
        self.use_inputs = use_inputs
        self.use_parameters = use_parameters
        self.force = force

    def __call__(self, state: Cached, inputs: Dict[str, Any], parameters: Dict[str, Any]):
        # Never use cache when force is set
        if self.force:
            logger.debug('Cache miss: forced is set')
            return False

        # When caching with inputs and outputs, defer to the corresponding
        # prefect cache validators
        if self.use_inputs and self.use_parameters:
            input_cache = all_inputs(state, inputs, parameters)
            param_cache = all_parameters(state, inputs, parameters)
            result = input_cache and param_cache
            logger.debug('Cache %s: inputs was %s, params was %s',
                         'hit' if result else 'miss',
                         input_cache, param_cache)

        elif self.use_inputs:
            input_cache = all_inputs(state, inputs, parameters)
            result = input_cache
            logger.debug('Cache %s: inputs was %s',
                         'hit' if result else 'miss',
                         input_cache)
            return input_cache

        elif self.use_parameters:
            param_cache = all_parameters(state, inputs, parameters)
            result = param_cache
            logger.debug('Cache %s: parameters was %s',
                         'hit' if result else 'miss',
                         param_cache)
            return param_cache
        else:
            result = False
            logger.debug('Cache miss: no cache configuration set')

        if result:
            logger.info('Cache hit!')

        return result
```

For us, it would probably be feasible to refactor this as a function: after some time, we realized that almost all of our tasks were going to use the same caching configuration. Moreover, after a bit more of experience, I am reluctant to use caches since it often leads to very weird situations, specially during development.

Independently from our workaround / refactor, I think the undefined variable is still a bug, don't you think?

Resolving patch:
diff --git a/src/prefect/utilities/serialization.py b/src/prefect/utilities/serialization.py
--- a/src/prefect/utilities/serialization.py
+++ b/src/prefect/utilities/serialization.py
@@ -379,10 +379,9 @@ def _serialize(self, value, attr, obj, **kwargs):  # type: ignore
         try:
             qual_name = to_qualified_name(value)
         except:
-            if self.reject_invalid:
-                raise ValidationError("Invalid function reference: {}".format(value))
-            else:
-                return qual_name
+            raise ValidationError(
+                f"Invalid function reference, function required, got {value}"
+            )
 
         # sort matches such that the longest / most specific match comes first
         valid_bases = sorted(


Test patch:
diff --git a/tests/utilities/test_serialization.py b/tests/utilities/test_serialization.py
--- a/tests/utilities/test_serialization.py
+++ b/tests/utilities/test_serialization.py
@@ -279,6 +279,14 @@ def test_deserialize_invalid_fn_without_validation(self):
         )
         assert deserialized["f_allow_invalid"] is None
 
+    def test_serialize_non_function_good_error(self):
+        class Foo(object):
+            def __call__(self, a, b):
+                return a + b
+
+        with pytest.raises(marshmallow.ValidationError, match="function required"):
+            self.Schema().dump(dict(f=Foo()))
+
     def test_serialize_none(self):
         with pytest.raises(marshmallow.ValidationError):
             self.Schema().dump({"f": None})