Repository: ramonhagenaars/jsons

Issue:
Unexpected results with Optional type and .load()
There seems to be a problem when marking a value as optional. When something is declared as optional, jsons ignores the data type and simply sets the value given. However, I would expect to have "None" or an exception raised. 

Using a custom deserializer and raising any Exceptions inside to not propagate back to the jsons.load call in this case.

I'm currently using version 1.1.2 from pypi.

``` python
from typing import Optional, Union
from dataclasses import dataclass

import jsons

# Strict mode does not matter
strict = True

print("\n----- Optional Int")

@dataclass
class TestOptionalInt:
    val: Optional[int]

print(jsons.load({"val": 10}, cls=TestOptionalInt, strict=strict))
print(jsons.load({"val": "bla"}, cls=TestOptionalInt, strict=strict)) # Does not raise error, returns the string!
print(jsons.load({"val": (1,"bla",[])}, cls=TestOptionalInt, strict=strict)) # Does not raise error, returns the tuple!
print(jsons.load({"val": None}, cls=TestOptionalInt, strict=strict))


print("\n----- Union Int, None")

@dataclass
class TestUnionInt:
    val: Union[int, None]

print(jsons.load({"val": 10}, cls=TestUnionInt, strict=strict))
print(jsons.load({"val": "bla"}, cls=TestUnionInt, strict=strict)) # Does not raise error, returns the string!
print(jsons.load({"val": (1,"bla",[])}, cls=TestUnionInt, strict=strict)) # Does not raise error, returns the tuple!
print(jsons.load({"val": None}, cls=TestUnionInt, strict=strict))


print("\n----- Int")

@dataclass
class TestInt:
    val: int

print(jsons.load({"val": 10}, cls=TestInt, strict=strict))
print(jsons.load({"val": "bla"}, cls=TestInt, strict=strict)) # Does raise exception (expected)
print(jsons.load({"val": (1,"bla",[])}, cls=TestInt, strict=strict)) # Does raise exception (expected)
print(jsons.load({"val": None}, cls=TestInt, strict=strict)) # Does raise exception (expected)
```

Output:
```
----- Optional Int
TestOptionalInt(val=10)
TestOptionalInt(val='bla')
TestOptionalInt(val=(1, 'bla', []))
TestOptionalInt(val=None)

----- Union Int, None
TestUnionInt(val=10)
TestUnionInt(val='bla')
TestUnionInt(val=(1, 'bla', []))
TestUnionInt(val=None)

----- Int
TestInt(val=10)
Could not cast "bla" into "int"
Could not deserialize value "(1, 'bla', [])" into "int". int() argument must be a string, a bytes-like object or a number, not 'tuple'
NoneType cannot be deserialized into int
```

Discussion:
codecov-io: # [Codecov](https://codecov.io/gh/ramonhagenaars/jsons/pull/101?src=pr&el=h1) Report
> Merging [#101](https://codecov.io/gh/ramonhagenaars/jsons/pull/101?src=pr&el=desc) into [master](https://codecov.io/gh/ramonhagenaars/jsons/commit/e335fd5ebc465b523cf153675ad2eeafbc37b5fb&el=desc) will **decrease** coverage by `0.14%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/ramonhagenaars/jsons/pull/101/graphs/tree.svg?width=650&height=150&src=pr&token=V3gpRP0Ean)](https://codecov.io/gh/ramonhagenaars/jsons/pull/101?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master     #101      +/-   ##
==========================================
- Coverage   99.41%   99.26%   -0.15%     
==========================================
  Files          55       55              
  Lines        1362     1360       -2     
==========================================
- Hits         1354     1350       -4     
- Misses          8       10       +2     
```


| [Impacted Files](https://codecov.io/gh/ramonhagenaars/jsons/pull/101?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [jsons/deserializers/default\_union.py](https://codecov.io/gh/ramonhagenaars/jsons/pull/101/diff?src=pr&el=tree#diff-anNvbnMvZGVzZXJpYWxpemVycy9kZWZhdWx0X3VuaW9uLnB5) | `100.00% <ø> (ø)` | |
| [jsons/\_meta.py](https://codecov.io/gh/ramonhagenaars/jsons/pull/101/diff?src=pr&el=tree#diff-anNvbnMvX21ldGEucHk=) | `100.00% <100.00%> (ø)` | |
| [jsons/deserializers/default\_dict.py](https://codecov.io/gh/ramonhagenaars/jsons/pull/101/diff?src=pr&el=tree#diff-anNvbnMvZGVzZXJpYWxpemVycy9kZWZhdWx0X2RpY3QucHk=) | `100.00% <100.00%> (ø)` | |
| [jsons/deserializers/default\_nonetype.py](https://codecov.io/gh/ramonhagenaars/jsons/pull/101/diff?src=pr&el=tree#diff-anNvbnMvZGVzZXJpYWxpemVycy9kZWZhdWx0X25vbmV0eXBlLnB5) | `100.00% <100.00%> (ø)` | |
| [jsons/serializers/default\_object.py](https://codecov.io/gh/ramonhagenaars/jsons/pull/101/diff?src=pr&el=tree#diff-anNvbnMvc2VyaWFsaXplcnMvZGVmYXVsdF9vYmplY3QucHk=) | `98.44% <100.00%> (ø)` | |
| [jsons/classes/verbosity.py](https://codecov.io/gh/ramonhagenaars/jsons/pull/101/diff?src=pr&el=tree#diff-anNvbnMvY2xhc3Nlcy92ZXJib3NpdHkucHk=) | `90.00% <0.00%> (-10.00%)` | :arrow_down: |
| [jsons/\_compatibility\_impl.py](https://codecov.io/gh/ramonhagenaars/jsons/pull/101/diff?src=pr&el=tree#diff-anNvbnMvX2NvbXBhdGliaWxpdHlfaW1wbC5weQ==) | `95.34% <0.00%> (-0.11%)` | :arrow_down: |
| [jsons/\_cache.py](https://codecov.io/gh/ramonhagenaars/jsons/pull/101/diff?src=pr&el=tree#diff-anNvbnMvX2NhY2hlLnB5) | `100.00% <0.00%> (ø)` | |
| [jsons/exceptions.py](https://codecov.io/gh/ramonhagenaars/jsons/pull/101/diff?src=pr&el=tree#diff-anNvbnMvZXhjZXB0aW9ucy5weQ==) | `100.00% <0.00%> (ø)` | |
| [jsons/\_common\_impl.py](https://codecov.io/gh/ramonhagenaars/jsons/pull/101/diff?src=pr&el=tree#diff-anNvbnMvX2NvbW1vbl9pbXBsLnB5) | `100.00% <0.00%> (ø)` | |
| ... and [2 more](https://codecov.io/gh/ramonhagenaars/jsons/pull/101/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/ramonhagenaars/jsons/pull/101?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/ramonhagenaars/jsons/pull/101?src=pr&el=footer). Last update [e335fd5...cccfc10](https://codecov.io/gh/ramonhagenaars/jsons/pull/101?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

codecov-commenter: # [Codecov](https://codecov.io/gh/ramonhagenaars/jsons/pull/101?src=pr&el=h1) Report
> Merging [#101](https://codecov.io/gh/ramonhagenaars/jsons/pull/101?src=pr&el=desc) into [master](https://codecov.io/gh/ramonhagenaars/jsons/commit/e335fd5ebc465b523cf153675ad2eeafbc37b5fb&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/ramonhagenaars/jsons/pull/101/graphs/tree.svg?width=650&height=150&src=pr&token=V3gpRP0Ean)](https://codecov.io/gh/ramonhagenaars/jsons/pull/101?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master     #101   +/-   ##
=======================================
  Coverage   99.41%   99.41%           
=======================================
  Files          55       55           
  Lines        1362     1366    +4     
=======================================
+ Hits         1354     1358    +4     
  Misses          8        8           
```


| [Impacted Files](https://codecov.io/gh/ramonhagenaars/jsons/pull/101?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [jsons/deserializers/default\_union.py](https://codecov.io/gh/ramonhagenaars/jsons/pull/101/diff?src=pr&el=tree#diff-anNvbnMvZGVzZXJpYWxpemVycy9kZWZhdWx0X3VuaW9uLnB5) | `100.00% <ø> (ø)` | |
| [jsons/\_meta.py](https://codecov.io/gh/ramonhagenaars/jsons/pull/101/diff?src=pr&el=tree#diff-anNvbnMvX21ldGEucHk=) | `100.00% <100.00%> (ø)` | |
| [jsons/deserializers/default\_dict.py](https://codecov.io/gh/ramonhagenaars/jsons/pull/101/diff?src=pr&el=tree#diff-anNvbnMvZGVzZXJpYWxpemVycy9kZWZhdWx0X2RpY3QucHk=) | `100.00% <100.00%> (ø)` | |
| [jsons/deserializers/default\_nonetype.py](https://codecov.io/gh/ramonhagenaars/jsons/pull/101/diff?src=pr&el=tree#diff-anNvbnMvZGVzZXJpYWxpemVycy9kZWZhdWx0X25vbmV0eXBlLnB5) | `100.00% <100.00%> (ø)` | |
| [jsons/serializers/default\_object.py](https://codecov.io/gh/ramonhagenaars/jsons/pull/101/diff?src=pr&el=tree#diff-anNvbnMvc2VyaWFsaXplcnMvZGVmYXVsdF9vYmplY3QucHk=) | `98.44% <100.00%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/ramonhagenaars/jsons/pull/101?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/ramonhagenaars/jsons/pull/101?src=pr&el=footer). Last update [e335fd5...26aa0c8](https://codecov.io/gh/ramonhagenaars/jsons/pull/101?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/README.md b/README.md
index 39b8014..432619c 100644
--- a/README.md
+++ b/README.md
@@ -84,6 +84,14 @@ list_of_tuples = jsons.load(some_dict, List[Tuple[AClass, AnotherClass]])
 
 ## Recent updates
 
+### 1.2.0
+
+- Bugfix: Fixed bug with postponed typehints (PEP-563).
+- Bugfix: Loading an invalid value targeting an optional did not raise.
+- Bugfix: Loading a dict did not properly pass key_transformers.
+- Bugfix: Loading a namedtuple did not properly use key_transformers.
+- Bugfix: Utilized `__annotations__` in favor `_field_types` because of deprecation as of 3.8.
+
 ### 1.1.2
 
 - Feature: Added `__version__` which can be imported from `jsons`
@@ -119,24 +127,22 @@ list_of_tuples = jsons.load(some_dict, List[Tuple[AClass, AnotherClass]])
   - Bugfix: Loading with `List` (no generic type) failed.
   - Bugfix: Loading with `Dict` (no generic type) failed.
   - Bugfix: Loading with `Tuple` (no generic type) failed.
-
-### 0.10.2
-
-  - Bugfix: Loading `Dict[K, V]` did not parse `K
-
+  
 
 ## Contributors
 
 Special thanks to the following contributors of code, discussions or
 suggestions:
 
-  - [finetuned89](https://github.com/finetuned89)
-  - [haluzpav](https://github.com/haluzpav)
-  - [jmolinski](https://github.com/jmolinski)
-  - [gastlich](https://github.com/gastlich)
-  - [cypreess](https://github.com/cypreess)
-  - [casparjespersen](https://github.com/casparjespersen)
-  - [ahmetkucuk](https://github.com/ahmetkucuk)
-  - [robinklaassen](https://github.com/robinklaassen)
-  - [jochembroekhoff](https://github.com/jochembroekhoff)
-  - [herdigiorgi](https://github.com/herdigiorgi)
+[tirkarthi](https://github.com/tirkarthi), 
+[marksomething](https://github.com/marksomething), 
+[herdigiorgi](https://github.com/herdigiorgi), 
+[jochembroekhoff](https://github.com/jochembroekhoff), 
+[robinklaassen](https://github.com/robinklaassen), 
+[ahmetkucuk](https://github.com/ahmetkucuk), 
+[casparjespersen](https://github.com/casparjespersen), 
+[cypreess](https://github.com/cypreess), 
+[gastlich](https://github.com/gastlich), 
+[jmolinski](https://github.com/jmolinski), 
+[haluzpav](https://github.com/haluzpav), 
+[finetuned89](https://github.com/finetuned89),
diff --git a/jsons/_meta.py b/jsons/_meta.py
index 0062608..8d5300b 100644
--- a/jsons/_meta.py
+++ b/jsons/_meta.py
@@ -1,5 +1,5 @@
 __title__ = 'jsons'
-__version__ = '1.1.2'
+__version__ = '1.2.0'
 __author__ = 'Ramon Hagenaars'
 __author_email__ = 'ramon.hagenaars@gmail.com'
 __description__ = 'For serializing Python objects to JSON (dicts) and back'
diff --git a/jsons/deserializers/default_dict.py b/jsons/deserializers/default_dict.py
index 182ce74..63f4e02 100644
--- a/jsons/deserializers/default_dict.py
+++ b/jsons/deserializers/default_dict.py
@@ -20,13 +20,14 @@ def default_dict_deserializer(
     """
     key_tfr = key_transformer or (lambda key: key)
     cls_args = get_args(cls)
+    kwargs_ = {**kwargs, 'key_transformer': key_transformer}
     if len(cls_args) == 2:
         cls_k, cls_v = cls_args
-        kwargs_k = {**kwargs, 'cls': cls_k}
-        kwargs_v = {**kwargs, 'cls': cls_v}
+        kwargs_k = {**kwargs_, 'cls': cls_k}
+        kwargs_v = {**kwargs_, 'cls': cls_v}
         res = {load(key_tfr(k), **kwargs_k): load(obj[k], **kwargs_v)
                for k in obj}
     else:
-        res = {key_tfr(key): load(obj[key], **kwargs)
+        res = {key_tfr(key): load(obj[key], **kwargs_)
                for key in obj}
     return res
diff --git a/jsons/deserializers/default_nonetype.py b/jsons/deserializers/default_nonetype.py
index 577eb14..588b5c6 100644
--- a/jsons/deserializers/default_nonetype.py
+++ b/jsons/deserializers/default_nonetype.py
@@ -1,5 +1,7 @@
 from typing import Optional
 
+from jsons.exceptions import DeserializationError
+
 
 def default_nonetype_deserializer(obj: object,
                                   cls: Optional[type] = None,
@@ -11,4 +13,7 @@ def default_nonetype_deserializer(obj: object,
     :param kwargs: not used.
     :return: ``obj``.
     """
+    if obj is not None:
+        raise DeserializationError('Cannot deserialize {} as NoneType'
+                                   .format(obj), source=obj, target=cls)
     return obj
diff --git a/jsons/deserializers/default_union.py b/jsons/deserializers/default_union.py
index f417a47..3015f14 100644
--- a/jsons/deserializers/default_union.py
+++ b/jsons/deserializers/default_union.py
@@ -25,5 +25,5 @@ def default_union_deserializer(obj: object, cls: Union, **kwargs) -> object:
         args_msg = ', '.join([get_class_name(cls_)
                               for cls_ in get_union_params(cls)])
         err_msg = ('Could not match the object of type "{}" to any type of '
-                   'the Union: {}'.format(str(cls), args_msg))
+                   'the Union: {}'.format(type(obj).__name__, args_msg))
         raise DeserializationError(err_msg, obj, cls)
diff --git a/jsons/serializers/default_object.py b/jsons/serializers/default_object.py
index 6d8664e..3a97210 100644
--- a/jsons/serializers/default_object.py
+++ b/jsons/serializers/default_object.py
@@ -202,7 +202,9 @@ def _get_attributes_and_types(cls: type,
     if '__slots__' in cls.__dict__:
         attributes = {attr: None for attr in cls.__slots__}
     elif hasattr(cls, '__annotations__'):
-        attributes = cls.__annotations__
+        # Fixme: remove code below if tests succeed on all platforms.
+        # attributes = cls.__annotations__
+        attributes = get_type_hints(cls)
     elif strict:
         hints = get_type_hints(cls.__init__)
         attributes = {k: hints[k] for k in hints if k != 'self'}


Test patch:
diff --git a/test_resources/version_37.py b/test_resources/version_37.py
index 9d48db4..2e4c38e 100644
--- a/test_resources/version_37.py
+++ b/test_resources/version_37.py
@@ -1,1 +1,15 @@
 from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import List
+
+
+@dataclass
+class A:
+    x: int
+
+
+@dataclass
+class B:
+    a: A
+    x: List[int]
diff --git a/tests/test_dict.py b/tests/test_dict.py
index dbbbd54..8d4fec0 100644
--- a/tests/test_dict.py
+++ b/tests/test_dict.py
@@ -44,6 +44,31 @@ class TestDict(TestCase):
         loaded2 = jsons.load(source2, Dict[Color, int], key_transformer=lambda x: x.upper())
         self.assertDictEqual(expected, loaded2)
 
+    def test_load_dict_with_key_transformers(self):
+
+        class A_:
+            def __init__(self, b_: int):
+                self.b_ = b_
+
+        class C:
+            def __init__(self, a_: A_):
+                self.a_ = a_
+
+        src = {
+            'a': {
+                'b': 42
+            }
+        }
+        loaded = jsons.load(src, key_transformer=lambda x: x + '_')
+
+        expected = {
+            'a_': {
+                'b_': 42
+            }
+        }
+
+        self.assertDictEqual(expected, loaded)
+
     def test_load_dict_with_generic(self):
         class A:
             def __init__(self):
diff --git a/tests/test_object.py b/tests/test_object.py
index e20003e..ee7c465 100644
--- a/tests/test_object.py
+++ b/tests/test_object.py
@@ -477,6 +477,15 @@ class TestObject(TestCase):
             self.assertTrue('y' in err.message)
             self.assertTrue('Told you so' in err.message)
 
+    def test_dump_object_with_str_hint(self):
+        class C:
+            def __init__(self, x: 'str'):
+                self.x = x
+
+        dumped = jsons.dump(C('test'), cls=C)
+
+        self.assertDictEqual({'x': 'test'}, dumped)
+
 
 class ParentDumpable:
     _par_c = 10
diff --git a/tests/test_specific_versions.py b/tests/test_specific_versions.py
index 37d43c2..b08b141 100644
--- a/tests/test_specific_versions.py
+++ b/tests/test_specific_versions.py
@@ -36,17 +36,20 @@ def only_version_3(minor_version: int, and_above: bool = False):
 
 class TestSpecificVersions(TestCase):
 
-    @only_version_3(7)
+    @only_version_3(7, and_above=True)
     def test_simple_dump_and_load_dataclass_with_future_import(self):
         import version_37
-        from version_with_dataclasses import Person
-        p = Person('John')
-        dumped = jsons.dump(p)
-        loaded = jsons.load(dumped, Person)
-        self.assertEqual(p.name, loaded.name)
 
-        with self.assertRaises(SignatureMismatchError):
-            jsons.load({'name': 'John', 'age': 88}, Person, strict=True)
+        dumped = jsons.dump(version_37.B(version_37.A(42), [1, 2, 3]))
+
+        expected = {
+            'a': {
+                'x': 42
+            },
+            'x': [1, 2, 3]
+        }
+
+        self.assertDictEqual(expected, dumped)
 
     @only_version_3(6, and_above=True)
     def test_simple_dump_and_load_dataclass(self):
diff --git a/tests/test_union.py b/tests/test_union.py
index 356eae5..de5634e 100644
--- a/tests/test_union.py
+++ b/tests/test_union.py
@@ -92,3 +92,26 @@ class TestUnion(TestCase):
         jsons.load({'x': 1, 'y': None}, cls=None, strict=True)  # Should not raise.
         jsons.load({'x': 1}, cls=None, strict=True)  # Should not raise.
         jsons.load(None)  # Should not raise.
+
+    def test_load_optional(self):
+        class TestOptionalInt:
+            def __init__(self, value: Optional[int]):
+                self.value = value
+
+        # This seems fine.
+        loaded1 = jsons.load({'value': 42}, cls=TestOptionalInt)
+        self.assertEqual(42, loaded1.value)
+
+        # Strings are parsed if possible.
+        loaded2 = jsons.load({'value': '42'}, cls=TestOptionalInt)
+        self.assertEqual(42, loaded2.value)
+
+        # No value or None will result in None.
+        loaded3 = jsons.load({}, cls=TestOptionalInt)
+        loaded4 = jsons.load({'value': None}, cls=TestOptionalInt)
+        self.assertEqual(None, loaded3.value)
+        self.assertEqual(None, loaded4.value)
+
+        # Now this will fail.
+        with self.assertRaises(DeserializationError):
+            jsons.load({'value': 'not good'}, cls=TestOptionalInt)