Repository: lebrice/SimpleParsing

Issue:
How to get rid of a warning for each used `dataclass`
**Describe the bug**

For each dataclass I use I get a warning like:

> .../python3.8/site-packages/simple_parsing/helpers/serialization/decoding.py:249: UserWarning: Unable to find a decoding function for the annotation <class '__main__.Hparams'> (of type <class 'type'>). Will try to use the type as a constructor. Consider registering a decoding function using `register_decoding_fn`, or posting an issue on GitHub.
>  warnings.warn(

**To Reproduce**

I reduced the problem to just this:

```python
from dataclasses import dataclass
import yaml
from simple_parsing import ArgumentParser, Serializable

@dataclass
class Hparams:
    seed: int = 13

@dataclass
class Parameters(Serializable):
    hparams: Hparams = Hparams()

if __name__ == "__main__":
    with open("conf.yaml", "w") as f: f.write("hparams:")
    file_config = Parameters.load("conf.yaml", load_fn=yaml.safe_load)
```

Run:
```
$ python config.py
/home/stas/anaconda3/envs/py38-pt113/lib/python3.8/site-packages/simple_parsing/helpers/serialization/decoding.py:249: 
UserWarning: Unable to find a decoding function for the annotation <class '__main__.Hparams'> (of type <class 'type'>). Will try to 
use the type as a constructor. Consider registering a decoding function using `register_decoding_fn`, or posting an issue on GitHub.
  warnings.warn(
```

It'd be awesome to receive an advice on how to resolve this warning as we have a whole slew of those.

I think it has to be one of:

```
register_decoding_fn(Hparams, type)
register_decoding_fn(Hparams, Hparams)
```

may be `dataclasses.asdict`?

Additionally a few issues that you might kindly consider:

1. I think this library surely should know how to decode a `dataclass` type of class, since it's used everywhere in the documentation and there is not a single `register_decoding_fn` used for such classes. i.e. is this a documentation issue, or the authors don't mind the warnings? 

   I dug a bit into `dataclasses` and the check is:
```
dataclasses.is_dataclass(Hparams)
```

   so auto-recognizing `dataclass` objects should be very helpful, no?

2. I'd also like to highlight that this warnings isn't very actionable as the user doesn't know what `<class 'type'>` means or if they do how to act on it. Surely, it comes from `type(Hparams)` - but how does it help to the user to fix the problem? i.e. would it be possible to guide the user from the problem to the solution w/o needing to file an issue? Perhaps writing a small section with examples? I diligently spent an hour reading your docs and a bit the code but I am still not sure if I'm doing it right.

3. The real complex program (not the repro) works despite those warnings. Is it possible that these warnings are misplaced and shouldn't be emitted in the first place?

  If it's a problem please assert, if it is not a problem then why emit a warning? Perhaps it'd help to understand the situation better with an example where `type` as a fallback doesn't work. But please feel free to ignore this curiosity question and let's focus on the actual solving of the problem first if this doesn't resonate.

---------------------

now let's go to a non-dataclass classes:

The real program also emitted this warning about `pathlib`  (not in the repro example), and this code removed the warning
```
register_decoding_fn(Path, str)
```

But I'm not at all sure if I did it correctly, does it want a constructor? `str` would be more of a encoding_fn - does it want:

```
register_decoding_fn(Path, Path)
```

I think it'd also help the user to have the context, what are you decoding from - is it a pickled string to object situation? or object to string?

Again, having a section with examples of common and custom classes would be very helpful and the warning could send the user to that document.

Thank you!

**Desktop (please complete the following information):**
 - Version: 0.0.20 / 0.0.21.post1
 - Python version: 3.8
 
 


Comments:

Patch:
diff --git a/simple_parsing/helpers/serialization/serializable.py b/simple_parsing/helpers/serialization/serializable.py
index 22bfb42..a3c15f3 100644
--- a/simple_parsing/helpers/serialization/serializable.py
+++ b/simple_parsing/helpers/serialization/serializable.py
@@ -361,6 +361,20 @@ def load(
 ) -> Dataclass:
     """Loads an instance of `cls` from the given file.
 
+    First, `load_fn` is used to get a potentially nested dictionary of python primitives from a
+    file. Then, a decoding function is applied to each value, based on the type annotation of the
+    corresponding field. Finally, the resulting dictionary is used to instantiate an instance of
+    the dataclass `cls`.
+
+    - string -> `load_fn` (json/yaml/etc) -> dict with "raw" python values -> decode -> \
+        dict with constructor arguments -> `cls`(**dict) -> instance of `cls`
+
+    NOTE: This does not save the types of the dataclass fields. This is usually not an issue, since
+    we can recover the right type to use by looking at subclasses of the annotated type. However,
+    in some cases (e.g. subgroups), it might be useful to save all the types of all the
+    fields, in which case you should probably use something like `yaml.dump`, directly passing it
+    the dataclass, instead of this.
+
     Args:
         cls (Type[D]): A dataclass type to load.
         path (Path | str): Path or Path string or open file.
@@ -529,7 +543,12 @@ def read_file(path: str | Path) -> dict:
 
 
 def save(obj: Any, path: str | Path, dump_fn: Callable[[dict, IO], None] | None = None) -> None:
-    """Save the given dataclass or dictionary to the given file."""
+    """Save the given dataclass or dictionary to the given file.
+
+    Note: The `encode` function is applied to all the object fields to get serializable values,
+    like so:
+    - obj -> encode -> "raw" values (dicts, strings, ints, etc) -> `dump_fn` ([json/yaml/etc].dumps) -> string
+    """
     path = Path(path)
 
     if not isinstance(obj, dict):


Test patch:
diff --git a/test/helpers/test_enum_serialization.py b/test/helpers/test_enum_serialization.py
new file mode 100644
index 0000000..8ae3b11
--- /dev/null
+++ b/test/helpers/test_enum_serialization.py
@@ -0,0 +1,138 @@
+import textwrap
+from dataclasses import dataclass, field
+from enum import Enum
+from pathlib import Path
+from typing import List, Optional
+
+import pytest
+import yaml
+
+from simple_parsing import Serializable
+from simple_parsing.helpers.serialization.serializable import dumps_yaml, loads_yaml
+
+
+class LoggingTypes(Enum):
+    JSONL = "jsonl"
+    PRINT = "print"
+
+
+@dataclass
+class Hparams:
+    seed: int = 13
+    xyz: List[LoggingTypes] = field(default_factory=list)
+
+
+@dataclass
+class Parameters(Serializable):
+    hparams: Hparams = field(default_factory=Hparams)
+    p: Optional[Path] = None
+
+
+@pytest.fixture(autouse=True)
+def clear_get_decoding_fn_cache():
+    """Clears the lru_cache of the `get_decoding_fn` function between tests.
+    This avoids any interference between runs."""
+    from simple_parsing.helpers.serialization.decoding import get_decoding_fn
+
+    get_decoding_fn.cache_clear()
+
+
+@pytest.mark.xfail(
+    raises=KeyError, match="'jsonl'", strict=True, reason="Enums are saved by name, not by value."
+)
+def test_decode_enum_saved_by_value_doesnt_work():
+    """Test to reproduce https://github.com/lebrice/SimpleParsing/issues/219#issuecomment-1437817369"""
+    with open("conf.yaml", "w") as f:
+        f.write(
+            textwrap.dedent(
+                """\
+                p: /tmp
+                hparams:
+                    xyz:
+                    - jsonl
+                """
+            )
+        )
+
+    file_config = Parameters.load_yaml("conf.yaml")
+    assert file_config == Parameters(hparams=Hparams(xyz=[LoggingTypes.JSONL]), p=Path("/tmp"))
+
+
+def test_decode_enum_saved_by_name():
+    with open("conf.yaml", "w") as f:
+        f.write(
+            textwrap.dedent(
+                """\
+                p: /tmp
+                hparams:
+                    xyz:
+                    - JSONL
+                """
+            )
+        )
+    file_config = Parameters.load("conf.yaml", load_fn=yaml.safe_load)
+    assert file_config == Parameters(hparams=Hparams(xyz=[LoggingTypes.JSONL]), p=Path("/tmp"))
+
+
+def test_round_trip():
+    p = Parameters(hparams=Hparams(xyz=[LoggingTypes.JSONL]), p=Path("/tmp"))
+    assert loads_yaml(Parameters, dumps_yaml(p)) == p
+    assert dumps_yaml(loads_yaml(Parameters, dumps_yaml(p))) == dumps_yaml(p)
+
+
+def test_decode_enum_saved_by_value_using_register():
+    from simple_parsing.helpers.serialization.decoding import register_decoding_fn
+    from simple_parsing.helpers.serialization.encoding import encode
+
+    register_decoding_fn(LoggingTypes, LoggingTypes)
+    encode.register(LoggingTypes, lambda x: x.value)
+
+    with open("conf.yaml", "w") as f:
+        f.write(
+            textwrap.dedent(
+                """\
+                p: /tmp
+                hparams:
+                    xyz:
+                    - jsonl
+                """
+            )
+        )
+
+    file_config = Parameters.load_yaml("conf.yaml")
+    assert file_config == Parameters(hparams=Hparams(xyz=[LoggingTypes.JSONL]), p=Path("/tmp"))
+
+
+def test_decode_enum_saved_by_value_using_field():
+    from simple_parsing.helpers import field
+
+    @dataclass
+    class HparamsWithField:
+        seed: int = 13
+        xyz: List[LoggingTypes] = field(
+            encoding_fn=lambda x: [e.value for e in x],
+            decoding_fn=lambda str_list: [LoggingTypes(e) for e in str_list],
+            default_factory=list,
+        )
+
+    @dataclass
+    class ParametersWithField(Serializable):
+        hparams: HparamsWithField = field(default_factory=HparamsWithField)
+        p: Optional[Path] = None
+
+    with open("conf.yaml", "w") as f:
+        f.write(
+            textwrap.dedent(
+                """\
+                p: /tmp
+                hparams:
+                    xyz:
+                    - jsonl
+                """
+            )
+        )
+
+    file_config = ParametersWithField.load_yaml("conf.yaml")
+    assert file_config == ParametersWithField(
+        hparams=HparamsWithField(xyz=[LoggingTypes.JSONL]), p=Path("/tmp")
+    )