Repository: danielgtaylor/python-betterproto

Issue:
TypeError in to_pydict with optional fields
See https://github.com/danielgtaylor/python-betterproto/issues/475#issuecomment-1512296041

<details>
<summary>Outdated</summary>

I am using betterproto to load a ton of messages at once from a binary file, however this operation is very slow so I wanted to cache/parse it into json using `json.dump` with `to_dict`, and then load it directly from json. Problem is that `from_json` is also very slow, so my idea was to use `to_dict(casing=Casing.SNAKE, include_default_values=true)` to generate a dictionary that matches the attributes of the message dataclass, and then wrap it in an utility class that redirects dict.attribute calls to dict["attribute"] so that for the consumer (which is using the message dataclasses exclusively for their data) there's no difference.

However the problem is that when serializing to dictionary, enum values are serialized based on their name instead of value. Maybe an optional parameter can be added to change this behaviour? Thank you


<details>
<summary>Utility dictionary class for redirecting attributes to items</summary>

```py
class AttrDict(dict):
    def __getattr__(self, key):
        try:
            return self[key]
        except KeyError:
            raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{key}'")

    def __setattr__(self, key, value):
        self[key] = value

    def __getitem__(self, key):
        value = super().__getitem__(key)
        if isinstance(value, dict) and not isinstance(value, AttrDict):
            value = AttrDict(value)
            self[key] = value
        if isinstance(value, list) and any(isinstance(x, dict) for x in value):
            value = [AttrDict(x) if isinstance(x, dict) else x for x in value]
            self[key] = value
        return value
```

</details>

</details>



Discussion:

Patch:
diff --git a/src/betterproto/__init__.py b/src/betterproto/__init__.py
index 82a5b14..40be27f 100644
--- a/src/betterproto/__init__.py
+++ b/src/betterproto/__init__.py
@@ -1373,6 +1373,9 @@ class Message(ABC):
                     value = [i.to_pydict(casing, include_default_values) for i in value]
                     if value or include_default_values:
                         output[cased_name] = value
+                elif value is None:
+                    if include_default_values:
+                        output[cased_name] = None
                 elif (
                     value._serialized_on_wire
                     or include_default_values


Test patch:
diff --git a/tests/test_features.py b/tests/test_features.py
index 2deef2b..940cd51 100644
--- a/tests/test_features.py
+++ b/tests/test_features.py
@@ -265,6 +265,32 @@ def test_optional_flag():
     assert Request().parse(b"\n\x00").flag is False
 
 
+def test_optional_datetime_to_dict():
+    @dataclass
+    class Request(betterproto.Message):
+        date: Optional[datetime] = betterproto.message_field(1, optional=True)
+
+    # Check dict serialization
+    assert Request().to_dict() == {}
+    assert Request().to_dict(include_default_values=True) == {"date": None}
+    assert Request(date=datetime(2020, 1, 1)).to_dict() == {
+        "date": "2020-01-01T00:00:00Z"
+    }
+    assert Request(date=datetime(2020, 1, 1)).to_dict(include_default_values=True) == {
+        "date": "2020-01-01T00:00:00Z"
+    }
+
+    # Check pydict serialization
+    assert Request().to_pydict() == {}
+    assert Request().to_pydict(include_default_values=True) == {"date": None}
+    assert Request(date=datetime(2020, 1, 1)).to_pydict() == {
+        "date": datetime(2020, 1, 1)
+    }
+    assert Request(date=datetime(2020, 1, 1)).to_pydict(
+        include_default_values=True
+    ) == {"date": datetime(2020, 1, 1)}
+
+
 def test_to_json_default_values():
     @dataclass
     class TestMessage(betterproto.Message):