Repository: python-telegram-bot/python-telegram-bot

Issue:
[FEATURE] Improve Backward Compatibility for pickling
### What kind of feature are you missing? Where do you notice a shortcoming of PTB?

When TG deprecates attributes, we usually convert them to properties in our classes and issue a warning on access. For this, we rename `Class.attribute` to `Class.._attribute`. This leads to [problems](https://t.me/pythontelegrambotgroup/723964) on un-pickling data that was pickled with previous versions. While this is okay acccording to our [stability policy](https://docs.python-telegram-bot.org/en/v20.8/stability_policy.html#what-doesn-t-this-policy-cover), it would still be nice to handle this better for the users

### Describe the solution you'd like

In `TelegramObjejct.__setstate__`, the current logic is

https://github.com/python-telegram-bot/python-telegram-bot/blob/1cf63c26c59c3db2c938bffe26898216283b60af/telegram/_telegramobject.py#L311-L322

I would like to change this to

1. first try set_attr
2. check if the attribute name is a property
3. if yes, try setting _attributename (unless the name already starts with underscore)
4. if that fails or the attribute is not a property, put the attribute into `api_kwargs` - unless the attribute has a leading underscore. This last part is for when we finally completely remove the property - pickling with the property in place and unpickling without the property in the new version will then simply remove the attribute.

Moreover, in `_apply_api_kwargs`

https://github.com/python-telegram-bot/python-telegram-bot/blob/1cf63c26c59c3db2c938bffe26898216283b60af/telegram/_telegramobject.py#L492-L494

we can first check if tha attribute is a property

### Describe alternatives you've considered

Do nothing as we didn't promise that this works.

### Additional context

https://t.me/pythontelegrambotgroup/723964


Discussion:
As a 'victim' of this unfortunate upgrading consequence, I would like to add that - while 'doing nothing' is of course a valid course of action, it would be nice to *at least* having the un-pickling continue, skipping the deprecated attributes - even with a loss of data - instead of totally crashing and being de-facto locked out of the pickle file.

> we can first check if tha attribute is a property

Do you mean like this?
```python
        for key in list(api_kwargs.keys()):
            if getattr(self, key, True) is None:
                try:
                    setattr(self, key, api_kwargs.pop(key))
                except AttributeError:
                    if hasattr(self, key) and not key.startswith("_"):
                        setattr(self, f"_{key}", api_kwargs.pop(key))
```
Agree with approach, I can't tell if there are any downsides to this without actually coding it up, so that's what I'll do this weekend.
@harshil21 For reference: it&#x27;s also possible to directly check if something is a property: https://stackoverflow.com/a/17735709
Try-except may still be viewed as more elegant, I guess. I&#x27;ll leave that up to you:)

Resolving patch:
diff --git a/telegram/_telegramobject.py b/telegram/_telegramobject.py
--- a/telegram/_telegramobject.py
+++ b/telegram/_telegramobject.py
@@ -17,6 +17,7 @@
 # You should have received a copy of the GNU Lesser Public License
 # along with this program.  If not, see [http://www.gnu.org/licenses/].
 """Base class for Telegram Objects."""
+import contextlib
 import datetime
 import inspect
 import json
@@ -312,7 +313,20 @@ def __setstate__(self, state: Dict[str, object]) -> None:
             try:
                 setattr(self, key, val)
             except AttributeError:
-                # catch cases when old attributes are removed from new versions
+                # So an attribute was deprecated and removed from the class. Let's handle this:
+                # 1) Is the attribute now a property with no setter? Let's check that:
+                if isinstance(getattr(self.__class__, key, None), property):
+                    # It is, so let's try to set the "private attribute" instead
+                    try:
+                        setattr(self, f"_{key}", val)
+                    # If this fails as well, guess we've completely removed it. Let's add it to
+                    # api_kwargs as fallback
+                    except AttributeError:
+                        api_kwargs[key] = val
+
+                # 2) The attribute is a private attribute, i.e. it went through case 1) in the past
+                elif key.startswith("_"):
+                    continue  # skip adding this to api_kwargs, the attribute is lost forever.
                 api_kwargs[key] = val  # add it to api_kwargs as fallback
 
         # For api_kwargs we first apply any kwargs that are already attributes of the object
@@ -490,7 +504,12 @@ def _apply_api_kwargs(self, api_kwargs: JSONDict) -> None:
         """
         # we convert to list to ensure that the list doesn't change length while we loop
         for key in list(api_kwargs.keys()):
-            if getattr(self, key, True) is None:
+            # property attributes are not settable, so we need to set the private attribute
+            if isinstance(getattr(self.__class__, key, None), property):
+                # if setattr fails, we'll just leave the value in api_kwargs:
+                with contextlib.suppress(AttributeError):
+                    setattr(self, f"_{key}", api_kwargs.pop(key))
+            elif getattr(self, key, True) is None:
                 setattr(self, key, api_kwargs.pop(key))
 
     def _get_attrs_names(self, include_private: bool) -> Iterator[str]:


Test patch:
diff --git a/tests/test_telegramobject.py b/tests/test_telegramobject.py
--- a/tests/test_telegramobject.py
+++ b/tests/test_telegramobject.py
@@ -358,6 +358,68 @@ async def test_pickle_backwards_compatibility(self):
         chat.id = 7
         assert chat.id == 7
 
+    def test_pickle_handle_properties(self):
+        # Very hard to properly test, can't use a pickle file since newer versions of the library
+        # will stop having the property.
+        # The code below uses exec statements to simulate library changes. There is no other way
+        # to test this.
+        # Original class:
+        v1 = """
+class PicklePropertyTest(TelegramObject):
+    __slots__ = ("forward_from", "to_be_removed", "forward_date")
+    def __init__(self, forward_from=None, forward_date=None, api_kwargs=None):
+        super().__init__(api_kwargs=api_kwargs)
+        self.forward_from = forward_from
+        self.forward_date = forward_date
+        self.to_be_removed = "to_be_removed"
+"""
+        exec(v1, globals(), None)
+        old = PicklePropertyTest("old_val", "date", api_kwargs={"new_attr": 1})  # noqa: F821
+        pickled_v1 = pickle.dumps(old)
+
+        # After some API changes:
+        v2 = """
+class PicklePropertyTest(TelegramObject):
+    __slots__ = ("_forward_from", "_date", "_new_attr")
+    def __init__(self, forward_from=None, f_date=None, new_attr=None, api_kwargs=None):
+        super().__init__(api_kwargs=api_kwargs)
+        self._forward_from = forward_from
+        self.f_date = f_date
+        self._new_attr = new_attr
+    @property
+    def forward_from(self):
+        return self._forward_from
+    @property
+    def forward_date(self):
+        return self.f_date
+    @property
+    def new_attr(self):
+        return self._new_attr
+        """
+        exec(v2, globals(), None)
+        v2_unpickle = pickle.loads(pickled_v1)
+        assert v2_unpickle.forward_from == "old_val" == v2_unpickle._forward_from
+        with pytest.raises(AttributeError):
+            # New attribute should not be available either as is always the case for pickle
+            v2_unpickle.forward_date
+        assert v2_unpickle.new_attr == 1 == v2_unpickle._new_attr
+        assert not hasattr(v2_unpickle, "to_be_removed")
+        assert v2_unpickle.api_kwargs == {"to_be_removed": "to_be_removed"}
+        pickled_v2 = pickle.dumps(v2_unpickle)
+
+        # After PTB removes the property and the attribute:
+        v3 = """
+class PicklePropertyTest(TelegramObject):
+    __slots__ = ()
+    def __init__(self, api_kwargs=None):
+        super().__init__(api_kwargs=api_kwargs)
+"""
+        exec(v3, globals(), None)
+        v3_unpickle = pickle.loads(pickled_v2)
+        assert v3_unpickle.api_kwargs == {"to_be_removed": "to_be_removed"}
+        assert not hasattr(v3_unpickle, "_forward_from")
+        assert not hasattr(v3_unpickle, "_new_attr")
+
     def test_deepcopy_telegram_obj(self, bot):
         chat = Chat(2, Chat.PRIVATE)
         user = User(3, "first_name", False)