LGTM Devlog 20: Python Abstract Base Class-based data/quest storage
Tags: devjournal, python
Published: 2021-01-18 15:36:54

I changed my mind about quest storage. Previously, I wanted to instantiate each quest as an object of the `Quest` class, so something like this:

```py
intro_quest = Quest("intro")
intro_quest.add_stage(...)
```

However, I realized a downside to this, which is that this abstraction is wrong for loading/unloading data. Since the game core loop has to process multiple users' quests in a loop, I would have to somehow copy this instantiated object, or have it cleanly unload user data:

```py
# inside game loop

intro_quest.load(user1_data)
intro_quest.process_whatever()

intro_quest.load(user2_data)
intro_quest.process_whatever()
```

While this works, I feel this is the wrong abstraction. Instead, I feel it is better if `intro_quest` was instead an actual Class, which means we would be able to instantiate user data as an object of this quest class, and not have to deal with unloadign data:

For example, if we subclassed Quest as IntroQuest:
```py
IntroQuest(Quest):
  def _init_(self):
    ...
```

```
# inside game loop
user1_quest = IntroQuest(user1_data)
user2_quest = IntroQuest(user2_data)
```

# ABCs
To do this, I'm going to turn the parent `Quest` class into an [Abstract Base Classes](https://docs.python.org/3/library/abc.html), which lets me define certain properties and methods which subclasses should have. ABCs were actually in [one of my earliest blog post](https://dev.to/meseta/factories-abstract-base-classes-and-python-s-new-protocols-structural-subtyping-20bm) on Dev.to!

The benefit of using an Abstract Base Class (ABC) in Python, is helps ensure the implementation for each Quest is correct - any errors in implementation - if the concrete implementation of a quest is missing needed methods that our game loop will call later, then the code will error out on instantiation, letting us know a function is missing.

So, I can re-define the same Quest from last post as an ABC, declaring some of the metadata I need as abstract class properties. I've also defined a Difficulty enum. `semver_safe` is the same as last time.

```py
from copy import deepcopy
from typing import Any, Dict, ClassVar
from abc import ABC, abstractmethod
from enum import Enum
from semver import VersionInfo # type:  ignore

from .exceptions import QuestLoadError

class Difficulty(Enum):
    RESERVED = 0
    BEGINNER = 1
    INTERMEDIATE = 2
    ADVANCED = 3
    EXPERT = 4
    HACKER = 5


def semver_safe(start: VersionInfo, dest: VersionInfo) -> bool:
    """ whether semver loading is going to be safe """
    if start.major != dest.major:
        return False

    # check it's not a downgrade of minor version
    if start.minor > dest.minor:
        return False

    return True


class Quest(ABC):
    @property
    @abstractmethod
    def version(cls) -> VersionInfo:
        ...

    @property
    @abstractmethod
    def difficulty(cls) -> Difficulty:
        return NotImplemented

    @property
    @abstractmethod
    def description(cls) -> str:
        return NotImplemented
    
    default_data: ClassVar[Dict[str, Any]] = {}
    quest_data: Dict[str, Any] = {}
    VERSION_KEY = "_version"

    def __init_subclass__(self):
        self.quest_data = deepcopy(self.default_data)

    def load(self, save_data: Dict[str, Any]) -> None:
        """ Load save data back into structure """

        # check save version is safe before upgrading
        save_semver = VersionInfo.parse(save_data[self.VERSION_KEY])
        if not semver_safe(save_semver, self.version):
            raise QuestLoadError(
                f"Unsafe version mismatch! {save_semver} -> {self.version}"
            )

        self.quest_data.update(save_data)

    def get_save_data(self) -> Dict[str, Any]:
        """ Updates save data with new version and output """

        self.quest_data[self.VERSION_KEY] = str(self.version)
        return self.quest_data

```

Our concrete implementation now looks like this:
```py
from typing import TYPE_CHECKING
from semver import VersionInfo # type:  ignore
from ..quest_system import Quest, Difficulty


class IntroQuest(Quest):
    version = VersionInfo.parse("0.1.0")
    difficulty = Difficulty.BEGINNER
    description = "The intro quest"

if TYPE_CHECKING:
    IntroQuest()
```

The `if TYPE_CHECKING` at the end is there because until you instantiate a class, it won't be checked, so I have to actually instantiate the class in the code, but also I only want to do this at type-checking time. The `typing` library therefore provides us with a `TYPE_CHECKING` boolean for this purpose.

An example of an incorrect implementation, where the `version` is missing
```py
class BrokenQuest(Quest):
    difficulty = Difficulty.BEGINNER
    description = "This quest is broken"

if TYPE_CHECKING:
    BrokenQuest()
```
Running mypy on this would give us the following error, telling us we're missing `version`
```
error: Cannot instantiate abstract class 'BrokenQuest' with abstract attribute 'version'
```

The load/save tests have been updated accordingly:

```py
def test_quest_load_fail():
    """ Tests a quest load fail due to semver mismatch """

    # generate a bad save data version
    save_data = deepcopy(DebugQuest.default_data)
    save_data[DebugQuest.VERSION_KEY] = str(DebugQuest.version.bump_major())

    # create a new game and try to load with the bad version
    quest = DebugQuest()
    with pytest.raises(QuestLoadError):
        quest.load(save_data)


def test_quest_load_save():
    """ Tests a successful load with matching semvar """

    # generate save data version
    save_data = deepcopy(DebugQuest.default_data)
    save_data[DebugQuest.VERSION_KEY] = str(DebugQuest.version)

    # create a new game and load the good version
    quest = DebugQuest()
    quest.load(save_data)
    assert quest.get_save_data() == save_data
```

It's a little clunky, as I am copying out the default_data property directly to generate save files.

# Auto-loading all the quests
The way I would like the quests to work is I add each quest as a Class (whose base class is `Quest`), and then the module automatically loads this, so that I don't have to manually maintain a list of quests somewhere.

The code I use for that is the following in the `__init__.py` file in the `quests` folder:

```py
from typing import Type
import os
import pkgutil
import importlib
import inspect

from ..system import Quest
from ..exceptions import QuestError

all_quests = {}
for _importer, _name, _ in pkgutil.iter_modules(path=[os.path.dirname(__file__)]):
    _module = importlib.import_module("." + _name, __package__)
    _classes = inspect.getmembers(_module, inspect.isclass)

    for _parent, _class in _classes:
        if Quest in _class.__bases__:
            if _class.__name__ in all_quests:
                raise ValueError(f"Duplicate quests found with name {_class.__name__}")
            all_quests[_class.__name__] = _class

def get_quest_by_name(name: str) -> Type[Quest]:
    try:
        return all_quests[name]
    except KeyError as err:
        raise QuestError(f"No quest name {name}") from err
```

This allows me to simply `from quests import all_quests` to fetch all the quests, or, use the `get_quest_by_name()` conevenience function to do the lookup

The tests can then loop through and instantiate all of the quest classes to double-check they're implemented correctly according to the abstract base class:

```py
def test_quest_class_fail():
    """ Try to load a non-existant class """
    with pytest.raises(QuestError):
        get_quest_by_name("_does not exist_")


def test_get_quest():
    """ A successful class fetch """
    assert get_quest_by_name(DebugQuest.__name__) == DebugQuest


def test_all_quest_subclasses():
    """ Instantiate all quests to check abstract base class implementation """

    for quest_class in all_quests.values():
        quest_class()  # should succeed if correctly implemented
```

This update to the quest system (still missing actual implementation of quest stages) should now be in a better position to have new quests defined