Repository: projectmesa/mesa
Pull request #2291 (2024-09-11 13:33:49)

Add support for Observables to MESA
see #2281 


This PR adds various new classes to MESA. The key public-facing ones are, at the moment, `Observable,` `ObservableList,` and `HasObservables`. `Observable` is a descriptor that is used to declare a given attribute to be observable. Being observable means changing the attribute using the assignment operator `=` willl result in the sending of a signal. Callbacks can subscribe to this signal. `ObservableList` is a proof of principle on how the idea of an observable can be expanded to collections. Future extensions could include, e.g., `ObservableDict`. `HasObservables` is a mixin that can be used with e.g., `Model` or `Agent`.  It is required if you want to use `Observable` or `ObservableList`. `HasObservable` contains the key logic for subscribing to signals and the emitting of signals.

`Observable` emits a `change` signal when its attribute value is changed via the assignment operator (i.e., `=`). `ObservableList` at the moment defines five signal types: `"replace", "remove", "insert", "append", "change"`. `"change"` is emmited when the entire list is changed via the assignment operator (i.e., `=`). The other signals are tied to the respective list operations. Currently, signals are following the approach used in traitlets. The main reason for making the signals identical to traitlets is that it should make integration with solara easier, which already supports traitlets.

The emitted signals are instances of an `AttributeDict`. This is in line with Traitlets. It is just a dict with attribute-style access to the fields. By default, a signal has four fields: "owner", "type", "old", and "new". It is possible to add additiona fields if the user wants. For example, the `ObservableList ` signals (accept `"change"`) all also include `"index"`.

I have also added a WIP `Computed` and `Computable` approach. In javascript-style signals, but, e.g., also in [solara](https://solara.dev/documentation/api/utilities/computed), a Computed is a callable that is dependent on one or more signals. When it receives a signal that any of its dependencies has changed, it will mark itself as being outdated. It will only update its value if it is called to do so. This can be used to set up a hybrid push/pull structure where Obervables push signals, but a Computed is only recalculated on a pull if it knows its underlying data has changed. A Computed is also Observable, so it will emit a signal if it knows it is outdated. This allows the chaining of observables and computed's. 

# API
Below, there is a quick sketch of the resulting API. We define Obervable attributes at the class level. We can use the attribute normally in the rest of the code. 
```python

class MyAgent(Agent, HasObservables):
    wealth = Observable()
    income = Observable() 

   def __init__(self, model)
       super().__init__(model)
       self.wealth = 5  # we can use wealth as a normal attribute, Observable hides all the magic

class MyModel(Model, HasObservables):
    gini = Computable()
    agent_wealth = ObservableList()

    def __init__(self, random=random):
        super().__init__(random=random)
        
        agent_wealth = [agent.wealth for agent in self.agents]  # ObservableList can be used as a normal list
        self.gini = Computed(calculate_gini, agent_wealth)

model = MyModel()
agent = MyAgent(model)

# whenever wealth is changed some_callback_function will be called
agent.observe("wealth", "change", some_callback_function)

# Whenever either income or wealth is changed, generic_callback_function
# is called. All() is a new helper class
agent.observe(All(), "change", generic_callback_function)

# observables is a class-level attribute with all observables defined for the class
print(agent.observables)

```



# Implementation Details
After a long discussion in #2281, I decided to implement everything from scratch. Existing libraries like traitlets or psygnal add too much overhead because they are not tailored to the specific use case of ABMs with potentially many updates to an attribute. 

# future extensions
This PR puts in place the basic machinery, but there are ample options for further improvements:

- At the moment, anything that wants to emit a signal needs to be registered as being observable. Ideally, something like `model.agents` should emit `agent_added` and `agent_removed` signals. At the moment, there is no machinery in place to make it easy to add this functionality. I don't think it is overly difficult, but I want to take a closer look soon.
- I currently include only an ObservableList as an example of how we can use observables to also observe changes in collections. Other collections can easily be added if and when desired.
- The current implementation of `Computed` is heavily inspired by signals and Solara. It involves the autodiscovery of dependencies. The params library, instead, forces users to pass the list of dependencies explicitly. I am not sure which approach is better. 
- I don't like the current API for `Computed`. It requires the user to do 2 things: declare an attribute as being computable at the class level and assign a computed instance to this attribute during object initialization. If I fix the first bullet, it might be possible to improve this API and eliminate the `Computable` declaration.  
- The way in which the signals that a given object can emit is declared is less than ideal. `Observable` and `ObservableList` both have a list of strings that enumerate the signals that can be sent. However, there is no explicit declaration anywhere of the fields that are part of any signal. A signal is an instance of an `AttributeDict`, following traitlets, but it is completely up to the user to decide what fields will be in it. This means that the user is dependend on the docstring, or some test code to discover what is in a signal, rather than that it can be declared and checked explicitly via e.g., a type checker. 
- At the moment, the `notify` method, which is responsible for emitting the signals requires the name of the observable (`name`), the old value (`old`), the new value (`new`), and the signal type (`type`). Further elements can be added as desired. For example, most signals from `ObservableList` include `index`. However, for e.g., an `append` signal, having `old` in there makes little sense. 




EwoutH: Could you update or rebase this branch on `main`?
quaquel: The most basic version of `Computed` now works. See [this simple Boltzman example for the draft API](https://github.com/projectmesa/mesa/blob/43b2a7bad937f898d2c42fe094cd7d9d65974f29/mesa/experimental/signals/test.py). 

What works is a simple chain of observable --> computed --> computed. So if the observable is updated, this emits a signal to the first computed, which then relays it to its children. In the example, this is `agent.wealth` --> `model.agent_wealth` --> `model.gini`. Now, when the data collector wants to retrieve `model.gini`, the Computed/Computable checks whether it is dirty (i.e., is outdated) and, if so, will recalculate. This requires, in turn, calling `model.agent_wealth`, which does the same check. 

What does not work at the moment is the correct detection of cycles. So Computable A depends on Computable B, which depends on C which in turn depends on A. At the moment, I guess this will trigger some kind of infinite recursion error. I hope to fix this soonish.
Corvince: Awesome stuff! Just by looking at the code I can tell you are having a lot of fun with this one, am I right?

I am just slightly worried about the code complexity this is getting to. When you are done with everything else you want to do with this, could you maybe add a very simple data collector prototype to the example? I would like to then replicate the example using the param library, to have a more holistic comparison about the performance difference. So don't think too much about a possible api for the data collection, just some object where data is collected/updated/persisted once per step
quaquel: > Awesome stuff! Just by looking at the code I can tell you are having a lot of fun with this one, am I right?
> 
> I am just slightly worried about the code complexity this is getting to. When you are done with everything else you want to do with this, could you maybe add a very simple data collector prototype to the example? I would like to then replicate the example using the param library, to have a more holistic comparison about the performance difference. So don't think too much about a possible api for the data collection, just some object where data is collected/updated/persisted once per step

It's been fun indeed. The basic Observable stuff is quite straightforward and not that complicated. Complications arise due to 2 things: adding in Computable, and avoiding memory leaks. 

I have followed the draft signals library in the current setup of Computable. This means that dependencies are automatically and on the fly discovered and can also dynamically change over the course of a running program. I am not sure this is actually needed for something like mesa. The alternative I see (and just noticed is used by `params.bind`) is to explicitly indicate the dependencies. This might make certain parts of the code quite a bit shorter and cleaner. But you freeze the dependencies explicitly. 

Memory leaks are another issue. Subscribing to signals involves references between objects. Ideally, you want all of this to be weak references to ensure that, for example, an agent who is listening to some other agent can still be properly removed from the simulation. Part of the tricky thing here is that weakreffed bound methods differ from normal weakrefs, and the off-the-shelve weakref collections (dict/set) don't play nice with bound methods. I hope to move some stuff into a separate utils package to keep the main code cleaner.

Regardless, a comparison to params would indeed be very nice. 
quaquel: > I am just slightly worried about the code complexity this is getting to. When you are done with everything else you want to do with this, could you maybe add a very simple data collector prototype to the example? I would like to then replicate the example using the param library, to have a more holistic comparison about the performance difference.

The code is basically ready for such a comparison. I have started doing some timings myself to see what the overhead is. It seems that the autodiscovery of Computable can become quite expensive. So @Corvince, do you think we need this or might be get by by the approach used by [`param.bind`](https://param.holoviz.org/reference/param/generated/param.bind.html#param.bind) to explicitly indicated what the arguments and keyword arguments to the callable are that must be observed?
Corvince: I started to prepare such comparison, and from the documentation it seemed straightforward to do in `param`, but it actually isn't. I also experienced some errors, that seemed to be caused by some initialization clash between param and agents. What I want to say is, it is not that easy and seemed quite advanced and different than "typical" python. I hope to still finish the comparison, but I am even less convinced that `param` is the way to go (Except for the param specification itself, that seems to be super useful). 

Regarding the necessity of "autodiscovery": I would say it depends on the API implications. I always prefer to stay simple and expand later, so even without "computables" this PR would be a nice start. Or do you want to keep computables, but make the dependency explicit? 


quaquel: > Regarding the necessity of "autodiscovery": I would say it depends on the API implications. I always prefer to stay simple and expand later, so even without "computables" this PR would be a nice start. Or do you want to keep computables, but make the dependency explicit?

I don't know yet. At a minimum, I want to give the  `param.bind` approach to computables a go to see how difficult it would be to implement and to see the performance implications. 

The rest of this PR is indeed ready for feedback and discussion. 
Corvince: I thought about this whole issue a lot more and the distinction between event listeners, signals, computables, etc. Here is how I would like this to work from the point of a data collector. Please let me know how much this aligns with the current state, because I kind of lost the current state due to the rising complexity. This by no means should sound authorative, this is just how I currently imagine things. 

So lets stay with the boltzmann example and asume we want to gather two kinds of data: The individual wealth of each agent and the gini coefficient. 

Now in an observable world the data collector observes each agents "wealth" and is notified each time that changes. This is good and simple if the attribute doesnt change very often. But when the wealth changes multiple times during each "step" the datacollector gets flooded with information that it doesnt need, because it wants to collect the data only once after each step. 

So in that case I think it would be more desirable to only notify the data collector once to let it know the wealth changed. After that the data collector marks the wealth as "dirty" and stops listening to events until it needs the current value. That means at the end of the step the datacollector retrieves the current value from the agent and starts to listen again for any changes. 
That is at least how I understand the signals API, is that correct so far?

Now for the gini coefficient the simplest solution for the data collector would be to just store it as a @property on the model. Then the data collector could just collect it as any other attribute and its lazily evaluated only when needed. However in a signal world it would probably be a computed value, that itself listens to each agents wealth. With observables this would become expensive again, but with computed it should only listen for the first change event and then stop listening. 

Is this somewhat along the lines you implemented it? Because circling back to the question if "computed" properties are needed, I don't know how else derived values like the gini coefficient should work?

 
quaquel: @Corvince yes this tracks largely with how I look at it. A few remarks

1. Currently, I don't unsubscribe upon marking a computable as dirty. I'll test the performance implications of this. The issue is that you would have to unsubscribe from many agents. It might be cheaper to receive all messages because the handling is cheap (I am already dirty, so ignore the signal). The draft python signals library did not ussubscribe after the first signal and from looking at Solara, I think that it also does not unsubscribe. 
2. Most signal libraries can pause or stop the emissions of signals. This might be a complementary mechanism, currently not implemented, that could be used to avoid flooding observers with signals. For example, if you have some agent method that repeatedly updates an observable attribute, observers should only receive a signal for the last and final update. In such a case, having a way of pausing signals is useful. Various Python libraries use a context manager for this.
3. Yes `Gini` is a `Computed` and would take, ideally, the data collected agent wealth as its input. The data collected agent wealth would then also be a `Computed`, and signals would propagate. So, upon the first `agent.wealth` update, the data collector for this would receive a signal, mark itself as dirty, and broadcast this to its dependents. Gini, as a dependent, would then also mark itself as dirty. This is in essence up and working already for the Boltzman benchmark model. 
Corvince: Thanks for the additional insights! 

1. Yes that makes sense I guess. Much simpler to just ingore messages
2. At some stage that might make sense. I will come back to that below
3. Interesting order. That makes a lot more sense. In my head it was agent.wealth -> gini -> data collector. So both data collector and gini would observer agent.wealth. But yours makes more sense and is less expensive.


I am still a bit skeptical about the performance implications. Overall seems like a lot of additional operations in case of "volatile" attributes that change often. Also with lots of agents its a lot of subscriptions. I am wondering if we can somewhat use `agentset` as a middle man. So data collector would subscribe to agentset. Agentset already is connected to all agents, so maybe we can use that connections to pass agent messages to agentset. And Agentset than relays a single message with all the agents/attributes that should be marked as dirty (or the latest value). There, supressing messages while executing a single "do" or "apply" method might make sense.
quaquel: To give you a sense of the overhead, Here is a quick benchmark based on the current version of the code for the Boltzman wealth model. This benchmark is not based on something that is feature-complete. I have added a Table to track agent wealth. This table is then used in the gini calculation. So this table at all times reflects the value of `wealth` for all agents. 

In this version, gini is not a computed because making that work with this table is a bit tricky (which suggests that the computed stuff needs a bit more thought to make it easy to extend and use). 

```python
class Table:
    """This table tracks the wealth of agents."""

    def __init__(self, model: BoltzmannWealth):
        self.data = {}
        for agent in model.agents:
            agent.observe("wealth", "on_change", self.update)
            self.data[agent.unique_id] = agent.wealth

    def update(self, signal):
        """handler for signal that updates the wealth for the agent in the table"""
        self.data[signal.owner.unique_id] = signal.new_value

    def get(self):
        """return the wealth of agents"""
        return self.data.values()

```

So, the startup is much slower (and I have not tried to improve this at all yet). Overhead while running is present but not bad. And remember, Boltzmann is a worst-case example, and none of the code here has been profiled. I have analyzed some parts a little bit for performance, but not in any great depth. 

| Model           | Size   | Init time [95% CI]            | Run time [95% CI]          |
|-----------------|--------|-------------------------------|----------------------------|
| BoltzmannWealth | small  | 🔴 +302.2% [+295.2%, +309.0%] | 🔴 +8.6% [+8.2%, +9.1%]    |
| BoltzmannWealth | large  | 🔴 +407.5% [+375.9%, +467.4%] | 🔴 +24.8% [+23.9%, +25.8%] |


So why is startup so slow? There is quite some overhead from initializing the agents with the additional data structures for emitting signals etc. Moreover, at the moment of object instantiation, the code figures out what observables have been defined and adds these to the instance. This is silly, and I have to figure out a way to do this once during class creation (instead of 100 times for each agent).  Also, the percentages mislead for startup time. We go from roughly 0.00026 seconds to 0.001 seconds. 

Corvince: I am positively surprised by the benchmark! It is really not bad for a worst case scenario. I agree that startup times are not so relevant and can probably be improved. @EwoutH maybe it makes sense to also include absolute times in the comment? I think its different for different models, but in case where startup time makes up less than 1% of total runtime its totally negligible.

But can you maybe run these benchmarks also for the "large" model? I want to see how it scales
quaquel: > But can you maybe run these benchmarks also for the "large" model? I want to see how it scales

I am not sure what you mean. The benchmark results include both the small and the large version of the Boltzman wealth model.
EwoutH: I like the API!
github-actions[bot]: Performance benchmarks:

| Model           | Size   | Init time [95% CI]      | Run time [95% CI]        |
|-----------------|--------|-------------------------|--------------------------|
| BoltzmannWealth | small  | 🔵 -2.1% [-3.5%, -0.6%] | 🔵 -2.6% [-2.8%, -2.4%]  |
| BoltzmannWealth | large  | 🔵 -0.9% [-1.5%, -0.3%] | 🔵 -3.8% [-5.8%, -1.8%]  |
| Schelling       | small  | 🟢 -3.6% [-4.0%, -3.3%] | 🔵 -1.8% [-2.1%, -1.4%]  |
| Schelling       | large  | 🔵 -3.0% [-3.4%, -2.6%] | 🟢 -5.7% [-8.2%, -3.2%]  |
| WolfSheep       | small  | 🔵 -0.3% [-0.9%, +0.3%] | 🔵 -0.5% [-1.1%, +0.2%]  |
| WolfSheep       | large  | 🔵 -3.4% [-4.5%, -2.0%] | 🟢 -9.1% [-12.3%, -5.1%] |
| BoidFlockers    | small  | 🔵 -2.3% [-3.2%, -1.3%] | 🔵 -0.9% [-2.0%, +0.2%]  |
| BoidFlockers    | large  | 🔵 -1.3% [-2.0%, -0.4%] | 🔵 -0.8% [-1.6%, -0.1%]  |

quaquel: > Let us know what you want to complete before we merge.

I want to do a bit of further code cleaning (like the tests.py file) and improve code coverage a bit further. Beyond that, any feedback or ideas are welcome here or in #2281.
Corvince: Not directly related to this PR, but I just watched [a video](https://www.youtube.com/watch?v=RHTP_NZcnl0) about [alien-signals](https://github.com/stackblitz/alien-signals/blob/master/src/system.ts), a new signals library for JavaScript. Its supposed to be very fast and will probably be used by Vue.js. Vue.js is interesting because it previously switched from a push-based system to a pull-based system and alien-signals is now push-pull-based. So maybe that is interesting for you @quaquel. 
quaquel: Thanks, I'll take a look. There might be useful ideas that we can borrow. 

Patch:
diff --git a/codecov.yaml b/codecov.yaml
index a19c3dae..5aa4e0da 100644
--- a/codecov.yaml
+++ b/codecov.yaml
@@ -7,7 +7,6 @@ coverage:
 
 ignore:
   - "benchmarks/**"
-  - "mesa/experimental/**"
   - "mesa/visualization/**"
 
 comment: off
diff --git a/mesa/experimental/__init__.py b/mesa/experimental/__init__.py
index 42a510cb..069e418a 100644
--- a/mesa/experimental/__init__.py
+++ b/mesa/experimental/__init__.py
@@ -1,5 +1,5 @@
 """Experimental init."""
 
-from mesa.experimental import cell_space, devs
+from mesa.experimental import cell_space, devs, mesa_signals
 
-__all__ = ["cell_space", "devs"]
+__all__ = ["cell_space", "devs", "mesa_signals"]
diff --git a/mesa/experimental/mesa_signals/__init__.py b/mesa/experimental/mesa_signals/__init__.py
new file mode 100644
index 00000000..a3a0b805
--- /dev/null
+++ b/mesa/experimental/mesa_signals/__init__.py
@@ -0,0 +1,13 @@
+"""Functionality for Observables."""
+
+from .mesa_signal import All, Computable, Computed, HasObservables, Observable
+from .observable_collections import ObservableList
+
+__all__ = [
+    "Observable",
+    "ObservableList",
+    "HasObservables",
+    "All",
+    "Computable",
+    "Computed",
+]
diff --git a/mesa/experimental/mesa_signals/mesa_signal.py b/mesa/experimental/mesa_signals/mesa_signal.py
new file mode 100644
index 00000000..18128ce3
--- /dev/null
+++ b/mesa/experimental/mesa_signals/mesa_signal.py
@@ -0,0 +1,470 @@
+"""Core classes for Observables."""
+
+from __future__ import annotations
+
+import contextlib
+import functools
+import weakref
+from abc import ABC, abstractmethod
+from collections import defaultdict, namedtuple
+from collections.abc import Callable
+from typing import Any
+
+from mesa.experimental.mesa_signals.signals_util import AttributeDict, create_weakref
+
+__all__ = ["Observable", "HasObservables", "All", "Computable"]
+
+_hashable_signal = namedtuple("_HashableSignal", "instance name")
+
+CURRENT_COMPUTED: Computed | None = None  # the current Computed that is evaluating
+PROCESSING_SIGNALS: set[tuple[str,]] = set()
+
+
+class BaseObservable(ABC):
+    """Abstract base class for all Observables."""
+
+    @abstractmethod
+    def __init__(self, fallback_value=None):
+        """Initialize a BaseObservable."""
+        super().__init__()
+        self.public_name: str
+        self.private_name: str
+
+        # fixme can we make this an inner class enum?
+        #  or some SignalTypes helper class?
+        #  its even more complicated. Ideally you can define
+        #  signal_types throughout the class hierarchy and they are just
+        #  combined together.
+        #  while we also want to  make sure that any signal being emitted is valid for that class
+        self.signal_types: set = set()
+        self.fallback_value = fallback_value
+
+    def __get__(self, instance: HasObservables, owner):
+        value = getattr(instance, self.private_name)
+
+        if CURRENT_COMPUTED is not None:
+            # there is a computed dependent on this Observable, so let's add
+            # this Observable as a parent
+            CURRENT_COMPUTED._add_parent(instance, self.public_name, value)
+
+            # fixme, this can be done more cleanly
+            #  problem here is that we cannot use self (i.e., the observable), we need to add the instance as well
+            PROCESSING_SIGNALS.add(_hashable_signal(instance, self.public_name))
+
+        return value
+
+    def __set_name__(self, owner: HasObservables, name: str):
+        self.public_name = name
+        self.private_name = f"_{name}"
+        # owner.register_observable(self)
+
+    @abstractmethod
+    def __set__(self, instance: HasObservables, value):
+        # this only emits an on change signal, subclasses need to specify
+        # this in more detail
+        instance.notify(
+            self.public_name,
+            getattr(instance, self.private_name, self.fallback_value),
+            value,
+            "change",
+        )
+
+    def __str__(self):
+        return f"{self.__class__.__name__}: {self.public_name}"
+
+
+class Observable(BaseObservable):
+    """Observable class."""
+
+    def __init__(self, fallback_value=None):
+        """Initialize an Observable."""
+        super().__init__(fallback_value=fallback_value)
+
+        self.signal_types: set = {
+            "change",
+        }
+
+    def __set__(self, instance: HasObservables, value):  # noqa D103
+        if (
+            CURRENT_COMPUTED is not None
+            and _hashable_signal(instance, self.public_name) in PROCESSING_SIGNALS
+        ):
+            raise ValueError(
+                f"cyclical dependency detected: Computed({CURRENT_COMPUTED.name}) tries to change "
+                f"{instance.__class__.__name__}.{self.public_name} while also being dependent it"
+            )
+
+        super().__set__(instance, value)  # send the notify
+        setattr(instance, self.private_name, value)
+
+        PROCESSING_SIGNALS.clear()  # we have notified our children, so we can clear this out
+
+
+class Computable(BaseObservable):
+    """A Computable that is depended on one or more Observables.
+
+    .. code-block:: python
+
+       class MyAgent(Agent):
+           wealth = Computable()
+
+           def __init__(self, model):
+                super().__init__(model)
+                wealth = Computed(func, args, kwargs)
+
+    """
+
+    # fixme, with new _register_observable thing
+    #  we can do computed without a descriptor, but then you
+    #  don't have attribute like access, you would need to do a call operation to get the value
+
+    def __init__(self):
+        """Initialize a Computable."""
+        super().__init__()
+
+        # fixme have 2 signal: change and is_dirty?
+        self.signal_types: set = {
+            "change",
+        }
+
+    def __get__(self, instance, owner):  # noqa: D105
+        computed = getattr(instance, self.private_name)
+        old_value = computed._value
+
+        if CURRENT_COMPUTED is not None:
+            CURRENT_COMPUTED._add_parent(instance, self.public_name, old_value)
+
+        new_value = computed()
+
+        if new_value != old_value:
+            instance.notify(
+                self.public_name,
+                old_value,
+                new_value,
+                "change",
+            )
+            return new_value
+        else:
+            return old_value
+
+    def __set__(self, instance: HasObservables, value: Computed):  # noqa D103
+        if not isinstance(value, Computed):
+            raise ValueError("value has to be a Computable instance")
+
+        setattr(instance, self.private_name, value)
+        value.name = self.public_name
+        value.owner = instance
+        getattr(
+            instance, self.public_name
+        )  # force evaluation of the computed to build the dependency graph
+
+
+class Computed:
+    def __init__(self, func: Callable, *args, **kwargs):
+        self.func = func
+        self.args = args
+        self.kwargs = kwargs
+        self._is_dirty = True
+        self._first = True
+        self._value = None
+        self.name: str = ""  # set by Computable
+        self.owner: HasObservables  # set by Computable
+
+        self.parents: weakref.WeakKeyDictionary[HasObservables, dict[str, Any]] = (
+            weakref.WeakKeyDictionary()
+        )
+
+    def __str__(self):
+        return f"COMPUTED: {self.name}"
+
+    def _set_dirty(self, signal):
+        if not self._is_dirty:
+            self._is_dirty = True
+            self.owner.notify(self.name, self._value, None, "change")
+
+    def _add_parent(
+        self, parent: HasObservables, name: str, current_value: Any
+    ) -> None:
+        """Add a parent Observable.
+
+        Args:
+            parent: the HasObservable instance to which the Observable belongs
+            name: the public name of the Observable
+            current_value: the current value of the Observable
+
+        """
+        parent.observe(name, All(), self._set_dirty)
+
+        try:
+            self.parents[parent][name] = current_value
+        except KeyError:
+            self.parents[parent] = {name: current_value}
+
+    def _remove_parents(self):
+        """Remove all parent Observables."""
+        # we can unsubscribe from everything on each parent
+        for parent in self.parents:
+            parent.unobserve(All(), All(), self._set_dirty)
+
+    def __call__(self):
+        global CURRENT_COMPUTED  # noqa: PLW0603
+
+        if self._is_dirty:
+            changed = False
+
+            if self._first:
+                # fixme might be a cleaner solution for this
+                #  basically, we have no parents.
+                changed = True
+                self._first = False
+
+            # we might be dirty but values might have changed
+            # back and forth in our parents so let's check to make sure we
+            # really need to recalculate
+            if not changed:
+                for parent in self.parents.keyrefs():
+                    # does parent still exist?
+                    if parent := parent():
+                        # if yes, compare old and new values for all
+                        # tracked observables on this parent
+                        for name, old_value in self.parents[parent].items():
+                            new_value = getattr(parent, name)
+                            if new_value != old_value:
+                                changed = True
+                                break  # we need to recalculate
+                        else:
+                            # trick for breaking cleanly out of nested for loops
+                            # see https://stackoverflow.com/questions/653509/breaking-out-of-nested-loops
+                            continue
+                        break
+                    else:
+                        # one of our parents no longer exists
+                        changed = True
+                        break
+
+            if changed:
+                # the dependencies of the computable function might have changed
+                # so, we rebuilt
+                self._remove_parents()
+
+                old = CURRENT_COMPUTED
+                CURRENT_COMPUTED = self
+
+                try:
+                    self._value = self.func(*self.args, **self.kwargs)
+                except Exception as e:
+                    raise e
+                finally:
+                    CURRENT_COMPUTED = old
+
+            self._is_dirty = False
+
+        return self._value
+
+
+class All:
+    """Helper constant to subscribe to all Observables."""
+
+    def __init__(self):  # noqa: D107
+        self.name = "all"
+
+    def __copy__(self):  # noqa: D105
+        return self
+
+    def __deepcopy__(self, memo):  # noqa: D105
+        return self
+
+
+class HasObservables:
+    """HasObservables class."""
+
+    # we can't use a weakset here because it does not handle bound methods correctly
+    # also, a list is faster for our use case
+    subscribers: dict[str, dict[str, list]]
+    observables: dict[str, set[str]]
+
+    def __init__(self, *args, **kwargs) -> None:
+        """Initialize a HasObservables."""
+        super().__init__(*args, **kwargs)
+        self.subscribers = defaultdict(functools.partial(defaultdict, list))
+        self.observables = dict(descriptor_generator(self))
+
+    def _register_signal_emitter(self, name: str, signal_types: set[str]):
+        """Helper function to register an Observable.
+
+        This method can be used to register custom signals that are emitted by
+        the class for a given attribute, but which cannot be covered by the Observable descriptor
+
+        Args:
+            name: the name of the signal emitter
+            signal_types: the set of signals that might be emitted
+
+        """
+        self.observables[name] = signal_types
+
+    def observe(
+        self,
+        name: str | All,
+        signal_type: str | All,
+        handler: Callable,
+    ):
+        """Subscribe to the Observable <name> for signal_type.
+
+        Args:
+            name: name of the Observable to subscribe to
+            signal_type: the type of signal on the Observable to subscribe to
+            handler: the handler to call
+
+        Raises:
+            ValueError: if the Observable <name> is not registered or if the Observable
+            does not emit the given signal_type
+
+        """
+        # fixme should name/signal_type also take a list of str?
+        if not isinstance(name, All):
+            if name not in self.observables:
+                raise ValueError(
+                    f"you are trying to subscribe to {name}, but this Observable is not known"
+                )
+            else:
+                names = [
+                    name,
+                ]
+        else:
+            names = self.observables.keys()
+
+        for name in names:
+            if not isinstance(signal_type, All):
+                if signal_type not in self.observables[name]:
+                    raise ValueError(
+                        f"you are trying to subscribe to a signal of {signal_type} "
+                        f"on Observable {name}, which does not emit this signal_type"
+                    )
+                else:
+                    signal_types = [
+                        signal_type,
+                    ]
+            else:
+                signal_types = self.observables[name]
+
+            ref = create_weakref(handler)
+            for signal_type in signal_types:
+                self.subscribers[name][signal_type].append(ref)
+
+    def unobserve(self, name: str | All, signal_type: str | All, handler: Callable):
+        """Unsubscribe to the Observable <name> for signal_type.
+
+        Args:
+            name: name of the Observable to unsubscribe from
+            signal_type: the type of signal on the Observable to unsubscribe to
+            handler: the handler that is unsubscribing
+
+        """
+        names = (
+            [
+                name,
+            ]
+            if not isinstance(name, All)
+            else self.observables.keys()
+        )
+
+        for name in names:
+            # we need to do this here because signal types might
+            # differ for name so for each name we need to check
+            if isinstance(signal_type, All):
+                signal_types = self.observables[name]
+            else:
+                signal_types = [
+                    signal_type,
+                ]
+            for signal_type in signal_types:
+                with contextlib.suppress(KeyError):
+                    remaining = []
+                    for ref in self.subscribers[name][signal_type]:
+                        if subscriber := ref():  # noqa: SIM102
+                            if subscriber != handler:
+                                remaining.append(ref)
+                    self.subscribers[name][signal_type] = remaining
+
+    def clear_all_subscriptions(self, name: str | All):
+        """Clears all subscriptions for the observable <name>.
+
+        if name is All, all subscriptions are removed
+
+        Args:
+            name: name of the Observable to unsubscribe for all signal types
+
+        """
+        if not isinstance(name, All):
+            with contextlib.suppress(KeyError):
+                del self.subscribers[name]
+                # ignore when unsubscribing to Observables that have no subscription
+        else:
+            self.subscribers = defaultdict(functools.partial(defaultdict, list))
+
+    def notify(
+        self,
+        observable: str,
+        old_value: Any,
+        new_value: Any,
+        signal_type: str,
+        **kwargs,
+    ):
+        """Emit a signal.
+
+        Args:
+            observable: the public name of the observable emitting the signal
+            old_value: the old value of the observable
+            new_value: the new value of the observable
+            signal_type: the type of signal to emit
+            kwargs: additional keyword arguments to include in the signal
+
+        """
+        signal = AttributeDict(
+            name=observable,
+            old=old_value,
+            new=new_value,
+            owner=self,
+            type=signal_type,
+            **kwargs,
+        )
+
+        self._mesa_notify(signal)
+
+    def _mesa_notify(self, signal: AttributeDict):
+        """Send out the signal.
+
+        Args:
+        signal: the signal
+
+        Notes:
+        signal must contain name and type attributes because this is how observers are stored.
+
+        """
+        # we put this into a helper method, so we can emit signals with other fields
+        # then the default ones in notify.
+        observable = signal.name
+        signal_type = signal.type
+
+        # because we are using a list of subscribers
+        # we should update this list to subscribers that are still alive
+        observers = self.subscribers[observable][signal_type]
+        active_observers = []
+        for observer in observers:
+            if active_observer := observer():
+                active_observer(signal)
+                active_observers.append(observer)
+        # use iteration to also remove inactive observers
+        self.subscribers[observable][signal_type] = active_observers
+
+
+def descriptor_generator(obj) -> [str, BaseObservable]:
+    """Yield the name and signal_types for each Observable defined on obj."""
+    # we need to traverse the entire class hierarchy to properly get
+    # also observables defined in super classes
+    for base in type(obj).__mro__:
+        base_dict = vars(base)
+
+        for entry in base_dict.values():
+            if isinstance(entry, BaseObservable):
+                yield entry.public_name, entry.signal_types
diff --git a/mesa/experimental/mesa_signals/observable_collections.py b/mesa/experimental/mesa_signals/observable_collections.py
new file mode 100644
index 00000000..1111a8e6
--- /dev/null
+++ b/mesa/experimental/mesa_signals/observable_collections.py
@@ -0,0 +1,126 @@
+"""This module defines observable collections classes.
+
+Observable collections behave like Observable but then for collections.
+
+
+"""
+
+from collections.abc import Iterable, MutableSequence
+from typing import Any
+
+from .mesa_signal import BaseObservable, HasObservables
+
+__all__ = [
+    "ObservableList",
+]
+
+
+class ObservableList(BaseObservable):
+    """An ObservableList that emits signals on changes to the underlying list."""
+
+    def __init__(self):
+        """Initialize the ObservableList."""
+        super().__init__()
+        self.signal_types: set = {"remove", "replace", "change", "insert", "append"}
+        self.fallback_value = []
+
+    def __set__(self, instance: "HasObservables", value: Iterable):
+        """Set the value of the descriptor attribute.
+
+        Args:
+            instance: The instance on which to set the attribute.
+            value: The value to set the attribute to.
+
+        """
+        super().__set__(instance, value)
+        setattr(
+            instance,
+            self.private_name,
+            SignalingList(value, instance, self.public_name),
+        )
+
+
+class SignalingList(MutableSequence[Any]):
+    """A basic lists that emits signals on changes."""
+
+    __slots__ = ["owner", "name", "data"]
+
+    def __init__(self, iterable: Iterable, owner: HasObservables, name: str):
+        """Initialize a SignalingList.
+
+        Args:
+            iterable: initial values in the list
+            owner: the HasObservables instance on which this list is defined
+            name: the attribute name to which  this list is assigned
+
+        """
+        self.owner: HasObservables = owner
+        self.name: str = name
+        self.data = list(iterable)
+
+    def __setitem__(self, index: int, value: Any) -> None:
+        """Set item to index.
+
+        Args:
+            index: the index to set item to
+            value: the item to set
+
+        """
+        old_value = self.data[index]
+        self.data[index] = value
+        self.owner.notify(self.name, old_value, value, "replace", index=index)
+
+    def __delitem__(self, index: int) -> None:
+        """Delete item at index.
+
+        Args:
+            index: The index of the item to remove
+
+        """
+        old_value = self.data
+        del self.data[index]
+        self.owner.notify(self.name, old_value, None, "remove", index=index)
+
+    def __getitem__(self, index) -> Any:
+        """Get item at index.
+
+        Args:
+            index: The index of the item to retrieve
+
+        Returns:
+            the item at index
+        """
+        return self.data[index]
+
+    def __len__(self) -> int:
+        """Return the length of the list."""
+        return len(self.data)
+
+    def insert(self, index, value):
+        """Insert value at index.
+
+        Args:
+            index: the index to insert value into
+            value: the value to insert
+
+        """
+        self.data.insert(index, value)
+        self.owner.notify(self.name, None, value, "insert", index=index)
+
+    def append(self, value):
+        """Insert value at index.
+
+        Args:
+            index: the index to insert value into
+            value: the value to insert
+
+        """
+        index = len(self.data)
+        self.data.append(value)
+        self.owner.notify(self.name, None, value, "append", index=index)
+
+    def __str__(self):
+        return self.data.__str__()
+
+    def __repr__(self):
+        return self.data.__repr__()
diff --git a/mesa/experimental/mesa_signals/signals_util.py b/mesa/experimental/mesa_signals/signals_util.py
new file mode 100644
index 00000000..f5152452
--- /dev/null
+++ b/mesa/experimental/mesa_signals/signals_util.py
@@ -0,0 +1,43 @@
+"""helper functions and classes for mesa signals."""
+
+import weakref
+
+__all__ = [
+    "AttributeDict",
+    "create_weakref",
+]
+
+
+class AttributeDict(dict):
+    """A dict with attribute like access.
+
+    Each value can be accessed as if it were an attribute with its key as attribute name
+
+    """
+
+    # I want our signals to act like traitlet signals, so this is inspired by trailets Bunch
+    # and some stack overflow posts.
+    __setattr__ = dict.__setitem__
+    __delattr__ = dict.__delitem__
+
+    def __getattr__(self, key):  # noqa: D105
+        try:
+            return self.__getitem__(key)
+        except KeyError as e:
+            # we need to go from key error to attribute error
+            raise AttributeError(key) from e
+
+    def __dir__(self):  # noqa: D105
+        # allows us to easily access all defined attributes
+        names = dir({})
+        names.extend(self.keys())
+        return names
+
+
+def create_weakref(item, callback=None):
+    """Helper function to create a correct weakref for any item."""
+    if hasattr(item, "__self__"):
+        ref = weakref.WeakMethod(item, callback)
+    else:
+        ref = weakref.ref(item, callback)
+    return ref


Test patch:
diff --git a/tests/test_mesa_signals.py b/tests/test_mesa_signals.py
new file mode 100644
index 00000000..a41afdf4
--- /dev/null
+++ b/tests/test_mesa_signals.py
@@ -0,0 +1,290 @@
+"""Tests for mesa_signals."""
+
+from unittest.mock import Mock
+
+import pytest
+
+from mesa import Agent, Model
+from mesa.experimental.mesa_signals import (
+    All,
+    Computable,
+    Computed,
+    HasObservables,
+    Observable,
+    ObservableList,
+)
+from mesa.experimental.mesa_signals.signals_util import AttributeDict
+
+
+def test_observables():
+    """Test Observable."""
+
+    class MyAgent(Agent, HasObservables):
+        some_attribute = Observable()
+
+        def __init__(self, model, value):
+            super().__init__(model)
+            some_attribute = value  # noqa: F841
+
+    handler = Mock()
+
+    model = Model(seed=42)
+    agent = MyAgent(model, 10)
+    agent.observe("some_attribute", "change", handler)
+
+    agent.some_attribute = 10
+    handler.assert_called_once()
+
+
+def test_HasObservables():
+    """Test Observable."""
+
+    class MyAgent(Agent, HasObservables):
+        some_attribute = Observable()
+        some_other_attribute = Observable()
+
+        def __init__(self, model, value):
+            super().__init__(model)
+            some_attribute = value  # noqa: F841
+            some_other_attribute = 5  # noqa: F841
+
+    handler = Mock()
+
+    model = Model(seed=42)
+    agent = MyAgent(model, 10)
+    agent.observe("some_attribute", "change", handler)
+
+    subscribers = {entry() for entry in agent.subscribers["some_attribute"]["change"]}
+    assert handler in subscribers
+
+    agent.unobserve("some_attribute", "change", handler)
+    subscribers = {entry() for entry in agent.subscribers["some_attribute"]["change"]}
+    assert handler not in subscribers
+
+    subscribers = {
+        entry() for entry in agent.subscribers["some_other_attribute"]["change"]
+    }
+    assert len(subscribers) == 0
+
+    # testing All()
+    agent.observe(All(), "change", handler)
+
+    for attr in ["some_attribute", "some_other_attribute"]:
+        subscribers = {entry() for entry in agent.subscribers[attr]["change"]}
+        assert handler in subscribers
+
+    agent.unobserve(All(), "change", handler)
+    for attr in ["some_attribute", "some_other_attribute"]:
+        subscribers = {entry() for entry in agent.subscribers[attr]["change"]}
+        assert handler not in subscribers
+        assert len(subscribers) == 0
+
+    # testing for clear_all_subscriptions
+    nr_observers = 3
+    handlers = [Mock() for _ in range(nr_observers)]
+    for handler in handlers:
+        agent.observe("some_attribute", "change", handler)
+        agent.observe("some_other_attribute", "change", handler)
+
+    subscribers = {entry() for entry in agent.subscribers["some_attribute"]["change"]}
+    assert len(subscribers) == nr_observers
+
+    agent.clear_all_subscriptions("some_attribute")
+    subscribers = {entry() for entry in agent.subscribers["some_attribute"]["change"]}
+    assert len(subscribers) == 0
+
+    subscribers = {
+        entry() for entry in agent.subscribers["some_other_attribute"]["change"]
+    }
+    assert len(subscribers) == 3
+
+    agent.clear_all_subscriptions(All())
+    subscribers = {entry() for entry in agent.subscribers["some_attribute"]["change"]}
+    assert len(subscribers) == 0
+
+    subscribers = {
+        entry() for entry in agent.subscribers["some_other_attribute"]["change"]
+    }
+    assert len(subscribers) == 0
+
+    # test raises
+    with pytest.raises(ValueError):
+        agent.observe("some_attribute", "unknonw_signal", handler)
+
+    with pytest.raises(ValueError):
+        agent.observe("unknonw_attribute", "change", handler)
+
+
+def test_ObservableList():
+    """Test ObservableList."""
+
+    class MyAgent(Agent, HasObservables):
+        my_list = ObservableList()
+
+        def __init__(
+            self,
+            model,
+        ):
+            super().__init__(model)
+            self.my_list = []
+
+    model = Model(seed=42)
+    agent = MyAgent(model)
+
+    assert len(agent.my_list) == 0
+
+    # add
+    handler = Mock()
+    agent.observe("my_list", "append", handler)
+
+    agent.my_list.append(1)
+    assert len(agent.my_list) == 1
+    handler.assert_called_once()
+    handler.assert_called_once_with(
+        AttributeDict(
+            name="my_list", new=1, old=None, type="append", index=0, owner=agent
+        )
+    )
+    agent.unobserve("my_list", "append", handler)
+
+    # remove
+    handler = Mock()
+    agent.observe("my_list", "remove", handler)
+
+    agent.my_list.remove(1)
+    assert len(agent.my_list) == 0
+    handler.assert_called_once()
+
+    agent.unobserve("my_list", "remove", handler)
+
+    # overwrite the existing list
+    a_list = [1, 2, 3, 4, 5]
+    handler = Mock()
+    agent.observe("my_list", "change", handler)
+    agent.my_list = a_list
+    assert len(agent.my_list) == len(a_list)
+    handler.assert_called_once()
+
+    agent.my_list = a_list
+    assert len(agent.my_list) == len(a_list)
+    handler.assert_called()
+    agent.unobserve("my_list", "change", handler)
+
+    # pop
+    handler = Mock()
+    agent.observe("my_list", "remove", handler)
+
+    index = 4
+    entry = agent.my_list.pop(index)
+    assert entry == a_list.pop(index)
+    assert len(agent.my_list) == len(a_list)
+    handler.assert_called_once()
+    agent.unobserve("my_list", "remove", handler)
+
+    # insert
+    handler = Mock()
+    agent.observe("my_list", "insert", handler)
+    agent.my_list.insert(0, 5)
+    handler.assert_called()
+    agent.unobserve("my_list", "insert", handler)
+
+    # overwrite
+    handler = Mock()
+    agent.observe("my_list", "replace", handler)
+    agent.my_list[0] = 10
+    assert agent.my_list[0] == 10
+    handler.assert_called_once()
+    agent.unobserve("my_list", "replace", handler)
+
+    # combine two lists
+    handler = Mock()
+    agent.observe("my_list", "append", handler)
+    a_list = [1, 2, 3, 4, 5]
+    agent.my_list = a_list
+    assert len(agent.my_list) == len(a_list)
+    agent.my_list += a_list
+    assert len(agent.my_list) == 2 * len(a_list)
+    handler.assert_called()
+
+    # some more non signalling functionality tests
+    assert 5 in agent.my_list
+    assert agent.my_list.index(5) == 4
+
+
+def test_AttributeDict():
+    """Test AttributeDict."""
+
+    class MyAgent(Agent, HasObservables):
+        some_attribute = Observable()
+
+        def __init__(self, model, value):
+            super().__init__(model)
+            self.some_attribute = value
+
+    def on_change(signal):
+        assert signal.name == "some_attribute"
+        assert signal.type == "change"
+        assert signal.old == 10
+        assert signal.new == 5
+        assert signal.owner == agent
+
+        items = dir(signal)
+        for entry in ["name", "type", "old", "new", "owner"]:
+            assert entry in items
+
+    model = Model(seed=42)
+    agent = MyAgent(model, 10)
+    agent.observe("some_attribute", "change", on_change)
+    agent.some_attribute = 5
+
+
+def test_Computable():
+    """Test Computable and Computed."""
+
+    class MyAgent(Agent, HasObservables):
+        some_attribute = Computable()
+        some_other_attribute = Observable()
+
+        def __init__(self, model, value):
+            super().__init__(model)
+            self.some_other_attribute = value
+            self.some_attribute = Computed(lambda x: x.some_other_attribute * 2, self)
+
+    model = Model(seed=42)
+    agent = MyAgent(model, 10)
+    assert agent.some_attribute == 20
+
+    handler = Mock()
+    agent.observe("some_attribute", "change", handler)
+    agent.some_other_attribute = 9  # we change the dependency of computed
+    handler.assert_called_once()
+    agent.unobserve("some_attribute", "change", handler)
+
+    handler = Mock()
+    agent.observe("some_attribute", "change", handler)
+    assert (
+        agent.some_attribute == 18
+    )  # this forces a re-evaluation of the value of computed
+    handler.assert_called_once()  # and so, our change handler should be called
+    agent.unobserve("some_attribute", "change", handler)
+
+    # cyclical dependencies
+    def computed_func(agent):
+        # this creates a cyclical dependency
+        # our computed is dependent on o1, but also modifies o1
+        agent.o1 = agent.o1 - 1
+
+    class MyAgent(Agent, HasObservables):
+        c1 = Computable()
+        o1 = Observable()
+
+        def __init__(self, model, value):
+            super().__init__(model)
+            self.o1 = value
+            self.c1 = Computed(computed_func, self)
+
+    model = Model(seed=42)
+    with pytest.raises(ValueError):
+        MyAgent(model, 10)
+
+    # parents disappearing