Repository: scikit-hep/pyhf

Issue:
Exceptions related to weakref in pyhf 0.6.2
# Description

With `pyhf` 0.6.2 I see a long trace of `weakref`-related exceptions at the end of a program using certain parts of `pyhf`. I have not extensively tested which parts cause this behavior, but have so far observed it with `pyhf.simplemodels.correlated_background` / `pyhf.simplemodels.uncorrelated_background`. I can also confirm that other functionality like `pyhf.set_backend("numpy")` does not trigger it.

# Expected Behavior

No stack of `weakref` exceptions after code finishes running.

# Actual Behavior

Large stack of `weakref`-related exceptions.

# Steps to Reproduce

Run a simple file using `pyhf.simplemodels.uncorrelated_background`, the example runs in a `python:3.8-slim` container with Python 3.8.11. I see the same locally in a conda environment with Python 3.8.10 on Mac OS. With `pyhf` 0.6.2 the stack of exceptions appears. When installing `pyhf==0.6.1` and using `pyhf.simplemodels.hepdata_like`, I do not see any exceptions. When running via `python -m pdb test.py` locally to debug, I do no longer see the exception. The current `pyhf` master also shows the same issue.

```python
$ cat test.py
import pyhf

model = pyhf.simplemodels.uncorrelated_background(
    signal=[5.0], bkg=[10.0], bkg_uncertainty=[2.0]
)
```
```console
$ docker run -it --rm -v$PWD:/test python:3.8-slim bash
root@8fd8a5f229c8:/# pip install pyhf
Collecting pyhf
  Downloading pyhf-0.6.2-py3-none-any.whl (140 kB)
     |████████████████████████████████| 140 kB 3.5 MB/s
Collecting pyyaml>=5.1
  Downloading PyYAML-5.4.1-cp38-cp38-manylinux1_x86_64.whl (662 kB)
     |████████████████████████████████| 662 kB 5.7 MB/s
Collecting scipy>=1.4.1
  Downloading scipy-1.7.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl (28.4 MB)
     |████████████████████████████████| 28.4 MB 16.4 MB/s
Collecting jsonpatch>=1.15
  Downloading jsonpatch-1.32-py2.py3-none-any.whl (12 kB)
Collecting tqdm>=4.56.0
  Downloading tqdm-4.61.1-py2.py3-none-any.whl (75 kB)
     |████████████████████████████████| 75 kB 2.9 MB/s
Collecting click>=7.0
  Downloading click-8.0.1-py3-none-any.whl (97 kB)
     |████████████████████████████████| 97 kB 6.2 MB/s
Collecting jsonschema>=3.0.0
  Downloading jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
     |████████████████████████████████| 56 kB 6.3 MB/s
Collecting jsonpointer>=1.9
  Downloading jsonpointer-2.1-py2.py3-none-any.whl (7.4 kB)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/site-packages (from jsonschema>=3.0.0->pyhf) (57.0.0)
Collecting six>=1.11.0
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting attrs>=17.4.0
  Downloading attrs-21.2.0-py2.py3-none-any.whl (53 kB)
     |████████████████████████████████| 53 kB 4.1 MB/s
Collecting pyrsistent>=0.14.0
  Downloading pyrsistent-0.18.0-cp38-cp38-manylinux1_x86_64.whl (118 kB)
     |████████████████████████████████| 118 kB 18.9 MB/s
Collecting numpy<1.23.0,>=1.16.5
  Downloading numpy-1.21.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.7 MB)
     |████████████████████████████████| 15.7 MB 18.2 MB/s
Installing collected packages: six, pyrsistent, numpy, jsonpointer, attrs, tqdm, scipy, pyyaml, jsonschema, jsonpatch, click, pyhf
Successfully installed attrs-21.2.0 click-8.0.1 jsonpatch-1.32 jsonpointer-2.1 jsonschema-3.2.0 numpy-1.21.0 pyhf-0.6.2 pyrsistent-0.18.0 pyyaml-5.4.1 scipy-1.7.0 six-1.16.0 tqdm-4.61.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
root@8fd8a5f229c8:/# python test/test.py
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf03a0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf01f0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf0280>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf0430>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf00d0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf0160>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf0670>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf05e0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf04c0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf0550>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf0820>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf0790>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf0700>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf09d0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf0940>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf08b0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf0c10>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf0b80>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf0a60>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf0af0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf0dc0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf0d30>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf0ca0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf0f70>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf0ee0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dcf0e50>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dc821f0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dc82160>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dc82040>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dc820d0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dc823a0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dc82280>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dc82310>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dc82430>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
Exception ignored in: <function WeakMethod.__new__.<locals>._cb at 0x7f7e8dc824c0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/weakref.py", line 58, in _cb
  File "/usr/local/lib/python3.8/weakref.py", line 74, in __eq__
TypeError: isinstance() arg 2 must be a type or tuple of types
```

# Checklist

- [X] Run `git fetch` to get the most up to date version of `master`
- [X] Searched through existing Issues to confirm this is not a duplicate issue
- [X] Filled out the Description, Expected Behavior, Actual Behavior, and Steps to Reproduce sections above or have edited/removed them in a way that fully describes the issue



Discussion:
@alexander-held what does `test/test.py` look like? The weakref error here famously occurs with matplotlib involved. The other one is actually a bug in python 3.7.
It's this one:
```python
import pyhf

model = pyhf.simplemodels.uncorrelated_background(
    signal=[5.0], bkg=[10.0], bkg_uncertainty=[2.0]
)
```
The reason that it is `test/test.py` and not just `test.py` is that I mount it into `/test` in the container.

I am using Python 3.8 in these tests, and have not tried 3.7.
The issue seems to be related to `click`. When I add a `pip install click~=7.0` after `pip install pyhf`, it disappears.
Thanks for this report @alexander-held. Yeah, I'm not sure why we hadn't seen this in tests before but this is definitley related to the `click` `v8.X` release series, as even 

```console
python -m pip install --upgrade "click<8.0.1"
```

giving `v8.0.0` doesn't help. :( 
Copying discussion from Slack, @kratsg points out:

>I'll dump what I do know about `weakref` and why this comes up. The crash occurs on these lines (https://github.com/python/cpython/blob/ae5dcf588c39915b43e391c738cf99f56a1b7fe2/Lib/weakref.py#L73-L78). This is because the `WeakMethod` is `NoneType`'d (because it got deleted during garbage collect?). To test this, I added a nice print statement `print(other, WeakMethod, type(other), type(WeakMethod))`.
>
> When it doesn't crash, I see this
> ```python
> <weakref at 0x15ecf1200; dead> <class 'weakref.WeakMethod'> <class 'weakref.WeakMethod'> <class 'type'>
> ```
> When it **does** crash, I see this
> ```python
> <weakref at 0x15ecf19e0; dead> None <class 'weakref.WeakMethod'> <class 'NoneType'>
> ```
> So indeed, `weakref.WeakMethod` is `NoneType` too early.
> Why is it being deleted too early? Why is this combination of our `pyhf` and `click` version causing this issue?

Resolving patch:
diff --git a/src/pyhf/events.py b/src/pyhf/events.py
--- a/src/pyhf/events.py
+++ b/src/pyhf/events.py
@@ -6,7 +6,6 @@
 
 __all__ = [
     "Callables",
-    "WeakList",
     "disable",
     "enable",
     "noop",
@@ -24,24 +23,79 @@ def noop(*args, **kwargs):
     pass
 
 
-class WeakList(list):
-    def append(self, item):
-        list.append(self, weakref.WeakMethod(item, self.remove))
+class Callables:
+    def __init__(self):
+        self._callbacks = []
+
+    @property
+    def callbacks(self):
+        """
+        Get the current list of living callbacks.
+        """
+        self._flush()
+        return self._callbacks
+
+    def append(self, callback):
+        """
+        Append a new bound method as a callback to the list of callables.
+        """
+        try:
+            # methods
+            callback_ref = weakref.ref(callback.__func__), weakref.ref(
+                callback.__self__
+            )
+        except AttributeError:
+            callback_ref = weakref.ref(callback), None
+        self._callbacks.append(callback_ref)
+
+    def _flush(self):
+        """
+        Flush the list of callbacks with those who are weakly-referencing deleted objects.
+
+        Note: must interact with the self._callbacks directly, and not
+        self.callbacks, to avoid infinite recursion.
+        """
+        _callbacks = []
+        for func, arg in self._callbacks:
+            if arg is not None:
+                arg_ref = arg()
+                if arg_ref is None:
+                    continue
+            _callbacks.append((func, arg))
+        self._callbacks = _callbacks
 
-
-class Callables(WeakList):
     def __call__(self, *args, **kwargs):
-        for func in self:
+        for func, arg in self.callbacks:
             # weakref: needs to be de-ref'd first before calling
-            func()(*args, **kwargs)
+            if arg is not None:
+                func()(arg(), *args, **kwargs)
+            else:
+                func()(*args, **kwargs)
+
+    def __iter__(self):
+        return iter(self.callbacks)
+
+    def __getitem__(self, index):
+        return self.callbacks[index]
+
+    def __len__(self):
+        return len(self.callbacks)
 
     def __repr__(self):
-        return "Callables(%s)" % list.__repr__(self)
+        return f"Callables({self.callbacks})"
 
 
 def subscribe(event):
     """
-    This is meant to be used as a decorator.
+    Subscribe a function or object method as a callback to an event.
+
+    Note: this is meant to be used as a decorator.
+
+    Args:
+        event (:obj:`str`): The name of the event to subscribe to.
+
+    Returns:
+        :obj:`function`: Decorated function.
     """
     # Example:
     #
@@ -62,9 +116,17 @@ def __decorator(func):
 
 def register(event):
     """
-    This is meant to be used as a decorator to register a function for triggering events.
+    Register a function or object method to trigger an event.  This creates two
+    events: ``{event_name}::before`` and ``{event_name}::after``.
+
+    Note: this is meant to be used as a decorator.
+
+    Args:
+        event (:obj:`str`): The name of the event to subscribe to.
+
+    Returns:
+        :obj:`function`: Decorated function.
 
-    This creates two events: "<event_name>::before" and "<event_name>::after"
     """
     # Examples:
     #


Test patch:
diff --git a/tests/test_events.py b/tests/test_events.py
--- a/tests/test_events.py
+++ b/tests/test_events.py
@@ -7,9 +7,11 @@ def test_subscribe_event():
 
     m = mock.Mock()
     events.subscribe(ename)(m.__call__)
-
     assert ename in events.__events
-    assert m.__call__ == events.__events.get(ename)[0]()
+    assert m.__call__.__func__ == events.__events.get(ename)[0][0]()
+    assert "weakref" in repr(events.trigger(ename))
+    assert list(events.trigger(ename))
+    assert len(list(events.trigger(ename))) == 1
     del events.__events[ename]
 
 
@@ -48,9 +50,9 @@ def test_disable_event():
     assert m.called is False
     assert ename in events.__disabled_events
     assert events.trigger(ename) == events.noop
-    assert events.trigger(ename)() == events.noop()
+    events.trigger(ename)()
     assert m.called is False
-    assert noop_m.is_called_once()
+    noop_m.assert_called_once()
     events.enable(ename)
     assert ename not in events.__disabled_events
     del events.__events[ename]
@@ -59,10 +61,52 @@ def test_disable_event():
 
 def test_trigger_noevent():
     noop, noop_m = events.noop, mock.Mock()
+    events.noop = noop_m
 
     assert 'fake' not in events.__events
     assert events.trigger('fake') == events.noop
-    assert events.trigger('fake')() == events.noop()
-    assert noop_m.is_called_once()
+    events.trigger('fake')()
+    noop_m.assert_called_once()
 
     events.noop = noop
+
+
+def test_subscribe_function(capsys):
+    ename = 'test'
+
+    def add(a, b):
+        print(a + b)
+
+    events.subscribe(ename)(add)
+    events.trigger(ename)(1, 2)
+
+    captured = capsys.readouterr()
+    assert captured.out == "3\n"
+
+    del events.__events[ename]
+
+
+def test_trigger_function(capsys):
+    ename = 'test'
+
+    def add(a, b):
+        print(a + b)
+
+    precall = mock.Mock()
+    postcall = mock.Mock()
+
+    wrapped_add = events.register(ename)(add)
+    events.subscribe(f'{ename}::before')(precall.__call__)
+    events.subscribe(f'{ename}::after')(postcall.__call__)
+
+    precall.assert_not_called()
+    postcall.assert_not_called()
+
+    wrapped_add(1, 2)
+    captured = capsys.readouterr()
+    assert captured.out == "3\n"
+    precall.assert_called_once()
+    postcall.assert_called_once()
+
+    del events.__events[f'{ename}::before']
+    del events.__events[f'{ename}::after']
diff --git a/tests/test_public_api_repr.py b/tests/test_public_api_repr.py
--- a/tests/test_public_api_repr.py
+++ b/tests/test_public_api_repr.py
@@ -67,7 +67,6 @@ def test_contrib_viz_public_api():
 def test_contrib_events_public_api():
     assert dir(pyhf.events) == [
         "Callables",
-        "WeakList",
         "disable",
         "enable",
         "noop",