Repository: canonical/operator
Pull request #1913 (2025-07-17 22:55:21)

docs: replace broken link in testing explanation
The files in the promtheus-k8s repository have moved around, breaking the example link the testing explanation doc.

This PR replaces that link, but rather than update it to the new prometheus-k8s location, uses the ubuntu-manpages charm instead. That charm should be more stable, the tests are much simpler (which better matches this specific part of the docs, which are explaining the basics), and the charm *only* uses Scenario, unlike prometheus-k8s, which has a mix of Harness and Scenario.

Fixes #1910


Patch:
diff --git a/ops/_main.py b/ops/_main.py
index 7399a11..cae01a2 100644
--- a/ops/_main.py
+++ b/ops/_main.py
@@ -268,6 +268,8 @@ class _Manager:
       - graceful teardown of the storage
     """
 
+    _framework_class = _framework.Framework
+
     def __init__(
         self,
         charm_class: type[_charm.CharmBase],
@@ -388,7 +390,7 @@ class _Manager:
             remote_unit_name=self._juju_context.remote_unit_name,
         )
         store = self._make_storage(dispatcher)
-        framework = _framework.Framework(
+        framework = self._framework_class(
             store,
             self._charm_root,
             self._charm_meta,


Test patch:
diff --git a/.github/workflows/observability-charm-tests.yaml b/.github/workflows/observability-charm-tests.yaml
index 66e70d8..75ac6b8 100644
--- a/.github/workflows/observability-charm-tests.yaml
+++ b/.github/workflows/observability-charm-tests.yaml
@@ -20,13 +20,10 @@ jobs:
         include:
           - charm-repo: canonical/alertmanager-k8s-operator
             commit: cc7bd1db35589fd63e44fb0e8813cd3b4362943e  # rev171 2025-06-22T16:01:16Z
-            disabled: true
           - charm-repo: canonical/prometheus-k8s-operator
             commit: ca0a31dab4497eac1bcdc3ca3eb47bbbc7365dc9  # rev253 2025-06-23T08:45:58Z
-            disabled: true
           - charm-repo: canonical/grafana-k8s-operator
             commit: a2770608526f2c82ddc3d225870f1a48eaa87179  # 2025-06-12T17:55:14Z
-            disabled: true
     steps:
       - name: Checkout the ${{ matrix.charm-repo }} repository
         uses: actions/checkout@v4
@@ -41,19 +38,15 @@ jobs:
       - name: Update 'ops' and 'ops-scenario' dependencies in test charm to latest
         if: ${{ !(matrix.disabled) }}
         run: |
-          if [ -e "requirements.txt" ]; then
-            sed -i -e "/^ops[ ><=]/d" -e "/canonical\/operator/d" -e "/#egg=ops/d" requirements.txt
-            echo -e "\ngit+$GITHUB_SERVER_URL/$GITHUB_REPOSITORY@$GITHUB_SHA#egg=ops" >> requirements.txt
-            sed -i -e "/^ops-scenario[ ><=]/d" -e "/^ops\[testing\][ ><=]/d" requirements.txt
-            echo -e "\ngit+$GITHUB_SERVER_URL/$GITHUB_REPOSITORY@$GITHUB_SHA#egg=ops-scenario&subdirectory=testing" >> requirements.txt
-          elif [ -e "uv.lock" ]; then
+          if [ -e "uv.lock" ]; then
             uv remove ops --optional dev --frozen || echo "maybe ops[testing] is not a dev dependency"
             uv remove ops-scenario --optional dev --frozen || echo "maybe ops-scenario is not a dev dependency"
             uv remove ops --frozen
+            sed -i 's/requires-python = "~=3.8"/requires-python = "~=3.10"/g' pyproject.toml
             uv add git+$GITHUB_SERVER_URL/$GITHUB_REPOSITORY@$GITHUB_SHA#subdirectory=testing --optional dev --raw-sources --prerelease=if-necessary-or-explicit
             uv add git+$GITHUB_SERVER_URL/$GITHUB_REPOSITORY@$GITHUB_SHA --raw-sources --prerelease=if-necessary-or-explicit
           else
-            echo "Error: no requirements.txt or uv.lock file found"
+            echo "Error: no uv.lock file found"
             exit 1
           fi
 
diff --git a/docs/explanation/testing.md b/docs/explanation/testing.md
index 2c625a8..764b5b1 100644
--- a/docs/explanation/testing.md
+++ b/docs/explanation/testing.md
@@ -44,7 +44,7 @@ The `ops.testing` framework provides `State`, which mocks inputs and outputs. Th
 
 ### Examples
 
-- [Prometheus K8s charm unit tests](https://github.com/canonical/prometheus-k8s-operator/blob/main/tests/scenario/test_alert_expression_labels.py)
+- [Ubuntu manpages charm unit tests](https://github.com/canonical/ubuntu-manpages-operator/blob/main/tests/unit/test_charm.py)
 
 (interface-tests)=
 ## Interface testing
@@ -63,7 +63,7 @@ Interface tests also:
 - Enable alternative implementations of an interface to validate themselves against the contractual specification stored in `charm-relation-interfaces`.
 - Help verify compliance with multiple versions of an interface.
 
-An interface test has the following pattern: 
+An interface test has the following pattern:
 
 1) **Given** - An initial state of the relation over the interface under test.
 2) **When** - A specific relation event fires.
@@ -130,8 +130,8 @@ basepython = python3.10
 
 It builds a fixture called `ops_test` that helps you interact with Juju through constructs that wrap around [`python-libjuju` ](https://pypi.org/project/juju/).
 
-> See more: 
-> - [`pytest-operator` > `ops_test`](https://github.com/charmed-kubernetes/pytest-operator/blob/main/docs/reference.md#ops_test) 
+> See more:
+> - [`pytest-operator` > `ops_test`](https://github.com/charmed-kubernetes/pytest-operator/blob/main/docs/reference.md#ops_test)
 > - [`pytest` > Fixtures](https://docs.pytest.org/en/6.2.x/fixture.html)
 
 It also provides convenient markers and command line parameters (e.g., the `@pytest.mark.skip_if_deployed` marker in combination with the `--no-deploy` configuration helps you skip, e.g., a deployment test in the case where you already have a deployment).
diff --git a/test/charms/test_main/src/charm.py b/test/charms/test_main/src/charm.py
index af24eea..6e32084 100755
--- a/test/charms/test_main/src/charm.py
+++ b/test/charms/test_main/src/charm.py
@@ -28,6 +28,12 @@ sys.path.append('lib')
 
 logger = logging.getLogger()
 
+# Note for ops developers: juju-log doesn't go anywhere useful much of the time
+# during unit tests, and test_main failures that subprocess out are often
+# difficult to debug. Uncomment this line to get more informative errors when
+# running the tests.
+# logger.addHandler(logging.StreamHandler(sys.stderr))
+
 
 class CustomEvent(ops.EventBase):
     pass
diff --git a/testing/src/scenario/_ops_main_mock.py b/testing/src/scenario/_ops_main_mock.py
index 18a347f..efd1641 100644
--- a/testing/src/scenario/_ops_main_mock.py
+++ b/testing/src/scenario/_ops_main_mock.py
@@ -4,6 +4,7 @@
 from __future__ import annotations
 
 import dataclasses
+import functools
 import logging
 import marshal
 import re
@@ -12,7 +13,9 @@ import warnings
 from typing import (
     TYPE_CHECKING,
     Any,
+    Generic,
     Sequence,
+    cast,
 )
 
 import ops
@@ -23,6 +26,7 @@ from ops.framework import _event_regex
 from ops._main import _Dispatcher, _Manager
 from ops._main import logger as ops_logger
 
+from .context import Context
 from .errors import BadOwnerPath, NoObserverError
 from .logger import logger as scenario_logger
 from .mocking import _MockModelBackend
@@ -111,7 +115,7 @@ class UnitStateDB:
             db.save_snapshot(stored_state._handle_path, stored_state.content)
 
 
-class Ops(_Manager):
+class Ops(_Manager, Generic[CharmType]):
     """Class to manage stepping through ops setup, event emission and framework commit."""
 
     def __init__(
@@ -127,6 +131,7 @@ class Ops(_Manager):
         self.context = context
         self.charm_spec = charm_spec
         self.store = None
+        self.captured_events: list[ops.EventBase] = []
         self.trace_data = []
 
         try:
@@ -163,6 +168,10 @@ class Ops(_Manager):
 
         return ops.CharmMeta.from_yaml(metadata, actions_metadata, config_metadata)
 
+    @property
+    def _framework_class(self):
+        return functools.partial(CapturingFramework, context=self.context)
+
     def _setup_root_logging(self):
         # The warnings module captures this in _showwarning_orig, but we
         # shouldn't really be using a private method, so capture it ourselves as
@@ -276,3 +285,41 @@ class Ops(_Manager):
             assert self._tracing_exporter
             self.trace_data = self._tracing_exporter.get_finished_spans()
             self._tracing_mock.__exit__(None, None, None)
+
+
+class CapturingFramework(ops.Framework):
+    def __init__(self, *args: Any, context: Context[CharmType], **kwargs: Any):
+        super().__init__(*args, **kwargs)
+        self._context = context
+
+    def _emit(self, event: ops.EventBase):
+        if self._context.capture_framework_events or not isinstance(
+            event,
+            (ops.PreCommitEvent, ops.CommitEvent, ops.CollectStatusEvent),
+        ):
+            # Dump/undump the event to ensure any custom attributes are (re)set by restore().
+            event.restore(event.snapshot())
+            self._context.emitted_events.append(event)
+
+        return super()._emit(event)
+
+    def _reemit(self, single_event_path: str | None = None):
+        if not self._context.capture_deferred_events or single_event_path is not None:
+            return super()._reemit(single_event_path)
+
+        # Load all notices from storage as events.
+        for event_path, _, _ in self._storage.notices(single_event_path):
+            event_handle = ops.Handle.from_path(event_path)
+            try:
+                event = self.load_snapshot(event_handle)
+            except ops.NoTypeError:
+                continue
+            event = cast('ops.EventBase', event)
+            event.deferred = False
+            self._forget(event)  # prevent tracking conflicts
+
+            # Dump/undump the event to ensure any custom attributes are (re)set by restore().
+            event.restore(event.snapshot())
+            self._context.emitted_events.append(event)
+
+        return super()._reemit(single_event_path)
diff --git a/testing/src/scenario/_runtime.py b/testing/src/scenario/_runtime.py
index fb9d94f..5187577 100644
--- a/testing/src/scenario/_runtime.py
+++ b/testing/src/scenario/_runtime.py
@@ -15,20 +15,10 @@ from pathlib import Path
 from typing import (
     TYPE_CHECKING,
     Type,
-    TypeVar,
 )
 
 import yaml
-from ops import (
-    CollectStatusEvent,
-    pebble,
-    CommitEvent,
-    EventBase,
-    Framework,
-    Handle,
-    NoTypeError,
-    PreCommitEvent,
-)
+from ops import pebble
 from ops.jujucontext import _JujuContext
 from ops._main import _Abort
 from ops._private.harness import ActionFailed
@@ -274,16 +264,6 @@ class Runtime:
             # charm_virtual_root is a tempdir
             typing.cast('tempfile.TemporaryDirectory', charm_virtual_root).cleanup()  # type: ignore
 
-    @contextmanager
-    def _exec_ctx(self, ctx: Context):
-        """python 3.8 compatibility shim"""
-        with self._virtual_charm_root() as temporary_charm_root:
-            with capture_events(
-                include_deferred=ctx.capture_deferred_events,
-                include_framework=ctx.capture_framework_events,
-            ) as captured:
-                yield (temporary_charm_root, captured)
-
     @contextmanager
     def exec(
         self,
@@ -310,7 +290,7 @@ class Runtime:
         output_state = copy.deepcopy(state)
 
         logger.info(' - generating virtual charm root')
-        with self._exec_ctx(context) as (temporary_charm_root, captured):
+        with self._virtual_charm_root() as temporary_charm_root:
             logger.info(' - preparing env')
             env = self._get_event_env(
                 state=state,
@@ -367,96 +347,5 @@ class Runtime:
                 os.environ.update(previous_env)
                 logger.info(' - exited ops.main')
 
-        context.emitted_events.extend(captured)
         logger.info('event dispatched. done.')
         context._set_output_state(ops.state)
-
-
-_T = TypeVar('_T', bound=EventBase)
-
-
-@contextmanager
-def capture_events(
-    *types: type[EventBase],
-    include_framework: bool = False,
-    include_deferred: bool = True,
-):
-    """Capture all events of type `*types` (using instance checks).
-
-    Arguments exposed so that you can define your own fixtures if you want to.
-
-    Example::
-    >>> from ops import StartEvent
-    >>> from scenario import Event, State
-    >>> from charm import MyCustomEvent, MyCharm  # noqa
-    >>>
-    >>> def test_my_event():
-    >>>     with capture_events(StartEvent, MyCustomEvent) as captured:
-    >>>         trigger(State(), ("start", MyCharm, meta=MyCharm.META)
-    >>>
-    >>>     assert len(captured) == 2
-    >>>     e1, e2 = captured
-    >>>     assert isinstance(e2, MyCustomEvent)
-    >>>     assert e2.custom_attr == 'foo'
-    """
-    allowed_types = types or (EventBase,)
-
-    captured: list[EventBase] = []
-    _real_emit = Framework._emit
-    _real_reemit = Framework.reemit
-
-    def _wrapped_emit(self: Framework, evt: EventBase):
-        if not include_framework and isinstance(
-            evt,
-            (PreCommitEvent, CommitEvent, CollectStatusEvent),
-        ):
-            return _real_emit(self, evt)
-
-        if isinstance(evt, allowed_types):
-            # dump/undump the event to ensure any custom attributes are (re)set by restore()
-            evt.restore(evt.snapshot())
-            captured.append(evt)
-
-        return _real_emit(self, evt)
-
-    def _wrapped_reemit(self: Framework):
-        # Framework calls reemit() before emitting the main juju event. We intercept that call
-        # and capture all events in storage.
-
-        if not include_deferred:
-            return _real_reemit(self)
-
-        # load all notices from storage as events.
-        for event_path, _, _ in self._storage.notices():
-            event_handle = Handle.from_path(event_path)
-            try:
-                event = self.load_snapshot(event_handle)
-            except NoTypeError:
-                continue
-            event = typing.cast('EventBase', event)
-            event.deferred = False
-            self._forget(event)  # prevent tracking conflicts
-
-            if not include_framework and isinstance(
-                event,
-                (PreCommitEvent, CommitEvent),
-            ):
-                continue
-
-            if isinstance(event, allowed_types):
-                captured.append(event)
-
-        return _real_reemit(self)
-
-    Framework._emit = _wrapped_emit  # type: ignore
-    Framework.reemit = _wrapped_reemit
-
-    # A finally block is needed here in case the code raises an exception that
-    # isn't a subclass of Exception (like SystemExit or KeyboardInterrupt).
-    # Those are captured by the exec() code that uses this context manager, but
-    # ones outside of Exception are not.
-    try:
-        yield captured
-    finally:
-        Framework._emit = _real_emit
-        Framework.reemit = _real_reemit
diff --git a/testing/tests/test_emitted_events_util.py b/testing/tests/test_emitted_events_util.py
index bd3b558..6792823 100644
--- a/testing/tests/test_emitted_events_util.py
+++ b/testing/tests/test_emitted_events_util.py
@@ -3,10 +3,8 @@ from __future__ import annotations
 from ops.charm import CharmBase, CharmEvents, CollectStatusEvent, StartEvent
 from ops.framework import CommitEvent, EventBase, EventSource, PreCommitEvent
 
-from scenario import State
+from scenario import Context, State
 from scenario.state import _Event
-from scenario._runtime import capture_events
-from .helpers import trigger
 
 
 class Foo(EventBase):
@@ -34,9 +32,10 @@ class MyCharm(CharmBase):
 
 
 def test_capture_custom_evt_nonspecific_capture_include_fw_evts():
-    with capture_events(include_framework=True) as emitted:
-        trigger(State(), 'start', MyCharm, meta=MyCharm.META)
+    ctx = Context(MyCharm, meta=MyCharm.META, capture_framework_events=True)
+    ctx.run(ctx.on.start(), State())
 
+    emitted = ctx.emitted_events
     assert len(emitted) == 5
     assert isinstance(emitted[0], StartEvent)
     assert isinstance(emitted[1], Foo)
@@ -46,24 +45,21 @@ def test_capture_custom_evt_nonspecific_capture_include_fw_evts():
 
 
 def test_capture_juju_evt():
-    with capture_events() as emitted:
-        trigger(State(), 'start', MyCharm, meta=MyCharm.META)
+    ctx = Context(MyCharm, meta=MyCharm.META)
+    ctx.run(ctx.on.start(), State())
 
+    emitted = ctx.emitted_events
     assert len(emitted) == 2
     assert isinstance(emitted[0], StartEvent)
     assert isinstance(emitted[1], Foo)
 
 
 def test_capture_deferred_evt():
-    # todo: this test should pass with ops < 2.1 as well
-    with capture_events() as emitted:
-        trigger(
-            State(deferred=[_Event('foo').deferred(handler=MyCharm._on_foo)]),
-            'start',
-            MyCharm,
-            meta=MyCharm.META,
-        )
+    ctx = Context(MyCharm, meta=MyCharm.META, capture_deferred_events=True)
+    deferred = [_Event('foo').deferred(handler=MyCharm._on_foo)]
+    ctx.run(ctx.on.start(), State(deferred=deferred))
 
+    emitted = ctx.emitted_events
     assert len(emitted) == 3
     assert isinstance(emitted[0], Foo)
     assert isinstance(emitted[1], StartEvent)
@@ -71,15 +67,11 @@ def test_capture_deferred_evt():
 
 
 def test_capture_no_deferred_evt():
-    # todo: this test should pass with ops < 2.1 as well
-    with capture_events(include_deferred=False) as emitted:
-        trigger(
-            State(deferred=[_Event('foo').deferred(handler=MyCharm._on_foo)]),
-            'start',
-            MyCharm,
-            meta=MyCharm.META,
-        )
+    ctx = Context(MyCharm, meta=MyCharm.META)
+    deferred = [_Event('foo').deferred(handler=MyCharm._on_foo)]
+    ctx.run(ctx.on.start(), State(deferred=deferred))
 
+    emitted = ctx.emitted_events
     assert len(emitted) == 2
     assert isinstance(emitted[0], StartEvent)
     assert isinstance(emitted[1], Foo)