Repository: endojs/endo
Pull request #2384 (2024-07-26 22:14:05)

test(cli): Add demo doc alignment test, serial implementation
Closes: #2368
Refs: #2372

## Description

A test suite which asserts that the cli demo produces its documented outputs.

Current weaknesses in the implementation include concurrency and independence. They are related and both should be resolved by a change to the daemon initialization in the test setups.

Note that despite these weaknesses, this setup is useful enough to reveal that certain cli commands do not behave as the demo supposes. Notably:
- `endo list --as alice-agent` is rejected because `--as` is not a supported option for `endo list`. This could have been caught with unit tests on the cli endpoints.
- `endo inbox` does not display messages as expected. In some cases the template string for the message has simply been altered, but the discrepancy in the mailboxes-are-symmetric section is unlikely to have been caught with a unit test.

### Testing Considerations

The `familiar-chat` section is not tested.
Three of the section tests are failing, which is correct test behavior.

#### Concurrency
Ideally each tested section would run concurrently, however as written the demo sections are implicitly serial. This is a nice UX for a demo and I don't suggest changing it.

Thus tests running on the same daemon which reference the same objects interfere with one another. Possible solutions:
1. Run a different endo daemon per test, configured to its own directory
2. Test a slightly modified version of each section which prefixes petnames with a per-test unique id
3. Run the tests serially
4. Some other solution of which I am unaware

In this PR I have opted for (3.) _test the sections serially_. The logic to test each section is written in a distinct file bearing the name of that section, but the routines are imported into a common `index.test.js` and declared to ava as serial.

In a future PR it may be nice to adopt (1.), reusing setup from `daemon/test/endo.test.js`. I think [prepareConfig](https://github.com/endojs/endo/blob/8bebd9416cca3199dba5d5655d98046e2c7781e7/packages/daemon/test/endo.test.js#L209) accomplishes what I have in mind.

#### Independence
Although the tests are run serially, each depended-upon test file `X` exports setup and teardown routines in a `context` object which can be passed to `withContext` to make a wrapper. This wrapper can be applied to a test routine `Y` to replay the state-changing commands from the test `X` before calling `Y`, then clean up the state changes from `X` afterward. 

The tests are all declared to ava in the same `index.test.js` file, but section tests can be selected independently with `yarn test --match <section-name-1> [--match <section-name-2> [...]]`, for example `yarn test --match doubler-agent --match messages-are-symmetric`. The independence is weak, as any breakage in the state changing commands in the previous sections will cause the setup for subsequent tests to fail. Stronger test independence is ideal, but probably a long way away.

### Upgrade Considerations

* Adds the execa dev dependency.


Patch:
diff --git a/packages/cli/package.json b/packages/cli/package.json
index 6f0c2997a..582574095 100644
--- a/packages/cli/package.json
+++ b/packages/cli/package.json
@@ -28,7 +28,7 @@
     "lint-fix": "eslint --fix .",
     "lint:eslint": "eslint .",
     "lint:types": "tsc",
-    "test": "exit 0"
+    "test": "ava"
   },
   "dependencies": {
     "@endo/bundle-source": "^3.3.0",
@@ -57,6 +57,7 @@
     "eslint-config-prettier": "^9.1.0",
     "eslint-plugin-eslint-comments": "^3.2.0",
     "eslint-plugin-import": "^2.29.1",
+    "execa": "^9.3.0",
     "prettier": "^3.2.5",
     "typescript": "5.5.2"
   },
diff --git a/yarn.lock b/yarn.lock
index 74bd617dd..1fcaf9170 100644
--- a/yarn.lock
+++ b/yarn.lock
@@ -288,6 +288,7 @@ __metadata:
     eslint-config-prettier: "npm:^9.1.0"
     eslint-plugin-eslint-comments: "npm:^3.2.0"
     eslint-plugin-import: "npm:^2.29.1"
+    execa: "npm:^9.3.0"
     open: "npm:^9.1.0"
     prettier: "npm:^3.2.5"
     ses: "npm:^1.7.0"
@@ -2582,6 +2583,13 @@ __metadata:
   languageName: node
   linkType: hard
 
+"@sec-ant/readable-stream@npm:^0.4.1":
+  version: 0.4.1
+  resolution: "@sec-ant/readable-stream@npm:0.4.1"
+  checksum: 10c0/64e9e9cf161e848067a5bf60cdc04d18495dc28bb63a8d9f8993e4dd99b91ad34e4b563c85de17d91ffb177ec17a0664991d2e115f6543e73236a906068987af
+  languageName: node
+  linkType: hard
+
 "@sinclair/typebox@npm:^0.25.16":
   version: 0.25.24
   resolution: "@sinclair/typebox@npm:0.25.24"
@@ -2596,6 +2604,13 @@ __metadata:
   languageName: node
   linkType: hard
 
+"@sindresorhus/merge-streams@npm:^4.0.0":
+  version: 4.0.0
+  resolution: "@sindresorhus/merge-streams@npm:4.0.0"
+  checksum: 10c0/482ee543629aa1933b332f811a1ae805a213681ecdd98c042b1c1b89387df63e7812248bb4df3910b02b3cc5589d3d73e4393f30e197c9dde18046ccd471fc6b
+  languageName: node
+  linkType: hard
+
 "@sinonjs/commons@npm:^2.0.0":
   version: 2.0.0
   resolution: "@sinonjs/commons@npm:2.0.0"
@@ -5642,6 +5657,26 @@ __metadata:
   languageName: node
   linkType: hard
 
+"execa@npm:^9.3.0":
+  version: 9.3.0
+  resolution: "execa@npm:9.3.0"
+  dependencies:
+    "@sindresorhus/merge-streams": "npm:^4.0.0"
+    cross-spawn: "npm:^7.0.3"
+    figures: "npm:^6.1.0"
+    get-stream: "npm:^9.0.0"
+    human-signals: "npm:^7.0.0"
+    is-plain-obj: "npm:^4.1.0"
+    is-stream: "npm:^4.0.1"
+    npm-run-path: "npm:^5.2.0"
+    pretty-ms: "npm:^9.0.0"
+    signal-exit: "npm:^4.1.0"
+    strip-final-newline: "npm:^4.0.0"
+    yoctocolors: "npm:^2.0.0"
+  checksum: 10c0/99ae08e7fb9172d25c453c2a9c414b54c7689e72f68263f6da7bc94c7011720dc8129cc64c2e3be44fb0c6ae8e37a08d346a61dbcfe9f1e79ad24364da2c48ce
+  languageName: node
+  linkType: hard
+
 "execution-time@npm:^1.2.0":
   version: 1.4.1
   resolution: "execution-time@npm:1.4.1"
@@ -5814,7 +5849,7 @@ __metadata:
   languageName: node
   linkType: hard
 
-"figures@npm:^6.0.1":
+"figures@npm:^6.0.1, figures@npm:^6.1.0":
   version: 6.1.0
   resolution: "figures@npm:6.1.0"
   dependencies:
@@ -6204,6 +6239,16 @@ __metadata:
   languageName: node
   linkType: hard
 
+"get-stream@npm:^9.0.0":
+  version: 9.0.1
+  resolution: "get-stream@npm:9.0.1"
+  dependencies:
+    "@sec-ant/readable-stream": "npm:^0.4.1"
+    is-stream: "npm:^4.0.1"
+  checksum: 10c0/d70e73857f2eea1826ac570c3a912757dcfbe8a718a033fa0c23e12ac8e7d633195b01710e0559af574cbb5af101009b42df7b6f6b29ceec8dbdf7291931b948
+  languageName: node
+  linkType: hard
+
 "get-symbol-description@npm:^1.0.0":
   version: 1.0.0
   resolution: "get-symbol-description@npm:1.0.0"
@@ -6758,6 +6803,13 @@ __metadata:
   languageName: node
   linkType: hard
 
+"human-signals@npm:^7.0.0":
+  version: 7.0.0
+  resolution: "human-signals@npm:7.0.0"
+  checksum: 10c0/ce0c6d62d2e9bfe529d48f7c7fdf4b8c70fce950eef7850719b4e3f5bc71795ae7d61a3699ce13262bed7847705822601cc81f1921ea6a2906852e16228a94ab
+  languageName: node
+  linkType: hard
+
 "humanize-ms@npm:^1.2.1":
   version: 1.2.1
   resolution: "humanize-ms@npm:1.2.1"
@@ -7354,6 +7406,13 @@ __metadata:
   languageName: node
   linkType: hard
 
+"is-plain-obj@npm:^4.1.0":
+  version: 4.1.0
+  resolution: "is-plain-obj@npm:4.1.0"
+  checksum: 10c0/32130d651d71d9564dc88ba7e6fda0e91a1010a3694648e9f4f47bb6080438140696d3e3e15c741411d712e47ac9edc1a8a9de1fe76f3487b0d90be06ac9975e
+  languageName: node
+  linkType: hard
+
 "is-plain-object@npm:^2.0.3, is-plain-object@npm:^2.0.4":
   version: 2.0.4
   resolution: "is-plain-object@npm:2.0.4"
@@ -7428,6 +7487,13 @@ __metadata:
   languageName: node
   linkType: hard
 
+"is-stream@npm:^4.0.1":
+  version: 4.0.1
+  resolution: "is-stream@npm:4.0.1"
+  checksum: 10c0/2706c7f19b851327ba374687bc4a3940805e14ca496dc672b9629e744d143b1ad9c6f1b162dece81c7bfbc0f83b32b61ccc19ad2e05aad2dd7af347408f60c7f
+  languageName: node
+  linkType: hard
+
 "is-string@npm:^1.0.5, is-string@npm:^1.0.7":
   version: 1.0.7
   resolution: "is-string@npm:1.0.7"
@@ -9077,6 +9143,15 @@ __metadata:
   languageName: node
   linkType: hard
 
+"npm-run-path@npm:^5.2.0":
+  version: 5.3.0
+  resolution: "npm-run-path@npm:5.3.0"
+  dependencies:
+    path-key: "npm:^4.0.0"
+  checksum: 10c0/124df74820c40c2eb9a8612a254ea1d557ddfab1581c3e751f825e3e366d9f00b0d76a3c94ecd8398e7f3eee193018622677e95816e8491f0797b21e30b2deba
+  languageName: node
+  linkType: hard
+
 "npmlog@npm:^5.0.1":
   version: 5.0.1
   resolution: "npmlog@npm:5.0.1"
@@ -10678,7 +10753,7 @@ __metadata:
   languageName: node
   linkType: hard
 
-"signal-exit@npm:^4.0.1":
+"signal-exit@npm:^4.0.1, signal-exit@npm:^4.1.0":
   version: 4.1.0
   resolution: "signal-exit@npm:4.1.0"
   checksum: 10c0/41602dce540e46d599edba9d9860193398d135f7ff72cab629db5171516cfae628d21e7bfccde1bbfdf11c48726bc2a6d1a8fb8701125852fbfda7cf19c6aa83
@@ -11223,6 +11298,13 @@ __metadata:
   languageName: node
   linkType: hard
 
+"strip-final-newline@npm:^4.0.0":
+  version: 4.0.0
+  resolution: "strip-final-newline@npm:4.0.0"
+  checksum: 10c0/b0cf2b62d597a1b0e3ebc42b88767f0a0d45601f89fd379a928a1812c8779440c81abba708082c946445af1d6b62d5f16e2a7cf4f30d9d6587b89425fae801ff
+  languageName: node
+  linkType: hard
+
 "strip-indent@npm:^2.0.0":
   version: 2.0.0
   resolution: "strip-indent@npm:2.0.0"
@@ -12521,3 +12603,10 @@ __metadata:
   checksum: 10c0/dceb44c28578b31641e13695d200d34ec4ab3966a5729814d5445b194933c096b7ced71494ce53a0e8820685d1d010df8b2422e5bf2cdea7e469d97ffbea306f
   languageName: node
   linkType: hard
+
+"yoctocolors@npm:^2.0.0":
+  version: 2.1.1
+  resolution: "yoctocolors@npm:2.1.1"
+  checksum: 10c0/85903f7fa96f1c70badee94789fade709f9d83dab2ec92753d612d84fcea6d34c772337a9f8914c6bed2f5fc03a428ac5d893e76fab636da5f1236ab725486d0
+  languageName: node
+  linkType: hard


Test patch:
diff --git a/packages/cli/test/daemon-context.js b/packages/cli/test/daemon-context.js
new file mode 100644
index 000000000..cf3b575c4
--- /dev/null
+++ b/packages/cli/test/daemon-context.js
@@ -0,0 +1,18 @@
+/** @import { Context } from './types' */
+
+/**
+ * Provides test setup and teardown hooks that purge the local endo
+ * daemon. In the future, we should create isolated daemon instances
+ * so that tests can be run in parallel.
+ *
+ * @type {Context}
+ */
+export const daemonContext = {
+  setup: async execa => {
+    await execa`endo purge -f`;
+    await execa`endo start`;
+  },
+  teardown: async execa => {
+    await execa`endo purge -f`;
+  },
+};
diff --git a/packages/cli/test/demo/confined-script.js b/packages/cli/test/demo/confined-script.js
new file mode 100644
index 000000000..e8ce912ed
--- /dev/null
+++ b/packages/cli/test/demo/confined-script.js
@@ -0,0 +1,9 @@
+/** @import {TestRoutine} from '../types */
+
+/** @type {TestRoutine} */
+export const section = async (execa, testLine) => {
+  // If a runlet returns a promise for some value, it will print that value before exiting gracefully.
+  await testLine(execa`endo run runlet.js a b c`, {
+    stdout: "Hello, World! [ 'a', 'b', 'c' ]\n42",
+  });
+};
diff --git a/packages/cli/test/demo/counter-example.js b/packages/cli/test/demo/counter-example.js
new file mode 100644
index 000000000..74cbfd42d
--- /dev/null
+++ b/packages/cli/test/demo/counter-example.js
@@ -0,0 +1,60 @@
+/** @import {Context, TestRoutine} from '../types' */
+
+/** @type {TestRoutine} */
+export const section = async (execa, testLine) => {
+  // We can create an instance of the counter and give it a name.
+  await testLine(execa`endo make counter.js --name counter`, {
+    stdout: 'Object [Alleged: Counter] {}',
+  });
+
+  // Then, we can send messages to the counter and see their responses...
+  await testLine(execa`endo eval E(counter).incr() counter`, {
+    stdout: '1',
+  });
+  await testLine(execa`endo eval E(counter).incr() counter`, {
+    stdout: '2',
+  });
+  await testLine(execa`endo eval E(counter).incr() counter`, {
+    stdout: '3',
+  });
+
+  // Aside: in all the above cases, we use counter both as the property name...
+  await testLine(execa`endo eval E(c).incr() c:counter`, {
+    stdout: '4',
+  });
+
+  // Endo preserves the commands that led to the creation of the counter value...
+  await testLine(execa`endo restart`);
+  await testLine(execa`endo eval E(counter).incr() counter`, {
+    stdout: '1',
+  });
+  await testLine(execa`endo eval E(counter).incr() counter`, {
+    stdout: '2',
+  });
+  await testLine(execa`endo eval E(counter).incr() counter`, {
+    stdout: '3',
+  });
+
+  // Aside, since Eventual Send, the machinery under the E operator...
+  await testLine(execa`endo spawn greeter`);
+  await testLine(
+    execa`endo eval --worker greeter '${'Hello, World!'}' --name greeting`,
+    {
+      stdout: 'Hello, World!',
+    },
+  );
+  await testLine(execa`endo show greeting`, {
+    stdout: 'Hello, World!',
+  });
+};
+
+/** @type {Context} */
+export const context = {
+  setup: async execa => {
+    await execa`endo make counter.js --name counter`;
+    await execa`endo eval E(counter).incr() counter`;
+    await execa`endo eval E(counter).incr() counter`;
+    await execa`endo eval E(counter).incr() counter`;
+    await execa`endo spawn greeter`;
+  },
+};
diff --git a/packages/cli/test/demo/doubler-agent.js b/packages/cli/test/demo/doubler-agent.js
new file mode 100644
index 000000000..b0faffc56
--- /dev/null
+++ b/packages/cli/test/demo/doubler-agent.js
@@ -0,0 +1,52 @@
+/** @import {Context, TestRoutine} from '../types' */
+
+/** @type {TestRoutine} */
+export const section = async (execa, testLine) => {
+  // We make a doubler mostly the same way we made the counter...
+  await testLine(execa`endo mkguest doubler-handle doubler-agent`, {
+    stdout: 'Object [Alleged: EndoGuest] {}',
+  });
+  await testLine(
+    execa`endo make doubler.js --name doubler --powers doubler-agent`,
+  );
+
+  // This creates a doubler, but the doubler cannot respond until we resolve...
+  await testLine(execa`endo inbox`, {
+    stdout:
+      /^0\. "doubler-handle" requested "a counter, suitable for doubling"/,
+  });
+  await testLine(execa`endo resolve 0 counter`);
+
+  // Now we can get a response from the doubler.
+  await testLine(execa`endo eval E(doubler).incr() doubler`, {
+    stdout: '8',
+  });
+  await testLine(execa`endo eval E(doubler).incr() doubler`, {
+    stdout: '10',
+  });
+  await testLine(execa`endo eval E(doubler).incr() doubler`, {
+    stdout: '12',
+  });
+
+  // Also, in the optional second argument to request, doubler.js names...
+  await testLine(execa`endo restart`);
+  await testLine(execa`endo eval E(doubler).incr() doubler`, {
+    stdout: '2',
+  });
+  await testLine(execa`endo eval E(doubler).incr() doubler`, {
+    stdout: '4',
+  });
+  await testLine(execa`endo eval E(doubler).incr() doubler`, {
+    stdout: '6',
+  });
+};
+
+/** @type {Context} */
+export const context = {
+  setup: async execa => {
+    await execa`endo mkguest doubler-handle doubler-agent`;
+    await execa`endo make doubler.js --name doubler --powers doubler-agent`;
+    await execa`endo inbox`;
+    await execa`endo resolve 0 counter`;
+  },
+};
diff --git a/packages/cli/test/demo/index.test.js b/packages/cli/test/demo/index.test.js
new file mode 100644
index 000000000..a531c7a99
--- /dev/null
+++ b/packages/cli/test/demo/index.test.js
@@ -0,0 +1,85 @@
+import test from 'ava';
+import { $ } from 'execa';
+import { makeSectionTest } from '../section.js';
+import { withContext } from '../with-context.js';
+import { daemonContext } from '../daemon-context.js';
+import * as counterExample from './counter-example.js';
+import * as doublerAgent from './doubler-agent.js';
+import * as confinedScript from './confined-script.js';
+import * as sendingMessages from './sending-messages.js';
+import * as namesInTransit from './names-in-transit.js';
+import * as mailboxesAreSymmetric from './mailboxes-are-symmetric.js';
+
+test.serial(
+  'trivial',
+  makeSectionTest(
+    $({ cwd: 'demo' }),
+    withContext(daemonContext)(async (execa, testLine) => {
+      const maxim = 'a failing test is better than failure to test';
+      await testLine(execa`echo ${maxim}`, { stdout: maxim });
+    }),
+  ),
+);
+
+test.serial(
+  'counter-example',
+  makeSectionTest(
+    $({ cwd: 'demo' }),
+    withContext(daemonContext)(counterExample.section),
+  ),
+);
+
+test.serial(
+  'doubler-agent',
+  makeSectionTest(
+    $({ cwd: 'demo' }),
+    withContext(daemonContext, counterExample.context)(doublerAgent.section),
+  ),
+);
+
+test.serial.failing(
+  'sending-messages',
+  makeSectionTest(
+    $({ cwd: 'demo' }),
+    withContext(
+      daemonContext,
+      counterExample.context,
+      doublerAgent.context,
+    )(sendingMessages.section),
+  ),
+);
+
+test.serial.failing(
+  'names-in-transit',
+  makeSectionTest(
+    $({ cwd: 'demo' }),
+    withContext(
+      daemonContext,
+      counterExample.context,
+      doublerAgent.context,
+      sendingMessages.context,
+    )(namesInTransit.section),
+  ),
+);
+
+test.serial.failing(
+  'mailboxes-are-symmetric',
+  makeSectionTest(
+    $({ cwd: 'demo' }),
+    withContext(
+      daemonContext,
+      counterExample.context,
+      doublerAgent.context,
+      sendingMessages.context,
+      namesInTransit.context,
+    )(mailboxesAreSymmetric.section),
+  ),
+);
+
+test.serial(
+  'confined-script',
+  makeSectionTest(
+    $({ cwd: 'demo' }),
+    withContext(daemonContext)(confinedScript.section),
+  ),
+);
diff --git a/packages/cli/test/demo/mailboxes-are-symmetric.js b/packages/cli/test/demo/mailboxes-are-symmetric.js
new file mode 100644
index 000000000..342d1bf2d
--- /dev/null
+++ b/packages/cli/test/demo/mailboxes-are-symmetric.js
@@ -0,0 +1,14 @@
+/** @import {TestRoutine} from '../types' */
+
+/** @type {TestRoutine} */
+export const section = async (execa, testLine) => {
+  // Guests can also send their host messages...
+  await testLine(
+    execa`endo send HOST --as alice-agent ${'This is the @doubler you sent me.'}`,
+  );
+  await testLine(execa`endo inbox`, {
+    stdout: /^0\. "alice" sent "This is the @doubler you sent me\."/,
+  });
+  await testLine(execa`endo adopt 0 doubler doubler-from-alice`);
+  await testLine(execa`endo dismiss 0`);
+};
diff --git a/packages/cli/test/demo/names-in-transit.js b/packages/cli/test/demo/names-in-transit.js
new file mode 100644
index 000000000..6f711a312
--- /dev/null
+++ b/packages/cli/test/demo/names-in-transit.js
@@ -0,0 +1,27 @@
+/** @import {Context, TestRoutine} from '../types' */
+
+/** @type {TestRoutine} */
+export const section = async (execa, testLine) => {
+  // In this example, we send alice our "doubler" but let it appear...
+  await testLine(
+    execa`endo send alice ${'Please enjoy this @counter:doubler.'}`,
+  );
+  await testLine(execa`endo inbox --as alice-agent`, {
+    stdout: /^1\. "HOST" sent "Please enjoy this @counter\."/,
+  });
+  await testLine(execa`endo adopt --as alice-agent 1 counter --name redoubler`);
+  await testLine(execa`endo list --as alice-agent`, {
+    stdout: 'redoubler',
+  });
+  await testLine(execa`endo dismiss --as alice-agent 1`);
+};
+
+/** @type {Context} */
+export const context = {
+  setup: async execa => {
+    await execa`endo send alice ${'Please enjoy this @counter:doubler.'}`;
+    await execa`endo inbox --as alice-agent`;
+    await execa`endo adopt --as alice-agent 1 counter --name redoubler`;
+    await execa`endo dismiss --as alice-agent 1`;
+  },
+};
diff --git a/packages/cli/test/demo/sending-messages.js b/packages/cli/test/demo/sending-messages.js
new file mode 100644
index 000000000..38522e1e6
--- /dev/null
+++ b/packages/cli/test/demo/sending-messages.js
@@ -0,0 +1,28 @@
+/** @import {Context, TestRoutine} from '../types' */
+
+/** @type {TestRoutine} */
+export const section = async (execa, testLine) => {
+  // So far, we have run guest programs like the doubler. Guests and hosts can exchange messages...
+  await testLine(execa`endo mkguest alice alice-agent`, {
+    stdout: 'Object [Alleged: EndoGuest] {}',
+  });
+  await testLine(execa`endo send alice ${'Please enjoy this @doubler.'}`);
+  await testLine(execa`endo inbox --as alice-agent`, {
+    stdout: /^0\. "HOST" sent "Please enjoy this @doubler\."/,
+  });
+  await testLine(execa`endo adopt --as alice-agent 0 doubler`);
+  await testLine(execa`endo list --as alice-agent`, {
+    stdout: 'doubler',
+  });
+  await testLine(execa`endo dismiss --as alice-agent 0`);
+};
+
+/** @type {Context} */
+export const context = {
+  setup: async execa => {
+    await execa`endo mkguest alice alice-agent`;
+    await execa`endo send alice ${'Please enjoy this @doubler.'}`;
+    await execa`endo adopt alice-agent 0 doubler`;
+    await execa`endo dismiss --as alice-agent 0`;
+  },
+};
diff --git a/packages/cli/test/section.js b/packages/cli/test/section.js
new file mode 100644
index 000000000..0e4d93a3a
--- /dev/null
+++ b/packages/cli/test/section.js
@@ -0,0 +1,32 @@
+/** @import {Execa} from 'execa' */
+/** @import {t} from 'ava' */
+/** @import {TestRoutine} from './types' */
+
+/**
+ * Transforms a testRoutine into an ava test.
+ * The testCommand function asserts that a given awaitable command produces the expected stdout and stderr.
+ *
+ * @param {Execa} execa - the command execution environment
+ * @param {TestRoutine} testRoutine - the test logic implementation
+ * @returns {(t: t) => Promise<void>}
+ */
+export function makeSectionTest(execa, testRoutine) {
+  return async t => {
+    const matchExpecation = (expectation, result, errMsg) => {
+      (expectation instanceof RegExp ? t.regex : t.is)(
+        result,
+        expectation ?? '',
+        errMsg,
+      );
+    };
+    const testCommand = async (command, expectation) => {
+      const result = await command;
+      if (expectation !== undefined) {
+        const errMsg = JSON.stringify({ expectation, result }, null, 2);
+        matchExpecation(expectation.stdout, result.stdout, errMsg);
+        matchExpecation(expectation.stderr, result.stderr, errMsg);
+      }
+    };
+    await testRoutine(execa, testCommand);
+  };
+}
diff --git a/packages/cli/test/types.d.ts b/packages/cli/test/types.d.ts
new file mode 100644
index 000000000..12f42ea6d
--- /dev/null
+++ b/packages/cli/test/types.d.ts
@@ -0,0 +1,18 @@
+import type { Execa } from 'execa';
+
+export type Expectation = {
+  stdout: RegExp | string | undefined;
+  stderr: RegExp | string | undefined;
+};
+export type TestCommand = (
+  command: ReturnType<Execa>,
+  expectation: Expectation,
+) => Promise<true>;
+export type TestRoutine = (
+  execa: Execa,
+  testCommnd: TestCommand,
+) => Promise<void>;
+export type Context = {
+  setup: (execa: Execa) => Promise<void>;
+  teardown?: (execa: Execa) => Promise<void>;
+};
diff --git a/packages/cli/test/with-context.js b/packages/cli/test/with-context.js
new file mode 100644
index 000000000..f63fce32a
--- /dev/null
+++ b/packages/cli/test/with-context.js
@@ -0,0 +1,50 @@
+/** @import {Context, TestRoutine} from '../types' */
+/**
+ * Creates a wrapper which wraps a test routine with an execa-curried setup and teardown routine.
+ *
+ * @param {Context} context
+ * @returns {(testRoutine: TestRoutine) => TestRoutine}
+ */
+const makeContextWrapper =
+  context => testRoutine => async (execa, testCommand) => {
+    await null;
+    try {
+      await context.setup(execa);
+      await testRoutine(execa, testCommand);
+    } finally {
+      await context.teardown?.(execa);
+    }
+  };
+
+/**
+ * Creates a wrapper which wraps a test routine with execa-curried setup and teardown routines.
+ *
+ * Context args are provided from outermost to inner-most wrapping, e.g.
+ *
+ * ```
+ * await makeContextWrappers(a, b)(c);
+ * ```
+ *
+ * is approximately equivalent to
+ *
+ * ```
+ * for ( var f of [a.setup, b.setup, c, b.teardown, a.teardown] ) {
+ *   await f();
+ * }
+ * ```
+ *
+ * with the important distinction that teardown routines execute as long as their corresponding
+ * setup was called, even if that setup or any calls in between failed.
+ *
+ * @param {...Context} contexts - the conjugations to be applied onion-wise
+ * @returns {(testRoutine: TestRoutine) => TestRoutine}
+ */
+export const withContext =
+  (...contexts) =>
+  testRoutine => {
+    let composition = testRoutine;
+    for (const context of contexts.reverse()) {
+      composition = makeContextWrapper(context)(composition);
+    }
+    return composition;
+  };