npm package: @saturnbtcio/arch-testkit
Description: Config-driven E2E orchestrator for Arch. It ships with a built-in Docker Compose file and a CLI that brings services up, seeds regtest, runs migrations, and deploys programs based on a simple YAML manifest.
Author: Saturn BTC
Latest version: 0.0.16

## Arch Testkit

Config-driven E2E orchestrator for Arch. It ships with a built-in Docker Compose file and a CLI that brings services up, seeds regtest, runs migrations, and deploys programs based on a simple YAML manifest.

### What changed recently

- Built-in compose is always used. External compose overrides are not supported.
- Profiles drive which services start. Core services are `bitcoind`, `titan`, and `local-validator`. Optional: `ord`, `oracle`.
- Indexer and indexer-server are no longer part of this compose; manage them from your indexing stack/testkits.
- New CLI commands: `run-all`, `logs`, `reset`, and granular `deploy` filtering.

## Prerequisites

- Docker + Docker Compose v2
- Node.js + pnpm
- Git
- jq (required for some wallet-seeding flows)
- Optional: ord binary if you enable the `ord` profile

## Images you need

- Titan: build from https://github.com/SaturnBTC/Titan
  - `docker build -t titan-titan .`
- Ord (optional): build from https://github.com/ordinals/ord
  - Update Rust in Dockerfile to 1.81.0, then `docker build -t ord .`
- Oracle (optional): build from your oracle repo
  - `docker build -t oracle .`

Verify with:

```
docker image ls titan-titan ord oracle
```

## Compose environment

Most variables now have sane defaults baked into the compose. You generally do not need a local `.env` anymore. If you want to override ports or usernames/passwords, you can still create `ts/packages/arch-testkit/compose/.env` and set any of the following:

```
BITCOIN_NETWORK=regtest
BITCOIN_RPC_PORT=18443
BITCOIN_RPC_USERNAME=bitcoin
BITCOIN_RPC_PASSWORD=bitcoinpass

TITAN_TCP_PORT=8080
TITAN_HTTP_PORT=3030
COMMIT_INTERVAL=100

ARCH_RPC_PORT=9002
ARCH_WS_PORT=9003
NETWORK_MODE=localnet

ORD_HTTP_PORT=8081
ORD_WALLET_NAME=ord
 
# Optional overrides for local-validator binary selection
# LOCAL_VALIDATOR_BIN=/app/local_validator_x64
# LOCAL_VALIDATOR_ARCH=amd64   # or arm64
```

Tip: set `ARCH_E2E_LOG_LEVEL=debug` for verbose orchestrator logs.

### Do I need a .env?

- In most cases, **no**. The testkit ships with sensible defaults and works out of the box.
- There are two optional places you might use environment files:
  - **Docker Compose overrides**: put variables in `ts/packages/arch-testkit/compose/.env` to change container ports/credentials used by compose. See the keys listed in the section above; we don’t repeat them here.
  - **Runtime CLI overrides**: the CLI loads environment variables at runtime (via `dotenv/config`). If a `.env` exists in your current working directory when running `arch-e2e`, its variables override defaults used by the orchestrator (e.g., for `bitcoin-cli`, `ord`). Prefer running `arch-e2e env` and sourcing its output to get the canonical set. See “Setup manifest environment” below.

## Manifest schema

The CLI consumes a YAML manifest that declares service profiles, seeding, migrations, and deployments.

```yaml
services:
  profiles: [core]  # core, ord, oracle, indexing (indexing not managed here)
  seed:
    bitcoin:
      wallet: ${BITCOIN_WALLET:-testwallet}
      blocks: 150

migrations:
  - name: example
    cmd: echo "run migrations"

deployments:
  - name: liquidity_pool
    type: arch-program
    build:
      cmd: cargo build-sbf
      cwd: ${LP_PROGRAM_DIR}
    deploy:
      cmd: cargo test -p e2e-test -- create_program_regtest --nocapture
      cwd: ${SATURN_ARCH_PROGRAMS_REPO}/e2e-test
```

See `ts/e2e/pool-sdk-e2e/arch-e2e.yaml` for a full, working example (including oracle deployments).

## Setup manifest environment

Program build/deploy steps often rely on environment variables. To help other packages discover the running endpoints, use the built-in env export:

```
arch-e2e env > .env-arch
set -a && source ./.env-arch && set +a
```

This emits standard variables (e.g., `ARCH_VALIDATOR_URL`, `BITCOIN_RPC_*`, `TITAN_*`, `ORD_*`) with defaults that match the compose.
If you need custom program paths for deployments, set `SATURN_ARCH_PROGRAMS_REPO`, `LP_PROGRAM_DIR`, and `ORACLE_*` as needed in your shell or CI.

## Build and install

```
pnpm --filter @saturnbtcio/arch-testkit install
pnpm --filter @saturnbtcio/arch-testkit run build
```

This exposes the `arch-e2e` binary at `ts/packages/arch-testkit/dist/cli.js` and via your workspace bin.

## CLI usage

```
arch-e2e - Config-driven E2E orchestrator

Usage:
  arch-e2e up -m <manifest> [--no-detach] [--profile <name>]...
  arch-e2e down [--volumes]
  arch-e2e seed -m <manifest>
  arch-e2e migrate -m <manifest>
  arch-e2e deploy -m <manifest> [--name <name>]... [--all]
  arch-e2e deploy-all -m <manifest>
  arch-e2e restart <service>
  arch-e2e run-all -m <manifest> [--profile <name>]...
  arch-e2e reset
  arch-e2e logs [--no-follow] [service...]
  arch-e2e env
```

Notes:

- Commands that DO require a manifest: `up`, `run-all`, `seed`, `migrate`, `deploy`, `deploy-all`.
- Commands that DO NOT require a manifest: `logs`, `down`, `reset`, `restart`.
- If omitted, `-m` defaults to `arch-e2e.yaml` in the current directory for commands that need it.
 - `arch-e2e env` does not require a manifest and prints canonical connection variables for consumers to source.

### Common flows

- Bring up core services (bitcoind, titan, local-validator):

```
arch-e2e up -m ts/e2e/pool-sdk-e2e/arch-e2e.yaml --profile core
```

- Full one-shot flow (start, seed chain, start titan + optional ord, start validator, deploy):

```
arch-e2e run-all -m ts/e2e/pool-sdk-e2e/arch-e2e.yaml --profile core --profile ord
```

- Run migrations:

```
arch-e2e migrate -m ts/e2e/pool-sdk-e2e/arch-e2e.yaml
```

- Deploy programs (all or selected by name):

```
arch-e2e deploy -m ts/e2e/pool-sdk-e2e/arch-e2e.yaml --all
arch-e2e deploy -m ts/e2e/pool-sdk-e2e/arch-e2e.yaml --name liquidity_pool --name oracle_mempool
```

- Tail logs (all or specific services):

```
arch-e2e logs
arch-e2e logs titan bitcoind local-validator
```

- Restart a service in-place:

```
arch-e2e restart titan
```

- Bring everything down and clean volumes:

```
arch-e2e down --volumes
arch-e2e reset
```

Notes:

- The CLI always runs Docker Compose from `ts/packages/arch-testkit/compose`. Put your compose `.env` there.
- `run-all` intentionally does not start `oracle` or indexing services. If you want `oracle`, include the `oracle` profile and run `arch-e2e up` or start it manually.

## Optional: Install ord

If you enable the `ord` profile, install the binary:

```sh
curl --proto '=https' --tlsv1.2 -fsLS https://ordinals.com/install.sh | bash -s
```

You should then be able to run `ord --version`.

## Troubleshooting

- Docker cannot find variables: ensure you created `compose/.env` (see above) and that your terminal session has any manifest variables exported (e.g., via `.env-e2e`).
- Services unhealthy: check `arch-e2e logs ...` for `bitcoind` and `titan` first. The orchestrator waits for `bitcoind` to become healthy before proceeding.
- Verbose logs: set `ARCH_E2E_LOG_LEVEL=debug`.