flywheel-bootstrap-staging 0.1.9.202602161939
Bootstrap runner for Flywheel provisioned GPU instances
License: MIT
Requires Python: >=3.11
Keywords: bootstrap, flywheel, gpu, machine-learning, ml

# Bootstrap

This package hosts the BYOC bootstrapper that:

- Ensures a pinned `codex` release is available (GitHub release assets).
- Fetches the bootstrap payload for a run from the Flywheel backend.
- Launches Codex app-server and streams notifications as logs.
- Collects artifacts (manifest-on-exit) and reports completion or error back to
  the backend.

## Configuration

Bootstrap reads the user's Codex `config.toml` (Codex schema). See:

- Codex config basics: <https://developers.openai.com/codex/config-basic>
- Codex config reference: <https://developers.openai.com/codex/config-reference>

Flywheel adds a small extension under `[flywheel]` and requires one of:

```toml
[flywheel]
# inline instructions (host-specific tips, paths, sandbox notes)
workspace_instructions = """
Use /mnt/work as your workspace. Write artifacts under ./artifacts.
"""

# or: reference a file (relative paths are resolved against the config file directory)
workspace_instructions_file = "workspace_notes.md"
```

Rules:

- At least one of `workspace_instructions` or `workspace_instructions_file` is
  required; otherwise bootstrap exits before contacting the server.
- If both are set, the file wins and the inline value is ignored (warns once).
- File contents must be non-empty; the path is resolved relative to the config
  file if not absolute.

Bootstrap also reads a few optional Codex config fields because they affect
workspace + sandbox behavior:

- `cd` / `workspace_dir` (run working directory; defaults to `~/.flywheel/runs/<run_id>`)
- `sandbox_mode` and `[sandbox_workspace_write].writable_roots`

Prompt assembly (sent as the first Codex `turn/start`):

1. Flywheel engineer context (logging/artifact expectations).
2. Task Description (prompt fetched from the server).
3. Workspace Instructions (resolved from config as above).

Full example config (recommended starting point):

- <https://github.com/paradigma-inc/paradigma/blob/main/project/bootstrap/examples/config.example.toml>

Update the paths and instructions for your machine.

## End-to-end flow (bootstrap.sh → Python bootstrapper)

1. User runs the bootstrapper on their BYOC machine:

   ```bash
   BOOTSTRAP_PACKAGE="${FLYWHEEL_BOOTSTRAP_PACKAGE:-flywheel-bootstrap}"
   uvx --no-cache --from "$BOOTSTRAP_PACKAGE" flywheel-bootstrap \
     --run-id <id> --token <token> --config /path/to/config.toml [--server <url>]
   ```

   Optional: if you have this repo checked out, you can run `project/bootstrap/bootstrap.sh`,
   which installs `uvx` if missing and then runs the same command. It defaults
   `FLYWHEEL_BOOTSTRAP_PACKAGE` to the local `project/bootstrap` path.

2. Python entrypoint (`python -m bootstrap`):
   - Parses args/env: requires run id + token, required `--config`, optional `--server` (default `http://localhost:8000`).
   - Loads Codex config.toml, enforces presence of workspace instructions (inline or file), extracts workspace/sandbox settings.
3. Workspace resolution:
   - Uses `cd`/`workspace_dir` from config if set; otherwise `~/.flywheel/runs/<run_id>`.
   - Creates the workspace and validates the artifact manifest path is inside sandbox `writable_roots` when sandboxing is enabled; else exits with an error.
4. Engineer runtime availability:
   - If `BOOTSTRAP_MOCK_ENGINEER` is set, skips agent launch and runs a mock flow (used in tests).
   - Else, download a pinned Codex release asset and mark it executable.
5. Fetch bootstrap payload:
   - `GET <server>/runs/<run_id>/bootstrap` with `X-Run-Token`; payload contains the task prompt.
6. Build prompt text:
   - Combine base Flywheel engineer context, “Task Description” (server prompt), and “Workspace Instructions” (user config) into one prompt string.
7. Launch engineer via Codex app-server:
   - Create a per-run `CODEX_HOME` directory at `<workspace>/.codex_home` containing a copied `config.toml` (with `~/` expanded).
   - Best-effort copy auth credentials from `~/.codex/auth.json` into `<workspace>/.codex_home/auth.json` so Codex can authenticate.
   - Launch `codex app-server` as a subprocess and speak JSON-RPC over stdio (newline-delimited JSON; `"jsonrpc":"2.0"` header omitted):
     - `initialize` + `initialized`
     - `thread/start`
     - `turn/start` (first prompt = full Flywheel prompt; subsequent prompts = autonomous resume)
     - `turn/interrupt` for takeover/resume
     - Stream `turn/*`, `item/*`, and delta notifications (`item/agentMessage/delta`, `item/commandExecution/outputDelta`, etc.)
   - Bootstrap auto-accepts command/file approvals (for-session) when Codex requests them.
   - Start a heartbeat thread posting `/runs/{id}/heartbeat`.
   - Start a run-control polling thread (`/runs/{id}/control/poll`) to handle:
     - `take_over` (interrupt autonomous prompt turn; enter interactive mode)
     - `send_input` (start a new Codex turn with user input in interactive mode)
     - `resume_autonomous` (return to autonomous mode)
   - Convert `thread/tokenUsage/updated` totals into a `turn.completed` usage log for billing.
   - Stream app-server notifications as logs to `/runs/{id}/logs`.
8. After the engineer prompt turn ends:
   - If code persistence is enabled, bootstrap commits/pushes non-artifact changes:
     - Excludes manifest-declared artifact paths from the git commit
     - Skips oversized files (default cap: 95 MiB, configurable via `FLYWHEEL_GIT_MAX_COMMIT_FILE_SIZE_BYTES`)
   - Artifact files referenced by `payload.path`/`payload.file` are embedded into artifact payloads
     when they are below the transfer threshold (default: 16 MiB, configurable via
     `FLYWHEEL_ARTIFACT_BLOB_THRESHOLD_BYTES`).
   - Read `flywheel_artifacts.json`; if missing/malformed/empty, retry up to `MAX_ARTIFACT_RETRIES` by prompting the engineer to fix/write the manifest.
   - POST artifacts to `/runs/{id}/artifacts`; POST `/complete` on exit 0, else `/error` with the exit code.
   - Stop/join the heartbeat thread.
9. Mock mode (`BOOTSTRAP_MOCK_ENGINEER=1`):
   - Sends a heartbeat, a few logs, writes a mock artifact manifest, returns 0 (used in e2e tests).

## Next steps

- Iterate on prompts / general polish.