npm package: @industream/flowmaker-worker-scripts
Description: A comprehensive CLI tool for managing FlowMaker workers - distributed data processing pipelines built with a source-pipe-sink architecture. This package provides everything you need to scaffold, build, test, and deploy workers across multiple runtimes (No
Latest version: 1.0.9


# FlowMaker Worker CLI

A comprehensive CLI tool for managing FlowMaker workers - distributed data processing pipelines built with a source-pipe-sink architecture. This package provides everything you need to scaffold, build, test, and deploy workers across multiple runtimes (Node.js, Python, .NET).

## What is FlowMaker?

FlowMaker is a framework for building industrial data processing workflows. Workers are self-contained processing units that can:
- **Source**: Ingest data from external systems (APIs, databases, file systems)
- **Pipe**: Transform and process data through custom logic
- **Sink**: Output processed data to destinations or downstream systems

Each worker includes Docker containerization, optional frontend UI components, and follows a standardized structure for consistency across your data pipeline.

## Worker CLI

The worker CLI is available as an npm package and can be run via `npx`:

```bash
npx worker-cli <command> [options]
```

### Commands

#### `init` - Initialize a repository

Initialize a new FlowMaker workers repository in the current directory by copying template files (README.md, .gitignore, .gitattributes, .env, AGENTS/).

```bash
# Initialize in current directory
npx worker-cli init
```

#### `update-repo-guidelines` - Update guidelines

Update `README.repo-guidelines.md` and `AGENTS/*` files from templates.

```bash
# Update guidelines in current directory
npx worker-cli update-repo-guidelines
```

#### `create` - Create a new worker

Scaffolds a new worker from templates. Runs interactively by default; switches to non-interactive mode when `--type`, `--name`, `--box-type`, and `--icon` are all provided (useful for LLMs and CI).

```bash
# Interactive
npx worker-cli create

# Non-interactive
npx worker-cli create --type node --name my-worker --box-type pipe --icon code
npx worker-cli create --type dotnet --name my-worker --box-type source --icon sensors --display-name "My Worker" --no-build
```

All workers are scaffolded with a [flowbox-ui](https://www.npmjs.com/package/@industream/flowmaker-flowbox-ui) Svelte frontend.

**Non-interactive flags:**

| Flag | Description |
|------|-------------|
| `--type <node\|python\|dotnet>` | Worker runtime (required) |
| `--name <name>` | Worker name — letters and hyphens only, no digits (required) |
| `--box-type <source\|pipe\|sink>` | Worker box role (required) |
| `--icon <icon>` | Material Symbols icon name (required) |
| `--display-name <name>` | Human-readable name (optional, defaults to title-cased name) |
| `--no-build` | Skip initial build |

#### `build` - Build Docker images

Build Docker images for workers with multi-arch support (linux/amd64, linux/arm64). Supports parallel builds for multiple workers.

**Pre-build checks:** Before building, the CLI runs consistency checks and version validation. Use `--no-check` to skip these checks.

```bash
# Build a specific worker
npx worker-cli build example-box

# Build multiple workers in parallel
npx worker-cli build example-box another-box

# Build all workers and push to registry
npx worker-cli build --all --push

# Build without running tests
npx worker-cli build example-box --no-test

# Skip pre-build consistency and version checks
npx worker-cli build example-box --no-check
```

**Options:**

| Option | Description |
|--------|-------------|
| `--all` | Build all workers |
| `--push` | Push to remote registry (default: load locally) |
| `--test` | Test container after build (default for local) |
| `--no-test` | Skip container testing |
| `--no-check` | Skip pre-build consistency and version checks |
| `--parallel` | Enable parallel builds (default for multiple workers) |

#### `test` - Test container registration

Test a container's registration against a mock config-hub server.

```bash
npx worker-cli test <worker-dir> <image-name>
```

#### `check` - Version alignment

Display version information across all workers.

```bash
npx worker-cli check
```

#### `consistency` - Validate workers

Check worker structure, version, syntax, and issues. Runs all checks: naming conventions, required files, version format, TypeScript/Python/.NET syntax, frontend, and issues.yaml.

```bash
# Check all workers
npx worker-cli consistency

# Check specific workers
npx worker-cli consistency example-box
```

See [README.repo-guidelines.md](README.repo-guidelines.md) for detailed requirements.

## Frontend CDN Dependencies

Worker frontends are built as self-contained IIFE bundles (Svelte, compiled by Vite). Dependencies that would be too large to inline — like `lodash-es`, `dayjs`, etc. — are loaded at runtime from a CDN rather than bundled.

### The `@cdn-cache/` import convention

Use the `@cdn-cache/` virtual prefix to declare a CDN-loaded dependency:

```js
// In your ConfigForm.svelte or main.js
const { default: dayjs } = await import('@cdn-cache/dayjs@1.11.10');
```

The Vite build plugin (`cdnCacheProd`) enforces two rules at build time:
1. The package must be listed in `frontend/package.json` under `peerDependencies` with the **exact same version**.
2. The version must be exact semver (`x.y.z`) — no ranges.

The import is kept external in the output bundle as `http://@cdn-cache/dayjs@1.11.10`. At runtime, FlowMaker replaces the `http://@cdn-cache/` prefix with the actual CDN base URL before executing the bundle.

### Self-hosted CDN (cdn-esm docker)

The CDN is served by a self-hosted [esm.sh](https://esm.sh) instance built from source — see `cdn-cache/cdn-esm/` in the repository root. This allows air-gapped deployments with no dependency on public CDNs.

Packages are resolved and cached on first request. No pre-registration of packages is needed: declaring `peerDependencies` in `package.json` is the only tracking mechanism required.

## Configuration

The `.env` file at the repository root configures local development settings. Running `npx worker-cli init` copies a template `.env` with all available variables commented out.

| Variable | Required | Description |
|----------|----------|-------------|
| `REGISTRY` | Yes (for `--push`) | Docker registry to push images to. Bring your own registry server or use Docker Hub (`docker.io/myorg`). |
| `IMAGE_PREFIX` | No | Prefix inserted between the registry and the worker name — e.g. `myteam/` → `registry.example.com/myteam/my-worker:1.0.0` |
| `CONFIG_HUB_MOCK_PORT` | No | Port for mock config-hub during container tests (default: `3120`) |

Example `.env`:

```bash
REGISTRY=registry.example.com
IMAGE_PREFIX=myteam/
```

## Setup

### Prerequisites

| Tool | Required for | Install |
|------|-------------|---------|
| Node.js 22+ | CLI, node workers | [nodejs.org](https://nodejs.org/) |
| pnpm | Node dependency management | `npm install -g pnpm` |
| Docker + buildx | Building container images | [docker.com](https://docs.docker.com/get-docker/) |
| .NET SDK 9.0+ | dotnet workers only | [dotnet.microsoft.com](https://dotnet.microsoft.com/download) |
| Python 3.11+ | python workers only | [python.org](https://www.python.org/downloads/) |

### Quick setup

```bash
# Verify prerequisites
npx worker-cli doctor
```

### Dependencies & registries

All npm packages (including `@industream/flowmaker-sdk` and `@industream/flowmaker-flowbox-ui`) are published on the **public npm registry** (npmjs.org). No private registry configuration or `.npmrc` is needed.

Docker images are pushed to whichever registry you configure in `REGISTRY` (your own registry server or Docker Hub).

### Adding a New Worker

The recommended way is to use the generator:

```bash
# Interactive
npx worker-cli create

# Non-interactive (for LLMs / CI)
npx worker-cli create --type node --name my-worker --box-type pipe --icon code
```

This scaffolds the worker directory (including a Svelte flowbox-ui frontend) and optionally runs the initial build.

To do it manually:

1. Create a new directory under `workers/` (lowercase letters and hyphens only — no digits; SDK constraint)
2. Add `Dockerfile` following the existing multi-stage pattern
3. Add `README.md` documenting the worker
4. Create `src/` directory for source code:
    - **Node.js**: Add `package.json` with name `@industream/flow-box-<worker-name>`
    - **.NET**: Add `*.csproj` with `<OutputType>Exe</OutputType>` and `<Version>`
    - **Python**: Add `pyproject.toml` with `name` and `version`
5. Optionally add frontend components in `frontend/`
6. Run `npx worker-cli consistency <worker-name>` to check structure

### Building Locally

```bash
# Build a specific worker (loads to local Docker)
npx worker-cli build example-box

# Build and push to registry
npx worker-cli build example-box --push
```