npm package: @safetnsr/md-pipe
Description: Event-driven automation for markdown directories. entr watches files. md-pipe understands them.
Author: safetnsr
Latest version: 0.3.0
Deprecated: merged into @safetnsr/md-kit — use: npx @safetnsr/md-kit pipe

# md-pipe

Markdown content pipelines.
Write markdown → auto-publish everywhere.

12 lines of YAML → your markdown folder auto-publishes, auto-indexes, and auto-archives based on frontmatter changes. Works with any `.md` directory — Obsidian vaults, agent workspaces, docs-as-code repos, digital gardens.

## Install

```bash
npx @safetnsr/md-pipe init
```

Or install globally:

```bash
npm install -g @safetnsr/md-pipe
```

## Quick Start

```bash
# 1. Create config
md-pipe init

# 2. Edit .md-pipe.yml with your pipelines

# 3. Start watching
md-pipe watch
```

## Pipelines (v0.3+)

Pipelines are multi-step content workflows that fire on frontmatter changes:

```yaml
watch: ./docs

pipelines:
  - name: publish-post
    trigger:
      path: "posts/**"
      frontmatter: { status: publish }
      frontmatter_changed: [status]
    steps:
      - run: "echo Publishing {{fm.title}}"
      - copy: { to: "./_site/posts" }
      - update-frontmatter:
          published_at: "{{now}}"
          published: "true"
      - webhook:
          url: "$WEBHOOK_URL"
          body: { title: "{{fm.title}}", slug: "{{slug}}" }
```

Steps execute in order. If one fails, the pipeline stops (unless `continue_on_error: true`).

### Step Types

| Step | Description |
|---|---|
| `run` | Shell command. Supports template vars and `$ENV` vars |
| `update-frontmatter` | Write fields back to the source file's YAML frontmatter |
| `copy` | Copy file to a destination directory |
| `webhook` | POST JSON to a URL |
| `template` | Render a template file with context data |

### Template Variables

Available in all step configs:

| Variable | Description |
|---|---|
| `{{now}}` | ISO timestamp |
| `{{date}}` | YYYY-MM-DD |
| `{{slug}}` | Filename without extension |
| `{{file}}` | Absolute path |
| `{{basename}}` | Filename |
| `{{relative}}` | Path relative to watch dir |
| `{{dir}}` | Parent directory |
| `{{tags}}` | Comma-separated tags |
| `{{fm.title}}` | Frontmatter field |
| `{{step.0.stdout}}` | Output from step 0 |

### Pipeline Context

Each step's output is available to subsequent steps:

```yaml
steps:
  - run: "deploy.sh $FILE"                    # stdout captured
  - update-frontmatter:
      url: "{{step.0.stdout}}"                # use output from step 0
      deployed_at: "{{now}}"
```

### Manual Run

Test a pipeline without watching:

```bash
md-pipe run publish-post posts/hello.md
```

Run against all matching files:

```bash
md-pipe run publish-post
```

## Legacy Triggers

Simple triggers with a single `run` command still work (backward compatible):

```yaml
watch: ./docs

triggers:
  - name: publish
    match:
      path: "posts/**"
      frontmatter:
        status: publish
    run: "echo Publishing $FILE"
```

Triggers and pipelines can coexist in the same config.

## Commands

| Command | Description |
|---|---|
| `md-pipe init` | Scaffold a `.md-pipe.yml` config file |
| `md-pipe watch` | Start watching for changes and trigger actions |
| `md-pipe once` | Run triggers against current files (CI/batch mode) |
| `md-pipe run <pipeline> [file]` | Manually trigger a pipeline |
| `md-pipe test <file>` | Show which triggers/pipelines match a file |

## Flags

| Flag | Description |
|---|---|
| `--config, -c <path>` | Path to config file |
| `--dry-run` | Show matches without executing actions |
| `--json` | Output in JSON format |
| `--verbose` | Show trigger + file + first line of command |
| `--debug` | Show full interpolated commands |
| `--state <path>` | State file for idempotent `once` mode |

## Trigger Matchers

All matchers work in both `triggers` (via `match:`) and `pipelines` (via `trigger:`).

### `path` — glob pattern matching
```yaml
path: "posts/**/*.md"
```

### `frontmatter` — match specific values
```yaml
frontmatter:
  status: publish
  type: blog
```

### Negation
```yaml
frontmatter:
  status: "!draft"    # matches anything except draft
```

### `frontmatter_changed` — fires when specific fields change
```yaml
frontmatter_changed:
  - title
  - tags
```
This is the key differentiator. md-pipe tracks frontmatter state and only triggers when the fields you care about actually change.

### `tags` — match files with specific tags
```yaml
tags:
  - urgent
```

### `content` / `content_regex` — match body text
```yaml
content: "TODO"
content_regex: "\\[ \\]"   # unchecked checkboxes
```

Matchers can be combined — all conditions must match.

## Step Details

### `update-frontmatter`

Write fields back to the source markdown file:

```yaml
- update-frontmatter:
    published_at: "{{now}}"
    published: "true"          # coerced to boolean
    url: "{{step.0.stdout}}"   # from previous step
```

Values are coerced: `"true"` → `true`, `"false"` → `false`, numeric strings → numbers.

### `copy`

```yaml
- copy: { to: "./_site/posts" }           # preserves directory structure
- copy: { to: "./_site", flatten: true }   # flat copy
```

### `webhook`

```yaml
- webhook:
    url: "$WEBHOOK_URL"                     # env var expansion
    method: POST                            # default
    headers:
      Authorization: "Bearer $API_KEY"
    body:
      title: "{{fm.title}}"
      file: "{{relative}}"
```

### `template`

```yaml
- template:
    src: "./templates/post.html"
    out: "./_site/{{slug}}.html"
```

Template files use the same `{{variable}}` syntax.

### Error Handling

```yaml
pipelines:
  - name: resilient
    continue_on_error: true     # pipeline-level
    steps:
      - run: "might-fail.sh"
        continue_on_error: true # step-level
      - run: "always-runs.sh"
```

## Environment Variables

Shell commands (`run` steps and legacy triggers) receive:

| Variable | Description |
|---|---|
| `$FILE` | Absolute path |
| `$DIR` | Directory |
| `$BASENAME` | Filename |
| `$RELATIVE` | Relative path |
| `$SLUG` | Filename without extension |
| `$FRONTMATTER` | JSON string of all frontmatter |
| `$DIFF` | JSON of changed fields |
| `$TAGS` | Comma-separated tags |
| `$FM_<field>` | Direct frontmatter access |
| `$STEP_OUTPUT` | stdout from the previous step |
| `$STEP_N_STDOUT` | stdout from step N |

## Idempotent Mode (--state)

```bash
md-pipe once --state .md-pipe-state.json
# Only processes files that changed since last run
```

## Use Cases

- **Content pipeline** — write markdown, auto-publish to static site + CMS
- **Docs-as-code** — deploy pages when `status: publish` is set
- **Agent workspace** — trigger reindexing when agents update files
- **Obsidian vault** — auto-archive, generate backlinks, push to git
- **Digital garden** — auto-generate RSS, index pages, deploy
- **CI/CD** — `md-pipe once --state .state.json` for idempotent processing

## License

MIT