npm package: @jondotsoy/ymake
Description: A Make-like task runner with YAML files, featuring modern capabilities like matrix builds and structured outputs.
Latest version: 1.5.3

# ymake

A Make-like task runner with YAML files, featuring modern capabilities like matrix builds and structured outputs.

## Key Features

- 🎯 **Simple YAML syntax**: Define tasks clearly and readably
- 🔗 **Dependency management**: Automatically executes tasks in the correct order
- 🔄 **Matrix builds**: Run tasks with multiple parameter combinations
- 📤 **Structured outputs**: Capture and reuse JSON data between tasks
- 🚀 **Smart detection**: Automatically finds your Ymakefile
- ⚡ **Fast**: Built with Bun for maximum performance

## Installation

### Global Installation (Recommended)

```bash
npm install -g @jondotsoy/ymake
```

Or using Bun:

```bash
bun install -g @jondotsoy/ymake
```

### Local Installation

```bash
npm install --save-dev @jondotsoy/ymake
```

Or using Bun:

```bash
bun add --dev @jondotsoy/ymake
```

## Usage

```bash
ymake [target] [options]
```

### Options

- `[target...]`: Name(s) of the target(s) to execute (optional, uses the first one if not specified). You can specify multiple targets separated by spaces.
- `--all`: Execute all targets defined in the Ymakefile in order
- `--verbose`, `-v`: Shows detailed execution information and outputs
- `--version`: Display the ymake version
- `--help`, `-h`: Show help message with available options
- `--file <path>`, `-f <path>`, `--ymakefile <path>`: Path to the Ymakefile to use (instead of auto-detecting)
- `--cwd <path>`: Change working directory before executing

### Examples

```bash
# Execute the first target
ymake

# Execute a specific target
ymake build

# Execute multiple targets in order
ymake test build

# Execute all targets in order
ymake --all

# Execute with verbose output
ymake test --verbose

# Use a custom Ymakefile location
ymake -f path/to/ymakefile.yml build

# Use a Ymakefile from a subdirectory
ymake --file config/prod/ymakefile.yml deploy

# Change working directory before executing
ymake --cwd dist build

# Combine --cwd with custom Ymakefile location
ymake --cwd dist -f prod/ymakefile.yml deploy
```

### Supported file names

ymake automatically searches for these files in order:

- `Ymakefile`
- `Ymakefile.yaml`
- `Ymakefile.yml`
- `ymakefile`
- `ymakefile.yaml`
- `ymakefile.yml`

### Custom Ymakefile Location

You can specify a custom Ymakefile location using the `--file`, `-f`, or `--ymakefile` flag. When you do this, ymake will:

1. Load the specified Ymakefile
2. Change the working directory to the directory containing that Ymakefile
3. Execute all commands relative to that directory

This is useful for:

- Managing multiple environments (dev, staging, prod)
- Organizing Ymakefiles in subdirectories
- Running builds from CI/CD pipelines

Example:

```bash
# Use a Ymakefile in a subdirectory
ymake -f environments/prod/ymakefile.yml deploy

# Files created by the target will be in environments/prod/
```

### Custom Working Directory

You can change the working directory before executing targets using the `--cwd` flag. This is particularly useful when:

- Running ymake from a different directory than where your Ymakefile is located
- Working with monorepo structures
- Executing builds in specific subdirectories

When using `--cwd`:

1. ymake changes to the specified directory first
2. Then loads the Ymakefile (either auto-detected or specified with `--file`)
3. If `--file` is used with `--cwd`, the working directory remains at the `--cwd` location (not the Ymakefile's directory)

Example:

```bash
# Execute in a subdirectory
ymake --cwd dist build

# Combine with custom Ymakefile location
# This will change to dist/, use prod/ymakefile.yml, and execute commands in dist/
ymake --cwd dist -f prod/ymakefile.yml deploy
```

Given this directory structure:

```
project/
├── ymake/
│   └── prod/
│       └── ymakefile.yml
```

And this ymakefile content (`ymake/prod/ymakefile.yml`):

```yaml
build:
  run: echo "hello" >> build.txt
```

Running from the project root:

```bash
ymake -f ymake/prod/ymakefile.yml build
```

Will create `build.txt` in `ymake/prod/build.txt` (not in the project root), because the working directory changes to `ymake/prod/`.

## Feature Guide

### 1. Basic Targets

Define targets with the command to execute. ymake supports two syntaxes:

#### Object Syntax

```yaml
build:
  run: echo "Building..."

test:
  need: [build]
  run: echo "Testing..."
```

#### String Syntax (Shorthand)

For simple targets without dependencies or options, you can use a string directly:

```yaml
# Single line
hello: echo "Hello, World!"

# Multi-line with pipe operator
build: |
  echo "Building..."
  mkdir -p dist
  echo "Done"
```

#### Mixed Syntax

You can mix both syntaxes in the same file:

```yaml
# String syntax
hello: echo "Hello"

# Object syntax with dependencies
test:
  need: [hello]
  run: echo "Testing..."
```

Each target requires either a `run` property (object syntax) or a command string (string syntax).

#### Target Environment Variable

The `$TARGET` environment variable is automatically available in all commands and contains the name of the current target:

```yaml
hello.txt:
  run: echo "hello" > $TARGET
```

This creates the file `hello.txt` using the target name. This is especially useful for file targets where the target name matches the output file.

### 2. Dependencies

Use `need` to specify dependencies between targets. You can use a single string or an array of strings:

```yaml
# Single dependency
test:
  need: build
  run: echo "Testing..."

# Multiple dependencies
deploy:
  need: [build, test]
  run: echo "Deploying..."
```

Dependencies are executed in order and only once, even if multiple targets need them.

#### File Dependencies

Dependencies can be either targets or files. If a dependency is not a target, ymake checks if it exists as a file:

```yaml
hello.txt:
  need: lang.txt
  run: echo "($(cat lang.txt)) hello" > hello.txt
```

In this example, `lang.txt` is not a target but a file that must exist. If the file doesn't exist, ymake will fail with an error. This allows you to specify file prerequisites without defining them as targets.

### 3. Phony Targets

Mark targets as phony so they always execute, without checking files:

```yaml
clean:
  phony: true
  run: rm -rf dist/
```

### 4. Matrix Builds

Execute a target with multiple parameter combinations:

```yaml
test:
  matrix:
    platform: [linux, macos, windows]
    version: [18, 20]
  run: echo "Testing on $platform with Node $version"
```

This will execute the command 6 times (3 platforms × 2 versions), with each combination available as environment variables.

#### File Targets with Matrix

When using matrix with file targets, ymake checks each file individually and only executes the necessary combinations:

```yaml
files:
  name: files/${foo}.txt
  matrix:
    foo: [tar, biz, fod]
  run: |
    mkdir -p files
    echo hello > files/${foo}.txt
```

First execution: creates `files/tar.txt`, `files/biz.txt`, `files/fod.txt`

If you add a new element to the matrix:

```yaml
files:
  name: files/${foo}.txt
  matrix:
    foo: [tar, biz, fod, woo] # new element: woo
  run: |
    mkdir -p files
    echo hello > files/${foo}.txt
```

Second execution: only creates `files/woo.txt` (the other files already exist)

This is useful for generating multiple files incrementally without regenerating existing ones.

### 5. Structured Outputs

#### Capturing Outputs

Capture JSON data from a target using the `$OUTPUTS` environment variable:

```yaml
prepare:
  outputs: true
  run: |
    echo '{"version": "1.0.0", "build": 42}' > $OUTPUTS
```

With `outputs: true`, all JSON data is captured but not automatically exposed.

#### Specific Outputs

Specify which keys you want to capture and reuse:

```yaml
prepare:
  outputs: [version, build]
  run: |
    echo '{"version": "1.0.0", "build": 42}' > $OUTPUTS

deploy:
  need: [prepare]
  run: |
    echo "Deploying version $prepare_version build $prepare_build"
```

Outputs are exposed as environment variables with the format `{target}_{key}`.

#### Outputs in Matrix

Use arrays in outputs to generate matrix combinations:

```yaml
prepare:
  outputs: [platforms]
  run: |
    echo '{"platforms": ["linux", "macos", "windows"]}' > $OUTPUTS

build:
  need: [prepare]
  matrix:
    platform: $prepare_platforms
  run: echo "Building for $platform"
```

This will execute the build 3 times, once for each platform.

#### Dynamic Matrix from Outputs with Wildcards

You can combine outputs, matrices, and wildcard patterns to create powerful dynamic workflows. When a pattern target's matrix references outputs from another target, ymake automatically:

1. Executes the dependency to get the outputs
2. Uses those outputs to populate the matrix
3. Expands wildcard dependencies based on the dynamic matrix

```yaml
all:
  phony: true
  need:
    - reports/d/*

generate_dates:
  outputs: true
  run: |
    echo '{"dates": ["2024-01-01", "2024-01-02", "2024-01-03"]}' > "${OUTPUTS}"

reports/d/{date}:
  need: generate_dates
  matrix:
    date: $generate_dates_dates
  run: |
    mkdir -p reports/d
    echo "Report for date: ${date}" > reports/d/${date}
```

When you run `ymake all`:

1. `generate_dates` executes first and produces the list of dates
2. The `reports/d/*` wildcard is expanded using the dates from the output
3. Each report is generated for the computed dates

This enables scenarios like:

- Generating reports for dynamically determined dates
- Building artifacts for regions/environments from a config service
- Processing files based on a manifest generated at runtime

See the [dynamic-matrix-from-outputs sample](samples/dynamic-matrix-from-outputs) for a complete example.

#### Mixing Outputs with Values

Combine output references with static values:

```yaml
prepare:
  outputs: [version]
  run: |
    echo '{"version": "1.0.0"}' > $OUTPUTS

test:
  need: [prepare]
  matrix:
    env: [dev, staging, $prepare_version]
  run: echo "Testing in $env"
```

Output:

```
Testing in dev
Testing in staging
Testing in 1.0.0
```

### 6. Output Persistence (Cache)

ymake automatically caches target outputs to avoid unnecessary re-executions, making your builds faster and more efficient.

#### How It Works

When a target with `outputs` completes successfully, ymake saves its output to `.ymake/info/outputs/<target>.json`. On subsequent runs, if the cached output exists, ymake reuses it without re-executing the target.

**Key Features:**

- **Automatic caching**: Outputs are persisted automatically after successful execution
- **Smart reuse**: Cached outputs are loaded and made available to dependent targets
- **Phony targets excluded**: Targets with `phony: true` never cache outputs (always re-execute)
- **Matrix support**: Each matrix combination gets its own cache file
- **Git-friendly**: `.ymake/.gitignore` is automatically created with `*` to exclude cache from version control

#### Cache File Location

Outputs are stored in:

- **Simple targets**: `.ymake/info/outputs/<target-name>.json`
- **Matrix targets**: `.ymake/info/outputs/<target-name>-<key1>_<value1>-<key2>_<value2>.json`

#### Example: Basic Caching

```yaml
prepare:
  outputs: [version, build_number]
  run: |
    # This expensive computation only runs once
    VERSION=$(git describe --tags)
    BUILD_NUM=$(date +%s)
    echo "{\"version\": \"$VERSION\", \"build_number\": $BUILD_NUM}" > $OUTPUTS

build:
  need: [prepare]
  run: echo "Building version $prepare_version (build $prepare_build_number)"
```

**First execution:**

- `prepare` runs and creates `.ymake/info/outputs/prepare.json`
- `build` uses the outputs from `prepare`

**Second execution:**

- `prepare` is skipped (cached output is used)
- `build` runs with the same cached outputs from `prepare`

#### Example: Matrix Caching

```yaml
test:
  outputs: [test_result]
  matrix:
    platform: [linux, macos, windows]
    node: [18, 20]
  run: |
    # Run tests for this combination
    npm test --platform=$platform --node=$node
    echo "{\"test_result\": \"passed\"}" > $OUTPUTS
```

This creates 6 cache files:

- `.ymake/info/outputs/test-node_18-platform_linux.json`
- `.ymake/info/outputs/test-node_18-platform_macos.json`
- `.ymake/info/outputs/test-node_18-platform_windows.json`
- `.ymake/info/outputs/test-node_20-platform_linux.json`
- `.ymake/info/outputs/test-node_20-platform_macos.json`
- `.ymake/info/outputs/test-node_20-platform_windows.json`

**Incremental builds**: If you add a new platform, only the new combinations execute.

#### Phony Targets and Caching

Targets marked as `phony: true` always execute and never cache their outputs:

```yaml
timestamp:
  phony: true # Always runs, never cached
  outputs: [current_time]
  run: |
    echo "{\"current_time\": \"$(date)\"}" > $OUTPUTS

deploy:
  need: [timestamp]
  run: echo "Deploying at $timestamp_current_time"
```

#### Cache Invalidation

To force re-execution and regenerate cached outputs:

1. Delete the `.ymake/info/outputs` directory
2. Delete specific cache files
3. Modify the target's command or dependencies

#### Static Outputs (No Caching)

Static outputs don't need caching since they're defined directly in the Ymakefile:

```yaml
config:
  outputs:
    version: "1.0.0"
    environment: "production"

deploy:
  need: [config]
  run: echo "Deploying $config_version to $config_environment"
```

## Complete Example

See `Ymakefile.example` for a complete example demonstrating all features.

```yaml
# Prepare build information
prepare:
  outputs: [version, platforms]
  run: |
    echo '{"version": "1.0.0", "platforms": ["linux", "macos"]}' > $OUTPUTS

# Build using outputs
build:
  need: [prepare]
  run: echo "Building version $prepare_version"

# Matrix tests with outputs
test:
  need: [build]
  matrix:
    platform: $prepare_platforms
    node: [18, 20]
  run: echo "Testing on $platform with Node $node"

# Deploy with multiple dependencies
deploy:
  need: [test]
  run: echo "Deploying version $prepare_version"

# Cleanup (always executes)
clean:
  phony: true
  run: rm -rf dist/
```

## Project Architecture

```
src/
├── ymake.ts       # Main CLI and entry point
├── parser.ts      # Ymakefile parser (YAML)
├── registry.ts    # Target registry
├── resolver.ts    # Dependency resolution
├── executor.ts    # Execution engine
├── matrix.ts      # Matrix build expansion
├── shell.ts       # Shell command execution
├── filesystem.ts  # File verification
└── errors.ts      # Custom error types
```

### Main Components

- **YmakefileParser**: Parses and validates YAML files
- **TargetRegistry**: Stores and manages target definitions
- **DependencyResolver**: Resolves execution order and detects cycles
- **ExecutionEngine**: Coordinates target execution
- **MatrixExpander**: Generates combinations for matrix builds
- **ShellExecutor**: Executes shell commands with output handling

## Exit Codes

- `0`: Successful execution
- `1`: Execution error (command failed)
- `2`: Configuration error (Ymakefile not found, invalid YAML, target doesn't exist, circular dependency)

## Development

This project was created using `bun init` with Bun v1.2.21. [Bun](https://bun.com) is a fast all-in-one JavaScript runtime.

### Setup for Development

```bash
# Clone the repository
git clone https://github.com/JonDotsoy/ymake.git
cd ymake

# Install dependencies
bun install
```

### Run Tests

```bash
bun test
```

### Run Locally

```bash
bun run src/ymake.ts [target] [options]
```

### Integration Tests

Integration tests cover:

- Dependency resolution
- Matrix builds
- Output capture and reuse
- Ymakefile name detection
- Error validation

## License

Private project.