choreography 0.0.2
A choreographed way to run tasks (shell or Python, async or sync) from the command line.
Requires Python: >=3.10

# Choreography 🩰

A choreographed way to run tasks (shell or Python, async or sync) from the command line.

Choreography is a lightweight task runner that lets you define and orchestrate tasks in YAML, with support for:
- **Shell scripts** and **Python functions** (both sync and async)
- **Task dependencies** with automatic execution ordering
- **Task groups** for organizing related workflows
- **Parallel execution** where possible
- **Custom action directories** for reusable scripts and functions

## Installation

### Using uv

```bash
uv tool install choreography
```

### Using pipx

```bash
pipx install choreography
```


### For development

```bash
# Clone the repository
git clone https://github.com/yourusername/choreography.git
cd choreography

# Install with uv
uv sync
```

## Quick Start

1. Create a `choreography.yaml` file in your project root:

```yaml
tasks:
  hello:
    type: shell
    script: |
      echo "Hello, World!"
  
  test:
    type: shell
    script: |
      pytest
    depends_on:
      - hello
```

2. Run your choreography:

```bash
dance
```

That's it! Choreography will execute your tasks in the correct order based on dependencies.

## Usage

The main command is `dance`, which reads from `choreography.yaml` by default.

### Basic Commands

```bash
# Run all tasks marked with always: true (default)
dance

# Run a specific task and its dependencies
dance <task-id>

# Run tasks from a specific group
dance --group ci

# Run tasks from multiple groups
dance --group local --group ci

# Use a custom choreography file
dance --choreography-file path/to/file.yaml

# Use a custom actions directory
dance --actions-path path/to/actions
```

### Command Options

- `--choreography-file`: Path to the choreography YAML file (default: `choreography.yaml`)
- `--actions-path`: Path to the actions directory (default: `actions/` relative to choreography file)
- `--group`: Group(s) of tasks to run (can be specified multiple times)
- Task ID (positional argument): Run a specific task and all its dependencies

## Choreography File Format

A choreography file defines tasks in YAML format:

```yaml
tasks:
  task-name:
    type: shell | python
    script: |
      # For shell tasks - inline script
    action: action-name  # For tasks using external action files
    depends_on:
      - other-task
    groups:
      - group-name
    always: true | false
    output: auto | always
```

### Task Properties

- **`type`** (required): Either `shell` or `python`
- **`script`**: Inline script content (mutually exclusive with `action`)
- **`action`**: Name of an action file in the actions directory (mutually exclusive with `script`)
- **`depends_on`**: List of task IDs that must complete before this task runs
- **`groups`**: List of group names this task belongs to
- **`always`**: Whether to run this task by default when no group or task ID is specified (default: `true`)
- **`output`**: Control output display - `auto` (default, shows on failure) or `always` (always shows)

**Note**: Each task must have either a `script` or an `action`, but not both.

## Task Types

### Shell Tasks

Shell tasks execute shell scripts. The script can include a shebang to specify the shell:

```yaml
tasks:
  build:
    type: shell
    script: |
      #!/bin/bash
      npm run build
```

Supported shells are detected from shebangs: `bash`, `zsh`, `sh`. If no shebang is provided, `sh` is used by default.

### Python Tasks

Python tasks execute Python functions from action files or small code snippets:

```yaml
tasks:
  process-data:
    type: python
    action: process  # Looks for actions/process.py
```

```yaml
task:
  say-hi:
    type: python
    script: |
     print("hi!")
```

The corresponding action file should export an `action` function:

```python
# actions/process.py

def action():
    """Synchronous Python function"""
    print("Processing data...")
    # Your code here
```

Or for async operations:

```python
# actions/async_process.py

async def action():
    """Asynchronous Python function"""
    import asyncio
    await asyncio.sleep(1)
    print("Async processing complete!")
```

## Task Dependencies

Tasks can depend on other tasks using the `depends_on` field:

```yaml
tasks:
  install:
    type: shell
    script: npm install
  
  build:
    type: shell
    script: npm run build
    depends_on:
      - install
  
  test:
    type: shell
    script: npm test
    depends_on:
      - build
  
  deploy:
    type: shell
    script: ./deploy.sh
    depends_on:
      - test
      - build
```

Choreography will:
- Detect and prevent circular dependencies
- Execute tasks in the correct order
- Run independent tasks in parallel when possible
- Stop execution if any task fails

## Task Groups

Groups let you organize tasks into logical collections:

```yaml
tasks:
  lint:
    type: shell
    groups: ["ci", "local"]
    script: ruff check .
  
  test:
    type: shell
    groups: ["ci", "local"]
    script: pytest
  
  deploy-staging:
    type: shell
    groups: ["deploy"]
    script: ./deploy-staging.sh
  
  deploy-prod:
    type: shell
    groups: ["deploy"]
    script: ./deploy-prod.sh
```

Run groups with:

```bash
dance --group ci       # Runs lint and test
dance --group deploy   # Runs both deploy tasks
```

When using groups, tasks' dependencies are automatically included even if they're not in the specified group.

## Actions Directory

The `actions/` directory (by default) contains reusable scripts and Python modules:

```
actions/
  format.sh           # Shell script action
  lint.sh             # Shell script action  
  process_data.py     # Python module with action() function
  analyze.py          # Python module with action() function
```

Reference these in your choreography:

```yaml
tasks:
  format-code:
    type: shell
    action: format  # Uses actions/format.sh
  
  process:
    type: python
    action: process_data  # Uses actions/process_data.py
```

For shell scripts, the file extension determines how it's executed (`.sh`, `.bash`, etc.), or you can use a shebang in the script.

## Output Control

Control when task output is displayed:

```yaml
tasks:
  quiet-task:
    type: shell
    output: auto  # Only shows output on failure (default)
    script: echo "You'll only see this if it fails"
  
  verbose-task:
    type: shell
    output: always  # Always shows output
    script: echo "You'll always see this"
```

## Examples

### Example: CI/CD Pipeline

```yaml
tasks:
  install-deps:
    type: shell
    groups: ["ci"]
    script: |
      npm ci
  
  lint:
    type: shell
    groups: ["ci"]
    output: always
    script: npm run lint
    depends_on:
      - install-deps
  
  test:
    type: shell
    groups: ["ci"]
    output: always
    script: npm test
    depends_on:
      - install-deps
  
  build:
    type: shell
    groups: ["ci"]
    script: npm run build
    depends_on:
      - lint
      - test
  
  deploy:
    type: shell
    always: false
    script: ./deploy.sh
    depends_on:
      - build
```

Run the CI pipeline:
```bash
dance --group ci
```

Deploy (including all dependencies):
```bash
dance deploy
```

### Example: Mixed Python and Shell Tasks

```yaml
tasks:
  fetch-data:
    type: python
    action: fetch
    groups: ["data-pipeline"]
  
  process-data:
    type: python
    action: process
    groups: ["data-pipeline"]
    depends_on:
      - fetch-data
  
  generate-report:
    type: shell
    groups: ["data-pipeline"]
    script: |
      python scripts/report_generator.py
    depends_on:
      - process-data
  
  upload-report:
    type: shell
    groups: ["data-pipeline"]
    script: aws s3 cp report.pdf s3://my-bucket/
    depends_on:
      - generate-report
```

### Example: Development Workflow

```yaml
tasks:
  format:
    type: shell
    groups: ["local"]
    action: fix-linting-issues
    output: always
  
  test:
    type: shell
    groups: ["local", "ci"]
    output: always
    script: |
      pytest --cov=src --cov-report=term-missing
  
  lint-check:
    type: shell
    groups: ["ci"]
    script: ruff check .
    depends_on:
      - test
```

Run local development checks:
```bash
dance --group local
```

## Development

### Running Tests

```bash
# With uv
uv run pytest

# With coverage
uv run pytest --cov=src/choreography --cov-report=term-missing --cov-branch -v

# Or if installed in development mode
pytest
```

### Project Structure

```
choreography/
├── src/choreography/
│   ├── __init__.py
│   ├── action.py          # Action loading and management
│   ├── choreograph.py     # Core orchestration logic
│   ├── cli.py             # CLI interface
│   ├── models.py          # Pydantic models and validation
│   ├── outputs.py         # Output formatting utilities
│   └── run.py             # Task execution
├── tests/
│   ├── actions/           # Test action files
│   ├── fixtures/          # Test choreography files
│   └── test_*.py          # Test modules
├── actions/               # Project's own actions
├── choreography.yaml      # Project's own choreography
└── pyproject.toml
```

### Running the Project's Own Choreography

This project uses itself for task management! Check out `choreography.yaml` for examples:

```bash
# Run local development tasks
uv run dance --group local

# Run CI tasks
uv run dance --group ci

# Build the project
uv run dance build
```

## Requirements

- Python 3.10+

## License

MIT License - see LICENSE file for details.

## Contributing

Contributions welcome! Please:

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Ensure all tests pass: `uv run pytest`
6. Submit a pull request

---

Made with ✨ by Meg Hicks