gomask-cli 1.0.0
GoMask CLI for configuration as code - manage synthetic data generation and masking routines
Requires Python: <4.0,>=3.9

# GoMask CLI

**Configuration-as-Code for Synthetic Data Generation and Data Masking**

GoMask CLI enables you to manage data generation and masking routines through YAML configuration files, bringing infrastructure-as-code principles to test data management.

## Features

- **YAML-based Configuration**: Define routines in version-controllable files
- **Secure Authentication**: Encrypted API secret contains all context
- **CI/CD Ready**: Integrate with GitHub Actions, GitLab CI, Jenkins, etc.
- **Bidirectional Sync**: Import from and export to YAML format
- **Environment Variables**: Full support for variable substitution

## Installation

### Via pip

```bash
pip install gomask-cli
```

### Via Poetry

```bash
poetry add gomask-cli
```

### From Source

```bash
git clone https://github.com/gomask/gomask-cli.git
cd gomask-cli
pip install -e .
```

## Quick Start

### 1. Initialize Configuration

Set up your GoMask CLI credentials from the web UI:

```bash
# Initialize CLI with your credentials (get secret from https://app.gomask.ai/settings/api-keys)
gomask init

# This creates a secure gomask.toml file in your current directory
```

### 2. Create a Database Connector

Connect to your database where you want to generate or mask data:

```bash
# Create a connector to your database
gomask connectors create \
  --name my-db \
  --type postgresql \
  --host localhost \
  --port 5432 \
  --database mydb \
  --username myuser

# The CLI will prompt for your password securely
# Or list existing connectors
gomask connectors list
```

### 3. Create a Routine (Recommended: Guided Setup)

Use the interactive wizard to automatically scan your database and generate a production-ready routine:

```bash
# Interactive wizard - automatically scans your database
gomask setup

# The wizard will:
# 1. Select your database connector
# 2. Introspect and display available schemas
# 3. Scan and list all tables in the selected schema
# 4. Detect foreign key relationships automatically
# 5. Let you choose tables (supports ranges like 1-5, or individual selections)
# 6. Ask for routine type (masking or synthetic data generation)
# 7. For masking: AI-powered detection of sensitive columns (PII, emails, etc.)
# 8. For synthetic: Automatically assign appropriate data generators per column
# 9. Generate and export a complete YAML configuration file

# Example output: routine_563_complete.yaml
```

The `setup` command creates a **fully configured, production-ready routine** with:

- Correct table hierarchy based on foreign keys
- Appropriate data generation functions for each column type
- Sensitive data detection for masking routines
- Valid YAML that's ready to run immediately

### 4. Edit Routine (Optional)

Customize the generated YAML configuration if needed:

```bash
# Open in your editor
vim routine_563_complete.yaml

# Common customizations:
# - Adjust record counts
# - Fine-tune generation functions
# - Add runtime parameters
# - Modify masking rules
# - Change data type mappings

# Validate your changes
gomask validate routine_563_complete.yaml
```

### 5. Run Your Routine

Execute the routine to generate or mask data:

```bash
# Execute the generated routine
gomask run routine_563_complete.yaml

# Run with live progress monitoring
gomask run routine_563_complete.yaml --watch

# Run with custom parameters
gomask run routine_563_complete.yaml --param RECORD_COUNT=5000 --watch

# Preview execution plan without running
gomask run routine_563_complete.yaml --dry-run
```

## Command Reference

### Configuration Commands

| Command | Description |
|---------|-------------|
| `gomask init` | Initialize CLI configuration and credentials |
| `gomask setup` | Interactive wizard to create routines (recommended) |
| `gomask version` | Display version information |

### Routine Commands

| Command | Description |
|---------|-------------|
| `gomask example` | Create example routine from template |
| `gomask validate` | Validate YAML configuration file |
| `gomask import` | Import YAML configuration to database |
| `gomask export` | Export routine from database to YAML |
| `gomask run` | Execute a routine from YAML configuration |

### Connector Commands

| Command | Description |
|---------|-------------|
| `gomask connectors list` | List all database connectors |
| `gomask connectors create` | Create a new database connector |
| `gomask connectors show` | Display connector details |
| `gomask connectors test` | Test connector connection |
| `gomask connectors delete` | Delete a connector |

### Function Commands

| Command | Description |
|---------|-------------|
| `gomask functions list` | List available data generation functions |
| `gomask functions show` | Show detailed function information |
| `gomask functions categories` | List function categories |
| `gomask functions tags` | List function tags |

### Routine Management

| Command | Description |
|---------|-------------|
| `gomask routines list` | List all routines for your team |
| `gomask routines show` | Display detailed routine information |

### Execution Commands

| Command | Description |
|---------|-------------|
| `gomask executions list` | List routine executions |
| `gomask executions show` | Display execution details and progress |
| `gomask executions logs` | Show execution logs |
| `gomask executions cancel` | Cancel a running execution |

### Global Options

| Option | Description |
|--------|-------------|
| `--debug` | Enable debug logging |
| `--api-url URL` | Override GoMask API URL |
| `--secret SECRET` | Override API authentication secret |
| `--version` | Display version information |
| `-h, --help` | Show help text |

Use `gomask <command> --help` for detailed options and examples.

For comprehensive documentation, visit: https://gomask.ai/docs/cli

## Authentication

The CLI uses `gomask init` to securely store your credentials:

```bash
# Initialize configuration
gomask init

# This creates a gomask.toml file with your API secret
# Get your secret from: https://app.gomask.ai/settings/api-keys
```

**Configuration Priority:**

1. Command-line flags (`--secret`, `--api-url`)
2. `gomask.toml` configuration file (created by `gomask init`)

**Important:** Add `gomask.toml` to your `.gitignore`!

## Environment Variables in YAML

YAML configuration files support environment variable substitution for sensitive data and runtime configuration:

### Using Variables in YAML

```yaml
connector:
  # Simple substitution
  host: ${DB_HOST}

  # With default value
  port: ${DB_PORT:-5432}

  # Required (errors if not set)
  password: ${DB_PASSWORD:?Password required}

  # Nested references
  database: ${ENV}_${DB_NAME}
```

### Loading from .env Files

```bash
# .env file
DB_HOST=production.example.com
DB_PORT=5432
DB_PASSWORD=secure-password

# Use with CLI
gomask run routine.yaml --env-file .env
```

**Note:** Authentication credentials are stored in `gomask.toml` (created via `gomask init`), not in `.env` files.

## YAML Configuration

### Synthetic Data Generation

```yaml
version: "1.0"
kind: SyntheticRoutine

metadata:
  id: "unique-routine-id"
  name: "Display Name"
  description: "Description"
  version: "1.0.0"

connector:
  ref: "connector-name" # Reference existing
  # OR inline:
  type: postgresql
  host: ${DB_HOST}
  port: 5432
  database: ${DB_NAME}
  username: ${DB_USER}
  password: ${DB_PASSWORD}

runtime_parameters:
  param_name:
    type: integer
    default: 1000
    description: "Parameter description"

tables:
  - name: table_name
    schema: public
    hierarchy_level: 0
    record_count: 1000
    columns:
      - name: column_name
        function: generation_function
        parameters:
          key: value
```

### Data Masking

```yaml
version: "1.0"
kind: MaskingRoutine

metadata:
  id: "unique-routine-id"
  name: "Display Name"

connector:
  ref: "connector-name"

masking:
  tables:
    - name: table_name
      schema: public
      columns:
        - name: column_name
          function: masking_function
          parameters:
            key: value
```

## CI/CD Integration

### GitHub Actions

```yaml
name: Data Generation

on:
  schedule:
    - cron: "0 0 * * *"

jobs:
  generate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Install GoMask CLI
        run: pip install gomask-cli

      - name: Setup GoMask
        run: |
          gomask init --secret ${{ secrets.GOMASK_SECRET }}

      - name: Run Data Generation
        run: |
          gomask validate routines/daily-data.yaml
          gomask run routines/daily-data.yaml --watch
```

### GitLab CI

```yaml
generate-test-data:
  stage: test
  script:
    - pip install gomask-cli
    - gomask init --secret $GOMASK_SECRET
    - gomask run test-data.yaml --param record_count=100
  variables:
    GOMASK_SECRET: $GOMASK_SECRET
```

## Best Practices

1. **Use `gomask setup`** - Always use the interactive wizard for creating new routines
2. **Version Control** - Keep YAML files in git (exclude `gomask.toml`)
3. **Environment Variables** - Use env vars for sensitive data
4. **Validate First** - Always run `gomask validate` before importing
5. **Dry Runs** - Use `--dry-run` to preview execution
6. **Test Small** - Start with small record counts before scaling up

## Troubleshooting

### Authentication Issues

```bash
# Reinitialize configuration
gomask init

# Test connectivity
gomask version
```

### YAML Validation Errors

```bash
# Validate with detailed output
gomask validate routine.yaml --detailed

# Check with environment variables
gomask validate routine.yaml --env-file .env
```

### Execution Problems

```bash
# Preview execution plan
gomask run routine.yaml --dry-run

# Test connector
gomask connectors test my-connector

# Monitor execution with live logs
gomask run routine.yaml --watch
gomask executions logs 123 --follow
```

## Support

- Documentation: https://gomask.ai/docs/cli/index
- Email: info@gomask.ai