iq-sdk 0.1.1
IQ SDK for creating and managing distributed workers in IQ Flow orchestration platform.
License: MIT
Requires Python: >=3.12
Keywords: cli, workers, distributed, socket.io, tools

# iq-sdk

A Python SDK for creating and managing distributed workers in IQ Flow orchestration platform.

[![PyPI version](https://badge.fury.io/py/iq-sdk.svg)](https://badge.fury.io/py/iq-sdk)
[![CI](https://github.com/iq-company/iq-sdk/actions/workflows/ci.yml/badge.svg)](https://github.com/iq-company/iq-sdk/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

## Quickstart

```bash
# Create project directory
mkdir my_iq_tools_project
cd my_iq_tools_project

# Set up virtual environment
python -m venv .venv
source .venv/bin/activate

# Install iq-sdk
pip install iq-sdk

# Initialize project
iq init .
```

This creates a complete project with configuration, Docker setup, GitHub Actions, and more.

**Next steps:**

```bash
# Set your password in .env
echo "IQ_SITE_DEFAULT_API_SECRET=secret" >> .env

# Create a tool
iq tool create hello

# Create a worker
iq worker create main --tools hello

# Start the worker
iq worker start main
```

→ [Full Quickstart Guide](docs/quickstart.md)

## Features

- **Site-Based Configuration** - Manage multiple sites with host/env/user/password
- **CLI for Project Scaffolding** - Complete setup with `iq init`
- **Socket.IO Worker Connectivity** - Workers connect to one or more sites
- **Secure Secrets Handling** - Passwords via ENV or K8s-style file references
- **Configurable Threading** - Set thread counts, enable autoscaling
- **Docker Integration** - Built-in support for [baker-cli](https://github.com/iq-company/baker-cli)

## Project Structure

After `iq init`, you get:

```
my-project/
├── config/
│   └── settings.yml      # Sites and tool configuration
├── .env                  # Secrets (passwords)
├── .gitignore
├── pyproject.toml
├── build-settings.yml    # Docker build config
├── tools/                # Your tool implementations
├── .devcontainer/        # VS Code DevContainer
├── .github/workflows/    # GitHub Actions
└── docker/               # Dockerfile & compose
```

## CLI Reference

### Site Management

```bash
iq site list              # List all sites
iq site list --check      # Check connectivity
iq site add prod --host http://prod:3000
iq site drop prod
```

### Tool Management

```bash
iq tool create NAME       # Create a tool
iq tool list              # List tools
```

### Worker Management

```bash
iq worker list            # Show configured tools/workers
iq worker list --check    # Check tool availability
iq worker run [OPTIONS]   # Start a worker

Options:
  --threads, -t INT      Thread count
  --max-threads INT      Max threads (autoscaling)
  --autoscale            Enable autoscaling
  --tool TOOL            Tools to register (repeatable)
  --site, -s SITE        Sites to connect (repeatable)
  --config, -c FILE      Config file
```

### Output Queue

When task output submission fails (host unreachable, errors), outputs are queued locally for retry:

```bash
iq worker queue           # List pending outputs
iq worker queue --failed  # List failed (max retries exceeded)
iq worker queue -t tool   # Filter by tool name
iq worker queue --retry   # Manually retry all pending
iq worker queue --retry -y  # Retry without confirmation
```

Queue files are stored in `./data/{site}/queue-output/`.

## Configuration

### Sites (config/settings.yml)

```yaml
site:
  id: "default"
  host: "http://localhost:3000"
  api_key: "worker"
  tools:
    - "my-tool"

sites:
  - id: "production"
    host: "http://prod:3000"
    api_key: "prod-worker"
    tools:
      - "my-tool"
```

### Secrets (.env)

```bash
# Direct password
IQ_SITE_DEFAULT_PASSWORD=secret
IQ_SITE_PRODUCTION_PASSWORD=prod-secret

# Or file reference (K8s/Docker secrets)
IQ_SITE_PRODUCTION_API_SECRET_FILE=/run/secrets/prod-password

# For JSON/YAML secrets, extract with _PASSWORD_KEY
IQ_SITE_STAGING_API_SECRET_FILE=/etc/secrets/creds.json
IQ_SITE_STAGING_API_SECRET_KEY=database.password
```

### Worker Settings (config/settings.yml)

```yaml
worker:
  threads: 2              # Default threads per tool
  autoscale: false        # Enable autoscaling
  min_threads: 1          # Min threads (autoscaling)
  max_threads: 16         # Max threads (autoscaling)

  # Retry settings for failed output submissions
  retry:
    max_retries: 10       # Max retry attempts before moving to failed queue
    initial_delay: 10     # Initial retry delay in seconds
    max_delay: 3600       # Maximum retry delay (1 hour)
    backoff_multiplier: 2.0  # Exponential backoff multiplier
```

### Environment Override

```bash
export IQ_SITE_PRODUCTION_HOST='http://new-host:3000'
export IQ_WORKER__THREADS=8
export IQ_WORKER__RETRY__MAX_RETRIES=20
```

## Documentation

- [Quickstart](docs/quickstart.md) - Get started in 5 minutes
- [Creating Tools](docs/tools.md) - Build your own tools
- [Worker Configuration](docs/workers.md) - Threading and autoscaling
- [Site Configuration](docs/configuration.md) - Multi-site setup
- [Docker Deployment](docs/docker.md) - Container deployment

## Development

For contributing or local development:

```bash
git clone https://github.com/iq-company/iq-sdk.git
cd iq-sdk
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

# Test your changes
iq init demo
cd demo
```

→ [Development Guide](DEVELOPMENT.md)

## License

MIT License - see [LICENSE](LICENSE) for details.