npm package: @schemaloom/cli
Description: Enterprise-grade CLI for SchemaLoom data pipeline management and deployment automation
Author: SchemaLoom Contributors
Latest version: 1.0.0

# SchemaLoom CLI

Enterprise-grade command-line interface for SchemaLoom data pipeline management and deployment automation.

## Overview

The SchemaLoom CLI provides a comprehensive suite of tools for creating, managing, and deploying AI-powered data extraction pipelines. Built with enterprise requirements in mind, it enables rapid development and deployment of production-ready data processing workflows.

## Features

- **Pipeline Scaffolding**: Generate production-ready pipeline projects with industry-standard structure
- **Multi-Schema Support**: Built-in templates for events, products, articles, contacts, and invoices
- **Docker Integration**: Automated containerization with optimized Docker configurations
- **Environment Management**: Secure configuration handling with environment variable templates
- **Health Monitoring**: Built-in health checks and monitoring endpoints
- **Production Ready**: TypeScript compilation, dependency management, and build optimization

## Installation

### Prerequisites

- Node.js 18.0.0 or higher
- npm 8.0.0 or higher
- Docker (for containerization features)

### Build from Source

```bash
git clone <repository-url>
cd schemaloom/cli
npm install
npm run build
```

### Global Installation (Optional)

```bash
npm install -g .
```

## Usage

### Command Structure

```bash
schemaloom <command> [options]
```

### Available Commands

#### `create-pipeline`

Creates a new SchemaLoom data pipeline project with configurable templates and deployment options.

**Syntax:**
```bash
schemaloom create-pipeline <name> [options]
```

**Options:**
- `--template <template>`: Schema template (event, product, article, contact, invoice)
- `--directory <path>`: Target directory for project creation
- `--with-docker`: Include Docker configuration files
- `--with-examples`: Include sample data and usage examples

**Examples:**
```bash
# Create basic event extraction pipeline
schemaloom create-pipeline event-processor --template event

# Create product pipeline with Docker support
schemaloom create-pipeline product-analyzer --template product --with-docker

# Create comprehensive pipeline with all features
schemaloom create-pipeline invoice-processor --template invoice --with-docker --with-examples
```

#### `docker-build`

Builds Docker images for existing SchemaLoom pipelines with configurable build options.

**Syntax:**
```bash
schemaloom docker-build <pipeline-path> [options]
```

**Options:**
- `--tag <tag>`: Docker image tag (default: latest)
- `--file <file>`: Dockerfile path (default: Dockerfile)
- `--no-cache`: Build without using cache
- `--push`: Push image to registry after building
- `--registry <registry>`: Target Docker registry

**Examples:**
```bash
# Build production image
schemaloom docker-build my-pipeline --tag v1.0.0

# Build and push to registry
schemaloom docker-build my-pipeline --tag v2.0.0 --push --registry my-registry.com
```

#### `info`

Displays CLI version and system information.

```bash
schemaloom info
```

## Pipeline Architecture

### Generated Project Structure

```
pipeline-name/
├── src/
│   └── index.ts          # Main pipeline logic
├── examples/              # Sample data and usage scripts
├── package.json           # Dependencies and build scripts
├── tsconfig.json          # TypeScript configuration
├── .env                   # Environment configuration
├── README.md              # Project documentation
├── Dockerfile             # Docker configuration
├── .dockerignore          # Docker ignore patterns
└── docker-compose.yml     # Docker Compose configuration
```

### Supported Schema Templates

| Template | Schema | Use Case |
|----------|--------|----------|
| `event` | EventListSchema | Event extraction and processing |
| `product` | ProductListSchema | Product catalog management |
| `article` | ArticleListSchema | Content analysis and extraction |
| `contact` | ContactListSchema | Contact information processing |
| `invoice` | InvoiceListSchema | Financial document processing |

## Configuration

### Environment Variables

The CLI generates environment templates with the following configuration options:

```bash
# Server Configuration
PORT=3000
HOST=0.0.0.0

# Google API Configuration
GOOGLE_API_KEY=your_api_key_here

# Extraction Configuration
EXTRACTION_TIMEOUT=30000
MAX_CHUNK_SIZE=100000
CHUNK_OVERLAP=0
TEMPERATURE=0
MODEL=gemini-2.5-flash

# Logging
LOG_LEVEL=info
```

### Security Considerations

- **API Key Management**: Store sensitive credentials in environment variables
- **Environment Isolation**: Use separate configurations for development, staging, and production
- **Access Control**: Implement appropriate network security for production deployments

## Deployment

### Docker Deployment

```bash
# Build image
docker build -t my-pipeline .

# Run container
docker run -p 3000:3000 -e GOOGLE_API_KEY=your_key my-pipeline

# Use docker-compose
docker-compose up --build
```

### Production Considerations

- **Resource Limits**: Configure appropriate CPU and memory limits
- **Health Checks**: Monitor pipeline health with built-in endpoints
- **Logging**: Implement centralized logging for production environments
- **Scaling**: Use container orchestration for high-availability deployments

## Development

### Local Development

```bash
# Install dependencies
npm install

# Development mode with hot reload
npm run dev

# Build for production
npm run build

# Start production server
npm start
```

### Testing

```bash
# Run health checks
curl http://localhost:3000/health

# Test extraction endpoints
curl -X POST http://localhost:3000/extract \
  -F "file=@sample.txt" \
  -F "schema=event"
```

## API Reference

### Health Endpoint

```http
GET /health
```

Returns service health status.

### Extraction Endpoint

```http
POST /extract
```

Processes file uploads for data extraction using predefined schemas.

**Parameters:**
- `file`: File to process
- `schema`: Schema template to use
- `chunkSize`: Text chunk size (1000-1000000)
- `temperature`: AI model creativity (0-2)

### Custom Schema Endpoint

```http
POST /extract/custom
```

Processes data using custom schema definitions.

**Body:**
```json
{
  "content": "Text content to process",
  "schemaDefinition": {
    "type": "object",
    "properties": {
      "name": {"type": "string"},
      "value": {"type": "number"}
    }
  }
}
```

## Troubleshooting

### Common Issues

**Docker Build Failures**
- Ensure Docker daemon is running
- Verify sufficient disk space for image building
- Check network connectivity for dependency downloads

**Environment Variable Issues**
- Confirm `.env` file exists and contains required variables
- Verify API key validity and permissions
- Check Docker environment variable passing

**TypeScript Compilation Errors**
- Ensure all dependencies are installed
- Verify TypeScript configuration
- Check source code syntax

### Debug Mode

Enable verbose logging for troubleshooting:

```bash
LOG_LEVEL=debug npm start
```

## Contributing

### Development Setup

1. Fork the repository
2. Create a feature branch
3. Implement changes with appropriate tests
4. Submit a pull request with detailed description

### Code Standards

- Follow TypeScript best practices
- Include comprehensive error handling
- Maintain backward compatibility
- Document all public APIs

## License

This project is licensed under the MIT License. See the [LICENSE](../LICENSE) file for details.

## Support

### Documentation

- [SchemaLoom Core Documentation](../core/README.md)
- [API Reference](../core/docs/api.md)
- [Deployment Guide](../docs/deployment.md)

### Community

- [GitHub Issues](https://github.com/schemaloom/schemaloom/issues)
- [Discussions](https://github.com/schemaloom/schemaloom/discussions)
- [Contributing Guidelines](CONTRIBUTING.md)

### Enterprise Support

For enterprise customers requiring dedicated support, please contact our sales team at [enterprise@schemaloom.com](mailto:enterprise@schemaloom.com).

## Version History

- **v1.0.0**: Initial release with basic pipeline creation and Docker support
- **v1.1.0**: Added multi-schema templates and environment management
- **v1.2.0**: Enhanced Docker integration and production optimizations