friday-gguf 0.1.0
Convert HuggingFace models to GGUF format for Ollama
License: MIT
Requires Python: >=3.8
Keywords: gguf, ollama, llm, model-conversion, huggingface

# Friday - Image Generation GGUF Converter

Convert Open Source Image Generation Models to GGUF format for use with Ollama.

## Overview

Friday enables you to run high-quality image generation models locally without API dependencies by converting them to GGUF format and integrating with Ollama. The system provides a complete pipeline for loading Stable Diffusion models, converting weights to GGUF format, applying quantization, and creating Ollama-compatible model files.

## Features

- **Model Loading**: Load Stable Diffusion models from local files or HuggingFace Hub
- **Weight Conversion**: Transform PyTorch/SafeTensors weights to GGUF-compatible format
- **Quantization Support**: Multiple quantization levels (FP32, FP16, INT8) for size optimization
- **Ollama Integration**: Automatic Modelfile generation for seamless Ollama integration
- **🚀 Ollama Diffusion Proxy**: Run GGUF diffusion models with Ollama-compatible API
- **Quality Validation**: Built-in validation to ensure conversion quality
- **Memory Optimization**: Chunked processing for large models
- **Performance Optimization**: Component caching, batch processing, and tensor optimization
- **Resume Capability**: Resume interrupted conversions from checkpoints

## Architecture

Friday consists of four main components:

```
┌─────────────────┐      ┌──────────────────┐      ┌─────────────────┐
│  Model Loader   │─────▶│  Converter       │─────▶│  GGUF Writer    │
│                 │      │  Pipeline        │      │                 │
└─────────────────┘      └──────────────────┘      └─────────────────┘
                                                             │
                                                             ▼
                                                    ┌─────────────────┐
                                                    │ Ollama          │
                                                    │ Integration     │
                                                    └─────────────────┘
```

1. **Model Loader**: Loads and validates Stable Diffusion models
2. **Converter Pipeline**: Converts weights, applies quantization, and maps layers
3. **GGUF Writer**: Writes GGUF-compliant binary files
4. **Ollama Integration**: Creates Modelfiles and registers models with Ollama

## System Requirements

### Minimum Requirements
- **Python**: 3.8 or higher
- **RAM**: 8GB (16GB recommended for larger models)
- **Storage**: 10GB free space for model files
- **OS**: Windows, Linux, or macOS

### Recommended Requirements
- **Python**: 3.10+
- **RAM**: 32GB for SDXL models
- **GPU**: CUDA-compatible GPU with 8GB+ VRAM (optional but recommended)
- **Storage**: SSD for faster processing

## Quick Start

Get started in 3 steps with our test model (Animagine XL 3.1):

```bash
# 1. Download test model
python download_animagine.py

# 2. Run quick test
python test_animagine_xl.py

# 3. View results in output/ directory
```

See **[QUICKSTART.md](QUICKSTART.md)** for detailed instructions.

## Testing with Animagine XL 3.1

We provide a complete testing setup with Animagine XL 3.1, a high-quality anime image generation model:

- **[QUICKSTART.md](QUICKSTART.md)** - Get started in 3 steps
- **[ANIMAGINE_SETUP.md](ANIMAGINE_SETUP.md)** - Detailed setup guide
- **[test_animagine_xl.py](test_animagine_xl.py)** - Automated test script
- **[download_animagine.py](download_animagine.py)** - Model download helper

**Why Animagine XL 3.1?**
- Based on Stable Diffusion XL
- Excellent for testing SDXL architecture
- High-quality results
- Active community support
- **RAM**: 16GB or more
- **GPU**: NVIDIA GPU with 6GB+ VRAM (CUDA support)
- **Storage**: 20GB+ SSD for faster I/O

### GPU Support
- **CUDA**: Recommended for faster conversion and inference
- **CPU-only**: Supported but slower (use INT8 quantization for better performance)

## Installation

### 1. Clone the Repository

```bash
git clone https://github.com/yourusername/friday.git
cd friday
```

### 2. Create Virtual Environment (Recommended)

```bash
# Windows
python -m venv .venv
.venv\Scripts\activate

# Linux/macOS
python -m venv .venv
source .venv/bin/activate
```

### 3. Install Dependencies

```bash
# Install Friday and all dependencies
pip install -e .

# Or install from requirements.txt
pip install -r requirements.txt
```

### 4. Install Ollama (Optional)

If you want to use Ollama for inference:

```bash
# Download from https://ollama.ai/download
# Or use package manager:

# macOS
brew install ollama

# Linux
curl -fsSL https://ollama.ai/install.sh | sh

# Windows
# Download installer from https://ollama.ai/download
```

### 5. Verify Installation

```bash
# Check Friday CLI
python -m friday.cli --help

# Check Ollama (if installed)
ollama --version
```

## Step-by-Step Conversion Guide

### Basic Conversion

Convert a Stable Diffusion model with default settings (FP16 quantization):

```bash
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --output ./output
```

### Conversion from Local Model

If you have a model downloaded locally:

```bash
python -m friday.cli \
  --model ./models/stable-diffusion-v1-5 \
  --output ./output/sd15-gguf
```

### Conversion with Quantization Options

Choose different quantization levels based on your needs:

```bash
# FP32 (no quantization, largest size, best quality)
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --output ./output \
  --quantization fp32

# FP16 (recommended, good balance)
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --output ./output \
  --quantization fp16

# INT8 (smallest size, faster on CPU, slight quality loss)
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --output ./output \
  --quantization int8
```

### Advanced Conversion Options

```bash
# Disable size optimization for maximum quality
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --output ./output \
  --no-optimize-size

# Skip validation (faster but not recommended)
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --output ./output \
  --skip-validation

# Resume interrupted conversion
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --output ./output \
  --resume

# Verbose logging for debugging
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --output ./output \
  --verbose
```

### Using Custom Configuration

Create a custom configuration file:

```yaml
# config/custom_config.yaml
conversion:
  quantization: fp16
  optimize_size: true
  preserve_quality: true
  target_platform: ollama

ollama:
  model_name: "my-custom-model"
  temperature: 0.7
  num_steps: 20
```

Then use it:

```bash
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --output ./output \
  --config ./config/custom_config.yaml
```

## Configuration Management

Friday provides a flexible configuration system that supports YAML files and CLI argument overrides.

### Configuration File Structure

Configuration files are organized into sections:

```yaml
# Model Configuration
model:
  type: "stable-diffusion-v1.5"
  source_path: ""
  output_path: "./output"

# Conversion Settings
conversion:
  quantization: "fp16"  # fp32, fp16, int8
  optimize_size: true
  preserve_quality: true
  target_platform: "ollama"
  max_quality_degradation: 0.05
  min_similarity_threshold: 0.95
  use_chunked_processing: true
  chunk_size_mb: 512

# GGUF Format Settings
gguf:
  version: 3
  include_metadata: true
  chunk_size: 100

# Ollama Integration
ollama:
  model_name: "friday-image-gen"
  temperature: 0.7
  num_ctx: 2048
  num_steps: 20
  guidance_scale: 7.5

# Image Generation
generation:
  default_width: 512
  default_height: 512
  default_steps: 20
  default_guidance_scale: 7.5

# Performance
performance:
  memory_optimization: true
  batch_size: 1
  use_gpu: true

# Logging
logging:
  level: "INFO"
  log_file: "./logs/friday.log"
  console: true
```

### Pre-configured Profiles

Friday includes several pre-configured profiles for common use cases:

#### High Quality Profile
```bash
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --config ./config/high_quality_config.yaml
```

- FP32 quantization (no compression)
- Maximum quality preservation
- 50 inference steps
- Best for: Final production images

#### Fast Profile
```bash
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --config ./config/fast_config.yaml
```

- INT8 quantization (maximum compression)
- Optimized for speed
- 15 inference steps
- Best for: Quick iterations, CPU-only systems

#### SDXL Profile
```bash
python -m friday.cli \
  --model stabilityai/stable-diffusion-xl-base-1.0 \
  --config ./config/sdxl_config.yaml
```

- FP16 quantization
- Larger chunk sizes for SDXL
- 1024x1024 default resolution
- Best for: SDXL models

### Configuration Priority

Settings are applied in the following order (later overrides earlier):

1. **Default values** (hardcoded in the system)
2. **Default config file** (`./config/default_config.yaml`)
3. **User config file** (if found in `~/.friday/config.yaml`)
4. **Specified config file** (`--config` argument)
5. **CLI arguments** (highest priority)

Example:
```bash
# Config file sets quantization to fp16
# CLI argument overrides it to int8
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --config ./config/high_quality_config.yaml \
  --quantization int8  # This overrides the config file
```

### Creating Custom Configurations

You can create and save custom configurations programmatically:

```python
from friday.config import ConfigLoader

# Load default config
config = ConfigLoader.load()

# Modify settings
config.conversion.quantization = "fp16"
config.ollama.model_name = "my-custom-model"
config.generation.default_steps = 25

# Save to file
ConfigLoader.save_yaml(config.to_dict(), "./my_config.yaml")
```

### Configuration Validation

All configurations are automatically validated. Invalid settings will produce clear error messages:

```bash
Configuration validation failed:
  - conversion.quantization: Invalid quantization: fp64. Must be fp32, fp16, or int8
  - generation.default_width: default_width must be positive and divisible by 8, got 513
```

For more details, see the [Configuration README](friday/config/README.md).

## Using with Ollama

### 1. Register the Model

After conversion, register the model with Ollama:

```bash
# The Modelfile is automatically created in the output directory
ollama create friday-sd15 -f ./output/Modelfile
```

### 2. Generate Images

```bash
# Generate an image from a text prompt
ollama run friday-sd15 "a beautiful sunset over mountains, highly detailed, 4k"

# With custom parameters
ollama run friday-sd15 \
  --parameter num_ctx 77 \
  --parameter temperature 0.7 \
  "a serene lake with mountains in the background"
```

### 3. Using Ollama API

```python
import requests
import json
from PIL import Image
import io

# Generate image via API
response = requests.post('http://localhost:11434/api/generate', 
    json={
        'model': 'friday-sd15',
        'prompt': 'a beautiful sunset over mountains',
        'stream': False
    }
)

# Save the generated image
if response.status_code == 200:
    result = response.json()
    # Process image data from result
```

## 🚀 Ollama Diffusion Proxy

**NEW!** Friday now includes an Ollama Diffusion Proxy that enables **native GGUF diffusion model support** for Ollama!

### Why Use the Proxy?

Ollama currently doesn't support diffusion models natively. The Ollama Diffusion Proxy solves this by:

- ✅ Emulating the complete Ollama API
- ✅ Loading and executing GGUF diffusion models
- ✅ Making image generation available to **all Ollama-compatible clients**
- ✅ Running on the same port as Ollama (11434)
- ✅ Drop-in replacement for Ollama

### Quick Start with Proxy

#### 1. Convert Your Model

```bash
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --output ./models \
  --quantization Q4_0
```

#### 2. Start the Proxy

```bash
# Auto-discover models in directory
python start_ollama_diffusion_proxy.py --models-dir ./models

# Or manually add models
python start_ollama_diffusion_proxy.py \
  --add-model sd15 ./models/sd15.gguf \
  --port 11434
```

#### 3. Generate Images

```bash
# Using curl
curl http://localhost:11434/api/generate -d '{
  "model": "sd15",
  "prompt": "a beautiful sunset over mountains",
  "num_inference_steps": 30,
  "guidance_scale": 7.5,
  "width": 512,
  "height": 512
}'

# Using Python
import requests
import base64
from PIL import Image
import io

response = requests.post(
    "http://localhost:11434/api/generate",
    json={
        "model": "sd15",
        "prompt": "a cat wearing sunglasses",
        "num_inference_steps": 30
    }
)

# Decode and save image
data = response.json()
image_data = base64.b64decode(data['response'])
image = Image.open(io.BytesIO(image_data))
image.save("output.png")
```

### Proxy Features

- **Ollama API Compatible**: Works with any Ollama client
- **GGUF Support**: Loads quantized GGUF diffusion models
- **Performance Optimized**: Component caching, batch processing
- **Memory Efficient**: Streaming weight loading for large models
- **Multiple Models**: Load and switch between multiple models
- **Flexible Parameters**: Control steps, guidance, size, seed, etc.

### Proxy API Endpoints

The proxy implements these Ollama endpoints:

- `GET /api/tags` - List available models
- `POST /api/generate` - Generate images
- `POST /api/show` - Show model details
- `GET /` - Server status

For complete documentation, see [Ollama Diffusion Proxy README](friday/ollama_diffusion_proxy/README.md).

### Example: Complete Workflow

```python
from friday.conversion_workflow import ConversionWorkflow
from friday.ollama_diffusion_proxy import OllamaDiffusionProxy
from pathlib import Path

# 1. Convert model
workflow = ConversionWorkflow()
result = workflow.execute(
    model_path="runwayml/stable-diffusion-v1-5",
    output_path=Path("./models/sd15.gguf")
)

# 2. Start proxy
proxy = OllamaDiffusionProxy(models_dir=Path("./models"))
proxy.run()

# 3. Now any Ollama client can generate images!
```

See [examples/ollama_proxy_example.py](examples/ollama_proxy_example.py) for a complete example
    print("Image generated successfully!")
```

## Example Commands for Common Use Cases

### Convert and Test Locally

```bash
# 1. Convert the model
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --output ./output

# 2. Validate the conversion
python validate_model.py --gguf-path ./output/model.gguf

# 3. Register with Ollama
ollama create friday-sd15 -f ./output/Modelfile

# 4. Test inference
ollama run friday-sd15 "test prompt"
```

### Batch Conversion

```bash
# Convert multiple models with different quantization levels
for quant in fp32 fp16 int8; do
  python -m friday.cli \
    --model runwayml/stable-diffusion-v1-5 \
    --output ./output/sd15-$quant \
    --quantization $quant \
    --model-name friday-sd15-$quant
done
```

### Production Deployment

```bash
# Convert with optimal settings for production
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --output ./production/models \
  --quantization fp16 \
  --optimize-size \
  --preserve-quality \
  --model-name friday-sd15-prod \
  --verbose
```

## Troubleshooting Common Issues

### Issue: Out of Memory During Conversion

**Symptoms**: Process crashes with "CUDA out of memory" or system freezes

**Solutions**:
1. Use a lower quantization level (INT8 instead of FP16)
2. Close other applications to free up RAM
3. Enable size optimization: `--optimize-size`
4. Use CPU-only mode by setting `CUDA_VISIBLE_DEVICES=""`

```bash
# Force CPU-only conversion
CUDA_VISIBLE_DEVICES="" python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --output ./output \
  --quantization int8
```

### Issue: Model Download Fails

**Symptoms**: "Connection timeout" or "Model not found" errors

**Solutions**:
1. Check your internet connection
2. Verify the model ID is correct on HuggingFace Hub
3. Try downloading manually first:

```bash
# Download model manually using git-lfs
git lfs install
git clone https://huggingface.co/runwayml/stable-diffusion-v1-5

# Then convert from local path
python -m friday.cli \
  --model ./stable-diffusion-v1-5 \
  --output ./output
```

### Issue: Conversion Hangs or Takes Too Long

**Symptoms**: Process appears stuck, no progress for extended time

**Solutions**:
1. Enable verbose logging to see what's happening: `--verbose`
2. Check system resources (RAM, CPU, disk I/O)
3. Try resuming if interrupted: `--resume`
4. Reduce model size with aggressive quantization

```bash
# Resume interrupted conversion
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --output ./output \
  --resume \
  --verbose
```

### Issue: GGUF File Validation Fails

**Symptoms**: "Invalid GGUF format" or "Checksum mismatch" errors

**Solutions**:
1. Re-run conversion without skipping validation
2. Check disk space (corrupted writes due to full disk)
3. Verify source model integrity
4. Try without size optimization: `--no-optimize-size`

```bash
# Clean conversion with validation
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --output ./output \
  --no-optimize-size \
  --preserve-quality
```

### Issue: Ollama Cannot Load Model

**Symptoms**: "Model not found" or "Invalid model format" when using Ollama

**Solutions**:
1. Verify Modelfile was created: `ls ./output/Modelfile`
2. Check GGUF file exists: `ls ./output/*.gguf`
3. Manually create the model:

```bash
# Verify Ollama is running
ollama list

# Create model with absolute path
ollama create friday-sd15 -f /absolute/path/to/output/Modelfile

# Test the model
ollama run friday-sd15 "test"
```

### Issue: Poor Image Quality After Conversion

**Symptoms**: Generated images are blurry, distorted, or low quality

**Solutions**:
1. Use higher quantization: FP16 or FP32 instead of INT8
2. Enable quality preservation: `--preserve-quality`
3. Disable size optimization: `--no-optimize-size`
4. Validate conversion quality:

```bash
# High-quality conversion
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --output ./output \
  --quantization fp32 \
  --no-optimize-size \
  --preserve-quality

# Run validation
python validate_model.py \
  --gguf-path ./output/model.gguf \
  --compare-original
```

### Issue: Permission Denied Errors

**Symptoms**: Cannot write to output directory or read model files

**Solutions**:
1. Check directory permissions
2. Run with appropriate user permissions
3. Use a different output directory

```bash
# Windows: Run as administrator or use user directory
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --output %USERPROFILE%\friday-output

# Linux/macOS: Check permissions
chmod -R u+w ./output
python -m friday.cli \
  --model runwayml/stable-diffusion-v1-5 \
  --output ./output
```

## Project Structure

```
friday/
├── model_loader/          # Model loading and validation
│   ├── loader.py         # ModelLoader class
│   ├── data_classes.py   # ModelData, ArchitectureInfo
│   └── downloader.py     # HuggingFace Hub integration
├── converter/            # Weight conversion logic
│   ├── pipeline.py       # ConversionPipeline class
│   ├── quantizer.py      # Quantization implementation
│   ├── layer_mapper.py   # Layer mapping system
│   └── metadata_generator.py  # GGUF metadata generation
├── gguf_writer/          # GGUF file generation
│   ├── writer.py         # GGUFWriter class
│   └── validator.py      # GGUF format validation
├── ollama_integration/   # Ollama integration
│   ├── integration.py    # OllamaIntegration class
│   └── data_classes.py   # ModelConfig, etc.
├── validation/           # Quality validation
│   └── validator.py      # Validation and testing
├── utils/                # Utility functions
│   └── helpers.py        # Common utilities
├── cli.py                # Command-line interface
└── conversion_workflow.py # End-to-end workflow

config/                   # Configuration files
├── default_config.yaml   # Default settings
└── custom_config.yaml    # User configurations

examples/                 # Example scripts
├── convert_model_example.py
├── full_conversion_example.py
├── load_model_example.py
├── ollama_integration_example.py
├── validation_example.py
└── write_gguf_example.py
```

## API Documentation

For detailed API documentation, see the individual component READMEs:

- [Model Loader](friday/model_loader/README.md)
- [Converter Pipeline](friday/converter/README.md)
- [GGUF Writer](friday/gguf_writer/README.md)
- [Ollama Integration](friday/ollama_integration/README.md)
- [Validation](friday/validation/README.md)

## Examples

See the `examples/` directory for complete working examples:

- `convert_model_example.py` - Basic model conversion
- `full_conversion_example.py` - Complete end-to-end workflow
- `ollama_integration_example.py` - Ollama integration and inference
- `validation_example.py` - Quality validation and testing

## Contributing

Contributions are welcome! Please see the design documentation in `.kiro/specs/friday-image-gen-gguf/` for architecture details.

## License

MIT License

## Support

For issues and questions:
- Check the troubleshooting section above
- Review the [design documentation](.kiro/specs/friday-image-gen-gguf/design.md)
- Open an issue on GitHub

## Acknowledgments

- Stable Diffusion by Stability AI
- GGUF format by ggerganov
- Ollama by Ollama team
- HuggingFace for model hosting and libraries