debrepomanager 0.5.0
Debian Repository Manager for multi-distribution package management
License: MIT
Requires Python: >=3.11
Keywords: debian, repository, apt, aptly, package-management

# Debian Repository Manager (debrepomanager)

A system for managing multiple Debian-like repositories with support for multiple distributions, architectures, and package collections, featuring atomic updates and GitHub Actions integration.

[![Tests](https://github.com/jethome-iot/repomanager/workflows/Tests/badge.svg)](https://github.com/jethome-iot/repomanager/actions)
[![PyPI version](https://badge.fury.io/py/debrepomanager.svg)](https://pypi.org/project/debrepomanager/)
[![Python](https://img.shields.io/pypi/pyversions/debrepomanager.svg)](https://pypi.org/project/debrepomanager/)
[![Coverage](https://img.shields.io/badge/coverage-81%25-brightgreen.svg)](https://github.com/jethome-iot/repomanager)

## Features

- 🚀 **Atomic updates** via aptly snapshots
- 🔄 **Multi-codename**: bookworm, noble, trixie, jammy, etc.
- 🏗️ **Multi-architecture**: amd64, arm64, riscv64
- 📦 **Multi-component**: various package collections
- 🔐 **GPG signing** for all repositories
- 🧹 **Retention policies**: automatic cleanup of old versions
- 🤖 **GitHub Actions**: CI/CD pipeline integration
- 🔍 **Verification**: repository consistency checks
- 🔀 **Dual format**: old and new APT URL formats support
- 🔄 **Release lifecycle**: integrated `release add/remove` commands (v0.4.0)
- 🌍 **Environment isolation**: stable/beta/test with `--environment` (v0.4.0)
- ⚙️ **Dynamic nginx**: auto-updated configuration (v0.3.7)

## Architecture

Built on top of [aptly](https://www.aptly.info/) with:
- Multi-root isolation (separate aptly root per codename)
- Snapshots for atomic operations
- Python CLI for easy management
- GitHub Actions for automation

See [ARCHITECTURE.md](docs/ARCHITECTURE.md) for detailed description.

## Requirements

### Server
- aptly >= 1.5.0
- gpg (GnuPG) >= 2.2
- Python >= 3.11
- rsync (for GitHub Actions)
- SSH server (for GitHub Actions)

### Development
- Python >= 3.11 (tested on 3.11, 3.12, 3.13)
- pip
- virtualenv (recommended)
- Docker (for integration tests)

## Installation

### From PyPI

```bash
pip install debrepomanager
```

### From source

```bash
git clone https://github.com/jethome-iot/repomanager.git
cd repomanager
pip install -e .
```

### For development

```bash
git clone https://github.com/jethome-iot/repomanager.git
cd repomanager

# Create virtual environment
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install -e ".[dev]"

# Run tests
pytest
```

## Quick Start

```bash
# Add new release
debrepomanager release add bookworm

# Add packages with environment isolation
debrepomanager add --environment stable --codename bookworm --package-dir /path/to/packages/

# Or traditional way
debrepomanager add --codename bookworm --component jethome-tools --package-dir /path/to/packages/

# List repositories
debrepomanager list

# Remove old release
debrepomanager release remove bullseye --backup --confirm
```

**See also:** [QUICK_START.md](QUICK_START.md) for 5-minute getting started guide.

## Configuration

### Auto-Detection (Recommended)

debrepomanager automatically finds configuration files:

```bash
# No --config needed! Searches standard locations:
# 1. /etc/debrepomanager/config.yaml
# 2. ~/.debrepomanager/config.yaml
# 3. ./config.yaml

debrepomanager list
debrepomanager create-repo --codename bookworm --component jethome-tools
```

### Configuration Priority

Priority (low → high):
1. `/etc/debrepomanager/config.yaml` - System-wide
2. `~/.debrepomanager/config.yaml` - User-level
3. `./config.yaml` - Local directory
4. `--config path` - Explicit override
5. `REPOMANAGER_*` env vars - Highest priority

### Minimal Configuration

Create `~/.debrepomanager/config.yaml`:

```yaml
gpg:
  key_id: "YOUR_GPG_KEY_ID"

aptly:
  root_base: "/srv/aptly"
  publish_base: "/srv/repo/public"

repositories:
  architectures: [amd64, arm64, riscv64]
```

**Note**: Codenames and components are created dynamically - no need to list them!

### Environment Variables

```bash
export REPOMANAGER_APTLY_ROOT_BASE=/custom/path
export REPOMANAGER_GPG_KEY_ID=TESTKEY123
debrepomanager list
```

See [`docs/CONFIG.md`](docs/CONFIG.md) and `config.yaml.example` for all options.

## Usage

### Release Management (v0.4.0+)

#### Add New Release

```bash
# Add new release with default components
debrepomanager release add trixie

# Add with custom components
debrepomanager release add oracular \
  --components jethome-tools \
  --components jethome-desktop
```

#### Remove Old Release

```bash
# Remove with backup (safe)
debrepomanager release remove bullseye --backup --confirm

# Remove without backup
debrepomanager release remove jammy --no-backup --force --confirm
```

### Environment Isolation (v0.4.0+)

#### Upload to Different Environments

```bash
# Stable (production)
debrepomanager add --environment stable \
  --codename bookworm \
  --package-dir /path/to/packages/

# Beta (testing)
debrepomanager add --environment beta \
  --codename bookworm \
  --package-dir /path/to/packages/

# Test (development)
debrepomanager add --environment test \
  --codename bookworm \
  --package-dir /path/to/packages/
```

Environment URLs:
- Stable: `http://repo.site.com/`
- Beta: `http://repo.site.com/beta/`
- Test: `http://repo.site.com/test/`

### Legacy Scripts (Deprecated in v0.4.0)

⚠️ **The following scripts are deprecated and will be removed in v0.5.0:**

- `scripts/repoadd` → Use `debrepomanager add --environment`
- `scripts/add-release.sh` → Use `debrepomanager release add`
- `scripts/remove-release.sh` → Use `debrepomanager release remove`

See [MIGRATION.md](docs/MIGRATION.md) for migration guide.

### Symlink Management

Check and fix symlinks for dual format APT support:

```bash
# Check all symlinks
debrepomanager fix-symlinks --check-only

# Fix all symlinks
debrepomanager fix-symlinks

# Fix specific codename
debrepomanager fix-symlinks --codename trixie
```

### Create repository

```bash
debrepomanager create-repo --codename bookworm --component jethome-tools
```

### Add packages

```bash
# Add specific packages
debrepomanager add --codename bookworm --component jethome-tools \
    --packages package1.deb package2.deb

# Add all packages from directory
debrepomanager add --codename bookworm --component jethome-tools \
    --package-dir /path/to/packages/
```

### List repositories and packages

```bash
# List all repositories (grouped by codename)
debrepomanager list

# List repositories with statistics (package count and size)
debrepomanager list --stats

# List repositories for specific codename
debrepomanager list --codename bookworm

# List repositories with statistics for specific codename
debrepomanager list --codename bookworm --stats

# List packages in repository
debrepomanager list --codename bookworm --component jethome-tools

# List packages with statistics
debrepomanager list --codename bookworm --component jethome-tools --stats
```

### Delete repository

```bash
# With confirmation
debrepomanager delete-repo --codename bookworm --component old-component --confirm

# Quick delete (for scripts)
debrepomanager delete-repo --codename bookworm --component old-component --yes
```

### Retention and cleanup

```bash
# Check what would be removed (dry-run)
debrepomanager cleanup --codename bookworm --component jethome-tools

# Apply cleanup
debrepomanager cleanup --codename bookworm --component jethome-tools --apply
```

## Client Configuration

### Supported APT URL Formats (v0.3.7+)

Both old and new formats work simultaneously:

```bash
# New format (recommended)
deb http://repo.site.com/bookworm jethome-tools main

# Old format (backward compatibility via nginx)
deb http://repo.site.com bookworm jethome-tools
```

**Requirements for old format**: Nginx rewrite rules (see [NGINX_OLD_FORMAT.md](docs/NGINX_OLD_FORMAT.md))

### Debian Bookworm

```bash
# Import GPG key
wget -qO - http://repo.site.com/pubkey.gpg | \
    sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/jethome.gpg

# Add repository (new format - recommended)
echo "deb http://repo.site.com/bookworm jethome-tools main" | \
    sudo tee /etc/apt/sources.list.d/jethome.list

# Or old format (if nginx configured)
echo "deb http://repo.site.com bookworm jethome-tools" | \
    sudo tee /etc/apt/sources.list.d/jethome.list

# Update
sudo apt update

# Install packages
sudo apt install your-package
```

### Ubuntu Noble

```bash
# Import GPG key
wget -qO - http://repo.site.com/pubkey.gpg | \
    sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/jethome.gpg

# Add repository
echo "deb http://repo.site.com/noble jethome-tools main" | \
    sudo tee /etc/apt/sources.list.d/jethome.list

# Update
sudo apt update
```

## Dual Format Support

The system supports both old and new APT URL formats simultaneously:

**Old format** (backward compatibility):
```
deb http://repo.site.com bookworm jethome-tools
```

**New format** (recommended):
```
deb http://repo.site.com/bookworm jethome-tools main
```

Both formats work simultaneously via symlinks. See [DUAL_FORMAT.md](docs/DUAL_FORMAT.md) for technical details.

## Repository URL Structure

Repositories are accessible via:
```
http://repo.site.com/{codename}/{component}
```

Examples:
```
http://repo.site.com/bookworm/jethome-tools
http://repo.site.com/noble/jethome-armbian
http://repo.site.com/trixie/jethome-bookworm
```

## Development

### Project Structure

```
debrepomanager/
├── debrepomanager/        # Main package
│   ├── __init__.py
│   ├── cli.py            # CLI interface
│   ├── aptly.py          # Aptly wrapper
│   ├── config.py         # Configuration management
│   ├── gpg.py            # GPG operations
│   └── utils.py          # Utilities
├── tests/                # Tests
├── docs/                 # Documentation
├── .github/workflows/    # GitHub Actions
└── config.yaml.example   # Configuration example
```

### Running Tests

```bash
# All tests
pytest

# With coverage
pytest --cov=debrepomanager --cov-report=html

# Specific module
pytest tests/test_config.py

# Verbose
pytest -v
```

### Code Quality

Project uses:
- **Black** for formatting (spaces, not tabs)
- **flake8** for linting
- **mypy** for type checking

```bash
# Format
black debrepomanager/

# Lint
flake8 debrepomanager/

# Type check
mypy debrepomanager/
```

## Documentation

### Quick Reference
- 📖 [Quick Start](QUICK_START.md) - Get started in 5 minutes
- 📋 [Command Cheat Sheet](docs/CHEATSHEET.md) - Command reference
- 🔄 [Common Workflows](docs/WORKFLOWS.md) - Typical scenarios
- 🔀 [Migration Guide](docs/MIGRATION.md) - Upgrade from v0.3.x to v0.4.0

### For Users
- [Deployment Guide](docs/DEPLOYMENT_GUIDE.md) - Complete deployment instructions
- [APT Configuration](docs/APT_CONFIGURATION.md) - Client setup for all systems
- [Release Lifecycle](docs/RELEASE_LIFECYCLE.md) - Managing releases

### For Administrators
- [Configuration Reference](docs/CONFIG.md) - Detailed config options
- [Nginx Configuration](docs/NGINX_OLD_FORMAT.md) - Old format APT support
- [Multi-Environment Setup](docs/NGINX_MULTI_ENV.md) - stable/beta/test environments
- [Dual Format Support](docs/DUAL_FORMAT.md) - Old and new URL formats

### For Developers
- [Architecture](docs/ARCHITECTURE.md) - Architecture decisions
- [Development Guide](docs/DEVELOPMENT.md) - Developer workflow
- [GPG Key Rotation](docs/GPG_ROTATION.md) - Zero-downtime key rotation
- [Changelog](docs/CHANGELOG.md) - Version history

## Troubleshooting

### GPG signing fails

```bash
# Check key availability
gpg --list-secret-keys

# Check gpg-agent
gpg-connect-agent 'keyinfo --list' /bye

# Restart gpg-agent
gpgconf --kill gpg-agent
gpg-connect-agent /bye
```

### Aptly errors

```bash
# Check repository status
debrepomanager list

# Check snapshots (via aptly directly)
aptly -config /srv/aptly/bookworm/aptly.conf snapshot list

# Check published repositories
aptly -config /srv/aptly/bookworm/aptly.conf publish list
```

### Permission issues

```bash
# Check permissions
ls -la /srv/aptly/
ls -la /srv/repo/public/

# Fix permissions
chown -R repomanager:repomanager /srv/aptly/
chown -R www-data:repomanager /srv/repo/public/
chmod -R g+w /srv/aptly/ /srv/repo/public/
```

## GitHub Actions Integration

For CI/CD workflows, configure GitHub Secrets:

- `SSH_PRIVATE_KEY`: SSH key for server access
- `SSH_HOST`: Server address (e.g., repo.site.com)
- `SSH_USER`: SSH user (e.g., repomanager)
- `GPG_PRIVATE_KEY`: GPG private key (base64 encoded)
- `GPG_PASSPHRASE`: GPG key password
- `GPG_KEY_ID`: GPG key ID

See [Quick Start Guide](docs/QUICKSTART.md#github-actions-setup) for GitHub Actions examples.

## License

MIT License - see [LICENSE](LICENSE)

## Authors

JetHome Team

## Contributing

Pull requests are welcome! Please make sure to:
1. Add tests for new functionality
2. Update documentation
3. Follow code style (black, flake8)
4. Ensure all tests pass

## Links

- **Documentation**: [docs/](docs/)
- **PyPI**: https://pypi.org/project/debrepomanager/
- **Issues**: https://github.com/jethome-iot/repomanager/issues
- **Discussions**: https://github.com/jethome-iot/repomanager/discussions

## Project Status

- **Version**: 0.4.0
- **Status**: Production Ready
- **Tests**: 300+ tests, 80%+ coverage
- **Python**: 3.11, 3.12, 3.13 supported

### What's New in v0.4.0

- ✅ **Release management commands**: `debrepomanager release add/remove`
- ✅ **Environment isolation**: `--environment stable/beta/test`
- ✅ **Auto-component generation**: from directory name
- ⚠️ **Deprecated**: bash scripts (use Python CLI instead)
- 📚 **Enhanced documentation**: CHEATSHEET, WORKFLOWS, MIGRATION guides

---

**Made with ❤️ by JetHome Team**