Skill: nissan/langfuse-backup (version 1.0.2)

--- name: langfuse-backup version: 1.0.2 metadata: openclaw: emoji: "💾" requires: bins: ["docker"] env: [] network: outbound: false reason: "Backs up local Docker volumes only. No data is sent to remote servers." security_notes: "All operations are local Docker volume copies. No data leaves the local machine. Backup files are stored in a local directory specified by the user." description: "Docker volume backup and restore for self-hosted Langfuse. Use when: backing up a self-hosted Langfuse instance, restoring Langfuse after a crash or migration, setting up daily cron backups for Langfuse postgres/clickhouse/minio/redis volumes. Backs up postgres (traces, scores), minio (blobs), with optional clickhouse and redis. Includes restore script with validation and 14-day retention pruning." --- **Last used:** 2026-03-24 **Memory references:** 2 **Status:** Active # langfuse-backup Backup and restore Docker volumes for a self-hosted Langfuse instance. ## Scripts - `scripts/backup_langfuse.sh` — back up postgres + minio volumes (primary data) - `scripts/restore_langfuse.sh` — restore from a specific backup date ## Quick start ```bash # Configure (env vars or edit the script defaults) export LANGFUSE_BACKUP_DIR="$HOME/.langfuse-backups" export LANGFUSE_COMPOSE_DIR="/path/to/langfuse" # directory with docker-compose.yml export LANGFUSE_DB_CONTAINER="langfuse-db-1" # postgres container name export LANGFUSE_MINIO_CONTAINER="langfuse-minio-1" # minio container name export LANGFUSE_DB_NAME="langfuse" # postgres database name export LANGFUSE_DB_USER="langfuse" # postgres user # Run a backup bash scripts/backup_langfuse.sh # List available backups ls "$LANGFUSE_BACKUP_DIR" # Restore from a date bash scripts/restore_langfuse.sh 2026-02-27 ``` ## Env vars | Var | Default | Description | |---------------------------|--------------------------------------|-------------------------------------| | `LANGFUSE_BACKUP_DIR` | `~/.langfuse-backups` | Root backup directory | | `LANGFUSE_COMPOSE_DIR` | `~/langfuse` | Docker Compose project directory | | `LANGFUSE_DB_CONTAINER` | `langfuse-db-1` | Postgres container name | | `LANGFUSE_MINIO_CONTAINER`| `langfuse-minio-1` | MinIO container name | | `LANGFUSE_DB_NAME` | `langfuse` | Postgres database name | | `LANGFUSE_DB_USER` | `langfuse` | Postgres user | | `LANGFUSE_RETENTION_DAYS` | `14` | How many days of backups to keep | ## What gets backed up | Volume | Backed up? | Notes | |-----------|------------|---------------------------------------------| | Postgres | ✅ Yes | `pg_dump` → `.sql.gz` — traces, scores, evals | | MinIO | ✅ Yes | `tar.gz` — uploaded blobs | | ClickHouse| ⚠️ Optional| Large; many users skip (replayed from traces)| | Redis | ⚠️ Skip | Cache only — safe to skip | ## Cron setup (macOS LaunchAgent) ```xml <!-- ~/Library/LaunchAgents/com.yourname.langfuse-backup.plist --> <key>StartCalendarInterval</key> <dict> <key>Hour</key><integer>2</integer> <key>Minute</key><integer>0</integer> </dict> <key>ProgramArguments</key> <array> <string>/bin/bash</string> <string>/path/to/scripts/backup_langfuse.sh</string> </array> <key>StandardOutPath</key> <string>/tmp/langfuse-backup.log</string> ``` ## Restore procedure ```bash # 1. Stop Langfuse cd $LANGFUSE_COMPOSE_DIR && docker compose down # 2. Restore from backup bash scripts/restore_langfuse.sh 2026-02-27 # 3. Start Langfuse cd $LANGFUSE_COMPOSE_DIR && docker compose up -d ```

path: scripts/backup_langfuse.sh
content: #!/usr/bin/env bash
# backup_langfuse.sh — Back up Langfuse Docker volumes to local backup dir.
# Backs up: postgres (traces, scores, evals) + minio (blob storage)
# ClickHouse and Redis are transient — optional.
#
# Usage:
#   bash scripts/backup_langfuse.sh
#
# Env vars (all optional — sensible defaults):
#   LANGFUSE_BACKUP_DIR       Root backup directory (default: ~/.langfuse-backups)
#   LANGFUSE_COMPOSE_DIR      Docker Compose project directory (default: ~/langfuse)
#   LANGFUSE_DB_CONTAINER     Postgres container name (default: langfuse-db-1)
#   LANGFUSE_MINIO_CONTAINER  MinIO container name (default: langfuse-minio-1)
#   LANGFUSE_DB_NAME          Postgres database name (default: langfuse)
#   LANGFUSE_DB_USER          Postgres user (default: langfuse)
#   LANGFUSE_RETENTION_DAYS   Days of backups to keep (default: 14)

set -euo pipefail

BACKUP_BASE="${LANGFUSE_BACKUP_DIR:-$HOME/.langfuse-backups}"
DATE=$(date +%Y-%m-%d)
BACKUP_DIR="$BACKUP_BASE/$DATE"
DB_CONTAINER="${LANGFUSE_DB_CONTAINER:-langfuse-db-1}"
MINIO_CONTAINER="${LANGFUSE_MINIO_CONTAINER:-langfuse-minio-1}"
DB_NAME="${LANGFUSE_DB_NAME:-langfuse}"
DB_USER="${LANGFUSE_DB_USER:-langfuse}"
RETENTION_DAYS="${LANGFUSE_RETENTION_DAYS:-14}"
LOG="/tmp/langfuse-backup.log"

log() { echo "$(date '+%H:%M:%S')  $*" | tee -a "$LOG"; }

log "=== Langfuse backup: $DATE ==="
mkdir -p "$BACKUP_DIR"

# ── Postgres (main database: traces, scores, sessions) ───────────────────────
log "Backing up Postgres..."
if docker ps --format '{{.Names}}' | grep -q "$DB_CONTAINER"; then
    docker exec "$DB_CONTAINER" \
        pg_dump -U "$DB_USER" "$DB_NAME" 2>/dev/null \
        | gzip > "$BACKUP_DIR/postgres-langfuse.sql.gz"
    SIZE=$(du -sh "$BACKUP_DIR/postgres-langfuse.sql.gz" | cut -f1)
    log "✅ Postgres: $SIZE"
else
    log "⚠️  Postgres container '$DB_CONTAINER' not running — skipping"
    log "    (run 'docker ps' to check container names, set LANGFUSE_DB_CONTAINER)"
fi

# ── MinIO (blob storage: uploaded files/media) ────────────────────────────────
log "Backing up MinIO..."
if docker ps --format '{{.Names}}' | grep -q "$MINIO_CONTAINER"; then
    MINIO_VOL=$(docker inspect "$MINIO_CONTAINER" \
        --format '{{range .Mounts}}{{if eq .Type "volume"}}{{.Name}}{{end}}{{end}}' 2>/dev/null | head -1)
    if [ -n "$MINIO_VOL" ]; then
        docker run --rm \
            -v "$MINIO_VOL":/data:ro \
            -v "$BACKUP_DIR":/backup \
            alpine tar czf /backup/minio-data.tar.gz -C /data . 2>/dev/null
        SIZE=$(du -sh "$BACKUP_DIR/minio-data.tar.gz" | cut -f1)
        log "✅ MinIO: $SIZE"
    else
        log "⚠️  Could not find MinIO volume — skipping"
    fi
else
    log "⚠️  MinIO container '$MINIO_CONTAINER' not running — skipping"
fi

# ── Manifest ─────────────────────────────────────────────────────────────────
FILES=$(ls "$BACKUP_DIR" | grep -v manifest.json | python3 -c "
import sys, json
lines = sys.stdin.read().strip().split('\n')
print(json.dumps([l for l in lines if l]))
" 2>/dev/null || echo "[]")

cat > "$BACKUP_DIR/manifest.json" << MANIFEST
{
  "date": "$DATE",
  "timestamp": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
  "host": "$(hostname)",
  "db_container": "$DB_CONTAINER",
  "minio_container": "$MINIO_CONTAINER",
  "files": $FILES,
  "total_size": "$(du -sh "$BACKUP_DIR" | cut -f1)"
}
MANIFEST
log "Manifest written: $BACKUP_DIR/manifest.json"

# ── Prune old backups ─────────────────────────────────────────────────────────
log "Pruning backups older than $RETENTION_DAYS days..."
find "$BACKUP_BASE" -maxdepth 1 -type d -name "20*" -mtime "+$RETENTION_DAYS" \
    -exec rm -rf {} \; 2>/dev/null || true

TOTAL=$(du -sh "$BACKUP_BASE" 2>/dev/null | cut -f1)
log "=== Done. Backup: $BACKUP_DIR | Total store: $TOTAL ==="


path: scripts/restore_langfuse.sh
content: #!/usr/bin/env bash
# restore_langfuse.sh — Restore Langfuse Docker volumes from a backup.
#
# Usage:
#   bash scripts/restore_langfuse.sh <YYYY-MM-DD>
#   bash scripts/restore_langfuse.sh latest
#
# Env vars (same as backup script):
#   LANGFUSE_BACKUP_DIR       Root backup directory (default: ~/.langfuse-backups)
#   LANGFUSE_COMPOSE_DIR      Docker Compose project directory (default: ~/langfuse)
#   LANGFUSE_DB_CONTAINER     Postgres container name (default: langfuse-db-1)
#   LANGFUSE_MINIO_CONTAINER  MinIO container name (default: langfuse-minio-1)
#   LANGFUSE_DB_NAME          Postgres database name (default: langfuse)
#   LANGFUSE_DB_USER          Postgres user (default: langfuse)

set -euo pipefail

BACKUP_BASE="${LANGFUSE_BACKUP_DIR:-$HOME/.langfuse-backups}"
COMPOSE_DIR="${LANGFUSE_COMPOSE_DIR:-$HOME/langfuse}"
DB_CONTAINER="${LANGFUSE_DB_CONTAINER:-langfuse-db-1}"
MINIO_CONTAINER="${LANGFUSE_MINIO_CONTAINER:-langfuse-minio-1}"
DB_NAME="${LANGFUSE_DB_NAME:-langfuse}"
DB_USER="${LANGFUSE_DB_USER:-langfuse}"

log() { echo "$(date '+%H:%M:%S')  $*"; }
err() { echo "❌  ERROR: $*" >&2; exit 1; }

# Resolve backup date
DATE_ARG="${1:-latest}"
if [ "$DATE_ARG" = "latest" ]; then
    BACKUP_DATE=$(ls -1 "$BACKUP_BASE" | grep -E '^20[0-9]{2}-[0-9]{2}-[0-9]{2}$' | sort | tail -1)
    if [ -z "$BACKUP_DATE" ]; then
        err "No backups found in $BACKUP_BASE"
    fi
    log "Resolved 'latest' → $BACKUP_DATE"
else
    BACKUP_DATE="$DATE_ARG"
fi

BACKUP_DIR="$BACKUP_BASE/$BACKUP_DATE"
[ -d "$BACKUP_DIR" ] || err "Backup directory not found: $BACKUP_DIR"

log "=== Langfuse restore: $BACKUP_DATE ==="

# ── Validate manifest ─────────────────────────────────────────────────────────
if [ -f "$BACKUP_DIR/manifest.json" ]; then
    log "Manifest:"
    cat "$BACKUP_DIR/manifest.json"
    echo ""
else
    log "⚠️  No manifest.json found — proceeding anyway"
fi

# ── Validate backup files exist ───────────────────────────────────────────────
PG_FILE="$BACKUP_DIR/postgres-langfuse.sql.gz"
MINIO_FILE="$BACKUP_DIR/minio-data.tar.gz"

[ -f "$PG_FILE" ] && log "✅ Found: postgres-langfuse.sql.gz" || log "⚠️  Missing: postgres-langfuse.sql.gz"
[ -f "$MINIO_FILE" ] && log "✅ Found: minio-data.tar.gz" || log "⚠️  Missing: minio-data.tar.gz"

# ── Confirm ───────────────────────────────────────────────────────────────────
echo ""
echo "⚠️  WARNING: This will overwrite the current Langfuse database and MinIO data."
echo "   Restore from: $BACKUP_DIR"
echo "   Langfuse must be STOPPED before restoring."
echo ""
read -r -p "Continue? [y/N] " CONFIRM
[ "$CONFIRM" = "y" ] || { log "Aborted."; exit 0; }

# ── Check Langfuse is stopped ─────────────────────────────────────────────────
if docker ps --format '{{.Names}}' | grep -q "langfuse"; then
    echo ""
    echo "⚠️  Langfuse containers appear to be running."
    echo "   Stop them first: cd $COMPOSE_DIR && docker compose down"
    echo ""
    read -r -p "Stop them now and continue? [y/N] " STOP_CONFIRM
    if [ "$STOP_CONFIRM" = "y" ]; then
        (cd "$COMPOSE_DIR" && docker compose down) || true
        sleep 3
    else
        log "Aborted — stop Langfuse first."
        exit 1
    fi
fi

# ── Start only the DB container for restore ───────────────────────────────────
log "Starting DB container for restore..."
(cd "$COMPOSE_DIR" && docker compose up -d langfuse-db) 2>/dev/null || \
(cd "$COMPOSE_DIR" && docker compose up -d db) 2>/dev/null || true
sleep 5

# ── Restore Postgres ──────────────────────────────────────────────────────────
if [ -f "$PG_FILE" ]; then
    log "Restoring Postgres from $PG_FILE..."
    
    # Drop and recreate the database
    docker exec "$DB_CONTAINER" psql -U "$DB_USER" postgres \
        -c "DROP DATABASE IF EXISTS $DB_NAME;" 2>/dev/null || true
    docker exec "$DB_CONTAINER" psql -U "$DB_USER" postgres \
        -c "CREATE DATABASE $DB_NAME;" 2>/dev/null || true
    
    # Restore
    zcat "$PG_FILE" | docker exec -i "$DB_CONTAINER" \
        psql -U "$DB_USER" "$DB_NAME" -q
    
    log "✅ Postgres restored"
else
    log "⚠️  Skipping Postgres restore (no backup file)"
fi

# ── Start MinIO for restore ───────────────────────────────────────────────────
if [ -f "$MINIO_FILE" ]; then
    log "Restoring MinIO from $MINIO_FILE..."
    
    (cd "$COMPOSE_DIR" && docker compose up -d minio) 2>/dev/null || \
    (cd "$COMPOSE_DIR" && docker compose up -d langfuse-minio) 2>/dev/null || true
    sleep 5
    
    MINIO_VOL=$(docker inspect "$MINIO_CONTAINER" \
        --format '{{range .Mounts}}{{if eq .Type "volume"}}{{.Name}}{{end}}{{end}}' 2>/dev/null | head -1)
    
    if [ -n "$MINIO_VOL" ]; then
        # Clear and restore
        docker run --rm \
            -v "$MINIO_VOL":/data \
            -v "$BACKUP_DIR":/backup:ro \
            alpine sh -c "rm -rf /data/* && tar xzf /backup/minio-data.tar.gz -C /data"
        log "✅ MinIO restored from volume $MINIO_VOL"
    else
        log "⚠️  Could not find MinIO volume — skipping"
    fi
else
    log "⚠️  Skipping MinIO restore (no backup file)"
fi

# ── Done ──────────────────────────────────────────────────────────────────────
log ""
log "=== Restore complete ==="
log "Start Langfuse: cd $COMPOSE_DIR && docker compose up -d"


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)