Skill: xueyetianya/genai-toolkit (version 1.0.0)

--- version: "1.0.0" name: Genai Toolbox description: "Bridge AI models to databases through MCP with config and evaluation tools. Use when setting up DB tools, comparing engines, or evaluating prompt quality." --- # Genai Toolkit Genai Toolkit v2.0.0 — an AI toolkit for managing generative AI workflows from the command line. Log configurations, benchmarks, prompts, evaluations, fine-tuning runs, cost tracking, and optimization notes. Each entry is timestamped and persisted locally. Works entirely offline — your data never leaves your machine. ## Why Genai Toolkit? - Works entirely offline — your data never leaves your machine - Simple command-line interface with no GUI dependency - Export to JSON, CSV, or plain text at any time for sharing or archival - Automatic activity history logging across all commands - Each domain command doubles as both a logger and a viewer ## Commands ### Domain Commands Each domain command works in two modes: **log mode** (with arguments) saves a timestamped entry, **view mode** (no arguments) shows the 20 most recent entries. | Command | Description | |---------|-------------| | `genai-toolkit configure <input>` | Log a configuration note such as model parameters, API keys, or environment settings. Use this to record setup changes and track which configurations were active during experiments. | | `genai-toolkit benchmark <input>` | Log a benchmark result or performance observation. Record latency, throughput, accuracy, or other metrics to compare across runs and model versions. | | `genai-toolkit compare <input>` | Log a comparison note between models, configurations, or approaches. Useful for side-by-side evaluations like GPT-4 vs Claude on specific tasks. | | `genai-toolkit prompt <input>` | Log a prompt template or prompt engineering note. Track iterations on prompt design, record what worked, and document prompt versioning. | | `genai-toolkit evaluate <input>` | Log an evaluation result or quality metric. Record accuracy scores, F1 metrics, human ratings, or any qualitative assessment of model outputs. | | `genai-toolkit fine-tune <input>` | Log a fine-tuning run or hyperparameter note. Track epochs, learning rates, dataset sizes, and resulting model performance after fine-tuning. | | `genai-toolkit analyze <input>` | Log an analysis observation or insight. Record patterns found in data, failure mode analysis, or trends across experiments. | | `genai-toolkit cost <input>` | Log cost tracking data including API costs, compute expenses, and token consumption. Essential for budget monitoring across projects and providers. | | `genai-toolkit usage <input>` | Log usage metrics or consumption data. Track request volumes, token counts, rate limit encounters, and daily/monthly consumption patterns. | | `genai-toolkit optimize <input>` | Log optimization attempts or performance improvements. Record what was changed, the expected vs actual impact, and next steps. | | `genai-toolkit test <input>` | Log test results or test case notes. Record pass/fail outcomes, edge cases discovered, and regression test results. | | `genai-toolkit report <input>` | Log a report entry or summary finding. Capture weekly summaries, milestone reports, or executive-level findings from AI workflows. | ### Utility Commands | Command | Description | |---------|-------------| | `genai-toolkit stats` | Show summary statistics across all log files, including entry counts per category and total data size on disk. | | `genai-toolkit export <fmt>` | Export all data to a file in the specified format. Supported formats: `json`, `csv`, `txt`. Output is saved to the data directory. | | `genai-toolkit search <term>` | Search all log entries for a term using case-insensitive matching. Results are grouped by log category for easy scanning. | | `genai-toolkit recent` | Show the 20 most recent entries from the unified activity log, giving a quick overview of recent work across all commands. | | `genai-toolkit status` | Health check showing version, data directory path, total entry count, disk usage, and last activity timestamp. | | `genai-toolkit help` | Show the built-in help message listing all available commands and usage information. | | `genai-toolkit version` | Print the current version (v2.0.0). | ## Data Storage All data is stored locally at `~/.local/share/genai-toolkit/`. Each domain command writes to its own log file (e.g., `configure.log`, `benchmark.log`). A unified `history.log` tracks all actions across commands. Use `export` to back up your data at any time. ## Requirements - Bash (4.0+) - No external dependencies — pure shell script - No network access required ## When to Use - Tracking AI model benchmarks and comparisons across different providers and versions over time - Logging prompt engineering iterations to understand what improvements actually moved the needle - Monitoring API costs and token usage across multiple projects and billing periods - Evaluating fine-tuning experiments with detailed hyperparameter and metric tracking - Building a searchable knowledge base of optimization attempts and analysis insights ## Examples ```bash # Log a benchmark result genai-toolkit benchmark "GPT-4o latency: avg 1.2s, p99 3.8s on summarization task, 500 samples" # Track a cost entry genai-toolkit cost "March batch processing: $42.50 across 15k requests, avg $0.0028/req" # Compare two models genai-toolkit compare "Claude 3.5 vs GPT-4o on code generation — Claude 15% faster, GPT-4o 5% more accurate" # Log a prompt iteration genai-toolkit prompt "v3: Added chain-of-thought instruction, reduced hallucination rate from 12% to 3%" # Record a fine-tuning run genai-toolkit fine-tune "SQL-gen model epoch 5: accuracy=0.96, loss=0.12, lr=2e-5, dataset=50k rows" # View all statistics genai-toolkit stats # Export everything to JSON genai-toolkit export json # Search for entries mentioning latency genai-toolkit search latency # Check recent activity genai-toolkit recent # Health check genai-toolkit status ``` --- Powered by BytesAgain | bytesagain.com | [REDACTED_SECRET]

path: scripts/script.sh
content: #!/usr/bin/env bash
# Genai Toolkit — ai tool
# Powered by BytesAgain | bytesagain.com | [REDACTED_SECRET]
set -euo pipefail

DATA_DIR="${HOME}/.local/share/genai-toolkit"
mkdir -p "$DATA_DIR"

_log() { echo "$(date '+%m-%d %H:%M') $1: $2" >> "$DATA_DIR/history.log"; }
_version() { echo "genai-toolkit v2.0.0"; }

_help() {
    echo "Genai Toolkit v2.0.0 — ai toolkit"
    echo ""
    echo "Usage: genai-toolkit <command> [args]"
    echo ""
    echo "Commands:"
    echo "  configure          Configure"
    echo "  benchmark          Benchmark"
    echo "  compare            Compare"
    echo "  prompt             Prompt"
    echo "  evaluate           Evaluate"
    echo "  fine-tune          Fine Tune"
    echo "  analyze            Analyze"
    echo "  cost               Cost"
    echo "  usage              Usage"
    echo "  optimize           Optimize"
    echo "  test               Test"
    echo "  report             Report"
    echo "  stats              Summary statistics"
    echo "  export <fmt>       Export (json|csv|txt)"
    echo "  search <term>      Search entries"
    echo "  recent             Recent activity"
    echo "  status             Health check"
    echo "  help               Show this help"
    echo "  version            Show version"
    echo ""
    echo "Data: $DATA_DIR"
}

_stats() {
    echo "=== Genai Toolkit Stats ==="
    local total=0
    for f in "$DATA_DIR"/*.log; do
        [ -f "$f" ] || continue
        local name=$(basename "$f" .log)
        local c=$(wc -l < "$f")
        total=$((total + c))
        echo "  $name: $c entries"
    done
    echo "  ---"
    echo "  Total: $total entries"
    echo "  Data size: $(du -sh "$DATA_DIR" 2>/dev/null | cut -f1)"
}

_export() {
    local fmt="${1:-json}"
    local out="$DATA_DIR/export.$fmt"
    case "$fmt" in
        json)
            echo "[" > "$out"
            local first=1
            for f in "$DATA_DIR"/*.log; do
                [ -f "$f" ] || continue
                local name=$(basename "$f" .log)
                while IFS='|' read -r ts val; do
                    [ $first -eq 1 ] && first=0 || echo "," >> "$out"
                    printf '  {"type":"%s","time":"%s","value":"%s"}' "$name" "$ts" "$val" >> "$out"
                done < "$f"
            done
            echo "\n]" >> "$out"
            ;;
        csv)
            echo "type,time,value" > "$out"
            for f in "$DATA_DIR"/*.log; do
                [ -f "$f" ] || continue
                local name=$(basename "$f" .log)
                while IFS='|' read -r ts val; do echo "$name,$ts,$val" >> "$out"; done < "$f"
            done
            ;;
        txt)
            echo "=== Genai Toolkit Export ===" > "$out"
            for f in "$DATA_DIR"/*.log; do
                [ -f "$f" ] || continue
                echo "--- $(basename "$f" .log) ---" >> "$out"
                cat "$f" >> "$out"
            done
            ;;
        *) echo "Formats: json, csv, txt"; return 1 ;;
    esac
    echo "Exported to $out ($(wc -c < "$out") bytes)"
}

_status() {
    echo "=== Genai Toolkit Status ==="
    echo "  Version: v2.0.0"
    echo "  Data dir: $DATA_DIR"
    echo "  Entries: $(cat "$DATA_DIR"/*.log 2>/dev/null | wc -l) total"
    echo "  Disk: $(du -sh "$DATA_DIR" 2>/dev/null | cut -f1)"
    echo "  Last: $(tail -1 "$DATA_DIR/history.log" 2>/dev/null || echo never)"
    echo "  Status: OK"
}

_search() {
    local term="${1:?Usage: genai-toolkit search <term>}"
    echo "Searching for: $term"
    for f in "$DATA_DIR"/*.log; do
        [ -f "$f" ] || continue
        local m=$(grep -i "$term" "$f" 2>/dev/null || true)
        if [ -n "$m" ]; then
            echo "  --- $(basename "$f" .log) ---"
            echo "$m" | sed 's/^/    /'
        fi
    done
}

_recent() {
    echo "=== Recent Activity ==="
    tail -20 "$DATA_DIR/history.log" 2>/dev/null | sed 's/^/  /' || echo "  No activity yet."
}

case "${1:-help}" in
    configure)
        shift
        if [ $# -eq 0 ]; then
            echo "Recent configure entries:"
            tail -20 "$DATA_DIR/configure.log" 2>/dev/null || echo "  No entries yet. Use: genai-toolkit configure <input>"
        else
            local input="$*"
            local ts=$(date '+%Y-%m-%d %H:%M')
            echo "$ts|$input" >> "$DATA_DIR/configure.log"
            local total=$(wc -l < "$DATA_DIR/configure.log")
            echo "  [Genai Toolkit] configure: $input"
            echo "  Saved. Total configure entries: $total"
            _log "configure" "$input"
        fi
        ;;
    benchmark)
        shift
        if [ $# -eq 0 ]; then
            echo "Recent benchmark entries:"
            tail -20 "$DATA_DIR/benchmark.log" 2>/dev/null || echo "  No entries yet. Use: genai-toolkit benchmark <input>"
        else
            local input="$*"
            local ts=$(date '+%Y-%m-%d %H:%M')
            echo "$ts|$input" >> "$DATA_DIR/benchmark.log"
            local total=$(wc -l < "$DATA_DIR/benchmark.log")
            echo "  [Genai Toolkit] benchmark: $input"
            echo "  Saved. Total benchmark entries: $total"
            _log "benchmark" "$input"
        fi
        ;;
    compare)
        shift
        if [ $# -eq 0 ]; then
            echo "Recent compare entries:"
            tail -20 "$DATA_DIR/compare.log" 2>/dev/null || echo "  No entries yet. Use: genai-toolkit compare <input>"
        else
            local input="$*"
            local ts=$(date '+%Y-%m-%d %H:%M')
            echo "$ts|$input" >> "$DATA_DIR/compare.log"
            local total=$(wc -l < "$DATA_DIR/compare.log")
            echo "  [Genai Toolkit] compare: $input"
            echo "  Saved. Total compare entries: $total"
            _log "compare" "$input"
        fi
        ;;
    prompt)
        shift
        if [ $# -eq 0 ]; then
            echo "Recent prompt entries:"
            tail -20 "$DATA_DIR/prompt.log" 2>/dev/null || echo "  No entries yet. Use: genai-toolkit prompt <input>"
        else
            local input="$*"
            local ts=$(date '+%Y-%m-%d %H:%M')
            echo "$ts|$input" >> "$DATA_DIR/prompt.log"
            local total=$(wc -l < "$DATA_DIR/prompt.log")
            echo "  [Genai Toolkit] prompt: $input"
            echo "  Saved. Total prompt entries: $total"
            _log "prompt" "$input"
        fi
        ;;
    evaluate)
        shift
        if [ $# -eq 0 ]; then
            echo "Recent evaluate entries:"
            tail -20 "$DATA_DIR/evaluate.log" 2>/dev/null || echo "  No entries yet. Use: genai-toolkit evaluate <input>"
        else
            local input="$*"
            local ts=$(date '+%Y-%m-%d %H:%M')
            echo "$ts|$input" >> "$DATA_DIR/evaluate.log"
            local total=$(wc -l < "$DATA_DIR/evaluate.log")
            echo "  [Genai Toolkit] evaluate: $input"
            echo "  Saved. Total evaluate entries: $total"
            _log "evaluate" "$input"
        fi
        ;;
    fine-tune)
        shift
        if [ $# -eq 0 ]; then
            echo "Recent fine-tune entries:"
            tail -20 "$DATA_DIR/fine-tune.log" 2>/dev/null || echo "  No entries yet. Use: genai-toolkit fine-tune <input>"
        else
            local input="$*"
            local ts=$(date '+%Y-%m-%d %H:%M')
            echo "$ts|$input" >> "$DATA_DIR/fine-tune.log"
            local total=$(wc -l < "$DATA_DIR/fine-tune.log")
            echo "  [Genai Toolkit] fine-tune: $input"
            echo "  Saved. Total fine-tune entries: $total"
            _log "fine-tune" "$input"
        fi
        ;;
    analyze)
        shift
        if [ $# -eq 0 ]; then
            echo "Recent analyze entries:"
            tail -20 "$DATA_DIR/analyze.log" 2>/dev/null || echo "  No entries yet. Use: genai-toolkit analyze <input>"
        else
            local input="$*"
            local ts=$(date '+%Y-%m-%d %H:%M')
            echo "$ts|$input" >> "$DATA_DIR/analyze.log"
            local total=$(wc -l < "$DATA_DIR/analyze.log")
            echo "  [Genai Toolkit] analyze: $input"
            echo "  Saved. Total analyze entries: $total"
            _log "analyze" "$input"
        fi
        ;;
    cost)
        shift
        if [ $# -eq 0 ]; then
            echo "Recent cost entries:"
            tail -20 "$DATA_DIR/cost.log" 2>/dev/null || echo "  No entries yet. Use: genai-toolkit cost <input>"
        else
            local input="$*"
            local ts=$(date '+%Y-%m-%d %H:%M')
            echo "$ts|$input" >> "$DATA_DIR/cost.log"
            local total=$(wc -l < "$DATA_DIR/cost.log")
            echo "  [Genai Toolkit] cost: $input"
            echo "  Saved. Total cost entries: $total"
            _log "cost" "$input"
        fi
        ;;
    usage)
        shift
        if [ $# -eq 0 ]; then
            echo "Recent usage entries:"
            tail -20 "$DATA_DIR/usage.log" 2>/dev/null || echo "  No entries yet. Use: genai-toolkit usage <input>"
        else
            local input="$*"
            local ts=$(date '+%Y-%m-%d %H:%M')
            echo "$ts|$input" >> "$DATA_DIR/usage.log"
            local total=$(wc -l < "$DATA_DIR/usage.log")
            echo "  [Genai Toolkit] usage: $input"
            echo "  Saved. Total usage entries: $total"
            _log "usage" "$input"
        fi
        ;;
    optimize)
        shift
        if [ $# -eq 0 ]; then
            echo "Recent optimize entries:"
            tail -20 "$DATA_DIR/optimize.log" 2>/dev/null || echo "  No entries yet. Use: genai-toolkit optimize <input>"
        else
            local input="$*"
            local ts=$(date '+%Y-%m-%d %H:%M')
            echo "$ts|$input" >> "$DATA_DIR/optimize.log"
            local total=$(wc -l < "$DATA_DIR/optimize.log")
            echo "  [Genai Toolkit] optimize: $input"
            echo "  Saved. Total optimize entries: $total"
            _log "optimize" "$input"
        fi
        ;;
    test)
        shift
        if [ $# -eq 0 ]; then
            echo "Recent test entries:"
            tail -20 "$DATA_DIR/test.log" 2>/dev/null || echo "  No entries yet. Use: genai-toolkit test <input>"
        else
            local input="$*"
            local ts=$(date '+%Y-%m-%d %H:%M')
            echo "$ts|$input" >> "$DATA_DIR/test.log"
            local total=$(wc -l < "$DATA_DIR/test.log")
            echo "  [Genai Toolkit] test: $input"
            echo "  Saved. Total test entries: $total"
            _log "test" "$input"
        fi
        ;;
    report)
        shift
        if [ $# -eq 0 ]; then
            echo "Recent report entries:"
            tail -20 "$DATA_DIR/report.log" 2>/dev/null || echo "  No entries yet. Use: genai-toolkit report <input>"
        else
            local input="$*"
            local ts=$(date '+%Y-%m-%d %H:%M')
            echo "$ts|$input" >> "$DATA_DIR/report.log"
            local total=$(wc -l < "$DATA_DIR/report.log")
            echo "  [Genai Toolkit] report: $input"
            echo "  Saved. Total report entries: $total"
            _log "report" "$input"
        fi
        ;;
    stats) _stats ;;
    export) shift; _export "$@" ;;
    search) shift; _search "$@" ;;
    recent) _recent ;;
    status) _status ;;
    help|--help|-h) _help ;;
    version|--version|-v) _version ;;
    *)
        echo "Unknown: $1 — run 'genai-toolkit help'"
        exit 1
        ;;
esac

Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)