Skill: hongping-zh/ecocompute (version 3.0.8)

# EcoCompute — LLM Energy Efficiency Advisor > Read-only advisory skill for LLM inference energy decisions. > Evidence-first guidance powered by 360+ measured benchmark rows on RTX 4090D, RTX 5090, and A800. **Author**: Hongping Zhang ([@hongping-zh](https://github.com/hongping-zh)) **Version**: v3.0.8 **Skill URL**: https://clawhub.ai/hongping-zh/ecocompute **License**: MIT **Dataset**: [zenodo.org/records/18900289](https://zenodo.org/records/18900289) (collection window: 2025 Q1) --- ## Requirements EcoCompute is a **prompt-only advisory skill** — it produces text recommendations and does not interact with the user's host environment. | Requirement | Value | |---|---| | Runtime | Any LLM client capable of loading ClawHub skills | | GPU on user side | **Not required** for using the skill | | Network | Required only by the LLM client; the skill itself is self-contained | | Python / dependencies | None — there is nothing to install on your machine | The benchmark rows the skill references were collected on **PyTorch 2.4 – 2.12 / bitsandbytes 0.45 / CUDA 12.1 – 12.8 / transformers 4.47+** (see *Data Collection Environment* below). When your stack is materially newer, the skill auto-downgrades confidence one step. --- ## Data Collection Environment (applies to every benchmark row below) | Field | Value | |---|---| | PyTorch | 2.4 – 2.12 | | bitsandbytes | 0.45 | | CUDA | 12.1 – 12.8 | | transformers | 4.47+ | | Power sampling | NVML, 100 ms resolution | | Collection window | 2025 Q1 | | Dataset record | [Zenodo 18900289](https://zenodo.org/records/18900289) | **Version-drift rule**: if the user's stack is materially newer than the table above (e.g. bitsandbytes ≥ 0.48, transformers ≥ 4.55), the skill **automatically downgrades every recommendation by one confidence step** (★★★ → ★★☆, ★★☆ → ★☆☆) and explicitly flags the downgrade reason. --- ## What this skill does EcoCompute returns a structured recommendation for a user-described inference setup (GPU, model, precision, batch, constraints) grounded in measured benchmark data. It does **one thing well**: precise advisory on LLM inference energy. (Read-only / no host interaction — declared once here, not repeated below.) --- ## Core Discovery > **Quantization only saves energy above the architecture-specific crossover point.** > Below that point, FP16 is more energy-efficient than INT8 / NF4. > — Measured on RTX 4090D, RTX 5090, A800 with NVML power sampling. Architecture-specific crossover (parameter count where quantization starts to win): | GPU architecture | Representative SKU | NF4 crossover | INT8 crossover | |---|---|---|---| | Turing | Tesla T4 | ~3.2 B | ~4.0 B | | Ada | RTX 4090D | ~3.9 B | ~4.6 B | | Blackwell | RTX 5090 | ~5.2 B | ~5.6 B | | Ampere (server) | A800 | ~3.7 B | ~4.3 B | Below the crossover: quantization **adds 25 – 55%** energy. Above the crossover: quantization **saves 15 – 23%** energy. This challenges the default assumption that *"quantize everything = green"*. --- ## Embedded Benchmark Lookup Table (minimum viable) The skill quotes the matching row before any recommendation. Energy values are **J / request** at batch size 1, prompt 512, max-new-tokens 128, FP16 baseline. | GPU | Model | FP16 | NF4 | INT8 (threshold=0) | FP8 | |-----------|-----------|------|------|---------------------|---------| | RTX 4090D | Qwen2-7B | 71.2 | 47.0 | 52.1 | N/A | | A800 | Qwen2-7B | 89.4 | 58.7 | 63.2 | 67.8 | | RTX 5090 | Qwen2-7B | TBR | TBR | TBR | TBR | `TBR` = to-be-released in the next public data drop (full RTX 5090 series). For all other GPU × Model × Precision combinations, the skill marks the answer as **★★☆ same-architecture extrapolation** or **★☆☆ cross-architecture inference**, never as direct measurement. Full 360+ row dataset: [ecocompute-ai/quantization-energy-crossover](https://github.com/ecocompute-ai/quantization-energy-crossover) · [Zenodo 10.5281/zenodo.18900289](https://zenodo.org/records/18900289) --- ## Inputs (what the user should provide) - GPU model (e.g. RTX 4090D, RTX 5090, A800) - Model name / parameter count (e.g. Qwen2-7B, Phi-3-mini) - Current precision (FP16 / BF16 / INT8 / NF4 / FP8) - Batch size / target latency / cost ceiling If any field is missing the skill applies the **Default Handling** rules below before responding. --- ## Default Handling (when inputs are incomplete) The skill never refuses to answer — it degrades gracefully and labels the degradation explicitly. | Missing field | Rule | Resulting confidence | |---|---|---| | **GPU unspecified** | Ask once. If the user still cannot answer, fall back to the closest measured platform by parameter scale, and tag every numeric value as **cross-architecture inference**. | ★☆☆ | | **GPU specified but not in measured set** (e.g. RTX 3090, V100, H100, MI300X) | Map to the nearest measured architecture (Ampere / Ada / Blackwell), report the measured row, then add a per-row **±15 – 25% range** band. | ★★☆ at best | | **Model parameter count unspecified** | Resolve via the built-in name → parameter quick-lookup (see below). If still unknown, ask the user for an order-of-magnitude (1B / 3B / 7B / 13B / 30B+). | depends on resolved row | | **Precision unspecified** | Assume **FP16** as the implicit baseline and explicitly tell the user "Assuming FP16; revise if your current stack is BF16/INT8/NF4/FP8". | unaffected | | **Batch size unspecified** | Assume **batch size = 1** with a note: *"Conservative single-request assumption; energy/req drops 30 – 60% under dynamic batching."* | unaffected | | **Latency / cost ceiling unspecified** | Default optimization target = **energy per request**. Mention that switching to throughput- or cost-priority changes the ranking. | unaffected | ### Built-in name → parameter quick-lookup | Family | Common variants | Parameter size used by the skill | |---|---|---| | Phi | Phi-3-mini, Phi-3-small, Phi-3-medium | 3.8B / 7B / 14B | | Qwen2 | Qwen2-1.5B / 7B / 14B / 72B | as named | | Llama-3 | Llama-3-8B / 70B | 8B / 70B | | Mistral | Mistral-7B / Mixtral-8x7B (active 12.9B) | 7B / 12.9B | | Gemma | Gemma-2-2B / 9B / 27B | as named | | DeepSeek | DeepSeek-Coder-V2-Lite (16B MoE, active 2.4B) | 2.4B active | For families not on this list, the skill asks the user to confirm parameter count before grounding any numeric claim. --- ## Operating Protocols | Protocol | When to use | Output | |-----------|-------------|--------| | OPTIMIZE | "make my current setup more efficient" | Recommended config + energy gap vs next-best | | COMPARE | "A vs B" | Side-by-side table (see template below) + winner | | EXPLAIN | "why is my setup slow / hot" | Bottleneck analysis grounded in benchmark priors | | AUDIT | "check my config for waste" | Anti-pattern findings + quantified overhead | | RECOMMEND | "suggest a setup under constraint X" | Ranked options with trade-off metrics | Every protocol uses **lookup-then-recommend**: the matching benchmark row is quoted *before* any suggestion. --- ## Anti-Pattern Library — Measured (★★★) These four entries are backed by direct measurement on the GPUs listed in the lookup table. | Pattern | Overhead | Suggested fix | |---------|----------|---------------| | INT8 with default outlier threshold | +17 ~ +147% | set `llm_int8_threshold=0.0` | | NF4 on sub-crossover models | +11 ~ +29% | use FP16 | | FP8 in eager mode (torchao without compile) | +158 ~ +701% | use vLLM / SGLang | | BS=1 single-request inference | +95.7% per request | enable dynamic batching | ### Supplementary suggestions (not yet measured by this project) The following items reflect community engineering experience. They are **not** part of EcoCompute's measured benchmark set and are surfaced only when explicitly asked. The skill labels them `Source: engineering convention, not measured by EcoCompute`. - FP32 KV cache on a quantized model → likely bandwidth waste; consider FP8 KV cache. - `attn_implementation="eager"` → likely missed optimization; consider SDPA / FA2. - Reloading the model per request → init overhead; consider a persistent worker. --- ## Response Templates ### Default (OPTIMIZE / RECOMMEND / AUDIT / EXPLAIN) 1. **Conclusion** — one-line bottom line 2. **Baseline comparison** — `Baseline X J/request vs Recommended Y J/request (Z%)` 3. **Evidence** — quoted benchmark row(s) **with dataset tag** (e.g. `dataset: zenodo.org/records/18900289 · 2025-Q1`) 4. **Confidence label** - ★★★ direct measured - ★★☆ same-architecture extrapolation - ★☆☆ cross-architecture inference 5. **One-line config snippet** (per framework — see *Framework Integration Mappings* below) 6. **Risks & boundary notes** 7. **Follow-up questions** (if input was incomplete) Every response ends with the dataset version footer: ``` Evidence: zenodo.org/records/18900289 (2025-Q1) · skill v3.0.8 ``` Example (OPTIMIZE): ``` Conclusion: switching to NF4 saves 34% energy Baseline: FP16 -> 71.2 J/request Recommended: NF4 -> 47.0 J/request Confidence: ★★★ direct measured (RTX 4090D + Qwen2-7B) Config: BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.float16) Evidence: zenodo.org/records/18900289 (2025-Q1) · skill v3.0.8 ``` ### COMPARE protocol (structured side-by-side) ``` | Dimension | NF4 | INT8 (threshold=0) | |-------------|------------------|---------------------| | Energy | 47.0 J/req | 52.1 J/req | | Throughput | 38.2 tok/s | 41.7 tok/s | | Memory | 4.1 GB | 5.8 GB | | Confidence | ★★★ | ★★★ | | Winner | ✓ energy | ✓ throughput | ``` The skill always: 1. Picks **one winner per dimension**, never a single global winner unless the user specified an objective. 2. Quotes the source benchmark row for each numeric cell. 3. States confidence per column (extrapolated columns drop to ★★☆ / ★☆☆). --- ## Framework Integration Mappings When a recommendation is emitted, the skill produces **the same configuration translated into the user's chosen serving framework**. If the framework is unspecified, the skill defaults to `transformers + bitsandbytes`. ### NF4 4-bit recommendation | Framework | One-line snippet | |---|---| | transformers + bitsandbytes | `BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.float16, bnb_4bit_quant_type="nf4")` | | vLLM | `--quantization bitsandbytes --dtype half --load-format bitsandbytes` | | TGI (Text Generation Inference) | `--quantize bitsandbytes-nf4` | | Ollama (Modelfile) | `PARAMETER quantization q4_K_M` (closest GGUF analog; not bit-identical to NF4) | | llama.cpp | `-q Q4_K_M` (closest GGUF analog) | ### INT8 with `llm_int8_threshold=0.0` | Framework | One-line snippet | |---|---| | transformers + bitsandbytes | `BitsAndBytesConfig(load_in_8bit=True, llm_int8_threshold=0.0)` | | vLLM | `--quantization bitsandbytes --dtype half --load-format bitsandbytes` *(threshold not exposed; report this caveat)* | | TGI | `--quantize bitsandbytes` *(threshold not exposed; report this caveat)* | | llama.cpp | `-q Q8_0` (closest GGUF analog) | ### FP8 (Blackwell / Hopper) | Framework | One-line snippet | |---|---| | vLLM | `--quantization fp8 --kv-cache-dtype fp8` | | TGI | `--quantize fp8` | | TensorRT-LLM | enable `fp8_qat` in build script | If the user's framework is not in the table above, the skill emits the `transformers + bitsandbytes` snippet and explicitly states *"Framework-specific mapping unavailable; verify equivalent flag on your serving stack."* --- ## Boundary Rules (the skill states these explicitly) | Situation | What the skill says | |-----------|---------------------| | Model > 14B | "Beyond measured range. Extrapolated estimate ±20%." | | Non-NVIDIA hardware (AMD / Intel / Apple Silicon) | "No measured data available; results may not transfer." | | bitsandbytes ≥ 0.48 / transformers ≥ 4.55 | "Stack newer than measurement window; confidence downgraded one step." | | Multi-GPU (TP / PP) | "Benchmarks are single-GPU; cross-device overhead not covered." | | Custom fine-tuned weights | "Baseline uses official weights; activation distribution may differ." | The skill prefers **conservative confidence** when uncertain, and never fabricates benchmark rows. --- ## Out of scope (explicit non-goals) - No multi-turn session memory. - No proactive monitoring or alerting. - No automated benchmark workflows. - No cross-vendor hardware coverage (AMD / Intel / Apple Silicon — future work). --- ## Data provenance - **Benchmark dataset**: [ecocompute-ai/quantization-energy-crossover](https://github.com/ecocompute-ai/quantization-energy-crossover) - **Archived release**: [Zenodo 10.5281/zenodo.18900289](https://zenodo.org/records/18900289) - **Live dashboard**: https://hongping-zh.github.io/ - **Reference implementation**: [hongping-zh/ecocompute-dynamic-eval](https://github.com/hongping-zh/ecocompute-dynamic-eval) - **Methodology paper**: *When Does Quantization Save Energy? Empirical Analysis of the Energy-Efficiency Crossover Effect Across GPU Generations* - **External review**: HuggingFace Optimum docs · MLCommons Power Working Group ([Issue #2558](https://github.com/mlcommons/inference/issues/2558)) All measurements use NVML power sampling at 100 ms resolution; raw CSVs are published alongside the dataset for reproducibility. --- ## Install ``` openclaw skills install ecocompute ``` The skill is prompt-only and **needs nothing else installed on your side** — see *Requirements* at the top of this document. --- ## Changelog (recent) - **v3.0.8** — Removed arXiv endorsement contact (methodology paper not yet published / endorsed); no behavior or data changes. - **v3.0.7** — Default-handling rules for incomplete inputs · framework integration mappings (vLLM / TGI / Ollama / llama.cpp) · dataset version footer in every response · Requirements section. - **v3.0.6** — Anti-pattern table split into measured vs. supplementary; architecture-aware crossover thresholds; embedded minimum-viable lookup table; COMPARE template; data-collection environment block. - **v3.0.5** — Documentation refactor and cleanup; align with v3.0.5 crossover findings; advisory tone. --- ## Contact - Design partners / pilots: [REDACTED_SECRET] 🌍 *Making AI development more sustainable, one model at a time.*


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)