Skill: ivangdavila/hugging-face (version 1.0.0)

--- name: Hugging Face slug: hugging-face version: 1.0.0 homepage: https://clawic.com/skills/hugging-face description: Discover, evaluate, and run Hugging Face models, datasets, and spaces with license checks, benchmark prompts, and reproducible integration plans. changelog: Initial release with discovery, evaluation, inference, and troubleshooting workflows for Hugging Face operations. metadata: {"clawdbot":{"emoji":"HF","requires":{"bins":["curl","jq"],"env":["HF_TOKEN"],"config":["~/hugging-face/"]},"os":["linux","darwin","win32"],"configPaths":["~/hugging-face/"]}} --- ## Setup On first use, read `setup.md` for integration guidelines and local memory initialization. ## When to Use User needs to find the right Hugging Face model, dataset, or Space for a concrete task and move from browsing to reliable execution. Agent handles discovery, filtering, license checks, quick benchmarking, and integration-ready inference plans. ## Architecture Memory and reusable artifacts live in `~/hugging-face/`. See `memory-template.md` for structure and status fields. ```text ~/hugging-face/ |- memory.md # Stable context, priorities, and defaults |- shortlists.md # Candidate models and datasets by use case |- evaluations.md # Benchmark runs, winners, and caveats |- endpoints.md # Approved endpoints and auth notes `- exports/ # Saved outputs and comparison snapshots ``` ## Quick Reference Load only one focused file at a time to keep context small and decisions explicit. | Topic | File | |-------|------| | Setup process | `setup.md` | | Memory template | `memory-template.md` | | Model and dataset discovery | `discovery.md` | | Inference execution patterns | `inference.md` | | Evaluation rubric and scoring | `evaluation.md` | | Common failures and recovery | `troubleshooting.md` | ## Core Rules ### 1. Lock Objective and Constraints First Before selecting any artifact, confirm task type, latency budget, cost boundary, and deployment target. Use this minimum scope packet: - Task type: chat, generation, embedding, classification, vision, or speech - Quality priority: best quality, best speed, or balanced - Runtime constraints: CPU only, specific GPU class, or hosted endpoint - Compliance constraints: license, region, or private data limits ### 2. Separate Discovery from Execution Do not run inference on the first candidate found. First create a shortlist of at least three candidates, then execute only on finalists that pass compatibility and license checks. ### 3. Validate License and Access Before Recommendation For every candidate, verify license, gated access status, model size, and framework compatibility. If any of these are unknown, mark the candidate as provisional and avoid production recommendation. ### 4. Benchmark with a Deterministic Mini Suite Use the same prompt set and output checks across candidates so results are comparable. Minimum benchmark set: - One typical request - One edge-case request - One failure-prone request ### 5. Minimize External Data Send only what is required for the selected endpoint. Never send credentials, local paths, or unrelated private context in request payloads. ### 6. Use a Fallback Ladder If the preferred model fails, apply ordered fallback: 1. Retry same endpoint with smaller payload 2. Switch to a compatible backup model 3. Switch to local-only workflow if available ### 7. Keep Runs Reproducible Log selected model id, endpoint, key parameters, and evaluation result in local memory so future runs are consistent and auditable. ## Common Traps - Picking the highest download count as the only criterion -> often misses license, latency, or domain fit. - Ignoring gated model requirements -> integration fails at runtime due to access restrictions. - Comparing models with different prompts -> quality conclusions become unreliable. - Sending full user context to inference endpoints -> unnecessary privacy exposure. - Skipping fallback design -> workflows fail hard on transient endpoint errors. ## External Endpoints Use discovery endpoints before inference so candidate selection remains explainable and reproducible. | Endpoint | Data Sent | Purpose | |----------|-----------|---------| | `https://huggingface.co/api/models` | Search terms, filter parameters | Discover model candidates | | `https://huggingface.co/api/datasets` | Search terms, filter parameters | Discover dataset candidates | | `https://huggingface.co/api/spaces` | Search terms, filter parameters | Discover runnable Spaces | | `https://api-inference.huggingface.co/models/{model_id}` | Prompt or task input payload, selected model id, auth token | Run hosted inference | No other data is sent externally. ## Security & Privacy **Data that leaves your machine:** - Search terms and filter inputs sent to Hugging Face discovery APIs. - Inference payloads sent to Hugging Face Inference API when execution is requested. **Data that stays local:** - Preferences, shortlists, evaluation notes, and endpoint decisions in `~/hugging-face/`. **This skill does NOT:** - Exfiltrate local files by default. - Send undeclared network requests. - Store raw secrets in local notes. - Modify its own skill definition file. ## Trust By using this skill, selected request data is sent to Hugging Face services. Only install if you trust Hugging Face with the inputs you choose to process. ## Related Skills Install with `clawhub install <slug>` if user confirms: - `ai` - general AI strategy and model-selection framing - `api` - API-first integration patterns and HTTP debugging - `data-analysis` - dataset inspection and quality interpretation - `data` - structured data workflows and extraction patterns - `code` - implementation support for scripts and adapters ## Feedback - If useful: `clawhub star hugging-face` - Stay updated: `clawhub sync`

path: discovery.md
content: # Discovery Workflow - Hugging Face

## Objective

Build a candidate shortlist that is technically compatible before any inference run.

## Model Discovery

1. Define task and constraints:
- Task family
- Max latency
- Max cost
- Required license type

2. Query model catalog:
```bash
curl -s "https://huggingface.co/api/models?search=<term>&limit=20" | jq '.[].id'
```

3. For each candidate, capture:
- `model_id`
- Task compatibility
- License
- Gated or public status
- Approximate size and framework

4. Keep at least three finalists with different tradeoffs:
- High quality
- Fastest
- Lowest cost

## Dataset Discovery

Use dataset catalog when the task requires domain data or benchmark sources:

```bash
curl -s "https://huggingface.co/api/datasets?search=<term>&limit=20" | jq '.[].id'
```

Record:
- Dataset license
- Update cadence
- Split availability
- Domain relevance

## Space Discovery

Use Spaces to validate runnable demos or workflows quickly:

```bash
curl -s "https://huggingface.co/api/spaces?search=<term>&limit=20" | jq '.[].id'
```

Record:
- Public availability
- Hardware assumptions
- Input and output format

## Exit Criteria

Do not move to inference until each finalist has:
- Known license
- Known access status
- Clear runtime compatibility
- Stated reason for inclusion


path: evaluation.md
content: # Evaluation Rubric - Hugging Face

## Goal

Select the best candidate for the user objective using a comparable and repeatable scoring method.

## Benchmark Set

Use the same mini-suite for every candidate:
- Typical case: primary user request
- Edge case: unusual formatting or long input
- Failure-prone case: ambiguity or low-context prompt

Keep prompts fixed during one evaluation round.

## Scoring Grid

| Dimension | Weight | What to check |
|-----------|--------|---------------|
| Output quality | 40% | Correctness, completeness, clarity |
| Reliability | 20% | Stable behavior across reruns |
| Latency | 20% | Time to first useful result |
| Cost efficiency | 10% | Expected compute or endpoint cost |
| Integration fit | 10% | Schema and tooling compatibility |

Use 1-10 scoring per dimension and compute weighted total.

## Decision Rules

- Recommend only candidates with no unresolved license or access risks.
- If top scores are within 0.5 points, prefer lower operational risk.
- If no candidate reaches acceptable quality, return "no-go" and propose alternatives.

## Output Template

Record each run as:

```text
date:
objective:
candidate:
benchmark_set:
scores:
decision:
notes:
```

## Anti-Patterns

- Changing prompts between candidates.
- Mixing hosted and local runs without labeling.
- Ignoring latency because one output looked slightly better.
- Recommending a gated model without access confirmation.


path: inference.md
content: # Inference Playbook - Hugging Face

## Authentication

Set token once per shell session:

```bash
export HF_TOKEN="<token>"
```

Never print full tokens in logs or transcripts.

## Generic Inference Request

```bash
MODEL_ID="google/flan-t5-large"

curl -s "https://api-inference.huggingface.co/models/${MODEL_ID}" \
  -H "Authorization: Bearer ${HF_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": "Summarize this article in five bullets.",
    "parameters": {
      "max_new_tokens": 220,
      "temperature": 0.2
    }
  }' | jq
```

## Parameter Defaults

Use conservative defaults for predictable outputs:
- `temperature`: `0.1` to `0.3`
- `max_new_tokens`: task-specific but bounded
- Deterministic prompt framing for benchmarks

## Response Handling

Always normalize outputs before comparison:
- Strip boilerplate prefixes
- Extract core answer payload
- Preserve raw response in `evaluations.md` summary

## Fallback Ladder

If request fails:
1. Retry once with shorter input
2. Retry with lower `max_new_tokens`
3. Switch to backup model with same task family
4. Return a local-only alternative when external inference remains unavailable

## Guardrails

- Do not send unrelated user context.
- Do not upload local files unless explicitly requested.
- Do not assume all models support the same payload schema.


path: memory-template.md
content: # Memory Template - Hugging Face

Create `~/hugging-face/memory.md` with this structure:

```markdown
# Hugging Face Memory

## Status
status: ongoing
version: 1.0.0
last: YYYY-MM-DD
integration: pending | complete | paused | never_ask

## Environment Context
- Project:
- Preferred runtime:
- Latency budget:
- Cost ceiling:
- License posture:

## Active Objectives
- Objective | Success metric | Deadline

## Preferred Sources
- Models:
- Datasets:
- Spaces:

## Model Shortlist
- model_id | task | license | hosting option | current rank

## Evaluation Log
- date | model_id | benchmark set | score | decision

## Endpoint Notes
- endpoint | auth mode | limits | last check

## Notes
- Short observations safe to persist.

---
*Updated: YYYY-MM-DD*
```

## Status Values

| Value | Meaning | Behavior |
|-------|---------|----------|
| `ongoing` | context still evolving | keep refining defaults from real tasks |
| `complete` | stable context available | run faster with stored defaults |
| `paused` | memory updates paused | read-only memory behavior |
| `never_ask` | no setup prompts | avoid setup follow-up prompts |

## Key Principles

- Store durable decisions, not full conversation transcripts.
- Record license and access constraints next to each shortlisted artifact.
- Keep benchmark logs minimal but reproducible.
- Never store tokens, secrets, or private keys in memory files.


path: setup.md
content: # Setup - Hugging Face

Read this on first activation when `~/hugging-face/` does not exist or is incomplete.

## Operating Attitude

- Answer the immediate request first.
- Keep setup short and non-blocking.
- Prefer practical defaults that can be refined through real tasks.

## First Activation

1. Propose local structure and ask for explicit approval before writing files:
```bash
mkdir -p ~/hugging-face/exports
touch ~/hugging-face/{memory.md,shortlists.md,evaluations.md,endpoints.md}
chmod 700 ~/hugging-face
chmod 600 ~/hugging-face/{memory.md,shortlists.md,evaluations.md,endpoints.md}
```
2. If approved and `memory.md` is empty, initialize it using `memory-template.md`.
3. Continue with the user task immediately after setup.

## Integration Priority

Within the first natural exchanges, clarify activation preference:
- Always for requests about Hugging Face models, datasets, Spaces, or inference endpoints
- Only when explicitly requested
- Restricted to one project or repository

Store this preference as plain language context in local memory.

## Baseline Context to Capture

Capture only context that improves future recommendations:
- Preferred task families (chat, embedding, vision, speech, multimodal)
- Runtime constraints (local CPU, local GPU, hosted endpoint)
- License posture (open-only, permissive-only, or case-by-case)
- Output priorities (quality, speed, cost, reproducibility)

If context is missing, proceed with explicit assumptions and keep moving.

## Runtime Defaults

- Start with discovery-first flow: shortlist before inference.
- Default to at least three candidates for comparison.
- Require license and access validation before final recommendation.

## Optional Depth

If the user wants deeper rigor, load:
- `discovery.md` for search and filtering strategy
- `evaluation.md` for comparable scoring rubric
- `inference.md` for endpoint execution patterns
- `troubleshooting.md` for failure recovery


path: troubleshooting.md
content: # Troubleshooting - Hugging Face

## 401 Unauthorized

Likely causes:
- Missing `HF_TOKEN`
- Expired or revoked token
- Typo in auth header

Actions:
1. Verify token is set in current shell
2. Regenerate token if needed
3. Re-run with minimal payload first

## 403 Forbidden or Gated Access

Likely causes:
- Model requires access approval
- Account lacks permission scope

Actions:
1. Check model card access requirements
2. Request access or switch to open alternative
3. Document gating status in shortlist

## 404 Model Not Found

Likely causes:
- Wrong model id
- Model moved or deleted
- Typo in endpoint path

Actions:
1. Re-validate model id via model API search
2. Update references in shortlist
3. Use fallback model immediately if production request is blocked

## 429 Rate Limited

Likely causes:
- Burst traffic or quota limits

Actions:
1. Back off with incremental delays
2. Reduce request frequency and payload size
3. Queue non-urgent evaluations instead of retrying aggressively

## 503 or Cold Start Timeout

Likely causes:
- Endpoint cold start
- Temporary infra saturation

Actions:
1. Retry with shorter input
2. Switch to backup model
3. Return partial result with clear status if blocking persists

## Bad Output Quality

Likely causes:
- Prompt mismatch for the model family
- Wrong task family selection
- Unstable parameters

Actions:
1. Lower temperature
2. Tighten prompt instructions
3. Re-run benchmark set and compare against shortlist alternatives


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)