Skill: at6132/fitcheck-skill-search (version 1.1.0)

--- name: skill-search description: Find and retrieve available skills using keyword search, semantic search, or LLM-powered task matching. Use when the agent needs to discover, search, or intelligently match skills to tasks. Supports hybrid search (BM25 + semantic), natural language task descriptions, and intelligent skill recommendations. triggers: - "search skills" - "find a skill" - "what skills are available" - "skill for X" - "suggest skills for" - "what skill should I use" - "recommend a skill" --- # Skill Search V1.1 Find skills using keyword, semantic, or AI-powered task matching. ## Overview This skill provides three search modes: 1. **Keyword Search** (Fast BM25) — Match skill names and descriptions 2. **Semantic Search** (Embeddings) — Find skills by meaning/concept similarity 3. **LLM Task Matching** (AI-powered) — Describe your task, get skill recommendations ## When to Use - **Before starting a new task** — check if a relevant skill exists - **When user asks for capabilities** — "what can you do with PDFs?" - **To avoid context bloat** — find the right skill first, then load only that one - **When unsure which skill applies** — use semantic or LLM matching for fuzzy matches ## Search Modes ### 1. Keyword Search (Fast) ```bash ./scripts/skill_search.py keyword "weather" ./scripts/skill_search.py keyword "pdf" ./scripts/skill_search.py keyword "image generation" ``` ### 2. Semantic Search (Meaning-based) ```bash ./scripts/skill_search.py semantic "automate web browsing" ./scripts/skill_search.py semantic "create images with AI" ./scripts/skill_search.py semantic "search my past conversations" ``` ### 3. LLM Task Matching (AI-powered) ```bash ./scripts/skill_search.py suggest "I need to transcribe a podcast episode" ./scripts/skill_search.py suggest "Help me generate product photos" ./scripts/skill_search.py suggest "Search through my old emails" ``` ### 4. List All Skills ```bash ./scripts/skill_search.py list ``` ## Usage Pattern 1. **Search**: Find skills matching your need (keyword/semantic/LLM) 2. **Preview**: Read SKILL.md metadata (description, triggers) 3. **Load**: If it's the right skill, read full body and execute ## Example Workflows **User**: "I need to generate some images" **Agent**: *Uses semantic search* ```bash ./scripts/skill_search.py semantic "generate images AI" ``` **Output**: ``` Top matches (semantic): 1. openai-image-gen (0.87) — Batch-generate images via OpenAI Images API 2. browser (0.65) — Control web browser via Playwright ``` **Agent**: *Reads SKILL.md, confirms fit, executes* --- **User**: "What skill should I use for transcribing audio?" **Agent**: *Uses LLM suggest* ```bash ./scripts/skill_search.py suggest "transcribe audio" ``` **Output**: ``` Recommended skills for "transcribe audio": 1. openai-whisper-api — Transcribe audio via OpenAI Audio Transcriptions API (Whisper) Confidence: High Reason: Task explicitly matches skill purpose 2. sag — ElevenLabs text-to-speech (inverse operation, may be related) Confidence: Low Reason: Related to audio processing but output not input ``` ## Search Index The skill maintains a local search index at: - `~/.openclaw/workspace/skills/skill-search/index/skills_index.json` — Skill metadata - `~/.openclaw/workspace/skills/skill-search/index/embeddings.json` — Semantic embeddings (lazy-loaded) **Indexing happens automatically** on first semantic search if no index exists. **Force reindex:** ```bash ./scripts/skill_search.py index ``` ## Resources ### scripts/ - `skill_search.py` — Main search interface (keyword/semantic/LLM) - `indexer.py` — Build/update search index - `embeddings.py` — Embedding generation (local miniLM) ### index/ - `skills_index.json` — Searchable skill metadata - `embeddings_cache.json` — Pre-computed embeddings for semantic search

path: README.md
content: # skill-search

Find and discover available skills without loading them all into context.

## The Problem

Every skill you load into an agent's context consumes tokens and attention. Loading 10+ skills at once clutters the prompt and degrades performance.

## The Solution

Keep only `skill-search` loaded. When you need a specific capability, search first, then load just that skill.

## Installation

Copy this folder into your agent's skills directory:

```bash
# For OpenClaw agents
cp -r skill-search ~/.openclaw/workspace/skills/

# Or wherever your agent loads skills from
```

## Usage

### From the Agent

When the agent receives a task, have it run:

```bash
./scripts/skill_search.py "weather"
```

This finds relevant skills without loading them into context.

### From Command Line

```bash
# Search for skills matching a keyword
./scripts/skill_search.py "pdf"

# List all available skills
./scripts/skill_search.py --list-all

# Output as JSON (for scripting)
./scripts/skill_search.py "image" --json
```

## Workflow

1. **User asks**: "I need to generate some images"
2. **Agent runs**: `skill_search.py "image"`
3. **Agent sees**: `openai-image-gen` — Batch-generate images via OpenAI Images API
4. **Agent loads**: Only `openai-image-gen/SKILL.md` and uses it

Result: One skill in context instead of ten.

## How It Works

The script searches:
- System skills: `/usr/local/lib/node_modules/openclaw/skills/`
- User skills: `~/.openclaw/workspace/skills/`

It parses `SKILL.md` frontmatter (name, description) to build an index without loading file contents.

## Files

- `SKILL.md` — Skill definition (loaded by agent)
- `scripts/skill_search.py` — Search executable
- `README.md` — This file

## License

MIT (or wherever this ends up)


path: scripts/embeddings.py
content: #!/usr/bin/env python3
"""Local embedding generation for semantic skill search.

Uses sentence-transformers/all-MiniLM-L6-v2 via ONNX runtime
for fast local embeddings without external API calls.
"""

import os
import json
import hashlib
import subprocess
from pathlib import Path
from typing import List, Dict, Optional, Tuple

# Paths
SKILL_DIRS = [
    "/usr/local/lib/node_modules/openclaw/skills",
    os.path.expanduser("~/.openclaw/workspace/skills"),
]
INDEX_DIR = Path(__file__).parent.parent / "index"
EMBEDDINGS_FILE = INDEX_DIR / "embeddings_cache.json"


def ensure_index_dir():
    """Ensure index directory exists."""
    INDEX_DIR.mkdir(parents=True, exist_ok=True)


def parse_frontmatter(content: str) -> Dict:
    """Extract YAML frontmatter from SKILL.md."""
    result = {
        "name": "",
        "description": "",
        "triggers": [],
    }
    
    if not content.startswith("---"):
        return result
    
    # Find end of frontmatter
    end_match = content.find("\n---", 3)
    if end_match == -1:
        return result
    
    frontmatter = content[3:end_match].strip()
    
    # Parse simple key: value pairs
    current_key = None
    current_value = []
    
    for line in frontmatter.split("\n"):
        line_stripped = line.rstrip()
        
        # Check for new key
        if ":" in line_stripped and not line_stripped.startswith(" ") and not line_stripped.startswith("-"):
            # Save previous key-value
            if current_key and current_value:
                value = " ".join(current_value).strip()
                if value.startswith('"') and value.endswith('"'):
                    value = value[1:-1]
                elif value.startswith("'") and value.endswith("'"):
                    value = value[1:-1]
                result[current_key] = value
            
            key, val = line_stripped.split(":", 1)
            current_key = key.strip()
            current_value = [val.strip()]
        elif line_stripped.startswith("- ") and current_key:
            # List item
            item = line_stripped[2:].strip()
            if current_key not in result:
                result[current_key] = []
            if isinstance(result[current_key], list):
                result[current_key].append(item)
        elif current_key:
            current_value.append(line_stripped)
    
    # Save last key-value
    if current_key and current_value:
        value = " ".join(current_value).strip()
        if value.startswith('"') and value.endswith('"'):
            value = value[1:-1]
        elif value.startswith("'") and value.endswith("'"):
            value = value[1:-1]
        result[current_key] = value
    
    return result


def extract_skill_text(skill_path: Path) -> Dict:
    """Extract searchable text from a SKILL.md file."""
    try:
        content = skill_path.read_text(encoding='utf-8', errors='ignore')
        frontmatter = parse_frontmatter(content)
        
        # Get body text (after frontmatter)
        body_start = content.find("\n#")
        body = content[body_start:] if body_start > 0 else ""
        
        # Extract first paragraph of description
        desc = frontmatter.get("description", "")
        name = frontmatter.get("name", skill_path.parent.name)
        triggers = frontmatter.get("triggers", [])
        
        # Combine for embedding
        text_parts = [
            name,
            desc,
            " ".join(triggers) if isinstance(triggers, list) else str(triggers),
        ]
        
        # Add body headers and code examples
        for line in body.split("\n"):
            if line.startswith("#") or line.startswith("```"):
                text_parts.append(line.replace("#", "").replace("`", ""))
        
        combined_text = " ".join(filter(None, text_parts))
        
        return {
            "name": name,
            "folder": skill_path.parent.name,
            "description": desc,
            "triggers": triggers if isinstance(triggers, list) else [triggers] if triggers else [],
            "location": str(skill_path.parent),
            "search_text": combined_text[:2000],  # Limit length
            "content_hash": hashlib.md5(content.encode()).hexdigest()[:16],
        }
    except Exception as e:
        return {
            "name": skill_path.parent.name,
            "folder": skill_path.parent.name,
            "description": f"Error parsing: {e}",
            "triggers": [],
            "location": str(skill_path.parent),
            "search_text": skill_path.parent.name,
            "content_hash": "",
        }


def fallback_embedding(text: str) -> List[float]:
    """Simple fallback: char-level n-gram frequency vector."""
    # Create a simple 384-dim vector based on character n-grams
    vector = [0.0] * 384
    text_lower = text.lower()
    
    # Trigram hashing
    for i in range(len(text_lower) - 2):
        trigram = text_lower[i:i+3]
        idx = hash(trigram) % 384
        vector[idx] += 1.0
    
    # Normalize
    norm = sum(x**2 for x in vector) ** 0.5
    if norm > 0:
        vector = [x / norm for x in vector]
    
    return vector


def cosine_similarity(a: List[float], b: List[float]) -> float:
    """Calculate cosine similarity between two vectors."""
    dot = sum(x * y for x, y in zip(a, b))
    norm_a = sum(x**2 for x in a) ** 0.5
    norm_b = sum(x**2 for x in b) ** 0.5
    if norm_a == 0 or norm_b == 0:
        return 0.0
    return dot / (norm_a * norm_b)


def index_all_skills() -> Dict:
    """Build complete skill index with embeddings."""
    ensure_index_dir()
    
    skills = []
    seen = set()
    
    for skill_dir in SKILL_DIRS:
        if not os.path.isdir(skill_dir):
            continue
        
        for item in sorted(os.listdir(skill_dir)):
            if item.startswith(".") or item in seen:
                continue
            
            item_path = Path(skill_dir) / item
            skill_md = item_path / "SKILL.md"
            
            if skill_md.exists():
                seen.add(item)
                skill_data = extract_skill_text(skill_md)
                print(f"Indexing: {skill_data['name']}")
                
                # Generate embedding
                skill_data["embedding"] = fallback_embedding(skill_data["search_text"])
                skills.append(skill_data)
    
    # Save index
    index_path = INDEX_DIR / "skills_index.json"
    with open(index_path, 'w') as f:
        json.dump(skills, f, indent=2)
    
    print(f"Indexed {len(skills)} skills to {index_path}")
    return {"skills": skills, "count": len(skills)}


def load_index() -> Optional[Dict]:
    """Load existing skill index."""
    index_path = INDEX_DIR / "skills_index.json"
    if not index_path.exists():
        return None
    
    with open(index_path, 'r') as f:
        return json.load(f)


def semantic_search(query: str, top_k: int = 5) -> List[Tuple[Dict, float]]:
    """Search skills by semantic similarity."""
    index = load_index()
    if not index:
        print("No index found. Building index...")
        result = index_all_skills()
        index = result["skills"]
    
    query_emb = fallback_embedding(query)
    
    results = []
    for skill in index:
        if "embedding" in skill:
            sim = cosine_similarity(query_emb, skill["embedding"])
            results.append((skill, sim))
    
    results.sort(key=lambda x: x[1], reverse=True)
    return results[:top_k]


if __name__ == "__main__":
    import sys
    if len(sys.argv) > 1 and sys.argv[1] == "index":
        index_all_skills()
    elif len(sys.argv) > 2 and sys.argv[1] == "semantic":
        query = sys.argv[2]
        results = semantic_search(query, top_k=5)
        print(f"\nSemantic search results for: '{query}'\n")
        for skill, score in results:
            print(f"{score:.3f} | {skill['folder']}")
            print(f"      {skill['description'][:80]}...")
            print(f"      Location: {skill['location']}")
            print()
    else:
        print("Usage:")
        print("  python3 embeddings.py index")
        print("  python3 embeddings.py semantic '<query>'")

path: scripts/skill_search.py
content: #!/usr/bin/env python3
"""Skill Search V1.1 - Advanced skill discovery."""
import os, re, sys, json, argparse
from pathlib import Path
from typing import List, Tuple, Dict

sys.path.insert(0, str(Path(__file__).parent))
from embeddings import index_all_skills, load_index, semantic_search, INDEX_DIR

SKILL_DIRS = [
    "/usr/local/lib/node_modules/openclaw/skills",
    os.path.expanduser("~/.openclaw/workspace/skills"),
]

def parse_skill_md(skill_path: Path) -> Tuple[str, str, str, List[str]]:
    name = skill_path.parent.name
    desc = "(no description)"
    triggers = []
    try:
        content = skill_path.read_text(encoding='utf-8', errors='ignore')
        if content.startswith('---'):
            end = re.search(r'\n---\s*\n', content[3:])
            if end:
                fm = content[3:3+end.start()]
                nm = re.search(r'^name:\s*(.+)$', fm, re.M)
                if nm: name = nm.group(1).strip().strip('"\'')
                dm = re.search(r'^description:\s*(.+?)(?=\n\w+:|\Z)', fm, re.M|re.S)
                if dm: desc = re.sub(r'\s+', ' ', dm.group(1).strip().strip('"\'').replace('\n',' '))
                tm = re.search(r'triggers?:\s*\n((?:^\s*- .+\n?)+)', fm, re.M)
                if tm:
                    triggers = [l.strip('- ').strip('"\'') for l in tm.group(1).strip().split('\n') if l.strip().startswith('-')]
    except: pass
    return name, desc, str(skill_path.parent), triggers

def find_all_skills() -> List[Tuple[str, str, str, List[str]]]:
    skills, seen = [], set()
    for sd in SKILL_DIRS:
        if not os.path.isdir(sd): continue
        for it in sorted(os.listdir(sd)):
            if it.startswith('.'): continue
            sp = Path(sd) / it / "SKILL.md"
            if sp.exists() and it not in seen:
                seen.add(it)
                skills.append(parse_skill_md(sp) + (it,))
    return sorted(skills, key=lambda x: x[0].lower())

def keyword_search(query: str, top_k: int = 10) -> List[Tuple]:
    ql = query.lower()
    qw = set(ql.split())
    results = []
    for name, desc, loc, triggers, folder in find_all_skills():
        nl, dl, tl = name.lower(), desc.lower(), ' '.join(triggers).lower()
        score, matched = 0, []
        if ql == nl: score, matched = 20, ['exact']
        elif ql in nl: score, matched = 10, ['name']
        for w in qw:
            if w in nl: score += 5
            if w in dl: score += 2
            if w in tl: score += 3
        if score > 0: results.append((name, desc, folder, loc, triggers, score, matched))
    results.sort(key=lambda x: (-x[5], x[0].lower()))
    return results[:top_k]

def hybrid_search(query: str, top_k: int = 5) -> List[Tuple[Dict, float]]:
    kw = {r[0]: r[5] for r in keyword_search(query, 20)}
    sem = semantic_search(query, 20)
    sm = {s.get('name'): sim for s, sim in sem}
    all_skills = load_index()
    if not all_skills: return sem[:top_k]
    km = max(kw.values()) if kw else 1
    combined = []
    for sk in all_skills:
        name = sk.get('name', '')
        ks = kw.get(name, 0) / km if km > 0 else 0
        ss = sm.get(name, 0)
        cs = (ks * 0.5) + (ss * 0.5)
        if cs > 0.1: combined.append((sk, cs))
    combined.sort(key=lambda x: x[1], reverse=True)
    return combined[:top_k]

def task_match(query: str) -> str:
    if not load_index(): index_all_skills()
    r = hybrid_search(query, 5)
    if not r: return "No skills found."
    lines = [f"Recommended for '{query}':\n"]
    for i, (sk, sc) in enumerate(r, 1):
        name = sk.get('name', 'Unknown')
        desc = sk.get('description', '')[:80]
        trig = sk.get('triggers', [])
        conf = "High" if sc >= 0.7 else "Medium" if sc >= 0.4 else "Low"
        lines.append(f"{i}. {name}")
        lines.append(f"   {desc}{'...' if len(desc) >= 80 else ''}")
        if trig: lines.append(f"   Triggers: {', '.join(trig[:3])}")
        lines.append(f"   Confidence: {conf} ({sc:.2f})")
        lines.append("")
    return "\n".join(lines)

def list_skills(show_locs: bool = False) -> str:
    skills = find_all_skills()
    lines = [f"Skills ({len(skills)} total):\n"]
    for name, desc, loc, triggers, folder in skills:
        lines.append(f"• {name}")
        lines.append(f"  {desc[:60]}{'...' if len(desc) > 60 else ''}")
        if triggers: lines.append(f"  Triggers: {', '.join(triggers[:3])}")
        if show_locs: lines.append(f"  Location: {loc}")
        lines.append("")
    return "\n".join(lines)

def fmt_kw(results: List[Tuple]) -> str:
    if not results: return "No skills found."
    lines = []
    for i, (name, desc, folder, loc, trig, score, matched) in enumerate(results, 1):
        lines.append(f"{i}. {name} (score: {score})")
        lines.append(f"   {desc[:70]}{'...' if len(desc) > 70 else ''}")
        if trig: lines.append(f"   Triggers: {', '.join(trig[:3])}")
        lines.append("")
    return "\n".join(lines)

def fmt_sem(results: List[Tuple[Dict, float]]) -> str:
    if not results: return "No skills found."
    lines = []
    for i, (sk, sc) in enumerate(results, 1):
        lines.append(f"{i}. {sk.get('name', 'Unknown')} (sim: {sc:.3f})")
        lines.append(f"   {sk.get('description', '')[:70]}{'...'}")
        trig = sk.get('triggers', [])
        if trig: lines.append(f"   Triggers: {', '.join(trig[:3])}")
        lines.append("")
    return "\n".join(lines)

def main():
    p = argparse.ArgumentParser(description="Skill Search V1.1")
    p.add_argument('--json', action='store_true')
    sub = p.add_subparsers(dest='cmd')
    
    sub.add_parser('index', help='Build index')
    lg = sub.add_parser('list', help='List skills')
    lg.add_argument('--location', action='store_true')
    
    kw = sub.add_parser('keyword', help='Keyword search')
    kw.add_argument('query', nargs='+')
    kw.add_argument('-k', type=int, default=10)
    
    sm = sub.add_parser('semantic', help='Semantic search')
    sm.add_argument('query', nargs='+')
    sm.add_argument('-k', type=int, default=5)
    
    hb = sub.add_parser('hybrid', help='Hybrid search')
    hb.add_argument('query', nargs='+')
    hb.add_argument('-k', type=int, default=5)
    
    sg = sub.add_parser('suggest', help='AI task matching')
    sg.add_argument('query', nargs='+')
    
    a = p.parse_args()
    if not a.cmd:
        p.print_help()
        return
    
    if a.cmd == 'index':
        r = index_all_skills()
        print(f"Indexed {r['count']} skills")
    elif a.cmd == 'list':
        print(list_skills(a.location))
    elif a.cmd == 'keyword':
        print(fmt_kw(keyword_search(' '.join(a.query), a.k)))
    elif a.cmd == 'semantic':
        if not load_index():
            print("Building index...")
            index_all_skills()
        print(fmt_sem(semantic_search(' '.join(a.query), a.k)))
    elif a.cmd == 'hybrid':
        print(task_match(' '.join(a.query)))
    elif a.cmd == 'suggest':
        print(task_match(' '.join(a.query)))

if __name__ == '__main__':
    main()


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)