Skill: toller892/memory-semantic-search (version 1.0.0)

--- name: memory-semantic-search description: "Semantic search over workspace markdown files using embedding API + SQLite vector store. Use when: (1) searching workspace notes/memory by meaning rather than exact keywords, (2) finding related markdown content across files, (3) recalling past decisions, context, or notes semantically. Requires an OpenAI-compatible embedding API. NOT for: searching non-markdown files, web search, or code search." --- # Memory Semantic Search Standalone semantic search over workspace `.md` files. Uses an OpenAI-compatible embedding API and SQLite for vector storage. No external dependencies beyond Python 3 stdlib + the embedding API. ## Setup Set these environment variables (or pass as CLI args): ```bash export EMBEDDING_API_KEY="sk-xxx" export EMBEDDING_API_BASE="https://api.openai.com/v1" # any OpenAI-compatible endpoint export EMBEDDING_MODEL="text-embedding-3-small" # optional, this is the default ``` ## Usage ### Index workspace ```bash python3 scripts/index.py /path/to/workspace ``` Options: - `--force` — full reindex (clear existing data) - `--db PATH` — custom SQLite path (default: `memory_search.sqlite` in skill dir) - `--api-base`, `--api-key`, `--model` — override env vars Incremental: only new/changed chunks are embedded. Deleted files are cleaned up automatically. ### Search ```bash python3 scripts/search.py "your query here" ``` Options: - `--top-k N` — number of results (default: 5) - `--min-score F` — minimum cosine similarity threshold (default: 0.3) - `--json` — output as JSON - `--db`, `--api-base`, `--api-key`, `--model` — same as index ### Typical agent workflow 1. Run `index.py` on the workspace (once, or after file changes) 2. Run `search.py "query"` to find relevant snippets 3. Use `read` tool to load full context from the returned file paths and line numbers

path: README.md
content: # memory-semantic-search

对 workspace 中的 Markdown 文件进行语义搜索的独立 Skill。基于 OpenAI 兼容的 Embedding API + SQLite 向量存储，纯 Python stdlib 实现，无额外依赖。

## 工作原理

```
.md 文件 → 切 chunk → Embedding API → 向量存入 SQLite
                                            ↓
用户查询 → Embedding API → 查询向量 → 余弦相似度 → Top-K 结果
```

## 快速开始

### 1. 配置环境变量

```bash
export EMBEDDING_API_KEY="your-api-key"
export EMBEDDING_API_BASE="https://api.openai.com/v1"   # 任何 OpenAI 兼容端点
export EMBEDDING_MODEL="text-embedding-3-small"          # 可选，这是默认值
```

### 2. 建索引

```bash
python3 scripts/index.py /path/to/workspace
```

### 3. 搜索

```bash
python3 scripts/search.py "你的查询"
```

## 命令详情

### index.py — 索引

扫描目录下所有 `.md` 文件，按 ~400 token 切 chunk，调用 Embedding API 获取向量，存入 SQLite。

```bash
python3 scripts/index.py /path/to/workspace [选项]
```

| 选项 | 说明 |
|------|------|
| `--db PATH` | SQLite 数据库路径（默认：skill 目录下 `memory_search.sqlite`） |
| `--api-base URL` | Embedding API 地址 |
| `--api-key KEY` | API Key |
| `--model NAME` | Embedding 模型名 |
| `--force` | 清空已有索引，全量重建 |

支持增量更新：只 embed 新增/修改的 chunk，自动清理已删除文件的索引。

### search.py — 搜索

将查询文本转为向量，与 SQLite 中已存的 chunk 向量计算余弦相似度，返回最相关的结果。

```bash
python3 scripts/search.py "查询内容" [选项]
```

| 选项 | 说明 |
|------|------|
| `--db PATH` | SQLite 数据库路径 |
| `--api-base URL` | Embedding API 地址 |
| `--api-key KEY` | API Key |
| `--model NAME` | Embedding 模型名 |
| `--top-k N` | 返回结果数（默认：5） |
| `--min-score F` | 最低相似度阈值（默认：0.3） |
| `--json` | 以 JSON 格式输出 |

## 环境变量

所有参数均可通过环境变量设置，命令行参数优先级更高：

| 环境变量 | 说明 |
|----------|------|
| `EMBEDDING_API_KEY` | API Key（必填） |
| `EMBEDDING_API_BASE` | API 地址（默认：`https://api.openai.com/v1`） |
| `EMBEDDING_MODEL` | 模型名（默认：`text-embedding-3-small`） |
| `MSS_DB` | SQLite 数据库路径 |
| `MSS_TOP_K` | 搜索返回数量 |
| `MSS_MIN_SCORE` | 最低相似度阈值 |

## 技术细节

- chunk 策略：按行切分，目标 ~400 token/chunk，80 token 重叠
- 向量存储：SQLite + 二进制 blob（`struct.pack` 打包 float 数组）
- 相似度计算：余弦相似度（cosine similarity）
- 增量索引：基于内容 MD5 哈希判断 chunk 是否变更
- 批量 embedding：每批 10 条，避免 API 超时

## 兼容的 Embedding API

任何兼容 OpenAI `/v1/embeddings` 接口的服务均可使用，包括但不限于：

- OpenAI（text-embedding-3-small / text-embedding-3-large）
- 讯飞星火（xop3qwen8bembedding）
- Gemini Embedding
- Ollama
- 各类 OpenAI 兼容代理

## License

MIT


path: scripts/index.py
content: #!/usr/bin/env python3
"""Index markdown files in workspace into SQLite with embeddings."""

import argparse
import hashlib
import json
import math
import os
import re
import sqlite3
import struct
import sys
import urllib.request
import urllib.error

# --- Config ---

DEFAULT_MODEL = "text-embedding-3-small"
DEFAULT_BASE_URL = "https://api.openai.com/v1"
DEFAULT_DB = os.path.join(os.path.dirname(__file__), "..", "memory_search.sqlite")
CHUNK_TARGET = 400  # target tokens (~chars * 0.75)
CHUNK_OVERLAP = 80
CHAR_PER_TOKEN = 4  # rough estimate

CHUNK_TARGET_CHARS = CHUNK_TARGET * CHAR_PER_TOKEN
CHUNK_OVERLAP_CHARS = CHUNK_OVERLAP * CHAR_PER_TOKEN


def init_db(db_path):
    conn = sqlite3.connect(db_path)
    conn.execute("PRAGMA journal_mode=WAL")
    conn.execute("""
        CREATE TABLE IF NOT EXISTS chunks (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            file_path TEXT NOT NULL,
            line_start INTEGER NOT NULL,
            line_end INTEGER NOT NULL,
            content TEXT NOT NULL,
            content_hash TEXT NOT NULL,
            embedding BLOB,
            UNIQUE(file_path, line_start, content_hash)
        )
    """)
    conn.execute("""
        CREATE TABLE IF NOT EXISTS meta (
            key TEXT PRIMARY KEY,
            value TEXT
        )
    """)
    conn.commit()
    return conn


def chunk_markdown(text, file_path):
    """Split markdown into overlapping chunks, tracking line numbers."""
    lines = text.split("\n")
    chunks = []
    i = 0
    while i < len(lines):
        chunk_lines = []
        char_count = 0
        start = i
        while i < len(lines) and char_count < CHUNK_TARGET_CHARS:
            chunk_lines.append(lines[i])
            char_count += len(lines[i]) + 1
            i += 1
        content = "\n".join(chunk_lines).strip()
        if content:
            chunks.append({
                "file_path": file_path,
                "line_start": start + 1,
                "line_end": start + len(chunk_lines),
                "content": content,
                "content_hash": hashlib.md5(content.encode()).hexdigest(),
            })
        # overlap: rewind
        overlap_chars = 0
        rewind = 0
        for j in range(len(chunk_lines) - 1, -1, -1):
            overlap_chars += len(chunk_lines[j]) + 1
            rewind += 1
            if overlap_chars >= CHUNK_OVERLAP_CHARS:
                break
        if rewind > 0 and i < len(lines):
            i -= rewind
    return chunks


def get_embeddings(texts, api_base, api_key, model, batch_size=10):
    """Call OpenAI-compatible embedding API in batches."""
    url = f"{api_base.rstrip('/')}/embeddings"
    headers = {
        "Content-Type": "application/json",
        "Authorization": f"Bearer {api_key}",
    }
    all_embeddings = []
    for start in range(0, len(texts), batch_size):
        batch = texts[start:start + batch_size]
        payload = json.dumps({"input": batch, "model": model}).encode()
        req = urllib.request.Request(url, data=payload, headers=headers, method="POST")
        try:
            with urllib.request.urlopen(req, timeout=120) as resp:
                data = json.loads(resp.read())
        except urllib.error.HTTPError as e:
            body = e.read().decode(errors="replace")
            print(f"Embedding API error {e.code}: {body}", file=sys.stderr)
            sys.exit(1)
        except Exception as e:
            print(f"Embedding request failed at batch {start}: {e}", file=sys.stderr)
            sys.exit(1)
        # sort by index to ensure order
        sorted_embs = sorted(data["data"], key=lambda x: x["index"])
        all_embeddings.extend([e["embedding"] for e in sorted_embs])
        done = start + len(batch)
        print(f"  Embedded {done}/{len(texts)} chunks", flush=True)
    return all_embeddings


def pack_vector(vec):
    """Pack float list to binary blob."""
    return struct.pack(f"{len(vec)}f", *vec)


def scan_markdown_files(workspace):
    """Find all .md files recursively."""
    md_files = []
    for root, dirs, files in os.walk(workspace):
        # skip hidden dirs and node_modules
        dirs[:] = [d for d in dirs if not d.startswith(".") and d != "node_modules"]
        for f in files:
            if f.endswith(".md"):
                md_files.append(os.path.join(root, f))
    return sorted(md_files)


def main():
    parser = argparse.ArgumentParser(description="Index markdown files for semantic search")
    parser.add_argument("workspace", nargs="?", default=os.environ.get("WORKSPACE", "."),
                        help="Workspace directory to scan (default: cwd or $WORKSPACE)")
    parser.add_argument("--db", default=os.environ.get("MSS_DB", DEFAULT_DB),
                        help="SQLite database path")
    parser.add_argument("--api-base", default=os.environ.get("EMBEDDING_API_BASE", DEFAULT_BASE_URL))
    parser.add_argument("--api-key", default=os.environ.get("EMBEDDING_API_KEY", ""))
    parser.add_argument("--model", default=os.environ.get("EMBEDDING_MODEL", DEFAULT_MODEL))
    parser.add_argument("--force", action="store_true", help="Force full reindex")
    args = parser.parse_args()

    if not args.[REDACTED_SECRET]"Error: No API key. Set EMBEDDING_API_KEY or --api-key", file=sys.stderr)
        sys.exit(1)

    workspace = os.path.abspath(args.workspace)
    db_path = os.path.abspath(args.db)
    print(f"Workspace: {workspace}")
    print(f"Database:  {db_path}")
    print(f"Provider:  {args.api_base}")
    print(f"Model:     {args.model}")

    conn = init_db(db_path)

    if args.force:
        conn.execute("DELETE FROM chunks")
        conn.commit()
        print("Cleared existing index (--force)")

    # scan files
    md_files = scan_markdown_files(workspace)
    print(f"Found {len(md_files)} markdown files")

    # chunk all files
    all_chunks = []
    for fpath in md_files:
        rel_path = os.path.relpath(fpath, workspace)
        try:
            with open(fpath, "r", encoding="utf-8", errors="replace") as f:
                text = f.read()
        except Exception as e:
            print(f"  Skip {rel_path}: {e}")
            continue
        chunks = chunk_markdown(text, rel_path)
        all_chunks.extend(chunks)

    print(f"Total chunks: {len(all_chunks)}")

    # filter out already-indexed chunks
    new_chunks = []
    for c in all_chunks:
        existing = conn.execute(
            "SELECT id FROM chunks WHERE file_path=? AND line_start=? AND content_hash=?",
            (c["file_path"], c["line_start"], c["content_hash"])
        ).fetchone()
        if not existing:
            new_chunks.append(c)

    # remove stale chunks (files deleted or content changed)
    indexed_files = set(c["file_path"] for c in all_chunks)
    db_files = set(r[0] for r in conn.execute("SELECT DISTINCT file_path FROM chunks").fetchall())
    stale_files = db_files - indexed_files
    if stale_files:
        for sf in stale_files:
            conn.execute("DELETE FROM chunks WHERE file_path=?", (sf,))
        conn.commit()
        print(f"Removed {len(stale_files)} stale files from index")

    if not new_chunks:
        print("Index is up to date. Nothing to embed.")
        conn.close()
        return

    print(f"New/changed chunks to embed: {len(new_chunks)}")

    # get embeddings
    texts = [c["content"] for c in new_chunks]
    embeddings = get_embeddings(texts, args.api_base, args.api_key, args.model)

    # insert
    for chunk, emb in zip(new_chunks, embeddings):
        conn.execute(
            "INSERT OR REPLACE INTO chunks (file_path, line_start, line_end, content, content_hash, embedding) VALUES (?, ?, ?, ?, ?, ?)",
            (chunk["file_path"], chunk["line_start"], chunk["line_end"],
             chunk["content"], chunk["content_hash"], pack_vector(emb))
        )
    conn.commit()

    total = conn.execute("SELECT COUNT(*) FROM chunks").fetchone()[0]
    print(f"Done. Total indexed chunks: {total}")
    conn.close()


if __name__ == "__main__":
    main()


path: scripts/search.py
content: #!/usr/bin/env python3
"""Semantic search over indexed markdown chunks."""

import argparse
import json
import math
import os
import sqlite3
import struct
import sys
import urllib.request
import urllib.error

DEFAULT_MODEL = "text-embedding-3-small"
DEFAULT_BASE_URL = "https://api.openai.com/v1"
DEFAULT_DB = os.path.join(os.path.dirname(__file__), "..", "memory_search.sqlite")
DEFAULT_TOP_K = 5
DEFAULT_MIN_SCORE = 0.3


def unpack_vector(blob):
    n = len(blob) // 4
    return struct.unpack(f"{n}f", blob)


def cosine_similarity(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    norm_a = math.sqrt(sum(x * x for x in a))
    norm_b = math.sqrt(sum(x * x for x in b))
    if norm_a == 0 or norm_b == 0:
        return 0.0
    return dot / (norm_a * norm_b)


def get_embedding(text, api_base, api_key, model):
    url = f"{api_base.rstrip('/')}/embeddings"
    headers = {
        "Content-Type": "application/json",
        "Authorization": f"Bearer {api_key}",
    }
    payload = json.dumps({"input": [text], "model": model}).encode()
    req = urllib.request.Request(url, data=payload, headers=headers, method="POST")
    try:
        with urllib.request.urlopen(req, timeout=30) as resp:
            data = json.loads(resp.read())
    except urllib.error.HTTPError as e:
        body = e.read().decode(errors="replace")
        print(f"Embedding API error {e.code}: {body}", file=sys.stderr)
        sys.exit(1)
    return data["data"][0]["embedding"]


def search(query_vec, conn, top_k, min_score):
    rows = conn.execute(
        "SELECT file_path, line_start, line_end, content, embedding FROM chunks WHERE embedding IS NOT NULL"
    ).fetchall()

    results = []
    for file_path, line_start, line_end, content, emb_blob in rows:
        vec = unpack_vector(emb_blob)
        score = cosine_similarity(query_vec, vec)
        if score >= min_score:
            results.append({
                "file": file_path,
                "lines": f"{line_start}-{line_end}",
                "score": round(score, 4),
                "snippet": content[:700],
            })

    results.sort(key=lambda x: x["score"], reverse=True)
    return results[:top_k]


def main():
    parser = argparse.ArgumentParser(description="Semantic search over indexed markdown")
    parser.add_argument("query", help="Search query text")
    parser.add_argument("--db", default=os.environ.get("MSS_DB", DEFAULT_DB))
    parser.add_argument("--api-base", default=os.environ.get("EMBEDDING_API_BASE", DEFAULT_BASE_URL))
    parser.add_argument("--api-key", default=os.environ.get("EMBEDDING_API_KEY", ""))
    parser.add_argument("--model", default=os.environ.get("EMBEDDING_MODEL", DEFAULT_MODEL))
    parser.add_argument("--top-k", type=int, default=int(os.environ.get("MSS_TOP_K", DEFAULT_TOP_K)))
    parser.add_argument("--min-score", type=float, default=float(os.environ.get("MSS_MIN_SCORE", DEFAULT_MIN_SCORE)))
    parser.add_argument("--json", action="store_true", dest="json_output", help="Output as JSON")
    args = parser.parse_args()

    if not args.[REDACTED_SECRET]"Error: No API key. Set EMBEDDING_API_KEY or --api-key", file=sys.stderr)
        sys.exit(1)

    db_path = os.path.abspath(args.db)
    if not os.path.exists(db_path):
        print(f"Error: Database not found at {db_path}. Run index.py first.", file=sys.stderr)
        sys.exit(1)

    conn = sqlite3.connect(db_path)
    total = conn.execute("SELECT COUNT(*) FROM chunks WHERE embedding IS NOT NULL").fetchone()[0]
    if total == 0:
        print("Error: No indexed chunks. Run index.py first.", file=sys.stderr)
        sys.exit(1)

    # embed query
    query_vec = get_embedding(args.query, args.api_base, args.api_key, args.model)

    # search
    results = search(query_vec, conn, args.top_k, args.min_score)
    conn.close()

    if args.json_output:
        print(json.dumps(results, ensure_ascii=False, indent=2))
    else:
        if not results:
            print("No results found.")
            return
        for i, r in enumerate(results, 1):
            print(f"\n--- Result {i} (score: {r['score']}) ---")
            print(f"File: {r['file']} (lines {r['lines']})")
            print(r["snippet"])


if __name__ == "__main__":
    main()


path: summary.html
content: <!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Memory Semantic Search - 项目总结</title>
<style>
  * { margin: 0; padding: 0; box-sizing: border-box; }
  body { font-family: -apple-system, "Segoe UI", "PingFang SC", "Microsoft YaHei", sans-serif; background: #0f172a; color: #e2e8f0; line-height: 1.7; padding: 40px 20px; }
  .container { max-width: 960px; margin: 0 auto; }
  
  h1 { font-size: 2.2em; text-align: center; margin-bottom: 8px; background: linear-gradient(135deg, #60a5fa, #a78bfa); -webkit-background-clip: text; -webkit-text-fill-color: transparent; }
  .subtitle { text-align: center; color: #94a3b8; font-size: 1.1em; margin-bottom: 48px; }
  
  .card { background: #1e293b; border-radius: 16px; padding: 32px; margin-bottom: 28px; border: 1px solid #334155; }
  .card h2 { font-size: 1.4em; margin-bottom: 16px; color: #60a5fa; }
  .card h3 { font-size: 1.1em; margin: 20px 0 10px; color: #a78bfa; }
  
  .one-liner { text-align: center; font-size: 1.3em; color: #fbbf24; padding: 20px; background: #1e293b; border-radius: 16px; margin-bottom: 28px; border: 1px solid #334155; }
  
  /* Flow diagram */
  .flow { display: flex; align-items: center; justify-content: center; flex-wrap: wrap; gap: 12px; margin: 24px 0; }
  .flow-step { background: #334155; border-radius: 12px; padding: 14px 20px; text-align: center; min-width: 140px; font-size: 0.95em; }
  .flow-step .icon { font-size: 1.8em; display: block; margin-bottom: 6px; }
  .flow-arrow { font-size: 1.5em; color: #60a5fa; }
  
  /* Example */
  .example-box { background: #0f172a; border-radius: 12px; padding: 20px; margin: 16px 0; border: 1px solid #475569; }
  .example-box .cmd { color: #34d399; font-family: "Fira Code", "Cascadia Code", monospace; font-size: 0.9em; margin: 8px 0; }
  .example-box .output { color: #94a3b8; font-family: "Fira Code", "Cascadia Code", monospace; font-size: 0.85em; white-space: pre-wrap; margin: 8px 0; padding: 12px; background: #1a1a2e; border-radius: 8px; }
  .example-box .label { color: #fbbf24; font-weight: 600; margin-top: 16px; }
  
  /* Comparison */
  .compare { display: grid; grid-template-columns: 1fr 1fr; gap: 16px; margin: 16px 0; }
  .compare-item { background: #0f172a; border-radius: 12px; padding: 20px; border: 1px solid #475569; }
  .compare-item.bad { border-color: #ef4444; }
  .compare-item.good { border-color: #34d399; }
  .compare-item .tag { display: inline-block; padding: 2px 10px; border-radius: 20px; font-size: 0.8em; margin-bottom: 10px; }
  .compare-item.bad .tag { background: #7f1d1d; color: #fca5a5; }
  .compare-item.good .tag { background: #064e3b; color: #6ee7b7; }
  
  /* Tech stack */
  .tech-pills { display: flex; flex-wrap: wrap; gap: 10px; margin: 12px 0; }
  .pill { background: #334155; border-radius: 20px; padding: 6px 16px; font-size: 0.9em; }
  
  /* Key numbers */
  .metrics { display: grid; grid-template-columns: repeat(3, 1fr); gap: 16px; margin: 16px 0; }
  .metric { text-align: center; background: #0f172a; border-radius: 12px; padding: 20px; border: 1px solid #475569; }
  .metric .num { font-size: 2em; font-weight: 700; color: #60a5fa; }
  .metric .desc { color: #94a3b8; font-size: 0.85em; margin-top: 4px; }

  .footer { text-align: center; color: #475569; margin-top: 40px; font-size: 0.85em; }
  a { color: #60a5fa; text-decoration: none; }
  a:hover { text-decoration: underline; }

  @media (max-width: 640px) {
    .compare, .metrics { grid-template-columns: 1fr; }
    .flow { flex-direction: column; }
    .flow-arrow { transform: rotate(90deg); }
  }
</style>
</head>
<body>
<div class="container">

<h1>🔍 Memory Semantic Search</h1>
<p class="subtitle">把 Markdown 笔记变成能理解语义的搜索引擎</p>

<div class="one-liner">💡 一句话：用 Embedding API + SQLite，让你的 Markdown 支持语义搜索</div>

<!-- 解决什么问题 -->
<div class="card">
  <h2>❓ 解决了什么痛点？</h2>
  <p style="margin-bottom:16px;">
    你有很多 Markdown 笔记（比如 MEMORY.md、项目文档），但传统的 Ctrl+F 只能搜<strong>字面关键字</strong>。如果你想找"怎么部署项目"，但笔记里写的是"deployed to Zeabur"，就搜不到。
  </p>
  <p>
    这个 Skill 把笔记内容变成向量，实现<strong>语义搜索</strong>——就算用词不同，只要意思相近，就能搜出来。
  </p>
</div>

<!-- 实际运行示例（会议讲解重点） -->
<div class="card">
  <h2>📋 实际运行示例</h2>
  
  <h3>步骤一：建索引（只需执行一次）</h3>
  <div class="example-box">
    <p class="cmd">$ python3 scripts/index.py ~/.openclaw/workspace</p>
    <div class="output">Workspace: /home/tony/.openclaw/workspace
Found 5 markdown files (MEMORY.md, SOUL.md, ...)
Total chunks: 15
  Embedded 10/15 chunks  ← 第1批：10个chunk调API
  Embedded 15/15 chunks  ← 第2批：剩下的5个
Done. Total indexed chunks: 15</div>
    <p style="color:#94a3b8; margin-top:12px; font-size:0.9em;">
      ✅ 这一步扫描所有 .md 文件，切成小块，调用 embedding API 把每块变成向量，存入 SQLite。文件改了会自动增量更新。
    </p>
  </div>

  <h3>步骤二：语义搜索</h3>
  <div class="example-box">
    <p class="cmd">$ python3 scripts/search.py "Claw Arena 部署在哪里"</p>
    <div class="output">--- Result 1 (score: 0.6655) ---
File: MEMORY.md (lines 29-81)
Claw Arena 从 API key 模式重构为 Skill + API 架构，
部署到 Zeabur，Hong Kong region...

--- Result 2 (score: 0.647) ---
File: MEMORY.md (lines 1-39)
2026-02-10: Claw Arena 重构完成，部署到 Zeabur，
Skill 发布到 ClawHub...</div>
    <p style="color:#94a3b8; margin-top:12px; font-size:0.9em;">
      👆 笔记里写的是"部署到 Zeabur"，搜的是"部署在哪里"——<strong>措辞不同但语义匹配</strong>，score 0.67。语义搜索的核心能力。
    </p>
  </div>
</div>

<!-- 工作流程 -->
<div class="card">
  <h2>⚙️ 工作流程</h2>
  
  <div class="flow">
    <div class="flow-step"><span class="icon">📄</span>扫描 .md 文件</div>
    <span class="flow-arrow">→</span>
    <div class="flow-step"><span class="icon">✂️</span>切块 (chunk)</div>
    <span class="flow-arrow">→</span>
    <div class="flow-step"><span class="icon">🤖</span>Embedding API<br><small>文本 → 向量</small></div>
    <span class="flow-arrow">→</span>
    <div class="flow-step"><span class="icon">💾</span>存入 SQLite</div>
  </div>

  <div style="text-align:center; margin:16px 0; color:#94a3b8; font-size:1.2em;">↓ 搜索时 ↓</div>

  <div class="flow">
    <div class="flow-step"><span class="icon">💬</span>用户提问</div>
    <span class="flow-arrow">→</span>
    <div class="flow-step"><span class="icon">🤖</span>问题 → 向量</div>
    <span class="flow-arrow">→</span>
    <div class="flow-step"><span class="icon">📐</span>余弦相似度<br><small>本地计算</small></div>
    <span class="flow-arrow">→</span>
    <div class="flow-step"><span class="icon">🎯</span>Top-K 结果</div>
  </div>
</div>

<!-- 余弦相似度解释 -->
<div class="card">
  <h2>📐 余弦相似度：如何判断"意思像不像"？</h2>
  
  <h3>🌍 直观理解（30秒）</h3>
  <div class="example-box">
    <div class="output">
  向量 = 指向语义空间的箭头

  "部署项目"     → 箭头指向 东北方向
  "deploy app"  → 箭头也指向 东北方向
  "中午吃什么"   → 箭头指向 西南方向
    </div>
    <p style="color:#94a3b8; margin-top:12px;">
      余弦相似度就是算两个箭头夹角：夹角越小（方向越近），相似度越高；夹角越大，越不相关。
    </p>
  </div>

  <h3>🔢 数学公式</h3>
  <div class="example-box">
    <p class="label">两个向量 A 和 B（例如都是 1024 维）</p>
    <div class="output">
  相似度 = (A·B) / (|A| × |B|)
         = (A₁×B₁ + A₂×B₂ + ... + A₁₀₂₄×B₁₀₂₄) 
           / (A的长度 × B的长度)
    </div>
    <p class="label">结果范围：-1 ~ 1</p>
    <div class="output">
  -1  → 完全相反方向（意思相反）
   0  → 互相垂直（毫无关系）
   1  → 方向完全一致（意思几乎一样）
    </div>
  </div>

  <h3>🔄 本 Skill 的计算过程</h3>
  <div class="example-box">
    <div class="output">
  1. 查询 "Claw Arena 部署在哪里" → Q_vec (1024维)
  2. 从 SQLite 读取所有 chunk 向量 C₁, C₂, ...
  3. for each C:
        score = cosine(Q_vec, C)
  4. 按 score 从高到低排序，取 top-5
    </div>
  </div>

  <h3>⚡ 为什么合适？</h3>
  <p style="color:#94a3b8; margin:16px 0;">
    • <strong>轻量</strong>：不需要训练，直接算余弦<br>
    • <strong>高效</strong>：SQLite 里存的向量，本地直接算，不调 API<br>
    • <strong>语义友好</strong>：embedding 后的向量天然把意思相近的句子聚在一起
  </p>
</div>

<!-- 技术栈 -->
<div class="card">
  <h2>🛠 技术栈</h2>
  <div class="tech-pills">
    <span class="pill">🐍 Python 3（纯 stdlib）</span>
    <span class="pill">💾 SQLite（向量存储）</span>
    <span class="pill">🔗 OpenAI 兼容 Embedding API</span>
    <span class="pill">📐 余弦相似度</span>
  </div>
  <p style="color:#94a3b8; margin-top:12px;">
    零外部依赖，不需要 pip install。API 兼容 OpenAI / 讯飞 / Gemini / Ollama 等。
  </p>
</div>

<!-- 关键数据 -->
<div class="card">
  <h2>📊 实测数据</h2>
  <div class="metrics">
    <div class="metric">
      <div class="num">15</div>
      <div class="desc">索引 chunks<br>(5 个 md 文件)</div>
    </div>
    <div class="metric">
      <div class="num">0.67</div>
      <div class="desc">最高相似度 score<br>(语义精准命中)</div>
    </div>
    <div class="metric">
      <div class="num">&lt;2s</div>
      <div class="desc">单次搜索耗时<br>(1 次 API 调用)</div>
    </div>
  </div>
  <p style="color:#94a3b8; font-size:0.9em; margin-top:12px;">
    💡 即使扩展到 1000+ chunks，搜索依然是"1 次 API 调用 + 本地向量计算"，速度不变，成本仅一次 embedding。
  </p>
</div>

<!-- 核心代码结构 -->
<div class="card">
  <h2>📁 项目结构</h2>
  <div class="example-box">
    <div class="output">memory-semantic-search/
├── SKILL.md          # Skill 描述和使用说明
├── README.md         # 项目文档（含安装、使用、环境变量）
├── .gitignore        # 排除敏感数据和临时文件
└── scripts/
    ├── index.py      # 索引：扫描 → 切块 → embedding → 存储
    └── search.py     # 搜索：查询 → embedding → 相似度计算 → 排序</div>
  </div>
  <p style="color:#94a3b8; margin-top:12px;">
    ⚠️ SQLite 数据库文件（*.sqlite）和 __pycache__ 已加入 .gitignore，不会上传敏感数据。
  </p>
</div>

<div class="footer">
  GitHub: <a href="https://github.com/toller892/memory-semantic-search" target="_blank">toller892/memory-semantic-search</a>
</div>

</div>
</body>
</html>


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (HIGH)