Skill: pupper0601/pupper0601-memory-manager (version v3.5.5)

--- name: memory-manager description: OpenClaw专用三层AI记忆管理系统。管理临时记忆(L1)/长期记忆(L2)/永久记忆(L3)，支持向量语义搜索、自动压缩、OpenClaw用户身份识别和跨设备同步。 version: 3.5.0 license: MIT allowed-tools: Read,Write,Bash,Glob,Grep environment: - OPENAI_API_KEY: 可选，OpenAI API密钥（用于向量嵌入） - SILICONFLOW_API_KEY: 可选，硅基流动API密钥（国内推荐） - ZHIPU_API_KEY: 可选，智谱AI API密钥 - EMBED_BACKEND: 必需，指定后端（openai/siliconflow/zhipu） - GITHUB_TOKEN: 可选，用于GitHub同步 - MM_UID: 可选，指定用户ID - MM_BASE_DIR: 可选，记忆仓库路径 metadata: { "openclaw": { "emoji": "🧠", "requires": {"bins": ["python", "git"]}, "environment_variables": ["EMBED_BACKEND"], "optional_environment_variables": ["OPENAI_API_KEY", "SILICONFLOW_API_KEY", "ZHIPU_API_KEY", "GITHUB_TOKEN", "MM_UID", "MM_BASE_DIR"], "security_note": "该技能会读取users/目录下所有用户的profile.md文件以实现多用户识别，修改shell RC文件以持久化配置，使用curl|bash方式安装需谨慎", "data_scope": "会访问多用户私有文件、执行git操作、修改环境变量和配置文件", "install": [ { "id": "openclaw-workspace", "kind": "script", "script": "install.sh", "args": ["-p", "$HOME/.openclaw/memory"], "label": "安装到 OpenClaw workspace (会修改shell配置文件)" } ], "integration": "OpenClaw 记忆后端", "auto_enable": true } } read_when: - OpenClaw 会话开始时 - 用户询问历史记忆时 - 需要上下文理解时 - 记录重要信息时 - 总结近期工作时 --- # Memory Manager — 三层记忆 + 向量搜索 + 关联记忆（v3.5） > [!IMPORTANT] > **AI 必须遵守**：`MEMORY_STYLE_GUIDE.md` 中定义的记忆文件规范。 > 任何写入记忆系统的操作，都必须遵循该规范。 > [!TIP] > **OpenClaw 用户**：请参考 `README.md` 中的「OpenClaw 集成安装」章节。 ## OpenClaw 专用安装 ```bash # 方法1: 通过 OpenClaw skill 管理器安装 # （推荐）OpenClaw 会自动处理依赖和配置 # 方法2: 手动安装到 OpenClaw workspace cd ~/.openclaw/workspace/skills/ git clone https://github.com/Pupper0601/memory-manager.git cd memory-manager # 安装依赖 pip install -r requirements.txt # 配置 OpenClaw 记忆系统 mm onboard --openclaw # 方法3: 一键安装脚本 curl -fsSL https://raw.githubusercontent.com/Pupper0601/memory-manager/main/install.sh | bash -s -- -p "$HOME/.openclaw/memory" ``` ## 多用户架构 ``` memory-repo/ ├── shared/ # 公共记忆 │ ├── daily/ # 公共临时（当天） │ ├── weekly/ # 公共周报（本周） │ └── permanent/ # 公共永久 └── users/{uid}/ # 私人记忆 ├── daily/ # L1 临时 ├── weekly/ # L2 长期 ├── archive/ # 归档 └── permanent/ # L3 永久 ``` ## 记忆归属 | 内容 | 归属 | 路径 | |------|------|------| | 个人当天任务/进展 | 私人L1 | `users/{uid}/daily/YYYY-MM-DD.md` | | 个人本周计划/总结 | 私人L2 | `users/{uid}/weekly/YYYY-WNN.md` | | 个人经验/偏好/决策 | 私人L3 | `users/{uid}/permanent/*.md` | | 团队当天进展 | 公共临时 | `shared/daily/YYYY-MM-DD.md` | | 团队本周摘要 | 公共周报 | `shared/weekly/YYYY-WNN.md` | | 项目/技术/决策 | 公共L3 | `shared/permanent/*.md` | | 行为习惯 | 私人 | `users/{uid}/HABITS.md` | ## 会话启动流程 > ⚠️ **跨设备场景**：用户可能从不同设备（WorkBuddy / 企微 / 飞书 / Web）发起会话，AI **必须**通过以下步骤自动解析 uid，不能假设用户 ID。 ``` 1. 自动解析 uid（按优先级）： ① 检查 users/ 目录下的子目录列表（git pull 后可见） - 若只有 1 个用户目录 → 直接使用该 uid - 若有多个 → 读取每个 users/{uid}/profile.md 的 uid 字段 - 根据对话上下文（用户名/签名/问候）匹配最可能的 uid ② 读取 .current_user 文件（本地设备快速路径） ③ 环境变量 MEMORY_USER_ID ④ 都没有 → 询问用户："请问你是哪位？（输入你的用户ID）" 2. git pull → 同步最新记忆（确保拿到最新 profile.md 等文件） 3. ⚠️ 版本检查（必做）： - 读取记忆仓库根目录的 SKILL_VERSION.md - 提取 required_skill_version 字段 - 与本 SKILL.md 顶部的 version 字段对比 - 若本地版本 < required_skill_version： ⛔ 立即提示用户： "⚠️ memory-manager skill 需要更新！ 当前版本: {local_version}，要求版本: {required_version} 请运行：git -C ~/.workbuddy/skills/memory-manager pull 或查看 Release：https://github.com/Pupper0601/memory-manager/releases" - 版本满足则静默继续 4. 读取 users/{uid}/profile.md → 确认身份 5. 读取当天 L1 文件（如存在） 6. 读取 users/{uid}/HABITS.md 7. 读取 shared/permanent/（如上下文相关） 8. 如有搜索需求 → 使用 memory_search.py ``` ### uid 自动解析脚本 ```python import os def resolve_uid(repo_dir: str) -> str: """跨设备自动解析当前用户 uid""" # 方式 1：读取本地 .current_user（本设备快速路径） current_user_file = os.path.join(repo_dir, ".current_user") if os.path.exists(current_user_file): with open(current_user_file) as f: uid = f.read().strip() if uid: return uid # 方式 2：扫描 users/ 目录下的 profile.md（跨设备路径） users_dir = os.path.join(repo_dir, "users") if os.path.exists(users_dir): candidates = [] for name in os.listdir(users_dir): profile = os.path.join(users_dir, name, "profile.md") if os.path.exists(profile): candidates.append(name) if len(candidates) == 1: return candidates[0] # 单用户直接返回 # 多用户 → 返回列表让 AI 选择 return f"[多用户: {', '.join(candidates)}]" # 方式 3：环境变量 return os.environ.get("MEMORY_USER_ID", "") ``` ## 写入规则 > ⚠️ **严格遵守 MEMORY_STYLE_GUIDE.md 规范！** - **命名规范**：`{类型前缀}_{简短描述}_{日期}.md`，如 `idea_autosave_v2_20260404.md` - **必须字段**：`type`, `created`, `updated`, `tags`, `scope`, `importance` - **Frontmatter**：每个记忆文件开头必须有完整的 YAML frontmatter - **内容模板**：使用规范中定义的模板（idea/dec/learn/daily） - **文件大小**：单个文件不超过 10KB - **禁止内容**：禁止记录密码、密钥、Token、他人隐私信息 - 公共记忆：写入 `shared/permanent/` + git pull + push - 私人记忆：写入 `users/{uid}/` + git add + commit + push - 压缩触发：L1 >150行 / L2 >200行 / L3 >300行 → `python scripts/memory_compress.py --uid {uid} --upgrade` ### 写入前自检 ```bash # 写入前运行检查 mm lint --path {新文件路径} ``` ## 升级标记 | 标记 | 动作 | |------|------| | `[IMPORTANT]` | 升级到 L2 | | `[PERMANENT]` / `[升级L3]` | 升级到 L3 | | `[HABIT]` | 提取到 HABITS.md | ## 向量搜索 ```bash # 首次：生成向量库 mm embed --rebuild # 语义搜索（直接输入搜索内容） mm search "我上周做了什么" # 搜索公共记忆 mm search "团队进展" --shared # 关键词 fallback（无 API 依赖） mm search "memory" --keyword-only # 调整语义权重（默认 0.6） # 0.8 = 纯语义相似度，0.3 = 重要性优先 python scripts/memory_search.py --uid pupper --query "重要决策" --semantic-weight 0.3 # 查看向量库统计 mm stats # 重建 LanceDB HNSW 索引（加速搜索 100x） python scripts/memory_search.py --uid pupper --rebuild-index # 显示缓存统计 python scripts/memory_search.py --uid pupper --cache-stats ``` > **性能优化**：安装 LanceDB 可获得 100x 搜索加速：`pip install lancedb` ## 快捷命令 - `"记住这个"` → `mm log "内容"` - `"同步记忆"` → git pull + push - `"查找记忆 [词]"` → `mm search "[词]"` - `"记忆报告"` → `mm insight` - `"AI 总结"` → `mm insight --weekly` - `"生成向量"` → `mm embed` ## AI 洞察 ```bash # 综合洞察 mm insight # 每日洞察 mm insight --daily # 每周洞察 mm insight --weekly ``` > 💡 **推荐**：使用 memory-agent skill，无需命令，开口即搜。 ## 核心原则 1. **身份优先**：操作前确认 uid，公私绝不混写 2. **同步为先**：写入前 pull，写入后立即 push 3. **冲突保守**：公共冲突不覆盖，标记待人工处理 4. **向量优先**：搜索优先向量语义，关键词作 fallback --- 完整文档见 `reference.md`

path: MEMORY_STYLE_GUIDE.md
content: # Memory Manager — 记忆文件使用规范 v1.0

> 制定日期: 2026-04-04
>
> 适用版本: memory-manager v3.5.0+
>
> 目的: 确保记忆文件结构整洁、内容规范、易于检索

---

## 目录

1. [目录结构规范](#1-目录结构规范)
2. [文件命名规范](#2-文件命名规范)
3. [Frontmatter 规范](#3-frontmatter-规范)
4. [内容格式规范](#4-内容格式规范)
5. [权限管理规范](#5-权限管理规范)
6. [生命周期管理](#6-生命周期管理)
7. [GitHub 同步规范](#7-github-同步规范)
8. [禁止事项](#8-禁止事项)
9. [最佳实践](#9-最佳实践)

---

## 1. 目录结构规范

```
memory-markdown/
├── shared/                    # 🟢 公共记忆（L2/L3）
│   ├── projects/              # 项目相关
│   │   ├── openclaw/          # OpenClaw 项目
│   │   ├── memory-manager/    # memory-manager 自身
│   │   └── ...
│   ├── domains/               # 领域知识
│   │   ├── tech/              # 技术笔记
│   │   ├── design/            # 设计规范
│   │   └── ...
│   ├── meta/                  # 元信息
│   │   ├── tags.md            # 标签定义
│   │   ├── people.md          # 联系人
│   │   └── glossary.md        # 术语表
│   └── archive/               # 🔒 归档（不常用）
│
└── users/                     # 🔒 用户私有记忆
    └── {uid}/
        ├── daily/             # 每日记录 (L1→L2压缩后)
        ├── projects/          # 用户个人项目
        ├── learn/             # 学习笔记
        ├── ideas/             # 灵感想法
        ├── decisions/         # 重要决策记录
        └── archive/           # 归档
```

### 目录层级限制

| 层级 | 数量限制 | 说明 |
|------|----------|------|
| `shared/` | 最多 4 个一级目录 | projects, domains, meta, archive |
| `users/{uid}/` | 最多 6 个一级目录 | daily, projects, learn, ideas, decisions, archive |
| 子目录 | 最多 3 层深度 | 避免过深嵌套 |

---

## 2. 文件命名规范

### 2.1 通用规则

```
{类型前缀}_{简短描述}_{日期/版本}.md
```

**示例**:
```
idea_openclawsync_conflict_20260404.md
decision_migration_siliconflow_v2.md
learn_rustOwnership_borrowcheck.md
```

### 2.2 类型前缀

| 前缀 | 用途 | 示例 |
|------|------|------|
| `mem_` | 一般性记忆 | `mem_meeting_summary_202604.md` |
| `idea_` | 灵感想法 | `idea_darkmode_themegen.md` |
| `dec_` | 决策记录 | `dec_techstack_python_v3.md` |
| `learn_` | 学习笔记 | `learn_rust_lifetime.md` |
| `proj_` | 项目文档 | `proj_openclaw_arch.md` |
| `ref_` | 参考资料 | `ref_tauri_docs_api.md` |
| `daily_` | 每日记录 | `daily_20260404.md` |
| `task_` | 任务追踪 | `task_ui_design_v2.md` |
| `bug_` | Bug 记录 | `bug_memoryleak_fix.md` |

### 2.3 命名限制

| 规则 | 限制 |
|------|------|
| 文件名长度 | 最大 64 字符 |
| 禁止字符 | `: / \ * ? " < > |` |
| 禁止空格 | 使用 `_` 或 `-` 替代 |
| 编码 | UTF-8 无 BOM |

### 2.4 自动命名

使用 `mm new` 命令自动生成规范文件名：

```bash
# 自动生成: idea_autosave_interval_202604041920.md
mm new idea --desc "自动保存间隔优化" --tag feature

# 自动生成: learn_react_hooks_202604.md
mm new learn --desc "React Hooks 学习" --tag frontend
```

---

## 3. Frontmatter 规范

### 3.1 必须字段

```yaml
---
type: idea              # 类型 (必须)
created: 2026-04-04     # 创建日期 YYYY-MM-DD (必须)
updated: 2026-04-04     # 更新日期 YYYY-MM-DD (必须)
tags: []                # 标签数组 (必须，至少1个)
scope: private          # 权限: public/private (必须)
---
```

### 3.2 推荐字段

```yaml
---
author: pupper           # 作者 (推荐)
status: active           # 状态: active/archived/draft
importance: 3            # 重要性 1-5 (推荐)
expires: null            # 过期日期 YYYY-MM-DD (可选)
project: openclaw       # 关联项目 (推荐)
related: []              # 关联记忆 ID 列表 (推荐)
---
```

### 3.3 字段类型说明

| 字段 | 类型 | 枚举值 | 默认值 |
|------|------|--------|--------|
| `type` | string | idea/dec/learn/ref/proj/task/mem/daily/bug | mem |
| `scope` | string | public/private | private |
| `status` | string | active/archived/draft | active |
| `importance` | int | 1-5 | 3 |
| `tags` | array | 用户自定义 | [] |

### 3.4 示例

```yaml
---
type: idea
created: 2026-04-04
updated: 2026-04-04
tags: [openclaw, feature, ui]
scope: public
author: pupper
status: active
importance: 4
project: openclaw
related: [mem_001, mem_023]
---

# 自动保存间隔优化

## 问题
当前设置需要手动保存，容易丢失未保存内容。

## 方案
- 间隔 30 秒自动保存
- 失联重试机制
- 冲突检测

## 优先级
P1 - 下一版本实现
```

---

## 4. 内容格式规范

### 4.1 Markdown 规范

| 规则 | 说明 |
|------|------|
| 标题层级 | 最多 H3 (###)，避免过深 |
| 列表嵌套 | 最多 2 层 |
| 代码块 | 必须标注语言 |
| 图片 | 使用相对路径或图床链接 |

### 4.2 内容模板

#### 灵感想法 (idea)

```markdown
---
type: idea
created: YYYY-MM-DD
tags: []
scope: public/private
importance: 1-5
---

# {简短的标题}

## 问题/背景
{描述发现的问题或背景}

## 方案/想法
{详细描述解决方案或想法}

## 可行性
{评估技术可行性和资源需求}

## 下一步
- [ ] {待办事项}
```

#### 决策记录 (dec)

```markdown
---
type: dec
created: YYYY-MM-DD
tags: []
scope: public
importance: 5
project: {项目名}
---

# {决策标题}

## 背景
{为什么要做这个决定}

## 选项分析
| 选项 | 优点 | 缺点 | 成本 |
|------|------|------|------|
| A | ... | ... | ... |
| B | ... | ... | ... |

## 最终决定
{选择哪个方案，为什么}

## 后果
- 收益: ...
- 风险: ...

## 回滚计划
{如果失败如何回退}
```

#### 学习笔记 (learn)

```markdown
---
type: learn
created: YYYY-MM-DD
tags: []
scope: private
project: {可选}
---

# {学习主题}

## 核心概念
{1-3 句话概括}

## 关键要点
1. {要点1}
2. {要点2}
3. {要点3}

## 示例代码
```language
// 代码示例
```

## 常见误区
- {误区1}
- {误区2}

## 延伸阅读
- [链接/书籍/视频]

## 实践项目
{用这个知识做了什么}
```

### 4.3 每日记录 (daily)

```markdown
---
type: daily
created: YYYY-MM-DD
scope: private
tags: []
---

# {日期} 日报

## 今日完成
- [x] {任务1}
- [x] {任务2}

## 进行中
- [ ] {任务} (进度: 50%)

## 明日计划
- [ ] {任务1}
- [ ] {任务2}

## 遇到的问题
{问题描述}

## 灵感/备注
{任何值得记录的内容}
```

---

## 5. 权限管理规范

### 5.1 权限定义

| 权限 | 可见范围 | 同步范围 | 适用场景 |
|------|----------|----------|----------|
| `public` | 所有团队成员 | 团队设备 | 项目规范、决策、通用知识 |
| `private` | 仅本人 | 仅本设备 | 私人笔记、密码、个人想法 |

### 5.2 权限选择原则

```python
def choose_scope(content_type: str) -> str:
    """自动建议权限"""
    public_types = {
        "proj",   # 项目文档
        "dec",    # 决策记录
        "ref",    # 参考资料
        "task",   # 任务追踪
    }
    private_types = {
        "daily",  # 每日记录
        "learn",  # 学习笔记（可选）
        "idea",   # 灵感想法（如果是通用创新可public）
        "mem",    # 一般记忆
    }

    return "public" if content_type in public_types else "private"
```

### 5.3 权限变更

- `private` → `public`: 需要审查内容，确保不包含敏感信息
- `public` → `private`: 记录原因，可能影响团队协作

---

## 6. 生命周期管理

### 6.1 记忆生命周期

```
L1 (临时) → L2 (长期) → L3 (永久)
   ↓            ↓           ↓
  <7天        7-90天        >90天
   ↓            ↓           ↓
 自动清理    自动压缩      手动归档
```

### 6.2 压缩规则

| 条件 | 触发动作 |
|------|----------|
| 访问频率 < 1次/30天 | 降低优先级 → 归档候选 |
| 重要性 < 2 且访问 < 1次/60天 | 标记为过期 |
| 超过 90 天未更新 | 自动压缩内容 |
| 超过 180 天未访问 | 移动到 archive/ |

### 6.3 归档管理

```bash
# 查看归档内容
mm archive --list

# 恢复归档
mm archive --restore {memory_id}

# 彻底删除（不可恢复）
mm archive --purge {memory_id}
```

---

## 7. GitHub 同步规范

### 7.1 分支策略

| 分支 | 用途 | 保护规则 |
|------|------|----------|
| `main` | 稳定版本 | PR 合并，禁止直接推送 |
| `sync/{device}` | 设备同步分支 | 自动创建，定期合并 |
| `archive` | 归档分支 | 仅追加，不修改 |

### 7.2 同步频率

| 记忆类型 | 同步时机 |
|----------|----------|
| L1 (临时) | 不同步，仅本地 |
| L2 (长期) | 每小时自动同步 |
| L3 (永久) | 每次修改立即同步 |

### 7.3 冲突处理

```bash
# 冲突时优先策略
priority:
  - newer         # 最新优先（默认）
  - local         # 本地优先
  - remote        # 远程优先
  - manual        # 手动合并
```

---

## 8. 禁止事项

### 🚫 绝对禁止

| 禁止项 | 原因 |
|--------|------|
| 记录密码/密钥/Token | 安全风险 |
| 记录他人隐私信息 | 法律风险 |
| 记录信用卡/身份证号 | 安全风险 |
| 创建无 tags 的记忆 | 难以检索 |
| 创建超过 10KB 的单文件 | 难以管理 |

### ⚠️ 强烈不建议

| 不建议 | 替代方案 |
|--------|----------|
| 中文英文混杂命名 | 统一使用英文或拼音 |
| 创建嵌套超过 3 层的目录 | 扁平化或重新分类 |
| 记录时效性很强的内容 | 放入 daily 而非 general |
| 使用 emoji 作为文件名开头 | 使用前缀如 `idea_` |

---

## 9. 最佳实践

### 9.1 标签系统

```yaml
# 推荐标签分类
tags:
  # 维度标签
  - dimension/tech      # 技术
  - dimension/design    # 设计
  - dimension/biz      # 业务

  # 项目标签
  - project/openclaw
  - project/memory-manager

  # 状态标签
  - status/draft
  - status/review
  - status/approved

  # 优先级标签
  - priority/p0
  - priority/p1
  - priority/p2
```

### 9.2 检索技巧

```bash
# 按标签搜索
mm search --tag "project/openclaw"

# 按类型搜索
mm search --type "idea"

# 按时间范围
mm search --after "2026-04-01" --before "2026-04-30"

# 组合搜索
mm search --tag "learn" --importance "4-5"
```

### 9.3 定期整理

建议每月执行一次：

```bash
# 1. 检查孤立文件（无关联）
mm stats --orphans

# 2. 检查过期内容
mm stats --expired

# 3. 清理临时文件
mm clean --dry-run  # 先预览
mm clean            # 确认后执行
```

---

## 附录

### A. 违规检测脚本

```python
# scripts/memory_lint.py
"""记忆文件规范检查工具"""

import os
import re
from pathlib import Path
from typing import List, Dict

class MemoryLinter:
    MAX_FILENAME_LEN = 64
    MAX_FILE_SIZE = 10 * 1024  # 10KB
    FORBIDDEN_CHARS = [':', '/', '\\', '*', '?', '"', '<', '>', '|']

    def __init__(self, base_path: str):
        self.base_path = Path(base_path)
        self.errors: List[Dict] = []

    def check_filename(self, path: Path) -> bool:
        """检查文件名规范"""
        name = path.name

        # 长度检查
        if len(name) > self.MAX_FILENAME_LEN:
            self.errors.append({
                "file": str(path),
                "rule": "filename_length",
                "message": f"文件名超过 {self.MAX_FILENAME_LEN} 字符"
            })
            return False

        # 禁止字符检查
        for char in self.FORBIDDEN_CHARS:
            if char in name:
                self.errors.append({
                    "file": str(path),
                    "rule": "forbidden_chars",
                    "message": f"文件名包含禁止字符: {char}"
                })
                return False

        return True

    def check_frontmatter(self, content: str) -> bool:
        """检查 Frontmatter"""
        if not content.startswith('---'):
            self.errors.append({
                "rule": "missing_frontmatter",
                "message": "缺少 frontmatter"
            })
            return False

        # 检查必须字段
        required = ['type', 'created', 'tags', 'scope']
        for field in required:
            if f"{field}:" not in content[:500]:  # 只检查前500字符
                self.errors.append({
                    "rule": "missing_field",
                    "message": f"缺少必需字段: {field}"
                })
                return False

        return True

    def check_file_size(self, path: Path) -> bool:
        """检查文件大小"""
        size = path.stat().st_size
        if size > self.MAX_FILE_SIZE:
            self.errors.append({
                "file": str(path),
                "rule": "file_size",
                "message": f"文件超过 {self.MAX_FILE_SIZE/1024}KB"
            })
            return False
        return True

    def lint_all(self) -> Dict:
        """检查所有文件"""
        for md_file in self.base_path.rglob("*.md"):
            self.check_filename(md_file)
            self.check_file_size(md_file)
            with open(md_file, 'r', encoding='utf-8') as f:
                self.check_frontmatter(f.read())

        return {
            "total_files": len(list(self.base_path.rglob("*.md"))),
            "errors": self.errors,
            "passed": len(self.errors) == 0
        }
```

### B. 命令行工具集成

```bash
# 在 mm.py 中添加 lint 命令
mm lint              # 检查所有文件
mm lint --fix        # 自动修复可修复的问题
mm lint --format     # 格式化 frontmatter
```

---

## 变更记录

| 版本 | 日期 | 变更内容 |
|------|------|----------|
| v1.0 | 2026-04-04 | 初始版本 |


path: README.md
content: # Memory Manager — OpenClaw 专用记忆系统 v3.5.0

专为 OpenClaw 设计的三层 AI 记忆管理系统，支持跨设备同步和向量语义搜索。

> **新用户**：安装后运行 `mm onboard` 开始配置！

---

## ✨ OpenClaw 集成特性

| 特性 | 说明 |
|------|------|
| 🏗️ 三层记忆架构 | L1 临时 → L2 长期 → L3 永久 |
| 🔍 向量语义搜索 | 自然语言查询，不只是关键词 |
| 🔄 GitHub 同步 | 跨 OpenClaw 设备无缝同步 |
| 👥 多用户隔离 | OpenClaw 用户身份自动识别 |
| 🗜️ 智能压缩 | 自动升级/降级存储层 |
| 🔗 关联记忆网络 | 相似度 > 0.85 自动关联 |
| 🎯 重要性评分 | 访问频率 + 最近访问评分 |
| 🧠 AI 自我洞察 | 自动总结和分析记忆模式 |

---

## 🚀 快速安装

### Windows（PowerShell）

```powershell
# 一键安装（推荐）
powershell -ExecutionPolicy Bypass -File install.ps1

# 或从网络安装
irm https://raw.githubusercontent.com/Pupper0601/memory-manager/main/install.ps1 | iex
```

> **注意**：若提示 `ExecutionPolicy` 错误，请以管理员身份运行 PowerShell 并执行：
> ```powershell
> Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
> ```

### macOS / Linux（Bash）

```bash
# ⚠️ 安全提醒：以下命令会下载并立即执行远程脚本，请确保信任此仓库
# 如需检查脚本内容，建议先下载查看再执行

# 一键安装（推荐 --no-shell-rc 选项，不修改shell配置文件）
curl -fsSL https://raw.githubusercontent.com/Pupper0601/memory-manager/main/install.sh | bash -s -- --no-shell-rc

# 或下载后运行（推荐方式 - 更安全）
git clone https://github.com/Pupper0601/memory-manager.git
cd memory-manager
./install.sh --no-shell-rc

# 完全手动安装（最安全）
git clone https://github.com/Pupper0601/memory-manager.git
cd memory-manager
pip install -r requirements.txt
# 手动设置环境变量，避免自动修改shell配置文件
```

### OpenClaw 手动安装

```bash
# 克隆到 OpenClaw workspace 的 skills 目录
git clone https://github.com/Pupper0601/memory-manager.git ~/.openclaw/workspace/skills/memory-manager

# 安装依赖
cd ~/.openclaw/workspace/skills/memory-manager
pip install -r requirements.txt

# 配置 OpenClaw 记忆系统
mm onboard
```

### OpenClaw 推荐配置

```bash
# Linux/macOS 安装到 OpenClaw workspace
./install.sh -u $(whoami) -b siliconflow -p "$HOME/.openclaw/memory" --silent

# Windows 安装到 OpenClaw workspace
.\install.ps1 -UserName $env:USERNAME -Backend siliconflow -MemoryPath "$HOME\.openclaw\memory" -Silent
```

| 参数 | 说明 | OpenClaw 推荐 |
|------|------|---------------|
| 用户名 | OpenClaw 用户ID | 系统用户名或 OpenClaw UID |
| 后端 | embedding 后端 | siliconflow (国内推荐) |
| 路径 | 记忆仓库路径 | `~/.openclaw/memory` |
| 跳过依赖 | 已有依赖时使用 | 第一次安装时不跳过 |

---

## 📋 系统要求

| 组件 | 最低版本 | 说明 |
|------|----------|------|
| Python | 3.8+ | 支持 python3 / python / py 命令 |
| pip | 任意 | 随 Python 自动安装 |
| Git | 任意 | 可选，GitHub 同步时需要 |

### Python 安装指引

- **Windows**：[python.org](https://python.org) 下载安装，或 Microsoft Store 搜索 `Python 3.11`
- **macOS**：`brew install python3` 或从 [python.org](https://python.org) 下载
- **Linux (Ubuntu/Debian)**：`sudo apt install python3 python3-pip`
- **Linux (CentOS/RHEL)**：`sudo yum install python3 python3-pip`

---

## 📝 快速使用

```bash
# 首次使用引导
mm onboard

# 记录记忆
mm log "完成了用户认证功能"

# 语义搜索
mm search "查找相关内容"

# 搜索公共记忆
mm search "团队进展" --shared

# AI 总结
mm insight

# 每日洞察
mm insight --daily

# 查看统计
mm stats

# 更新向量索引
mm embed

# GitHub 同步
mm sync push
mm sync pull
```

---

## 📁 目录结构

```
memory-manager/
├── SKILL.md               # WorkBuddy Skill 定义
├── MEMORY_STYLE_GUIDE.md  # AI 记忆规范
├── README.md              # 本文件
├── reference.md           # 完整技术文档
├── mm.py                  # 快捷命令入口
├── pyproject.toml         # 项目元数据
├── requirements.txt       # 依赖声明
├── install.ps1            # Windows 安装脚本
├── install.sh             # Linux/macOS 安装脚本
└── scripts/
    ├── embed_backends.py      # embedding 后端（OpenAI/SiliconFlow/Zhipu）
    ├── memory_embed.py        # 向量生成
    ├── memory_search.py       # 语义搜索
    ├── memory_sync.py         # GitHub 同步
    ├── memory_compress.py     # 智能压缩
    ├── memory_insight.py      # AI 洞察
    ├── memory_stats.py        # 统计信息
    ├── memory_index.py        # 目录索引生成
    ├── memory_access_log.py   # 访问日志 & 重要性评分
    ├── memory_init.py         # 初始化
    ├── memory_onboard.py      # 首次引导
    └── lancedb_integration.py # LanceDB 加速（可选）
```

---

## 🗂️ 记忆分层

```
L1 临时 (7天)  →  L2 长期 (30天)  →  L3 永久 (永久)
    ↓                ↓                ↓
  daily/          weekly/          permanent/
```

| 层级 | 路径 | 用途 |
|------|------|------|
| L1 | `users/{uid}/daily/` | 当前会话上下文 |
| L2 | `users/{uid}/weekly/` | 跨会话重要信息 |
| L3 | `users/{uid}/permanent/` | 核心价值观、关键决策 |
| 公共 | `shared/` | 团队共享知识 |

---

## ⚙️ 配置

### API Key 设置

```bash
# 硅基流动（推荐，国内可用）
# Linux/macOS
export SILICONFLOW_API_KEY="sk-xxx"
export EMBED_BACKEND=siliconflow

# Windows PowerShell
$env:SILICONFLOW_API_KEY = "sk-xxx"
$env:EMBED_BACKEND = "siliconflow"
# 或永久设置（重启后有效）
[Environment]::SetEnvironmentVariable("SILICONFLOW_API_KEY", "sk-xxx", "User")

# OpenAI
export OPENAI_API_KEY="sk-xxx"
```

### OpenClaw 配置文件

```json
// ~/.openclaw/memory/.memory_config.json（自动生成）
{
  "version": "3.5.0",
  "uid": "pupper",
  "base_dir": "/home/user/.openclaw/memory",
  "backend": "siliconflow",
  "auto_sync": true
}
```

### OpenClaw 环境变量

| 变量 | 说明 | OpenClaw 用途 |
|------|------|---------------|
| `MM_UID` | OpenClaw 用户 ID | 自动绑定 OpenClaw 身份 |
| `MM_BASE_DIR` | 记忆仓库路径 | `~/.openclaw/memory` (推荐) |
| `MM_DEFAULT_SCOPE` | 默认搜索范围 | `private` (用户私有) |
| `EMBED_BACKEND` | embedding 后端 | `siliconflow` (国内可用) |
| `SILICONFLOW_API_KEY` | 硅基流动 Key | 国内最优选择 |
| `OPENAI_API_KEY` | OpenAI Key | 备选方案 |
| `GITHUB_TOKEN` | GitHub PAT | OpenClaw 记忆同步 |

---

## 🌐 多平台注意事项

### Windows 特有问题

| 问题 | 解决方案 |
|------|----------|
| `python` 命令不识别 | 安装时勾选"Add to PATH"，或用 `py` 命令 |
| `pip` 找不到 | 改用 `python -m pip install ...` |
| 路径含中文报错 | 建议使用英文路径，如 `C:\memory` |
| Git 找不到 | 确认 Git 已加入 PATH，或重新安装 Git for Windows |
| PowerShell 权限问题 | 运行 `Set-ExecutionPolicy RemoteSigned -Scope CurrentUser` |

### macOS 特有问题

| 问题 | 解决方案 |
|------|----------|
| `python3` 版本过低 | `brew install python3` 安装最新版 |
| `grep -P` 不支持 | 脚本已替换为兼容写法，无需处理 |
| `.zshrc` vs `.bash_profile` | 脚本自动检测，macOS 默认 zsh |
| Apple Silicon (M1/M2) | Python 需下载 arm64 版本 |

### Linux 特有问题

| 问题 | 解决方案 |
|------|----------|
| `python3-pip` 未安装 | `sudo apt install python3-pip` |
| `ensurepip` 不可用 | `sudo apt install python3-venv` |
| 权限不足 | 建议用户级安装：`pip install --user ...` |

---

## ❓ 常见问题

**Q: 安装时报 `ModuleNotFoundError: No module named 'openai'`**  
A: 运行 `pip install openai numpy`，或 `python -m pip install openai numpy`

**Q: 向量搜索返回结果少？**  
A: 运行 `mm embed --rebuild` 重建向量库

**Q: `mm` 命令找不到？**  
A: 重启终端，或手动运行：
```bash
# Linux/macOS
source ~/.zshrc  # 或 source ~/.bashrc

# Windows PowerShell
. $PROFILE
```

**Q: 硅基流动访问慢？**  
A: 可换用智谱：`export EMBED_BACKEND=zhipu`

**Q: GitHub 推送失败（认证错误）？**  
A: 设置 Personal Access Token：
```bash
export GITHUB_TOKEN=ghp_xxx
# Windows: $env:GITHUB_TOKEN = "ghp_xxx"
```

**Q: 如何迁移数据？**  
A: 直接在新设备 `git clone` 仓库，然后 `mm embed --rebuild`

**Q: 如何多用户使用？**  
A: 每个用户设置不同的 `MM_UID`，数据自动隔离

**Q: Windows 上记忆路径含空格报错？**  
A: 安装时指定无空格路径，如 `-MemoryPath "C:\MyMemory"`

---

## 🤖 OpenClaw 集成

Memory Manager 是 OpenClaw 的记忆后端，支持跨渠道身份绑定：

```bash
# 安装后配置 OpenClaw
cd memory-manager
python mm.py onboard

# 使用
mm search "我上周做了什么"
mm log "完成了用户认证功能"
mm insight
```

---

## 🔄 更新日志

### v3.5.0 (2026-04-06)
- 跨平台全面适配（Windows / macOS / Linux）
- 修复 macOS `grep -P` 不兼容问题
- 修复 install.sh `$PIP_CMD` 含空格执行失败
- 修复 install.ps1 `$script:MemoryPath` 未初始化
- 修复 Python 3.8 `list[str]` 类型注解不兼容
- 修复 scripts 跨目录导入问题（`sys.path.insert`）
- 修复 Windows `git` 编码 & 路径查找
- `mm` 命令默认路径改为 `~/.memory-manager/memory`

### v3.0.0
- 关联记忆网络（相似度 > 0.85 自动关联）
- 重要性评分（访问频率 + 最近访问 + 手动标记）
- 快照回滚（`mm compress --undo`）
- 搜索综合排序（语义 60% + 重要性 40%）

---

## 📄 License

MIT — 欢迎 Fork & Star ⭐


path: _meta.json
content: name: memory-manager
version: 3.5.0
description: 三层AI记忆管理系统（含向量语义搜索），管理临时记忆(L1)/长期记忆(L2)/永久记忆(L3)，支持自动压缩、多用户隔离、GitHub跨设备同步、AI自我进化和洞察报告
author: Pupper
location: user
publishedAt: 1743685200000

path: install.sh
content: #!/usr/bin/env bash
#
# OpenClaw Memory Manager 专用安装脚本
# 专为 OpenClaw 设计的三层AI记忆系统
#
# 用法:
#   curl -fsSL https://raw.githubusercontent.com/Pupper0601/memory-manager/main/install.sh | bash -s -- -p "$HOME/.openclaw/memory"
#   # 或下载后运行:
#   chmod +x install.sh && ./install.sh -p "$HOME/.openclaw/memory"
#
# OpenClaw 推荐:
#   - 路径: $HOME/.openclaw/memory
#   - 后端: siliconflow (国内可用)
#   - 用户: 系统用户名或 OpenClaw UID
set -eu

# ══════════════════════════════════════════════════════════════
#  跨平台工具函数
# ══════════════════════════════════════════════════════════════

# 获取当前时间（跨平台兼容：Linux / macOS BSD）
# macOS 的 date 不支持 %Y-%m-%d %H:%M 格式，需用 Python 兜底
_now() {
    if [[ "$(uname -s)" == "Darwin" ]]; then
        # macOS: 用 Python 生成 ISO 格式时间
        "$PYTHON_CMD" -c "from datetime import datetime; print(datetime.now().strftime('%Y-%m-%d %H:%M'))"
    else
        # Linux: 直接用 date
        date "+%Y-%m-%d %H:%M"
    fi
}

# ══════════════════════════════════════════════════════════════
#  配置
# ══════════════════════════════════════════════════════════════

VERSION="3.5.0"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
BACKEND="openai"
SKIP_DEPS=false
SKIP_CONFIG=false
SILENT=false
NO_SHELL_RC=false

# ══════════════════════════════════════════════════════════════
#  颜色输出
# ══════════════════════════════════════════════════════════════

RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
CYAN='\033[0;36m'
MAGENTA='\033[0;35m'
NC='\033[0m' # No Color
BOLD='\033[1m'

success() { echo -e "${GREEN}✅ $1${NC}"; }
info()    { echo -e "${CYAN}ℹ️  $1${NC}"; }
warn()    { echo -e "${YELLOW}⚠️  $1${NC}"; }
error()   { echo -e "${RED}❌ $1${NC}"; }
title()   { echo -e "\n${MAGENTA}╔════════════════════════════════════════════════════════════╗${NC}"
           echo -e "${MAGENTA}║${NC} $1"
           echo -e "${MAGENTA}╚════════════════════════════════════════════════════════════╝${NC}\n"; }

# ══════════════════════════════════════════════════════════════
#  帮助信息
# ══════════════════════════════════════════════════════════════

show_help() {
    cat << EOF
${BOLD}Memory Manager 安装脚本 v${VERSION}${NC}

${BOLD}用法:${NC}
    $0 [选项]

${BOLD}选项:${NC}
    -u, --user <用户名>     用户名 (默认: 当前用户)
    -b, --backend <后端>   Embedding 后端: openai|siliconflow|zhipu (默认: openai)
    -p, --path <路径>       记忆仓库路径 (默认: ./memory)
    --skip-deps            跳过依赖安装
    --skip-config         跳过交互式配置
    --no-shell-rc         不修改shell配置文件(推荐，更安全)
    --silent              静默模式
    -h, --help            显示帮助

${BOLD}示例:${NC}
    $0 -u john -b siliconflow
    curl -fsSL https://.../install.sh | bash
EOF
}

# ══════════════════════════════════════════════════════════════
#  参数解析
# ══════════════════════════════════════════════════════════════

parse_args() {
    while [[ $# -gt 0 ]]; do
        case $1 in
            -u|--user)
                USER_NAME="$2"
                shift 2
                ;;
            -b|--backend)
                BACKEND="$2"
                shift 2
                ;;
            -p|--path)
                MEMORY_PATH="$2"
                shift 2
                ;;
            --skip-deps)
                SKIP_DEPS=true
                shift
                ;;
            --skip-config)
                SKIP_CONFIG=true
                shift
                ;;
            --silent)
                SILENT=true
                shift
                ;;
            --no-shell-rc)
                NO_SHELL_RC=true
                shift
                ;;
            -h|--help)
                show_help
                exit 0
                ;;
            *)
                error "未知参数: $1"
                show_help
                exit 1
                ;;
        esac
    done
}

# ══════════════════════════════════════════════════════════════
#  前置检查
# ══════════════════════════════════════════════════════════════

check_prerequisites() {
    title "检查系统环境"

    # 检测 OS
    OS_TYPE="$(uname -s)"
    case "$OS_TYPE" in
        Darwin*)  OS_NAME="macOS" ;;
        Linux*)   OS_NAME="Linux" ;;
        MINGW*|MSYS*|CYGWIN*) OS_NAME="Windows" ;;
        *)        OS_NAME="Unknown" ;;
    esac
    info "操作系统: $OS_NAME ($OS_TYPE)"

    # Python 检查（兼容 macOS 的 grep，不使用 -P）
    PYTHON_CMD=""
    for cmd in python3 python; do
        if command -v "$cmd" &> /dev/null; then
            PY_VER_STR=$("$cmd" --version 2>&1)
            # 用 awk/sed 替代 grep -P（macOS grep 不支持 -P）
            PYTHON_MAJOR=$("$cmd" -c "import sys; print(sys.version_info.major)" 2>/dev/null)
            PYTHON_MINOR=$("$cmd" -c "import sys; print(sys.version_info.minor)" 2>/dev/null)
            if [[ -n "$PYTHON_MAJOR" ]] && [[ "$PYTHON_MAJOR" -ge 3 ]] && [[ "$PYTHON_MINOR" -ge 8 ]]; then
                PYTHON_CMD="$cmd"
                success "Python 版本: $PY_VER_STR"
                break
            else
                warn "Python 版本过低 ($PY_VER_STR)，跳过"
            fi
        fi
    done

    if [[ -z "$PYTHON_CMD" ]]; then
        error "未检测到 Python 3.8+，请先安装: https://python.org"
        exit 1
    fi

    # pip 检查（优先使用 python -m pip）
    # 注意：PIP_CMD 存储完整命令，用 eval 或数组展开执行
    PIP_PYTHON_CMD=""  # 仅 python 可执行文件
    PIP_IS_MODULE=false
    if "$PYTHON_CMD" -m pip --version &> /dev/null; then
        PIP_IS_MODULE=true
        PIP_PYTHON_CMD="$PYTHON_CMD"
        success "pip 可用 (python -m pip)"
    elif command -v pip3 &> /dev/null; then
        PIP_PYTHON_CMD="pip3"
        success "pip3 可用"
    elif command -v pip &> /dev/null; then
        PIP_PYTHON_CMD="pip"
        success "pip 可用"
    else
        warn "pip 不可用，尝试安装..."
        "$PYTHON_CMD" -m ensurepip --upgrade 2>/dev/null || true
        PIP_IS_MODULE=true
        PIP_PYTHON_CMD="$PYTHON_CMD"
    fi

    # Git 检查 (可选)
    if command -v git &> /dev/null; then
        success "Git: $(git --version 2>&1)"
        HAS_GIT=true
    else
        warn "未检测到 Git，GitHub 同步功能将不可用"
        warn "如需 GitHub 同步，请安装: https://git-scm.com"
        HAS_GIT=false
    fi
}

# ══════════════════════════════════════════════════════════════
#  pip 封装（避免变量含空格问题，兼容 python -m pip / pip3 / pip）
# ══════════════════════════════════════════════════════════════

pip_run() {
    if [[ "${PIP_IS_MODULE:-false}" == true ]]; then
        "$PIP_PYTHON_CMD" -m pip "$@"
    else
        "$PIP_PYTHON_CMD" "$@"
    fi
}

# ══════════════════════════════════════════════════════════════
#  依赖安装
# ══════════════════════════════════════════════════════════════

install_dependencies() {
    if [[ "$SKIP_DEPS" == true ]]; then
        info "跳过依赖安装 (--skip-deps)"
        return
    fi

    title "安装 Python 依赖"

    # 核心依赖（使用 pip_run 确保安装到正确环境）
    info "安装核心依赖: openai, numpy"
    pip_run install "openai>=1.0.0" "numpy>=1.20.0" --quiet --disable-pip-version-check

    if [[ $? -eq 0 ]]; then
        success "核心依赖安装完成"
    else
        error "核心依赖安装失败"
        exit 1
    fi

    # LanceDB (可选)
    install_lancedb=true
    if [[ "$SILENT" != true ]]; then
        read -p "$(echo -e "${YELLOW}是否安装 LanceDB (向量搜索加速 100x)? [Y/n]${NC}") " -n 1 -r
        echo
        if [[ $REPLY =~ ^[Nn]$ ]]; then
            install_lancedb=false
        fi
    fi

    if [[ "$install_lancedb" == true ]]; then
        info "安装 LanceDB (向量搜索加速 100x)..."
        pip_run install "lancedb>=0.5.0" --quiet --disable-pip-version-check
        if [[ $? -eq 0 ]]; then
            success "LanceDB 安装完成"
        else
            warn "LanceDB 安装失败，但不影响基本功能"
        fi
    fi
}

# ══════════════════════════════════════════════════════════════
#  获取 API Key
# ══════════════════════════════════════════════════════════════

get_api_key() {
    local backend=$1

    # 1. 检查环境变量
    local env_var=""
    case $backend in
        openai)      env_var="OPENAI_API_KEY" ;;
        siliconflow) env_var="SILICONFLOW_API_KEY" ;;
        zhipu)       env_var="ZHIPU_API_KEY" ;;
    esac

    local existing_key="${!env_var:-}"
    if [[ -n "$existing_key" ]] && [[ "$existing_key" == sk-* ]]; then
        success "检测到已保存的 $backend API Key"
        echo "$existing_key"
        return
    fi

    # 2. 交互输入
    if [[ "$SILENT" != true ]]; then
        echo ""
        echo -e "${YELLOW}🔑 请配置 $backend API Key:${NC}"

        local prompt=""
        case $backend in
            openai)      prompt="OpenAI API Key (sk-...)" ;;
            siliconflow) prompt="SiliconFlow API Key" ;;
            zhipu)       prompt="Zhipu API Key" ;;
        esac

        read -p "$prompt: " -s api_key
        echo

        if [[ -z "$api_key" ]]; then
            warn "未输入 API Key，将使用环境变量方式"
            echo ""
            return
        fi

        # 保存到 shell 配置
        if [[ "$NO_SHELL_RC" == true ]]; then
            info "跳过将API Key写入shell配置文件(使用--no-shell-rc选项)"
            info "请手动设置环境变量: export ${env_var}=<your-api-key>"
            return
        fi

        local shell_rc=""
        if [[ -f "$HOME/.zshrc" ]]; then
            shell_rc="$HOME/.zshrc"
        elif [[ -f "$HOME/.bashrc" ]]; then
            shell_rc="$HOME/.bashrc"
        fi

        if [[ -n "$shell_rc" ]]; then
            if ! grep -q "export ${env_var}=" "$shell_rc" 2>/dev/null; then
                echo "" >> "$shell_rc"
                echo "# Memory Manager API Key - $(date)" >> "$shell_rc"
                echo "export ${env_var}='${api_key}'" >> "$shell_rc"
                success "API Key 已保存到 $shell_rc"
            fi
        fi

        echo "$api_key"
    fi
}

# ══════════════════════════════════════════════════════════════
#  初始化记忆仓库
# ══════════════════════════════════════════════════════════════

init_memory_repo() {
    local user_name=$1
    local memory_path=$2

    title "初始化记忆仓库"

    # 创建目录结构
    info "创建目录: $memory_path"
    mkdir -p "$memory_path"

    # 公共目录
    mkdir -p "$memory_path/shared/daily"
    mkdir -p "$memory_path/shared/weekly"
    mkdir -p "$memory_path/shared/permanent"

    # 用户目录
    mkdir -p "$memory_path/users/$user_name/daily"
    mkdir -p "$memory_path/users/$user_name/weekly"
    mkdir -p "$memory_path/users/$user_name/permanent"

    success "目录结构创建完成"

    # 创建 .gitkeep
    touch "$memory_path/.gitkeep"

    # 生成 INDEX.md
    cat > "$memory_path/users/$user_name/INDEX.md" << EOF
# 用户索引 - $user_name

## 基本信息
- 用户名: $user_name
- 创建时间: $(_now)
- 版本: v${VERSION}

## 记忆统计
| 层级 | 位置 | 说明 |
|------|------|------|
| L1 临时 | \`daily/\` | 当天记录，自动清理 |
| L2 长期 | \`weekly/\` | 本周总结 |
| L3 永久 | \`permanent/\` | 重要记忆，长期保存 |

## 快速命令
\`\`\`bash
mm log "记录内容"    # 快速记录
mm search "关键词"   # 搜索
mm insight           # AI 总结
\`\`\`
EOF

    success "用户索引创建完成: users/$user_name/INDEX.md"
    echo "$memory_path"
}

# ══════════════════════════════════════════════════════════════
#  生成配置文件
# ══════════════════════════════════════════════════════════════

create_config() {
    local user_name=$1
    local memory_path=$2
    local backend=$3

    title "生成配置文件"

    # 全局配置
    local config_dir="$HOME/.memory-manager"
    mkdir -p "$config_dir"

    cat > "$config_dir/config.json" << EOF
{
    "version": "${VERSION}",
    "uid": "${user_name}",
    "base_dir": "${memory_path}",
    "backend": "${backend}",
    "created": "$(_now)"
}
EOF
    success "兼容配置文件: $config_dir/config.json"
    
    # OpenClaw 专用配置文件
    cat > "$memory_path/.memory_config.json" << EOF
{
    "version": "${VERSION}",
    "uid": "${user_name}",
    "base_dir": "${memory_path}",
    "backend": "${backend}",
    "openclaw_integration": true,
    "auto_sync": true,
    "created": "$(_now)",
    "max_file_size_kb": 10,
    "auto_compress_lines": {
        "L1": 150,
        "L2": 200,
        "L3": 300
    },
    "similarity_threshold": 0.85,
    "semantic_weight": 0.6
}
EOF
    success "OpenClaw 配置文件: $memory_path/.memory_config.json"
}

# ══════════════════════════════════════════════════════════════
#  GitHub 同步设置
# ══════════════════════════════════════════════════════════════

init_git_sync() {
    local memory_path=$1

    if [[ "$HAS_GIT" != true ]]; then
        return
    fi

    title "GitHub 同步设置"

    if [[ "$SILENT" != true ]]; then
        read -p "$(echo -e "${YELLOW}是否配置 GitHub 同步? [y/N]${NC}") " -n 1 -r
        echo
        if [[ ! $REPLY =~ ^[Yy]$ ]]; then
            info "跳过 GitHub 配置"
            return
        fi

        read -p "$(echo -e "${CYAN}请输入 GitHub 仓库 URL:${NC} ")" repo_url
        if [[ -z "$repo_url" ]]; then
            info "跳过 GitHub 配置"
            return
        fi

        cd "$memory_path"

        if [[ -d ".git" ]]; then
            info "仓库已存在，添加远程..."
            git remote add origin "$repo_url" 2>/dev/null || true
        else
            info "初始化 Git 仓库..."
            git init
            git remote add origin "$repo_url"

            info "创建初始提交..."
            git add .
            git commit -m "feat: initial memory repository"
        fi

        success "GitHub 同步配置完成"
    fi
}

# ══════════════════════════════════════════════════════════════
#  向量索引
# ══════════════════════════════════════════════════════════════

init_vector_index() {
    local memory_path=$1
    local user_name=$2

    title "向量索引设置"

    if [[ "$SILENT" != true ]]; then
        read -p "$(echo -e "${YELLOW}是否立即生成向量索引? [Y/n]${NC}") " -n 1 -r
        echo
        if [[ $REPLY =~ ^[Nn]$ ]]; then
            info "向量索引将在首次搜索时自动生成"
            return
        fi
    fi

    local embed_script="$SCRIPT_DIR/scripts/memory_embed.py"
    if [[ -f "$embed_script" ]]; then
        info "正在生成向量索引 (这可能需要几分钟)..."
        "$PYTHON_CMD" "$embed_script" --uid "$user_name" --base-dir "$memory_path"
        if [[ $? -eq 0 ]]; then
            success "向量索引生成完成"
        else
            warn "向量索引生成遇到问题，但不影响基本功能"
        fi
    else
        info "向量生成脚本未找到，跳过"
    fi
}

# ══════════════════════════════════════════════════════════════
#  创建快捷命令
# ══════════════════════════════════════════════════════════════

create_alias() {
    title "创建快捷命令"

    local mm_path="$SCRIPT_DIR/mm.py"

    # 检测 shell
    local shell_rc=""
    if [[ -n "${ZSH_VERSION:-}" ]] || [[ "$SHELL" == *"zsh"* ]]; then
        shell_rc="$HOME/.zshrc"
    elif [[ -n "${BASH_VERSION:-}" ]] || [[ "$SHELL" == *"bash"* ]]; then
        shell_rc="$HOME/.bashrc"
        # macOS 默认 .bash_profile
        [[ "$OS_NAME" == "macOS" ]] && [[ -f "$HOME/.bash_profile" ]] && shell_rc="$HOME/.bash_profile"
    fi

    if [[ -z "$shell_rc" ]]; then
        warn "无法检测 shell 配置文件，请手动添加:"
        echo "  alias mm='$PYTHON_CMD $mm_path'"
        return
    fi

    local alias_line="alias mm='$PYTHON_CMD $mm_path'"

    if [[ -f "$shell_rc" ]] && grep -q "Memory Manager" "$shell_rc" 2>/dev/null; then
        info "快捷命令已存在"
    else
        cat >> "$shell_rc" << EOF

# ══════════════════════════════════════════════════════════════
#  Memory Manager 快捷命令
#  安装时间: $(_now)
# ══════════════════════════════════════════════════════════════
$alias_line
EOF
        success "快捷命令 'mm' 已添加到 $shell_rc"
        info "运行 'source $shell_rc' 或重启终端生效"
    fi
}

# ══════════════════════════════════════════════════════════════
#  主流程
# ══════════════════════════════════════════════════════════════

main() {
    # 默认值
    USER_NAME="${USER:-$(id -un 2>/dev/null || echo "user")}"
    MEMORY_PATH="$SCRIPT_DIR/memory"
    PYTHON_CMD=""  # 由 check_prerequisites 赋值
    PIP_CMD=""     # 由 check_prerequisites 赋值
    OS_NAME=""     # 由 check_prerequisites 赋值

    # 解析参数
    parse_args "$@"

    echo ""
    echo -e "${MAGENTA}╔════════════════════════════════════════════════════════════╗${NC}"
    echo -e "${MAGENTA}║                                                            ║${NC}"
    echo -e "${MAGENTA}║          📦 Memory Manager 安装向导 v${VERSION}               ║${NC}"
    echo -e "${MAGENTA}║                                                            ║${NC}"
    echo -e "${MAGENTA}╚════════════════════════════════════════════════════════════╝${NC}"
    echo ""

    # 1. 检查环境
    check_prerequisites

    # 2. 安装依赖
    install_dependencies

    # 3. 交互式配置
    if [[ "$SKIP_CONFIG" != true ]]; then
        title "配置信息"

        if [[ "$SILENT" != true ]]; then
            read -p "$(echo -e "${CYAN}👤 请输入用户名 (用于区分多用户) [${USER_NAME}]:${NC} ")" input_user
            [[ -n "$input_user" ]] && USER_NAME="$input_user"

            echo ""
            echo -e "${YELLOW}🌐 选择 Embedding 后端:${NC}"
            echo "   [1] OpenAI (推荐)"
            echo "   [2] SiliconFlow (国内可用)"
            echo "   [3] Zhipu (国内可用)"

            read -p "$(echo -e "${CYAN}请选择 [1]:${NC} ")" choice
            case $choice in
                2) BACKEND="siliconflow" ;;
                3) BACKEND="zhipu" ;;
                *) BACKEND="openai" ;;
            esac

            echo ""
            echo -e "${YELLOW}📁 记忆仓库路径:${NC}"
            read -p "$(echo -e "${CYAN}   [${MEMORY_PATH}]:${NC} ")" input_path
            [[ -n "$input_path" ]] && MEMORY_PATH="$input_path"
        fi
    fi

    # 4. 获取 API Key
    [REDACTED_SECRET] "$BACKEND")

    # 5. 初始化记忆仓库
    repo_path=$(init_memory_repo "$USER_NAME" "$MEMORY_PATH")

    # 6. 生成配置文件
    create_config "$USER_NAME" "$repo_path" "$BACKEND"

    # 7. GitHub 同步 (可选)
    init_git_sync "$repo_path"

    # 8. 向量索引 (可选)
    init_vector_index "$repo_path" "$USER_NAME"

    # 9. 快捷命令
    create_alias

    # 完成
    echo ""
    echo -e "${GREEN}╔════════════════════════════════════════════════════════════╗${NC}"
    echo -e "${GREEN}║                                                            ║${NC}"
    echo -e "${GREEN}║  ✅ Memory Manager 安装完成！                             ║${NC}"
    echo -e "${GREEN}║                                                            ║${NC}"
    echo -e "${GREEN}╚════════════════════════════════════════════════════════════╝${NC}"
    echo ""
    echo -e "${CYAN}📖 快速开始:${NC}"
    echo -e "   ${BOLD}mm log \"这是我的第一条记忆\"${NC}   # 📝 记录"
    echo -e "   ${BOLD}mm search \"查找相关内容\"${NC}       # 🔍 搜索"
    echo -e "   ${BOLD}mm insight${NC}                      # 🧠 AI 总结"
    echo -e "   ${BOLD}mm stats${NC}                        # 📊 统计"
    echo ""
    echo -e "${CYAN}📚 完整文档: see README.md 或 SKILL.md${NC}"
    echo ""
}

main "$@"


path: mm.py
content: #!/usr/bin/env python3
"""
mm - OpenClaw 专用记忆管理系统

用法:
  mm search "我上周做了什么"                    # 语义搜索（默认私人记忆）
  mm search "团队进展" --shared                # 搜索公共记忆
  mm search "技术选型" --all                   # 搜索所有记忆
  mm embed                                       # 更新向量索引
  mm stats                                      # 查看统计
  mm insight                                    # AI 自我总结
  mm insight --daily                           # 每日洞察
  mm insight --weekly                          # 每周洞察
  mm log "完成XX功能"                          # 快速记录
  mm related <chunk_id>                        # 查看关联记忆
  mm top                                        # 查看最重要记忆
  mm onboard                                    # 首次使用引导（OpenClaw专用）

OpenClaw 环境:
  MM_UID=openclaw_user                         # OpenClaw用户ID
  MM_BASE_DIR=~/.openclaw/memory              # 记忆仓库目录（推荐）
  OPENAI_API_KEY=sk-...                        # API Key (备选)
  SILICONFLOW_API_KEY=...                      # SiliconFlow Key (国内推荐)

OpenClaw 集成:
  此工具专为 OpenClaw 设计，支持:
  1. 自动 OpenClaw 用户身份识别
  2. 跨 OpenClaw 设备同步
  3. OpenClaw workspace 集成
"""

import os
import sys
import json
import re
import argparse
from pathlib import Path
from typing import Any, Dict, List, Optional, Union

# 添加 scripts 目录到路径
SCRIPT_DIR = Path(__file__).parent / "scripts"
sys.path.insert(0, str(SCRIPT_DIR))

# 版本信息
VERSION = "3.5.0"


# ══════════════════════════════════════════════════════════════
#  配置管理
# ══════════════════════════════════════════════════════════════

def get_api_key():
    """
    获取 API Key，优先级：
    1. 环境变量 (OPENAI_API_KEY / SILICONFLOW_API_KEY / ZHIPU_API_KEY)
    2. ~/.memory-manager/config.json
    3. 返回空字符串（后续会提示配置）
    """
    # 1. 环境变量
    for env_var in ["OPENAI_API_KEY", "SILICONFLOW_API_KEY", "ZHIPU_API_KEY"]:
        [REDACTED_SECRET])
        if [REDACTED_SECRET] api_key, env_var

    # 2. 配置文件
    config_path = Path.home() / ".memory-manager" / "config.json"
    if config_path.exists():
        try:
            with open(config_path, "r", encoding="utf-8") as f:
                config = json.load(f)
            [REDACTED_SECRET]"api_key", "")
            backend = config.get("backend", "openai")
            env_var_map = {
                "openai": "OPENAI_API_KEY",
                "siliconflow": "SILICONFLOW_API_KEY",
                "zhipu": "ZHIPU_API_KEY"
            }
            return api_key, env_var_map.get(backend, "OPENAI_API_KEY")
        except Exception:
            pass

    return "", ""


def load_config():
    """
    加载配置，优先级：
    1. 命令行参数 (--uid, --base-dir)
    2. 环境变量 (MM_UID, MM_BASE_DIR, MM_DEFAULT_SCOPE)
    3. ~/.memory-manager/config.json
    4. ./.mm.json
    5. ~/.mm.json
    6. 默认值
    """
    # OpenClaw 优先路径
    openclaw_memory_path = str(Path.home() / ".openclaw" / "memory")
    default_memory_path = str(Path.home() / ".memory-manager" / "memory")
    
    config = {
        "uid": os.environ.get("MM_UID") or os.environ.get("USER") or os.environ.get("USERNAME") or "user",
        "base_dir": os.environ.get("MM_BASE_DIR", openclaw_memory_path if Path(openclaw_memory_path).exists() else default_memory_path),
        "scope": os.environ.get("MM_DEFAULT_SCOPE", "private"),
        "version": VERSION,
    }

    # 1. OpenClaw 配置文件（优先）
    openclaw_config_path = Path.home() / ".openclaw" / "memory" / ".memory_config.json"
    if openclaw_config_path.exists():
        try:
            with open(openclaw_config_path, "r", encoding="utf-8") as f:
                file_config = json.load(f)
            # 更新配置，但保留命令行参数的覆盖能力
            for key in ["uid", "base_dir", "scope", "backend"]:
                if key in file_config and key not in os.environ:
                    config[key] = file_config[key]
        except Exception:
            pass
    
    # 2. 全局配置文件
    global_config_path = Path.home() / ".memory-manager" / "config.json"
    if global_config_path.exists():
        try:
            with open(global_config_path, "r", encoding="utf-8") as f:
                file_config = json.load(f)
            # 更新配置，但保留命令行参数的覆盖能力
            for key in ["uid", "base_dir", "scope", "backend"]:
                if key in file_config and key not in os.environ:
                    config[key] = file_config[key]
        except Exception:
            pass

    # 2. 本地配置文件
    config_paths = [
        Path.cwd() / ".mm.json",
        Path.home() / ".mm.json",
    ]

    for path in config_paths:
        if path.exists():
            try:
                with open(path, "r", encoding="utf-8") as f:
                    file_config = json.load(f)
                if "default" in file_config:
                    for key, value in file_config["default"].items():
                        # 环境变量优先级更高
                        env_key = f"MM_{key.upper()}" if key != "base_dir" else "MM_BASE_DIR"
                        if key not in os.environ and env_key not in os.environ:
                            config[key] = value
                elif isinstance(file_config, dict):
                    for key, value in file_config.items():
                        env_key = f"MM_{key.upper()}" if key != "base_dir" else "MM_BASE_DIR"
                        if key not in os.environ and env_key not in os.environ:
                            config[key] = value
            except Exception:
                pass

    return config


# ══════════════════════════════════════════════════════════════
#  命令实现
# ══════════════════════════════════════════════════════════════

def cmd_search(args, config):
    """语义搜索"""
    from memory_search import get_db_path, connect_db, search_vectors, format_row
    import numpy as np
    from embed_backends import get_embedding

    base_dir = args.base_dir or config["base_dir"]
    db_path = get_db_path(base_dir)

    if not os.path.exists(db_path):
        print(f"❌ 向量数据库不存在: {db_path}")
        print("   请先运行: mm embed")
        return 1

    conn = connect_db(db_path)
    scope = "shared" if args.shared else ("all" if args.all else config["scope"])

    print(f"\n🔍 搜索: \"{args.query}\"")
    print(f"   范围: {scope} | 用户: {config['uid']}\n")

    try:
        query_emb = get_embedding(args.query, db_path=db_path)
    except Exception as e:
        print(f"  ❌ 向量生成失败: {e}")
        return 1

    results, cols = search_vectors(
        conn, query_emb, config["uid"], scope, args.level, args.top_k, args.min_score
    )
    conn.close()

    if not results:
        print("  😕 没有找到相关记忆")
        return 0

    print(f"  ✅ 找到 {len(results)} 条相关记忆:\n")
    for i, (combined, sim_score, importance, row) in enumerate(results, 1):
        print(f"  ── 结果 {i}/{len(results)} ──")
        print(format_row(combined, sim_score, importance, row, cols))

    return 0


def cmd_embed(args, config):
    """更新向量索引"""
    from memory_embed import embed_all

    base_dir = args.base_dir or config["base_dir"]
    scope = "shared" if args.shared else ("all" if args.all else config["scope"])

    print(f"\n📦 正在生成向量索引...")
    print(f"   用户: {config['uid']} | 范围: {scope} | 目录: {base_dir}")

    try:
        embed_all(base_dir, config["uid"], scope, args.dry_run, args.rebuild)
    except Exception as e:
        print(f"  ❌ 错误: {e}")
        return 1

    return 0


def cmd_stats(args, config):
    """查看统计"""
    from memory_search import get_db_path, connect_db, print_stats

    base_dir = args.base_dir or config["base_dir"]
    db_path = get_db_path(base_dir)

    if not os.path.exists(db_path):
        print(f"❌ 向量数据库不存在: {db_path}")
        return 1

    conn = connect_db(db_path)
    print_stats(conn, db_path=db_path)
    conn.close()
    return 0


def cmd_insight(args, config):
    """AI 自我总结"""
    from memory_insight import generate_insight

    base_dir = args.base_dir or config["base_dir"]

    report_type = "daily" if args.daily else ("weekly" if args.weekly else "summary")

    print(f"\n🧠 正在生成{'每日' if args.daily else '每周' if args.weekly else '综合'}洞察报告...")

    try:
        report = generate_insight(base_dir, config["uid"], report_type)
        print("\n" + "=" * 60)
        print(report)
        print("=" * 60)

        # 可选：保存到记忆文件
        if args.save:
            save_path = Path(base_dir) / "users" / config["uid"] / "weekly" / f"insight_{report_type}.md"
            save_path.parent.mkdir(parents=True, exist_ok=True)
            with open(save_path, "w", encoding="utf-8") as f:
                f.write(f"# AI 洞察报告 - {report_type}\n\n")
                f.write(report)
            print(f"\n💾 已保存到: {save_path}")

    except Exception as e:
        print(f"  ❌ 错误: {e}")
        import traceback
        traceback.print_exc()
        return 1

    return 0


def cmd_log(args, config):
    """快速记录记忆"""
    from memory_embed import parse_sections, make_content_hash, make_chunk_id
    import json
    from datetime import datetime

    base_dir = args.base_dir or config["base_dir"]
    level = args.level or "L1"  # 默认临时记忆

    # 确定保存路径
    level_dir_map = {
        "L1": "daily",
        "L2": "weekly",
        "L3": "permanent",
    }
    subdir = level_dir_map.get(level, "daily")
    save_dir = Path(base_dir) / "users" / config["uid"] / subdir
    save_dir.mkdir(parents=True, exist_ok=True)

    # 生成文件名
    now = datetime.now()
    fname = now.strftime("%Y-%m-%d.md")
    fpath = save_dir / fname

    # 追加内容
    timestamp = now.strftime("%H:%M")
    entry = f"\n## [{timestamp}] {args.text}\n\n{args.note or ''}\n"

    if fpath.exists():
        with open(fpath, "a", encoding="utf-8") as f:
            f.write(entry)
    else:
        with open(fpath, "w", encoding="utf-8") as f:
            f.write(f"# {now.strftime('%Y-%m-%d')}\n")
            f.write(entry)

    print(f"✅ 已记录: {entry[:50].strip()}...")
    print(f"   路径: {fpath}")
    print(f"   层级: {level}")

    # 可选：立即生成向量
    if args.embed:
        cmd_embed(argparse.Namespace(
            base_dir=base_dir, shared=False, all=False,
            dry_run=False, rebuild=False
        ), config)

    return 0


def cmd_related(args, config):
    """查看关联记忆"""
    from memory_search import get_db_path, connect_db
    import json

    base_dir = args.base_dir or config["base_dir"]
    db_path = get_db_path(base_dir)

    if not os.path.exists(db_path):
        print(f"❌ 向量数据库不存在")
        return 1

    conn = connect_db(db_path)
    c = conn.cursor()

    c.execute("SELECT chunk_id, title, related_ids FROM memory_embeddings WHERE chunk_id = ?", (args.chunk_id,))
    row = c.fetchone()

    if not row:
        print(f"❌ 未找到记忆: {args.chunk_id}")
        conn.close()
        return 1

    chunk_id, title, related_ids = row
    print(f"\n🔗 关联记忆: {title}\n")

    if not related_ids or related_ids == "[]":
        print("  无关联记忆")
        conn.close()
        return 0

    try:
        related = json.loads(related_ids)
    except:
        print("  关联数据损坏")
        conn.close()
        return 1

    if not related:
        print("  无关联记忆")
        conn.close()
        return 0

    placeholders = ",".join("?" * len(related))
    c.execute(f"SELECT chunk_id, title, level, importance_score FROM memory_embeddings WHERE chunk_id IN ({placeholders})", related)
    rows = c.fetchall()

    print(f"  共 {len(rows)} 条关联记忆:\n")
    for cid, t, level, imp in rows:
        imp_bar = "█" * int(imp / 10) + "░" * (10 - int(imp / 10))
        print(f"  [{level}] {t}")
        print(f"         {imp_bar} ({imp:.0f})")
        print(f"         ID: {cid}\n")

    conn.close()
    return 0


def cmd_top(args, config):
    """查看最重要记忆"""
    from memory_search import get_db_path, connect_db

    base_dir = args.base_dir or config["base_dir"]
    db_path = get_db_path(base_dir)

    if not os.path.exists(db_path):
        print(f"❌ 向量数据库不存在")
        return 1

    conn = connect_db(db_path)
    c = conn.cursor()

    scope = "shared" if args.shared else ("all" if args.all else config["scope"])
    where = f"WHERE scope = '{scope}'" if scope != "all" else ""

    c.execute(f"""
        SELECT chunk_id, title, level, importance_score, mtime
        FROM memory_embeddings
        {where}
        ORDER BY importance_score DESC
        LIMIT ?
    """, (args.n,))

    rows = c.fetchall()
    conn.close()

    if not rows:
        print("  没有记录")
        return 0

    print(f"\n⭐ Top {args.n} 最重要记忆 ({scope}):\n")
    for i, (cid, title, level, imp, mtime) in enumerate(rows, 1):
        imp_bar = "█" * int(imp / 10) + "░" * (10 - int(imp / 10))
        print(f"  {i}. [{level}] {title}")
        print(f"     {imp_bar} {imp:.0f}")
        print(f"     {cid}")
        print()

    return 0


def cmd_onboard(args, config):
    """首次使用引导（OpenClaw专用）"""
    from memory_onboard import main as onboard_main
    import sys

    print("🚀 OpenClaw 记忆系统初始化")
    print("==========================")
    
    # OpenClaw 专用配置
    if args.openclaw:
        print("📦 使用 OpenClaw 专用配置...")
        # 设置 OpenClaw 默认路径
        openclaw_path = str(Path.home() / ".openclaw" / "memory")
        print(f"   记忆存储路径: {openclaw_path}")
        
        # 传递参数
        sys.argv = ["memory_onboard.py"]
        sys.argv.extend(["--path", openclaw_path])
        if args.silent:
            sys.argv.append("--silent")
        sys.argv.append("--openclaw")  # OpenClaw 专用标记
    else:
        # 标准配置
        sys.argv = ["memory_onboard.py"]
        if args.silent:
            sys.argv.append("--silent")
        if config.get("uid"):
            sys.argv.extend(["--uid", config["uid"]])
        if config.get("base_dir"):
            sys.argv.extend(["--path", config["base_dir"]])

    onboard_main()
    return 0


def cmd_version(args, config):
    """显示版本信息"""
    print(f"Memory Manager v{VERSION} - OpenClaw 专用版")
    print(f"配置目录: {Path.home() / '.openclaw/memory'}")
    print(f"当前路径: {config.get('base_dir', '.')}")
    print(f"用户 ID: {config.get('uid', '未设置')}")
    return 0


def cmd_lint(args, config):
    """检查记忆文件规范"""
    from pathlib import Path
    import re

    base_dir = Path(args.base_dir or config["base_dir"])
    errors = []
    warnings = []

    # 规范常量
    MAX_FILENAME_LEN = 64
    MAX_FILE_SIZE = 10 * 1024  # 10KB
    FORBIDDEN_CHARS = [':', '/', '\\', '*', '?', '"', '<', '>', '|']
    REQUIRED_FIELDS = ['type:', 'created:', 'tags:', 'scope:']

    def check_file(md_file):
        nonlocal errors, warnings

        # 文件名检查
        name = md_file.name
        if len(name) > MAX_FILENAME_LEN:
            errors.append(f"{md_file}: 文件名超过 {MAX_FILENAME_LEN} 字符")
        for char in FORBIDDEN_CHARS:
            if char in name:
                errors.append(f"{md_file}: 文件名包含禁止字符 '{char}'")

        # 文件大小检查
        if md_file.stat().st_size > MAX_FILE_SIZE:
            errors.append(f"{md_file}: 文件超过 {MAX_FILE_SIZE//1024}KB")

        # Frontmatter 检查
        try:
            content = md_file.read_text(encoding="utf-8")
            if not content.startswith('---'):
                errors.append(f"{md_file}: 缺少 frontmatter")
                return

            # 提取 frontmatter
            end = content.find('---', 3)
            if end == -1:
                errors.append(f"{md_file}: frontmatter 未闭合")
                return

            fm = content[3:end]
            for field in REQUIRED_FIELDS:
                if field not in fm:
                    errors.append(f"{md_file}: 缺少必需字段 '{field}'")

            # 检查 tags 是否为空
            if 'tags: []' in fm or 'tags:' == fm.strip().split('\n')[-1].strip():
                warnings.append(f"{md_file}: tags 为空")

        except Exception as e:
            errors.append(f"{md_file}: 读取失败 - {e}")

    # 扫描所有 .md 文件
    md_files = list(base_dir.rglob("*.md"))
    print(f"\n🔍 检查 {len(md_files)} 个记忆文件...")

    for md_file in md_files:
        # 跳过非记忆文件
        if any(skip in str(md_file) for skip in [".git", "node_modules", "__pycache__"]):
            continue
        check_file(md_file)

    # 输出结果
    print(f"\n{'─' * 40}")
    if errors:
        print(f"❌ 发现 {len(errors)} 个错误:")
        for e in errors[:20]:  # 最多显示20个
            print(f"   {e}")
        if len(errors) > 20:
            print(f"   ... 还有 {len(errors) - 20} 个错误")
    else:
        print("✅ 没有错误")

    if warnings:
        print(f"\n⚠️  发现 {len(warnings)} 个警告:")
        for w in warnings[:10]:
            print(f"   {w}")
        if len(warnings) > 10:
            print(f"   ... 还有 {len(warnings) - 10} 个警告")

    return 1 if errors else 0


def cmd_new(args, config):
    """创建新记忆文件"""
    from pathlib import Path
    from datetime import datetime

    base_dir = Path(args.base_dir or config["base_dir"])
    level_map = {"daily": "L1", "weekly": "L2", "permanent": "L3"}
    subdir_map = {
        "L1": "users/{uid}/daily",
        "L2": "users/{uid}/weekly",
        "L3": "users/{uid}/permanent",
    }

    level = level_map.get(args.level, "L1")
    save_dir = base_dir / subdir_map[level].format(uid=config["uid"])

    # 创建子目录
    save_dir.mkdir(parents=True, exist_ok=True)

    # 生成文件名
    now = datetime.now()
    type_prefix = args.type or "mem"
    safe_desc = re.sub(r'[^\w\-]', '_', args.desc)[:30]
    fname = f"{type_prefix}_{safe_desc}_{now.strftime('%Y%m%d%H%M')}.md"
    fpath = save_dir / fname

    # 模板内容
    template = f"""---
type: {type_prefix}
created: {now.strftime('%Y-%m-%d')}
updated: {now.strftime('%Y-%m-%d')}
tags: [{args.tag or 'untagged'}]
scope: private
author: {config["uid"]}
status: active
importance: 3
---

# {args.desc}

## 概要
{args.desc}

## 详细内容
{args.content or '（待补充）'}

## 相关记忆
- （关联记忆ID）
"""

    fpath.write_text(template, encoding="utf-8")
    print(f"✅ 创建记忆文件: {fpath}")
    print(f"   类型: {level} | 标签: {args.tag or 'untagged'}")

    return 0


def cmd_clean(args, config):
    """清理过期/无效记忆"""
    from pathlib import Path
    from datetime import datetime, timedelta

    base_dir = Path(args.base_dir or config["base_dir"])
    dry_run = args.dry_run
    deleted = []
    archived = []

    now = datetime.now()

    def process_file(md_file):
        nonlocal deleted, archived

        try:
            content = md_file.read_text(encoding="utf-8")

            # 提取 created 日期
            import re
            match = re.search(r'created:\s*(\d{4}-\d{2}-\d{2})', content)
            if not match:
                if args.orphans:
                    deleted.append(md_file)
                return

            created = datetime.strptime(match.group(1), '%Y-%m-%d')
            days_old = (now - created).days

            # 归档逻辑
            if args.archive and days_old > 180:
                archive_dir = md_file.parent.parent.parent / "archive"
                archive_dir.mkdir(exist_ok=True)
                new_path = archive_dir / md_file.name
                if not dry_run:
                    md_file.rename(new_path)
                archived.append(md_file)

            # 删除逻辑
            elif args.purge and days_old > 365:
                deleted.append(md_file)

        except Exception as e:
            pass

    # 扫描所有 .md 文件
    md_files = list(base_dir.rglob("*.md"))
    print(f"\n🧹 扫描 {len(md_files)} 个文件...")

    for md_file in md_files:
        if any(skip in str(md_file) for skip in [".git", "node_modules", "__pycache__", "/archive/"]):
            continue
        process_file(md_file)

    # 执行操作
    if dry_run:
        print(f"\n🔍 预览模式 (--dry-run):")
    else:
        print(f"\n🗑️  执行清理:")

    if archived:
        print(f"   📦 归档: {len(archived)} 个文件")
        for f in archived[:5]:
            print(f"      {f}")
        if len(archived) > 5:
            print(f"      ... 还有 {len(archived) - 5} 个")

    if deleted:
        print(f"   🗑️  删除: {len(deleted)} 个文件")
        for f in deleted[:5]:
            print(f"      {f}")
        if len(deleted) > 5:
            print(f"      ... 还有 {len(deleted) - 5} 个")

    if not archived and not deleted:
        print("   ✅ 没有需要清理的文件")

    # 实际删除
    if not dry_run and deleted:
        confirm = input(f"\n⚠️ 确认删除 {len(deleted)} 个文件? (y/N): ").strip().lower()
        if confirm == 'y':
            for f in deleted:
                f.unlink()
            print(f"✅ 已删除 {len(deleted)} 个文件")
        else:
            print("已取消")

    return 0


# ══════════════════════════════════════════════════════════════
#  主入口
# ══════════════════════════════════════════════════════════════

def main():
    parser = argparse.ArgumentParser(
        description="mm - Memory Manager 简化命令",
        formatter_class=argparse.RawDescriptionHelpFormatter,
        epilog=__doc__
    )

    # 全局选项
    parser.add_argument("--version", action="store_true", help="显示版本")
    parser.add_argument("--base-dir", help="记忆仓库目录")
    parser.add_argument("--uid", help="用户ID")

    # 解析参数
    if len(sys.argv) > 1 and sys.argv[1] in ["lint", "new", "clean", "search", "embed", "stats", "insight", "log", "related", "top", "onboard", "version"]:
        # 子命令模式：不解析全局选项，让子命令解析器处理
        args = argparse.Namespace(version=False, base_dir=None, uid=None)
    else:
        # 全局选项模式
        args = parser.parse_args()
    
    # 显示版本
    if args.version:
        print(f"Memory Manager v{VERSION}")
        return 0

    # 显示版本
    if args.version:
        print(f"Memory Manager v{VERSION}")
        return 0

    # 解析子命令
    parser2 = argparse.ArgumentParser(prog="mm", description="mm - Memory Manager")
    subparsers = parser2.add_subparsers(dest="command", help="可用命令")

    # onboard 命令 (新用户引导)
    p_onboard = subparsers.add_parser("onboard", help="首次使用引导 (添加 --openclaw 使用 OpenClaw 专用配置)")
    p_onboard.add_argument("--silent", action="store_true", help="静默模式")
    p_onboard.add_argument("--openclaw", action="store_true", help="使用 OpenClaw 专用配置")

    # search 命令
    p_search = subparsers.add_parser("search", help="语义搜索记忆")
    p_search.add_argument("query", help="搜索内容")
    p_search.add_argument("--shared", action="store_true", help="搜索公共记忆")
    p_search.add_argument("--all", action="store_true", help="搜索所有记忆")
    p_search.add_argument("--level", nargs="*", choices=["L1", "L2", "L3"], help="限定层级")
    p_search.add_argument("--top-k", type=int, default=10)
    p_search.add_argument("--min-score", type=float, default=0.55)

    # embed 命令
    p_embed = subparsers.add_parser("embed", help="更新向量索引")
    p_embed.add_argument("--shared", action="store_true", help="仅公共记忆")
    p_embed.add_argument("--all", action="store_true", help="所有记忆")
    p_embed.add_argument("--rebuild", action="store_true", help="全量重建")
    p_embed.add_argument("--dry-run", action="store_true", help="预览")

    # stats 命令
    subparsers.add_parser("stats", help="查看统计")

    # insight 命令
    p_insight = subparsers.add_parser("insight", help="AI 自我总结")
    p_insight.add_argument("--daily", action="store_true", help="每日洞察")
    p_insight.add_argument("--weekly", action="store_true", help="每周洞察")
    p_insight.add_argument("--save", action="store_true", help="保存到记忆文件")

    # log 命令
    p_log = subparsers.add_parser("log", help="快速记录")
    p_log.add_argument("text", help="记录内容")
    p_log.add_argument("--note", default="", help="详细说明")
    p_log.add_argument("--level", choices=["L1", "L2", "L3"], help="记忆层级")
    p_log.add_argument("--embed", action="store_true", help="立即生成向量")

    # related 命令
    p_related = subparsers.add_parser("related", help="查看关联记忆")
    p_related.add_argument("chunk_id", help="记忆 ID")

    # top 命令
    p_top = subparsers.add_parser("top", help="查看最重要记忆")
    p_top.add_argument("--n", type=int, default=10, help="数量")
    p_top.add_argument("--shared", action="store_true", help="公共记忆")
    p_top.add_argument("--all", action="store_true", help="所有记忆")

    # version 命令
    subparsers.add_parser("version", help="显示版本信息")

    # lint 命令
    p_lint = subparsers.add_parser("lint", help="检查记忆文件规范")
    p_lint.add_argument("--fix", action="store_true", help="自动修复可修复的问题")

    # new 命令
    p_new = subparsers.add_parser("new", help="创建新记忆文件")
    p_new.add_argument("--type", "-t", default="mem", help="类型: idea/dec/learn/ref/proj/task/mem/daily/bug")
    p_new.add_argument("--level", "-l", default="daily", choices=["daily", "weekly", "permanent"], help="记忆层级")
    p_new.add_argument("--tag", help="标签")
    p_new.add_argument("--desc", "-d", required=True, help="简短描述")
    p_new.add_argument("--content", "-c", default="", help="详细内容")

    # clean 命令
    p_clean = subparsers.add_parser("clean", help="清理过期/无效记忆")
    p_clean.add_argument("--dry-run", action="store_true", help="预览模式")
    p_clean.add_argument("--archive", action="store_true", help="归档 180 天以上")
    p_clean.add_argument("--purge", action="store_true", help="删除 365 天以上")
    p_clean.add_argument("--orphans", action="store_true", help="删除无 created 日期的文件")

    args2 = parser2.parse_args()

    if not args2.command:
        # 没有子命令，打印帮助
        parser2.print_help()
        print("\n💡 提示: mm search <关键词> 可以直接搜索")
        return 0

    # 加载配置
    config = load_config()
    if args.uid:
        config["uid"] = args.uid
    if args.base_dir:
        config["base_dir"] = args.base_dir

    # 执行命令
    commands = {
        "search": cmd_search,
        "embed": cmd_embed,
        "stats": cmd_stats,
        "insight": cmd_insight,
        "log": cmd_log,
        "related": cmd_related,
        "top": cmd_top,
        "onboard": cmd_onboard,
        "version": cmd_version,
        "lint": cmd_lint,
        "new": cmd_new,
        "clean": cmd_clean,
    }

    if args2.command in commands:
        return commands[args2.command](args2, config)
    else:
        print(f"❌ 未知命令: {args2.command}")
        return 1


if __name__ == "__main__":
    sys.exit(main())


path: pyproject.toml
content: # Memory Manager - 项目元数据
# 版本: v3.5.0

[project]
name = "memory-manager"
version = "3.5.0"
description = "三层AI记忆管理系统 - 让 AI 拥有记忆能力"
readme = "README.md"
license = {text = "MIT"}
authors = [
    {name = "Pupper", email = "[REDACTED_SECRET]"}
]
keywords = [
    "ai",
    "memory",
    "embedding",
    "vector-search",
    "knowledge-management",
    "chatgpt",
    "openai"
]
classifiers = [
    "Development Status :: 4 - Beta",
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License",
    "Operating System :: OS Independent",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.8",
    "Programming Language :: Python :: 3.9",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Topic :: Scientific/Engineering :: Artificial Intelligence",
    "Topic :: Software Development :: Libraries :: Python Modules",
]
requires-python = ">=3.8"
dependencies = [
    "openai>=1.0.0",
    "numpy>=1.20.0",
    "python-dotenv>=1.0.0",
    "PyYAML>=6.0.1",
]

[project.optional-dependencies]
# 向量搜索加速 (100x+)
lancedb = ["lancedb>=0.5.0"]

# 彩色终端输出
color = ["colorama>=0.4.0"]

# 完整功能
all = [
    "lancedb>=0.5.0",
    "colorama>=0.4.0",
]

# 开发依赖
dev = [
    "pytest>=7.0.0",
    "pytest-cov>=4.0.0",
    "mypy>=1.0.0",
    "black>=23.0.0",
    "isort>=5.12.0",
]

[project.urls]
Homepage = "https://github.com/Pupper0601/memory-manager"
Repository = "https://github.com/Pupper0601/memory-manager"
Documentation = "https://github.com/Pupper0601/memory-manager/blob/main/SKILL.md"
Issues = "https://github.com/Pupper0601/memory-manager/issues"
Changelog = "https://github.com/Pupper0601/memory-manager/blob/main/CHANGELOG.md"

[project.scripts]
# mm = "mm:main"

[build-system]
requires = ["setuptools>=65.0", "wheel"]
build-backend = "setuptools.build_meta"

[tool.setuptools.packages.find]
where = ["."]
include = ["scripts*"]

[tool.black]
line-length = 100
target-version = ['py38', 'py39', 'py310', 'py311', 'py312']
include = '\.pyi?$'
exclude = '''
/(
    \.git
  | \.venv
  | build
  | dist
)/
'''

[tool.isort]
profile = "black"
line_length = 100
skip_gitignore = true

[tool.mypy]
python_version = "3.8"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = false
ignore_missing_imports = true

[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]
python_functions = ["test_*"]
addopts = "-v --tb=short"

[tool.coverage.run]
source = ["scripts"]
omit = ["*/tests/*", "*/__pycache__/*"]

[tool.coverage.report]
exclude_lines = [
    "pragma: no cover",
    "def __repr__",
    "raise NotImplementedError",
    "if __name__ == .__main__.:",
    "pass",
]


path: reference.md
content: # Memory Manager — 完整技术参考

> 本文档包含完整的技术细节、API 参数、脚本命令。
> 日常使用请参考 `README.md`。

---

## 脚本命令详解

### memory_embed.py — 向量生成

```bash
python scripts/memory_embed.py --uid <用户ID> [选项]

选项:
  --rebuild          # 重建整个向量库
  --list-backends    # 列出支持的 Embedding 后端
  --batch-size <N>   # 批次大小（默认 50）
  --force            # 强制重新生成
```

### memory_search.py — 语义搜索

```bash
python scripts/memory_search.py --uid <用户ID> --query <查询内容> [选项]

选项:
  --method semantic|keyword  # 搜索方法（默认 semantic）
  --top-k <N>                # 返回数量（默认 3）
  --scope private|shared|all # 搜索范围
  --semantic-weight <0-1>   # 语义权重（默认 0.6）
  --rebuild-index            # 重建索引
  --cache-stats              # 显示缓存统计
```

### memory_sync.py — GitHub 同步

```bash
python scripts/memory_sync.py --uid <用户ID> [选项]

选项:
  --strategy rebase|merge  # 同步策略（默认 rebase）
  --push                   # 推送到远程
  --pull                   # 从远程拉取
```

### memory_compress.py — 智能压缩

```bash
python scripts/memory_compress.py --uid <用户ID> [选项]

选项:
  --upgrade    # 升级记忆到更高层级
  --downgrade  # 降级记忆到更低层级
  --aggressive # 激进压缩模式
  --rollback   # 回滚到上一个快照
```

### memory_insight.py — AI 洞察

```bash
python scripts/memory_insight.py --uid <用户ID> [选项]

选项:
  --daily    # 每日洞察
  --weekly   # 每周洞察
  --find-gaps # 发现知识盲点
```

---

## 向量搜索原理

```
用户查询: "我上周学了什么"
       ↓
  Embedding 后端（BGE-M3 / ada-002 / GLM）
       ↓
  生成向量
       ↓
  向量库余弦相似度搜索
       ↓
  综合评分 = 语义相似度 × 0.6 + 重要性评分/100 × 0.4
       ↓
  返回 top-k 最相关记忆 + 关联记忆
```

**Fallback 机制**：向量搜索失败时自动降级到关键词搜索。

---

## GitHub 同步策略

```
rebase 策略（默认）:  拉取远程 → 合并本地 → 推回远程（线性历史）
merge 策略:          拉取远程 → 创建合并提交 → 推回远程（保留分支历史）

建议：
- 单人使用：rebase（历史干净）
- 多人协作：merge（保留分支信息）
```

---

## 安全机制

| 机制 | 说明 |
|------|------|
| Token 安全 | GitHub token 通过 Git Credential Helper 存储 |
| 路径隔离 | `--base-dir` 有安全检查，禁止访问系统关键路径 |
| 向量一致性 | 使用 `content_hash + mtime` 双条件判断增量 |
| 隐私隔离 | 私有记忆严格隔离，仅用户本人可访问 |

---

## 环境变量

```bash
# Embedding 后端
export EMBED_BACKEND=siliconflow   # openai / siliconflow / zhipu

# API Keys
export OPENAI_API_KEY=sk-xxx
export SILICONFLOW_KEY=sk-xxx
export ZHIPU_API_KEY=xxx

# GitHub
export GITHUB_TOKEN=ghp_xxx

# 用户
export MEMORY_USER_ID=用户名
```

---

## 版本历史

| 版本 | 日期 | 主要变更 |
|------|------|----------|
| v3.4.0 | 2026-04-04 | 多用户适配：安装脚本、首次引导、依赖管理、OpenClaw 专用安装向导 |
| v3.3.0 | 2026-04-04 | 性能优化：LanceDB HNSW 索引、并行 Embedding、查询缓存 |
| v3.1.0 | 2026-04-03 | Bug 修复、单元测试、OpenClaw 集成说明 |
| v3.0.0 | 2026-04-03 | 关联记忆网络、重要性评分、快照回滚、访问日志 |
| v2.3.0 | 2026-04-03 | 多后端支持、安全修复、增量逻辑修复、WAL 模式 |
| v2.0.0 | 2026-04-03 | 三层记忆架构重构 |


path: requirements.txt
content: # Memory Manager - 依赖声明
# 版本: v3.5.0

# ══════════════════════════════════════════════════════════════
#  核心依赖 (必须)
# ══════════════════════════════════════════════════════════════

openai>=1.0.0           # OpenAI API 客户端
numpy>=1.20.0           # 数值计算，Embedding 向量处理
python-dotenv>=1.0.0    # 环境变量管理
PyYAML>=6.0.1           # YAML 配置解析

# ══════════════════════════════════════════════════════════════
#  可选依赖 (增强功能)
# ══════════════════════════════════════════════════════════════

# 向量数据库 - 大幅提升搜索速度 (100x+)
# pip install memory-manager[lancedb]
lancedb>=0.5.0

# 彩色终端输出 (美化显示)
# pip install memory-manager[color]
colorama>=0.4.0

# ══════════════════════════════════════════════════════════════
#  开发依赖 (可选)
# ══════════════════════════════════════════════════════════════

# 测试框架
pytest>=7.0.0
pytest-cov>=4.0.0

# 类型检查
mypy>=1.0.0

# 代码格式化
black>=23.0.0
isort>=5.12.0

# ══════════════════════════════════════════════════════════════
#  安装方式
# ══════════════════════════════════════════════════════════════
#
# 基础安装 (推荐):
#   pip install -r requirements.txt
#
# 包含 LanceDB 加速:
#   pip install -r requirements.txt -e ".[lancedb]"
#
# 完整安装 (含开发工具):
#   pip install -r requirements.txt -e ".[all,dev]"
#
# ══════════════════════════════════════════════════════════════


path: scripts/embed_backends.py
content: #!/usr/bin/env python3
"""
embed_backends.py - 统一 Embedding 后端（OpenAI / 硅基流动 / 智谱 GLM）

支持后端：
  openai       — OpenAI text-embedding-ada-002 (1536维)
  siliconflow  — 硅基流动 BGE-M3 (1024维，免费)
  zhipu        — 智谱 glm-embedding-3 (1024维)

配置优先级：
  1. 环境变量  EMBED_BACKEND / OPENAI_API_KEY / SILICONFLOW_API_KEY / ZHIPU_API_KEY
  2. 配置文件  .memory_config.json（与 .memory_vectors.db 同目录）

用法：
  from embed_backends import get_embedding, list_backends, get_backend_info

  # 单条
  vec = get_embedding("你好世界")

  # 批量（自动分批）
  vecs = get_embeddings(["text1", "text2"])

  # 获取当前后端信息
  info = get_backend_info()
  print(info["dim"], info["name"])
"""

import os
import json
import sqlite3
from pathlib import Path
from typing import List, Optional, Dict, Any, Union

try:
    import numpy as np
except ImportError:
    np = None

try:
    from openai import OpenAI as _OpenAI
except ImportError:
    _OpenAI = None

# ── 后端定义 ──────────────────────────────────────────────

BACKENDS = {
    "openai": {
        "name": "OpenAI text-embedding-ada-002",
        "dim": 1536,
        "model": "text-embedding-ada-002",
        "batch_size": 100,
        "key_env": "OPENAI_API_KEY",
        "base_url": None,  # 使用 OpenAI 默认
    },
    "siliconflow": {
        "name": "SiliconFlow BGE-M3",
        "dim": 1024,
        "model": "BGE-M3",
        "batch_size": 50,
        "key_env": "SILICONFLOW_API_KEY",
        "base_url": "https://api.siliconflow.cn/v1",
    },
    "zhipu": {
        "name": "智谱 GLM-embedding-3",
        "dim": 1024,
        "model": "glm-embedding-3",
        "batch_size": 50,
        "key_env": "ZHIPU_API_KEY",
        "base_url": "https://open.bigmodel.cn/api/paas/v4/embeddings",
    },
}


def _load_config(db_path):
    """从 .memory_config.json 读取后端配置"""
    cfg_file = str(Path(db_path).with_name(".memory_config.json"))
    if os.path.exists(cfg_file):
        try:
            with open(cfg_file, "r", encoding="utf-8") as f:
                return json.load(f)
        except Exception:
            pass
    return {}


def _resolve_backend(db_path=None):
    """
    解析使用哪个后端。
    优先级：环境变量 EMBED_BACKEND > .memory_config.json > 默认 openai
    """
    backend = os.environ.get("EMBED_BACKEND", "").lower()
    if not backend:
        cfg = _load_config(db_path) if db_path else {}
        backend = cfg.get("backend", "openai")

    if backend not in BACKENDS:
        raise ValueError(
            f"未知后端: {backend}，可用: {', '.join(BACKENDS.keys())}"
        )
    return backend


def _get_api_key(backend_key_env):
    """获取 API key，支持多个环境变量名"""
    for env_var in [backend_key_env, "OPENAI_API_KEY", "ANTHROPIC_API_KEY"]:
        key = os.environ.get(env_var)
        if key:
            return key
    return None


# ── 向量维度查询 ──────────────────────────────────────────

def get_embedding_dim(db_path=None, backend=None):
    """获取当前后端的向量维度（不调用 API）"""
    if backend is None:
        backend = _resolve_backend(db_path)
    return BACKENDS[backend]["dim"]


def get_backend_info(db_path=None):
    """获取当前后端的元信息"""
    backend = _resolve_backend(db_path)
    info = BACKENDS[backend].copy()
    info["backend"] = backend
    # 不暴露 key
    info.pop("key_env", None)
    return info


# ── Embed 调用 ────────────────────────────────────────────

def get_embedding(text: str, db_path: Optional[str] = None, backend: Optional[str] = None):
    """
    获取单条文本的 embedding 向量（numpy array, float32）
    """
    vecs = get_embeddings([text], db_path=db_path, backend=backend)
    return vecs[0]


def get_embeddings(texts: List[str], db_path: Optional[str] = None, backend: Optional[str] = None) -> List:
    """
    批量获取 embeddings。

    返回: List[numpy.ndarray]，每个向量 dtype=float32

    自动分批：按 backend["batch_size"] 分批调用 API。
    支持 OpenAI 兼容接口（siliconflow、zhipu 均可用 base_url 切换）。
    """
    if not texts:
        return []

    if np is None:
        raise RuntimeError("需要安装 numpy: pip install numpy")

    # 解析后端
    if backend is None:
        backend = _resolve_backend(db_path)
    cfg = BACKENDS[backend]

    [REDACTED_SECRET]"key_env"])
    if not api_key:
        raise RuntimeError(
            f"需要设置 API key：环境变量 {cfg['key_env']}，"
            f"或创建 .memory_config.json {{'backend': '{backend}'}}"
        )

    # 构建 client（OpenAI 兼容接口）
    if _OpenAI is None:
        raise RuntimeError("需要安装 openai: pip install openai")

    client_kwargs = {"api_key": api_key}
    if cfg["base_url"]:
        client_kwargs["base_url"] = cfg["base_url"]

    client = _OpenAI(**client_kwargs)

    all_embeddings = []
    batch_size = cfg["batch_size"]

    for i in range(0, len(texts), batch_size):
        batch = texts[i:i + batch_size]
        resp = client.embeddings.create(
            model=cfg["model"],
            input=batch,
        )
        for item in resp.data:
            vec = np.array(item.embedding, dtype=np.float32)
            # 归一化（余弦相似度搜索时更快）
            norm = np.linalg.norm(vec)
            if norm > 0:
                vec = vec / norm
            all_embeddings.append(vec)

        done = min(i + batch_size, len(texts))
        print(f"    embedded {done}/{len(texts)}...")

    return all_embeddings


def list_backends():
    """列出所有可用后端"""
    return {
        bid: {k: v for k, v in info.items() if k != "key_env"}
        for bid, info in BACKENDS.items()
    }


path: scripts/lancedb_integration.py
content: #!/usr/bin/env python3
"""
lancedb_integration.py - LanceDB 向量索引集成

功能：
  1. HNSW 索引加速向量搜索
  2. 查询缓存避免重复搜索
  3. 并行 embedding 加速写入

安装依赖：
  pip install lancedb  # ~50MB

配置：
  # 方式1：环境变量
  export LANCEDB_ENABLED=true
  export LANCEDB_INDEX_TYPE=HNSW
  
  # 方式2：代码配置
  from lancedb_integration import LanceDBManager
  manager = LanceDBManager(enabled=True, index_type="HNSW")
"""

import os
import sqlite3
import json
import time
import hashlib
import threading
from pathlib import Path
from concurrent.futures import ThreadPoolExecutor, as_completed
from datetime import datetime, timedelta
from typing import List, Tuple, Optional, Dict, Any

# LanceDB 懒加载（可选依赖）
LANCEDB_AVAILABLE = False
try:
    import lancedb
    import pyarrow as pa
    LANCEDB_AVAILABLE = True
except ImportError:
    lancedb = None
    pa = None

# ─────────────────────────────────────────────────────────────
# 配置
# ─────────────────────────────────────────────────────────────

# 是否启用 LanceDB（通过环境变量或代码控制）
ENABLED = os.environ.get("LANCEDB_ENABLED", "false").lower() == "true"

# HNSW 参数
HNSW_M = int(os.environ.get("HNSW_M", "16"))           # 每个节点的最大连接数
HNSW_EF_CONSTRUCTION = int(os.environ.get("HNSW_EF_CONSTRUCTION", "200"))  # 构建时候选集
HNSW_EF_SEARCH = int(os.environ.get("HNSW_EF_SEARCH", "100"))              # 搜索时候选集

# 并行 embedding 配置
PARALLEL_EMBED_ENABLED = os.environ.get("PARALLEL_EMBED", "true").lower() == "true"
PARALLEL_WORKERS = int(os.environ.get("PARALLEL_WORKERS", "4"))  # 并行线程数
EMBED_BATCH_SIZE = int(os.environ.get("EMBED_BATCH_SIZE", "20"))  # 每批大小

# 缓存配置
CACHE_ENABLED = os.environ.get("CACHE_ENABLED", "true").lower() == "true"
CACHE_TTL_SECONDS = int(os.environ.get("CACHE_TTL_SECONDS", "3600"))  # 缓存有效期
CACHE_DB_NAME = ".memory_search_cache.db"


# ─────────────────────────────────────────────────────────────
# LanceDB Manager
# ─────────────────────────────────────────────────────────────

class LanceDBManager:
    """
    LanceDB 向量索引管理器
    
    功能：
    - HNSW 索引加速搜索
    - 增量索引更新
    - 与 SQLite 向量库共存
    """
    
    def __init__(self, db_path: str, enabled: bool = None, index_type: str = "HNSW"):
        self.db_path = db_path
        self.enabled = enabled if enabled is not None else ENABLED
        self.index_type = index_type
        self.db = None
        self.table = None
        
        if self.enabled and not LANCEDB_AVAILABLE:
            print("[Warning] LanceDB not installed. Run: pip install lancedb")
            self.enabled = False
    
    def connect(self):
        """连接到 LanceDB"""
        if not self.enabled:
            return False
        
        try:
            # LanceDB 支持 SQLite 后端
            uri = f"sqlite:///{self.db_path}"
            self.db = lancedb.connect(uri)
            
            # 打开或创建表
            table_name = "memory_embeddings"
            if table_name in self.db.table_names():
                self.table = self.db.open_table(table_name)
            else:
                self.table = self._create_table()
            
            return True
        except Exception as e:
            print(f"[Warning] LanceDB connection failed: {e}")
            self.enabled = False
            return False
    
    def _create_table(self):
        """创建 LanceDB 表（含 HNSW 索引）"""
        # 定义 schema
        schema = pa.schema([
            pa.field("chunk_id", pa.string()),
            pa.field("uid", pa.string()),
            pa.field("scope", pa.string()),
            pa.field("level", pa.string()),
            pa.field("filepath", pa.string()),
            pa.field("section_idx", pa.int32()),
            pa.field("title", pa.string()),
            pa.field("semantic_key", pa.string()),
            pa.field("content", pa.string()),
            pa.field("raw_content", pa.string()),
            pa.field("content_hash", pa.string()),
            pa.field("mtime", pa.float64()),
            pa.field("created_at", pa.string()),
            pa.field("importance_score", pa.float32()),
            pa.field("related_ids", pa.string()),
            # 向量字段
            pa.field("vector", pa.list_(pa.float32())),
        ])
        
        tbl = self.db.create_table("memory_embeddings", schema=schema)
        
        # 创建 HNSW 索引
        tbl.create_index(
            vector_column_name="vector",
            index_type=self.index_type,
            metric="cosine",
            parameters={
                "M": HNSW_M,
                "efConstruction": HNSW_EF_CONSTRUCTION,
                "ef": HNSW_EF_SEARCH,
            }
        )
        
        return tbl
    
    def add_vectors(self, records: List[Dict]):
        """
        批量添加向量记录
        
        Args:
            records: List of dict with keys: chunk_id, uid, scope, level, 
                    filepath, title, semantic_key, content, vector, etc.
        """
        if not self.enabled or not self.table:
            return False
        
        try:
            self.table.add(records)
            return True
        except Exception as e:
            print(f"[Warning] LanceDB add failed: {e}")
            return False
    
    def search(self, query_vector: List[float], top_k: int = 10, 
                uid: str = None, scope: str = None, level: str = None,
                filter_str: str = None) -> List[Dict]:
        """
        HNSW 向量搜索
        
        Args:
            query_vector: 查询向量
            top_k: 返回数量
            uid/scope/level: 过滤条件
            
        Returns:
            搜索结果列表
        """
        if not self.enabled or not self.table:
            return []
        
        try:
            # 构建过滤条件
            where_parts = []
            if uid and uid != "*":
                where_parts.append(f"uid = '{uid}'")
            if scope and scope != "*":
                where_parts.append(f"scope = '{scope}'")
            if level:
                where_parts.append(f"level = '{level}'")
            where_clause = " AND ".join(where_parts) if where_parts else None
            
            # 执行搜索
            results = self.table.search(query_vector) \
                .limit(top_k) \
                .select(["chunk_id", "uid", "scope", "level", "filepath", 
                        "title", "semantic_key", "content", "importance_score", 
                        "related_ids"]) \
                .to_list()
            
            return results
        except Exception as e:
            print(f"[Warning] LanceDB search failed: {e}")
            return []
    
    def rebuild_index(self, sqlite_db_path: str):
        """
        从 SQLite 重建 LanceDB 索引
        
        Args:
            sqlite_db_path: SQLite 数据库路径
        """
        if not self.enabled:
            return False
        
        try:
            # 读取 SQLite 数据
            conn = sqlite3.connect(sqlite_db_path)
            c = conn.cursor()
            c.execute("SELECT chunk_id, uid, scope, level, filepath, section_idx, "
                     "title, semantic_key, content, raw_content, content_hash, mtime, "
                     "created_at, importance_score, related_ids, embedding "
                     "FROM memory_embeddings")
            
            records = []
            for row in c.fetchall():
                emb = row[15]
                # 转换 bytes 到 list
                import struct
                vec = list(struct.unpack('f' * (len(emb) // 4), emb))
                
                records.append({
                    "chunk_id": row[0],
                    "uid": row[1],
                    "scope": row[2],
                    "level": row[3],
                    "filepath": row[4],
                    "section_idx": row[5],
                    "title": row[6],
                    "semantic_key": row[7],
                    "content": row[8],
                    "raw_content": row[9],
                    "content_hash": row[10],
                    "mtime": row[11],
                    "created_at": row[12],
                    "importance_score": row[13] or 50.0,
                    "related_ids": row[14] or "[]",
                    "vector": vec,
                })
            conn.close()
            
            if not records:
                return True
            
            # 清空并重建
            self.table.delete()
            self.table.add(records)
            
            print(f"  [LanceDB] Indexed {len(records)} vectors")
            return True
        except Exception as e:
            print(f"[Warning] LanceDB rebuild failed: {e}")
            return False
    
    def close(self):
        """关闭连接"""
        if self.db:
            self.db.close()


# ─────────────────────────────────────────────────────────────
# 查询缓存
# ─────────────────────────────────────────────────────────────

class SearchCache:
    """
    SQLite 实现的查询缓存
    
    避免重复查询同一条搜索，节省 API 调用
    """
    
    def __init__(self, base_path: str):
        self.db_path = os.path.join(os.path.dirname(base_path), CACHE_DB_NAME)
        self._init_cache_db()
        self._lock = threading.Lock()
    
    def _init_cache_db(self):
        """初始化缓存数据库"""
        conn = sqlite3.connect(self.db_path)
        c = conn.cursor()
        c.execute("""
            CREATE TABLE IF NOT EXISTS search_cache (
                cache_key    TEXT PRIMARY KEY,
                query        TEXT NOT NULL,
                uid          TEXT,
                scope        TEXT,
                level        TEXT,
                results      TEXT NOT NULL,  -- JSON 序列化
                created_at   TEXT NOT NULL,
                hit_count    INTEGER DEFAULT 1
            )
        """)
        c.execute("CREATE INDEX IF NOT EXISTS idx_created ON search_cache(created_at)")
        conn.close()
    
    def _make_key(self, query: str, uid: str, scope: str, level: str) -> str:
        """生成缓存键"""
        raw = f"{query}|{uid}|{scope}|{level}"
        return hashlib.md5(raw.encode()).hexdigest()
    
    def get(self, query: str, uid: str = None, scope: str = None, 
            level: str = None) -> Optional[List[Dict]]:
        """获取缓存结果"""
        if not CACHE_ENABLED:
            return None
        
        key = self._make_key(query, uid, scope, level)
        
        with self._lock:
            conn = sqlite3.connect(self.db_path)
            c = conn.cursor()
            c.execute("SELECT results, created_at, hit_count FROM search_cache WHERE cache_key = ?", (key,))
            row = c.fetchone()
            
            if row:
                results_json, created_at, hit_count = row
                created = datetime.fromisoformat(created_at)
                age = (datetime.now() - created).total_seconds()
                
                if age < CACHE_TTL_SECONDS:
                    # 更新命中次数
                    c.execute("UPDATE search_cache SET hit_count = hit_count + 1 WHERE cache_key = ?", (key,))
                    conn.commit()
                    conn.close()
                    return json.loads(results_json)
                else:
                    # 过期，删除
                    c.execute("DELETE FROM search_cache WHERE cache_key = ?", (key,))
            
            conn.close()
            return None
    
    def set(self, query: str, results: List[Dict], 
            uid: str = None, scope: str = None, level: str = None):
        """设置缓存"""
        if not CACHE_ENABLED:
            return
        
        key = self._make_key(query, uid, scope, level)
        
        with self._lock:
            conn = sqlite3.connect(self.db_path)
            c = conn.cursor()
            c.execute("""
                INSERT OR REPLACE INTO search_cache 
                (cache_key, query, uid, scope, level, results, created_at, hit_count)
                VALUES (?, ?, ?, ?, ?, ?, ?, 1)
            """, (key, query, uid, scope, level, json.dumps(results, ensure_ascii=False),
                  datetime.now().isoformat()))
            conn.close()
    
    def clear_expired(self):
        """清理过期缓存"""
        if not CACHE_ENABLED:
            return
        
        cutoff = (datetime.now() - timedelta(seconds=CACHE_TTL_SECONDS)).isoformat()
        
        with self._lock:
            conn = sqlite3.connect(self.db_path)
            c = conn.cursor()
            c.execute("DELETE FROM search_cache WHERE created_at < ?", (cutoff,))
            deleted = c.rowcount
            conn.commit()
            conn.close()
        
        if deleted > 0:
            print(f"  [Cache] Cleared {deleted} expired entries")
    
    def stats(self) -> Dict[str, Any]:
        """获取缓存统计"""
        if not CACHE_ENABLED:
            return {"enabled": False}
        
        conn = sqlite3.connect(self.db_path)
        c = conn.cursor()
        c.execute("SELECT COUNT(*), SUM(hit_count) FROM search_cache")
        total, hits = c.fetchone()
        conn.close()
        
        return {
            "enabled": True,
            "total_entries": total or 0,
            "total_hits": hits or 0,
            "ttl_seconds": CACHE_TTL_SECONDS,
        }


# ─────────────────────────────────────────────────────────────
# 并行 Embedding
# ─────────────────────────────────────────────────────────────

def parallel_embed(texts: List[str], get_embeddings_func, 
                  workers: int = None, batch_size: int = None) -> List:
    """
    并行执行 embedding
    
    Args:
        texts: 文本列表
        get_embeddings_func: embedding 函数 (接受 list 返回 list)
        workers: 并行线程数
        batch_size: 每批大小
        
    Returns:
        向量列表
    """
    if not PARALLEL_EMBED_ENABLED or len(texts) < 10:
        # 数据量小，直接串行
        return get_embeddings_func(texts)
    
    workers = workers or PARALLEL_WORKERS
    batch_size = batch_size or EMBED_BATCH_SIZE
    
    # 分批
    batches = [texts[i:i+batch_size] for i in range(0, len(texts), batch_size)]
    
    results = [None] * len(batches)
    
    def process_batch(batch_idx, batch_texts):
        vectors = get_embeddings_func(batch_texts)
        return batch_idx, vectors
    
    with ThreadPoolExecutor(max_workers=workers) as executor:
        futures = {
            executor.submit(process_batch, i, batch): i 
            for i, batch in enumerate(batches)
        }
        
        for future in as_completed(futures):
            batch_idx, vectors = future.result()
            results[batch_idx] = vectors
    
    # 合并结果
    all_vectors = []
    for vectors in results:
        all_vectors.extend(vectors)
    
    return all_vectors


# ─────────────────────────────────────────────────────────────
# 搜索入口（整合 LanceDB + 缓存）
# ─────────────────────────────────────────────────────────────

def smart_search(query_vector: List[float], query_text: str,
                 lancedb_manager: LanceDBManager,
                 search_cache: SearchCache,
                 uid: str = None, scope: str = None, level: str = None,
                 top_k: int = 10, use_cache: bool = True) -> List[Dict]:
    """
    智能搜索：优先使用 LanceDB + 缓存
    
    搜索流程：
    1. 检查缓存 → 命中直接返回
    2. LanceDB HNSW 搜索 → 快速准确
    3. 回退到 SQLite 全表扫描
    """
    # 1. 缓存查询
    if use_cache and CACHE_ENABLED:
        cached = search_cache.get(query_text, uid, scope, level)
        if cached:
            return cached
    
    # 2. LanceDB 搜索
    if lancedb_manager.enabled:
        results = lancedb_manager.search(
            query_vector, top_k, uid, scope, level
        )
        if results:
            if use_cache and CACHE_ENABLED:
                search_cache.set(query_text, results, uid, scope, level)
            return results
    
    # 3. 回退到 SQLite（返回空，让调用方处理）
    return []


# ─────────────────────────────────────────────────────────────
# 便捷函数
# ─────────────────────────────────────────────────────────────

def get_lancedb_manager(db_path: str) -> LanceDBManager:
    """获取 LanceDB 管理器（懒加载）"""
    return LanceDBManager(db_path)


def get_search_cache(base_path: str) -> SearchCache:
    """获取搜索缓存"""
    return SearchCache(base_path)


if __name__ == "__main__":
    # 测试
    print("LanceDB Integration Module")
    print(f"  LanceDB available: {LANCEDB_AVAILABLE}")
    print(f"  Enabled: {ENABLED}")
    print(f"  Parallel embed: {PARALLEL_EMBED_ENABLED}")
    print(f"  Cache enabled: {CACHE_ENABLED}")
    
    if LANCEDB_AVAILABLE:
        print("\nHNSW Parameters:")
        print(f"  M: {HNSW_M}")
        print(f"  efConstruction: {HNSW_EF_CONSTRUCTION}")
        print(f"  efSearch: {HNSW_EF_SEARCH}")


path: scripts/memory_access_log.py
content: #!/usr/bin/env python3
"""
memory_access_log.py - 记忆访问日志，记录每次记忆读取以计算重要性评分

用法:
  python scripts/memory_access_log.py --uid pupper --chunk-id xxx    # 记录访问
  python scripts/memory_access_log.py --uid pupper --report          # 打印评分报告
  python scripts/memory_access_log.py --uid pupper --get-importance  # 输出 JSON（供其他脚本调用）

重要性评分公式:
  score = 50 + (access_count_normalized × 0.4 + recency_normalized × 0.4 + importance_manual × 0.2) × 50

依赖:
  pip install openai numpy
"""

import os
import sys
import sqlite3
import argparse
import json
from datetime import datetime, timezone

ACCESS_DB = ".memory_access_log.db"

# 全局默认重要性（来自手动标记）
MANUAL_IMPORTANCE = {
    "L3": 100,  # 永久记忆默认最高
    "L2": 60,   # 长期记忆
    "L1": 40,   # 临时记忆
}


def get_db_path(base_dir):
    return os.path.join(os.path.abspath(base_dir), ACCESS_DB)


def init_access_db(db_path):
    conn = sqlite3.connect(db_path)
    c = conn.cursor()
    c.execute("PRAGMA journal_mode=WAL")
    c.execute("""
        CREATE TABLE IF NOT EXISTS memory_access_log (
            id          INTEGER PRIMARY KEY AUTOINCREMENT,
            chunk_id    TEXT    NOT NULL,
            uid         TEXT    NOT NULL,
            access_time REAL    NOT NULL,  -- Unix timestamp
            access_type TEXT    NOT NULL,  -- 'read' | 'search' | 'write' | 'manual_pin'
            extra       TEXT    DEFAULT '{}'  -- JSON extra info
        )
    """)
    # 手动重要性标记表（用户可手动 pin 重要记忆）
    c.execute("""
        CREATE TABLE IF NOT EXISTS manual_importance (
            chunk_id    TEXT PRIMARY KEY,
            uid         TEXT    NOT NULL,
            score       REAL    NOT NULL,  -- 0-100
            note        TEXT    DEFAULT '',
            updated_at  REAL    NOT NULL
        )
    """)
    c.execute("""
        CREATE INDEX IF NOT EXISTS idx_chunk_time
        ON memory_access_log(chunk_id, access_time DESC)
    """)
    conn.commit()
    return conn


def log_access(db_path, chunk_id, uid, access_type="read", extra=None):
    """记录一次记忆访问"""
    conn = sqlite3.connect(db_path)
    c = conn.cursor()
    c.execute(
        "INSERT INTO memory_access_log (chunk_id, uid, access_time, access_type, extra) VALUES (?, ?, ?, ?, ?)",
        (chunk_id, uid, datetime.now(timezone.utc).timestamp(), access_type, json.dumps(extra or {}))
    )
    conn.commit()
    conn.close()


def get_access_stats(db_path, uid, chunk_ids=None):
    """
    获取 chunk_ids 的访问统计。
    返回: {chunk_id: {"access_count": N, "last_accessed": ts, "access_types": [...]}}
    """
    conn = sqlite3.connect(db_path)
    c = conn.cursor()

    result = {}
    target_ids = chunk_ids or []

    if target_ids:
        placeholders = ",".join(["?"] * len(target_ids))
        rows = c.execute(f"""
            SELECT chunk_id, COUNT(*), MAX(access_time),
                   GROUP_CONCAT(DISTINCT access_type) as types
            FROM memory_access_log
            WHERE uid = ? AND chunk_id IN ({placeholders})
            GROUP BY chunk_id
        """, [uid] + target_ids).fetchall()

        for chunk_id, count, last_ts, types in rows:
            result[chunk_id] = {
                "access_count": count,
                "last_accessed": last_ts,
                "access_types": types.split(",") if types else [],
            }

    conn.close()
    return result


def get_manual_importance(db_path, uid, chunk_ids=None):
    """获取手动重要性评分"""
    conn = sqlite3.connect(db_path)
    c = conn.cursor()

    result = {}
    target_ids = chunk_ids or []

    if target_ids:
        placeholders = ",".join(["?"] * len(target_ids))
        rows = c.execute(f"""
            SELECT chunk_id, score FROM manual_importance
            WHERE uid = ? AND chunk_id IN ({placeholders})
        """, [uid] + target_ids).fetchall()
        result = {cid: score for cid, score in rows}

    conn.close()
    return result


def set_manual_importance(db_path, chunk_id, uid, score, note=""):
    """手动设置/更新记忆的重要性"""
    conn = sqlite3.connect(db_path)
    c = conn.cursor()
    c.execute("""
        INSERT INTO manual_importance (chunk_id, uid, score, note, updated_at)
        VALUES (?, ?, ?, ?, ?)
        ON CONFLICT(chunk_id) DO UPDATE SET score=excluded.score, note=excluded.note, updated_at=excluded.updated_at
    """, (chunk_id, uid, score, note, datetime.now(timezone.utc).timestamp()))
    conn.commit()
    conn.close()


def calculate_importance_score(db_path, uid, chunk_id, level, base_score=50):
    """
    计算单条记忆的重要性评分（0-100）。

    公式:
      score = base + (freq_w × 0.4 + recency_w × 0.4 + manual_w × 0.2) × 50

    其中:
      - freq_w: 访问频率标准化 (min-max, 0-1)
      - recency_w: 最近访问时间标准化 (指数衰减, 0-1)
      - manual_w: 手动标记分数 (0-1, 来自 manual_importance 表)
    """
    import math

    conn = sqlite3.connect(db_path)
    c = conn.cursor()

    now = datetime.now(timezone.utc).timestamp()

    # 获取该用户所有 chunk 的访问情况（用于标准化）
    rows = c.execute("""
        SELECT chunk_id, COUNT(*), MAX(access_time)
        FROM memory_access_log
        WHERE uid = ?
        GROUP BY chunk_id
    """, (uid,)).fetchall()

    if not rows:
        conn.close()
        return base_score + MANUAL_IMPORTANCE.get(level, 50) * 0.2

    counts = {cid: cnt for cid, cnt, _ in rows}
    last_times = {cid: ts for cid, _, ts in rows}

    all_counts = list(counts.values())
    max_count = max(all_counts) if all_counts else 1
    min_count = min(all_counts) if all_counts else 0

    count_range = max(max_count - min_count, 1)

    # 访问频率标准化
    freq_w = (counts.get(chunk_id, 0) - min_count) / count_range

    # 最近访问时间：指数衰减，半衰期 7 天
    last_ts = last_times.get(chunk_id, 0)
    if last_ts > 0:
        age_days = (now - last_ts) / 86400
        recency_w = math.exp(-age_days / 7)  # 0-1, 越新越接近 1
    else:
        recency_w = 0

    # 手动重要性
    row = c.execute(
        "SELECT score FROM manual_importance WHERE chunk_id = ? AND uid = ?",
        (chunk_id, uid)
    ).fetchone()
    manual_w = (row[0] / 100) if row else (MANUAL_IMPORTANCE.get(level, 50) / 100)

    conn.close()

    score = base_score + (freq_w * 0.4 + recency_w * 0.4 + manual_w * 0.2) * 50
    return min(100.0, max(0.0, score))


def get_all_importance_scores(db_path, uid):
    """
    返回该用户所有 chunk 的重要性评分字典。
    {chunk_id: {"score": float, "access_count": int, "last_accessed": float}}
    """
    conn = sqlite3.connect(db_path)
    c = conn.cursor()

    rows = c.execute("""
        SELECT chunk_id, COUNT(*) as cnt, MAX(access_time) as last_ts
        FROM memory_access_log
        WHERE uid = ?
        GROUP BY chunk_id
    """, (uid,)).fetchall()

    manual_rows = dict(c.execute(
        "SELECT chunk_id, score FROM manual_importance WHERE uid = ?", (uid,)
    ).fetchall())

    conn.close()

    result = {}
    for chunk_id, cnt, last_ts in rows:
        # 从 vector DB 获取 level（需要传入 base_dir）
        result[chunk_id] = {
            "access_count": cnt,
            "last_accessed": last_ts,
            "manual_score": manual_rows.get(chunk_id),
        }

    return result


def print_report(db_path, uid, top_n=20):
    """打印访问报告"""
    conn = sqlite3.connect(db_path)
    c = conn.cursor()

    print(f"\nfrom typing import List, Dict, Optional, Union, Any, Tuple\n📊 记忆访问报告 — {uid}")
    print("=" * 50)

    # 访问最多
    rows = c.execute("""
        SELECT chunk_id, COUNT(*) as cnt, MAX(access_time)
        FROM memory_access_log
        WHERE uid = ?
        GROUP BY chunk_id
        ORDER BY cnt DESC
        LIMIT ?
    """, (uid, top_n)).fetchall()

    print(f"\n🔥 最活跃记忆 (Top {top_n}):")
    print(f"  {'排名':<4} {'chunk_id':<20} {'访问次数':>8} {'最后访问':>12}")
    print("  " + "-" * 48)
    for i, (cid, cnt, last_ts) in enumerate(rows, 1):
        ts = datetime.fromtimestamp(last_ts, tz=timezone.utc).strftime("%m-%d %H:%M") if last_ts else "从未"
        print(f"  {i:<4} {cid:<20} {cnt:>8} {ts:>12}")

    # 总访问量
    total = c.execute(
        "SELECT COUNT(*) FROM memory_access_log WHERE uid = ?", (uid,)
    ).fetchone()[0]
    print(f"\n📈 总访问次数: {total}")

    conn.close()


def main():
    parser = argparse.ArgumentParser(description="记忆访问日志与重要性评分")
    parser.add_argument("--base-dir", default=".", help="记忆仓库根目录")
    parser.add_argument("--uid", help="用户ID")
    parser.add_argument("--chunk-id", help="记忆块 ID（记录访问时用）")
    parser.add_argument("--access-type", default="read",
                        choices=["read", "search", "write", "manual_pin"],
                        help="访问类型")
    parser.add_argument("--report", action="store_true", help="打印访问报告")
    parser.add_argument("--get-importance", action="store_true",
                        help="输出 JSON 重要性评分（供其他脚本调用）")
    parser.add_argument("--set-importance", type=float, metavar="SCORE",
                        help="手动设置重要性 (0-100)")
    parser.add_argument("--set-note", default="", help="手动设置备注")
    args = parser.parse_args()

    uid = args.uid or os.environ.get("MEMORY_USER_ID", "default")
    db_path = get_db_path(args.base_dir)

    if not os.path.exists(db_path):
        init_access_db(db_path)

    if args.report:
        print_report(db_path, uid)
        return

    if args.get_importance:
        # 输出 JSON（从向量库读取 level）
        import sqlite3 as sqlite_internal
        vec_db = os.path.join(os.path.abspath(args.base_dir), ".memory_vectors.db")
        scores = get_all_importance_scores(db_path, uid)

        if os.path.exists(vec_db):
            vconn = sqlite_internal.connect(vec_db)
            vconn.execute("PRAGMA journal_mode=WAL")
            vcur = vconn.cursor()
            for cid in scores:
                row = vcur.execute(
                    "SELECT level FROM memory_embeddings WHERE chunk_id = ? AND uid = ?",
                    (cid, uid)
                ).fetchone()
                if row:
                    scores[cid]["level"] = row[0]
                    scores[cid]["score"] = calculate_importance_score(
                        db_path, uid, cid, row[0]
                    )
            vconn.close()

        print(json.dumps(scores))
        return

    if args.set_importance is not None:
        if not args.chunk_id:
            print("❌ --set-importance 需要 --chunk-id")
            sys.exit(1)
        set_manual_importance(db_path, args.chunk_id, uid, args.set_importance, args.set_note)
        print(f"✅ 已设置 {args.chunk_id} 重要性 = {args.set_importance}")
        return

    if args.chunk_id:
        log_access(db_path, args.chunk_id, uid, args.access_type)
        print(f"✅ 记录访问: {args.chunk_id} ({args.access_type})")
        return

    print("❌ 请指定操作: --report / --get-importance / --chunk-id <id>")
    parser.print_help()


if __name__ == "__main__":
    main()


path: scripts/memory_compress.py
content: #!/usr/bin/env python3
"""
memory_compress.py - 智能压缩记忆文件，防止上下文过长

用法:
  python scripts/memory_compress.py --uid pupper --level ALL      # 压缩所有层级
  python scripts/memory_compress.py --uid pupper --dry-run       # 仅分析，不实际压缩
  python scripts/memory_compress.py --uid pupper --upgrade        # 主题重复性升级（L1→L2）
  python scripts/memory_compress.py --uid pupper --undo          # 回滚到最近一次操作

选项:
  --uid               用户ID（默认 MEMORY_USER_ID 环境变量或 "default"）
  --level             压缩层级（默认: ALL，检查所有层级）
  --dry-run           仅分析，不实际压缩
  --threshold-l1      L1 压缩触发行数（默认: 150）
  --threshold-l2      L2 压缩触发行数（默认: 200）
  --archive-days      超过 N 天归档 L1 文件（默认: 30）
  --upgrade           执行 L1→L2→L3 主题升级
  --rebase-threshold  同一主题出现 N 次触发升级（默认: 3）
  --undo              回滚最近一次压缩/归档/升级操作（从快照恢复）
  --undo-id           回滚指定快照 ID（默认: 最近一次）
  --list-snapshots    列出所有可用的快照
  --keep-snapshots N  快照保留天数（默认: 7）
"""

import os
import re
import sys
import json
import shutil
import argparse
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import List, Dict, Optional, Union, Any, Tuple


COMPRESS_HEADER = (
    "<!-- [已压缩] {date} | 原始: {original}行/{orig_entries}条 "
    "→ 压缩后: {compressed}行/{comp_entries}条 | 压缩率: {ratio:.0%} -->\n\n"
)

UPGRADE_L2_RE = re.compile(r"\[IMPORTANT\]|\[重要\]|\[WEEKLY\]|\[升级L2\]", re.IGNORECASE)
UPGRADE_L3_RE = re.compile(r"\[IMPORTANT\]|\[重要\]|\[升级L3\]|\[PERMANENT\]", re.IGNORECASE)
HABIT_RE = re.compile(r"\[HABIT\]|\[习惯\]", re.IGNORECASE)
TODO_RE = re.compile(r"^- \[ \] .+$", re.MULTILINE)


# ══════════════════════════════════════════════════════════════
#  快照系统
# ══════════════════════════════════════════════════════════════

def get_snapshots_dir(base_dir):
    return os.path.join(os.path.abspath(base_dir), ".snapshots")


def create_snapshot(base_dir, uid, operation_type, description="", files_changed=None):
    """
    创建操作前快照，返回 snapshot_id。
    files_changed: [{"path": str, "content": str, "mtime": float}]
    """
    snap_dir = get_snapshots_dir(base_dir)
    date_str = datetime.now().strftime("%Y-%m-%d")
    snap_subdir = os.path.join(snap_dir, date_str)
    os.makedirs(snap_subdir, exist_ok=True)

    snapshot_id = datetime.now().strftime("%H%M%S")  # HHMMSS 作为 ID
    snap_data = {
        "snapshot_id": snapshot_id,
        "created_at": datetime.now(timezone.utc).isoformat(),
        "uid": uid,
        "operation_type": operation_type,
        "description": description,
        "files": files_changed or [],
    }

    snap_path = os.path.join(snap_subdir, f"{snapshot_id}.json")
    with open(snap_path, "w", encoding="utf-8") as f:
        json.dump(snap_data, f, ensure_ascii=False, indent=2)

    print(f"  💾 快照已创建: .snapshots/{date_str}/{snapshot_id}.json "
          f"({len(files_changed or [])} 个文件)")
    return snapshot_id


def list_snapshots(base_dir, uid=None, keep_days=7):
    """列出所有快照，删除过期快照"""
    snap_dir = get_snapshots_dir(base_dir)
    if not os.path.exists(snap_dir):
        return []

    # 清理过期快照
    cutoff = datetime.now() - timedelta(days=keep_days)
    all_snapshots = []

    for date_dir in os.listdir(snap_dir):
        date_path = os.path.join(snap_dir, date_dir)
        if not os.path.isdir(date_path):
            continue

        try:
            dir_date = datetime.strptime(date_dir, "%Y-%m-%d")
        except ValueError:
            continue

        if dir_date < cutoff:
            shutil.rmtree(date_path)
            continue

        for fname in os.listdir(date_path):
            if not fname.endswith(".json"):
                continue
            fpath = os.path.join(date_path, fname)
            with open(fpath, "r", encoding="utf-8") as f:
                data = json.load(f)
            if uid and data.get("uid") != uid:
                continue
            data["_file_path"] = fpath
            all_snapshots.append(data)

    all_snapshots.sort(key=lambda x: x["created_at"], reverse=True)
    return all_snapshots


def undo_snapshot(base_dir, snapshot_id=None):
    """从快照恢复文件"""
    snapshots = list_snapshots(base_dir)
    if not snapshots:
        print("❌ 无可用快照")
        return False

    if snapshot_id:
        target = next((s for s in snapshots if s["snapshot_id"] == snapshot_id), None)
        if not target:
            print(f"❌ 未找到快照: {snapshot_id}")
            return False
    else:
        target = snapshots[0]

    print(f"\n↩️  回滚操作")
    print(f"   快照 ID: {target['snapshot_id']}")
    print(f"   创建时间: {target['created_at']}")
    print(f"   操作类型: {target['operation_type']}")
    print(f"   描述: {target['description'] or '(无)'}")
    print(f"   包含文件: {len(target['files'])} 个")

    restored = 0
    for file_entry in target["files"]:
        fpath = file_entry["path"]
        if not os.path.exists(fpath):
            print(f"  ⚠️  文件已不存在，跳过: {fpath}")
            continue

        with open(fpath, "w", encoding="utf-8") as f:
            f.write(file_entry["content"])

        # 恢复 mtime
        if "mtime" in file_entry:
            try:
                os.utime(fpath, (file_entry["mtime"], file_entry["mtime"]))
            except OSError:
                pass

        print(f"  ✅ 已恢复: {os.path.relpath(fpath, base_dir)}")
        restored += 1

    print(f"\n✅ 回滚完成，恢复了 {restored} 个文件")
    print(f"   快照文件: {target['_file_path']}（保留，可手动删除）")
    return True


# ══════════════════════════════════════════════════════════════
#  核心压缩逻辑
# ══════════════════════════════════════════════════════════════

def backup_file(filepath):
    """创建文件备份到 .compress_backup/"""
    backup_dir = os.path.join(os.path.dirname(filepath), ".compress_backup")
    os.makedirs(backup_dir, exist_ok=True)
    fname = os.path.basename(filepath)
    ts = datetime.now().strftime("%Y%m%d_%H%M%S")
    backup_path = os.path.join(backup_dir, f"{fname}.{ts}.bak")
    shutil.copy2(filepath, backup_path)
    return backup_path


def parse_sections(content):
    sections = []
    parts = re.split(r"(?=^## )", content, flags=re.MULTILINE)

    for part in parts:
        part = part.strip()
        if not part:
            continue
        lines = part.split("\n")
        title = lines[0].lstrip("#").strip() if lines else ""
        body = "\n".join(lines[1:]).strip() if len(lines) > 1 else ""

        sections.append({
            "title": title,
            "body": body,
            "raw": part,
            "has_todo": bool(TODO_RE.search(part)),
            "is_important_l2": bool(UPGRADE_L2_RE.search(part)),
            "is_important_l3": bool(UPGRADE_L3_RE.search(part)),
            "is_habit": bool(HABIT_RE.search(part)),
            "line_count": len(part.split("\n")),
        })
    return sections


def compress_l1_section(section):
    body = section["body"]
    todos = TODO_RE.findall(body)
    key_lines = []
    for line in body.split("\n"):
        stripped = line.strip()
        if not stripped:
            continue
        if (
            stripped.startswith("**")
            or stripped.startswith("- ")
            or stripped.startswith("- [")
            or stripped.startswith("#")
            or re.match(r"^\*\*[^\*]*\*\*(?:$|:)", stripped)
        ):
            key_lines.append(line)
    compressed_body = "\n".join(key_lines)
    if todos and "**待办" not in compressed_body:
        compressed_body += "\n**待办（保留）**:\n" + "\n".join(todos)
    return compressed_body


def compress_l2_section(section):
    body = section["body"]
    title = section["title"]
    todos = TODO_RE.findall(body)
    preserve_full = any(k in title for k in ["决策", "里程碑", "决定", "decision", "milestone"])
    if preserve_full or todos:
        return body
    key_lines = [
        line for line in body.split("\n")
        if line.strip() and (line.strip().startswith("**")
                               or line.strip().startswith("- ")
                               or line.strip().startswith("- ["))
    ]
    compressed_body = "\n".join(key_lines)
    if key_lines:
        compressed_body += "\n_(已压缩详情)_"
    return compressed_body


def compress_file(filepath, threshold=150, dry_run=False, level="L1",
                  snapshot_context=None):
    """
    压缩单个记忆文件。
    snapshot_context: [files_changed_list] — append file info here for snapshot
    返回 (是否压缩了, 升级条目列表)
    """
    with open(filepath, "r", encoding="utf-8") as f:
        content = f.read()

    original_lines = len(content.split("\n"))
    if original_lines <= threshold:
        return False, []

    sections = parse_sections(content)
    orig_entries = len(sections)

    preserved = []
    compressed = []
    upgraded_l2 = []
    upgraded_l3 = []
    habit_entries = []

    for s in sections:
        if s["has_todo"] or s["is_important_l3"]:
            preserved.append(s["raw"])
        elif s["is_important_l2"]:
            compressed.append(f"## {s['title']}\n{compress_l1_section(s)}")
            upgraded_l2.append(s)
        elif s["is_habit"]:
            habit_entries.append(s)
            compressed.append(f"## {s['title']}\n_(已提取到习惯追踪)_")
        else:
            comp = compress_l2_section(s) if level == "L2" else compress_l1_section(s)
            compressed.append(f"## {s['title']}\n{comp}")

    all_sections = preserved + compressed
    new_content = "\n\n---\n\n".join(all_sections)
    compressed_lines = len(new_content.split("\n"))
    ratio = (original_lines - compressed_lines) / original_lines if original_lines > 0 else 0
    comp_entries = len(all_sections)

    header = COMPRESS_HEADER.format(
        date=datetime.now().strftime("%Y-%m-%d %H:%M"),
        original=original_lines,
        orig_entries=orig_entries,
        compressed=compressed_lines,
        comp_entries=comp_entries,
        ratio=ratio,
    )
    final_content = header + new_content

    if not dry_run:
        # 记录快照
        if snapshot_context is not None:
            import os as _os
            stat = _os.stat(filepath)
            snapshot_context.append({
                "path": filepath,
                "content": content,
                "mtime": stat.st_mtime,
            })
        backup = backup_file(filepath)
        with open(filepath, "w", encoding="utf-8") as f:
            f.write(final_content)
        print(f"  ✅ {Path(filepath).name}: {original_lines}行/{orig_entries}条 "
              f"→ {compressed_lines}行/{comp_entries}条 ({ratio:.0%})")
        print(f"     💾 备份: {Path(backup).name}")
    else:
        print(f"  [DRY-RUN] {Path(filepath).name}: {original_lines}行 "
              f"→ {compressed_lines}行 ({ratio:.0%})")

    return True, upgraded_l2 + upgraded_l3


def upgrade_repeated_themes(base_dir, uid, threshold=3, dry_run=False,
                             snapshot_context=None):
    """发现同一主题在多个 L1 文件中重复出现 → 升级到 L2"""
    print(f"\n🔍 主题重复性检查（阈值: {threshold} 次）...")

    daily_dir = os.path.join(base_dir, "users", uid, "daily")
    if not os.path.exists(daily_dir):
        print("  (无 L1 文件)")
        return

    theme_counts = {}
    theme_examples = {}

    for fname in os.listdir(daily_dir):
        if not fname.endswith(".md"):
            continue
        with open(os.path.join(daily_dir, fname), "r", encoding="utf-8") as f:
            content = f.read()
        titles = re.findall(r"^## (.+)$", content, re.MULTILINE)
        for t in titles:
            clean = re.sub(r"^\[\d{2}:\d{2}\]\s*", "", t).strip().lower()
            if len(clean) < 3:
                continue
            theme_counts[clean] = theme_counts.get(clean, 0) + 1
            theme_examples.setdefault(clean, []).append(t)

    upgrades = {k: v for k, v in theme_counts.items() if v >= threshold}

    if not upgrades:
        print("  ✅ 无重复主题，无需升级")
        return

    print(f"  📋 发现 {len(upgrades)} 个重复主题:")
    for theme, count in sorted(upgrades.items(), key=lambda x: -x[1]):
        print(f"     [{count}次] {theme}")

    if dry_run:
        print("  [DRY-RUN] 跳过实际升级")
        return

    weekly_dir = os.path.join(base_dir, "users", uid, "weekly")
    this_week = datetime.now().strftime("%Y-W%V") + ".md"
    weekly_path = os.path.join(weekly_dir, this_week)

    # 快照：记录原周报内容
    original_weekly_content = ""
    weekly_existed = os.path.exists(weekly_path)
    if weekly_existed:
        with open(weekly_path, "r", encoding="utf-8") as f:
            original_weekly_content = f.read()

    upgrade_block = f"\n---\n\n## 主题升级（{datetime.now().strftime('%Y-%m-%d')}）\n\n"
    for theme, count in sorted(upgrades.items(), key=lambda x: -x[1]):
        examples = theme_examples[theme][:5]
        upgrade_block += f"### [{theme.title()}] 出现 {count} 次\n"
        upgrade_block += f"- **主题**: {theme}\n"
        upgrade_block += f"- **出现次数**: {count} 次\n"
        upgrade_block += f"- **示例**: {' | '.join(examples[:3])}\n\n"

    if weekly_existed:
        new_weekly_content = original_weekly_content + upgrade_block
    else:
        os.makedirs(weekly_dir, exist_ok=True)
        new_weekly_content = f"# Weekly Memory — {this_week.replace('.md', '')}\n\n{upgrade_block}"

    # 记录快照
    if snapshot_context is not None and weekly_existed:
        import os as _os
        stat = _os.stat(weekly_path)
        snapshot_context.append({
            "path": weekly_path,
            "content": original_weekly_content,
            "mtime": stat.st_mtime,
        })

    with open(weekly_path, "w", encoding="utf-8") as f:
        f.write(new_weekly_content)

    print(f"  ✅ 主题升级写入: users/{uid}/weekly/{this_week}")


def archive_old_l1(base_dir, uid, days=30, dry_run=False, snapshot_context=None):
    """归档超过 N 天的 L1 文件到 archive/daily/"""
    daily_dir = os.path.join(base_dir, "users", uid, "daily")
    archive_dir = os.path.join(base_dir, "users", uid, "archive", "daily")
    cutoff = datetime.now() - timedelta(days=days)

    if not os.path.exists(daily_dir):
        return 0

    archived = 0
    for fname in os.listdir(daily_dir):
        if not fname.endswith(".md"):
            continue
        try:
            file_date = datetime.strptime(fname.replace(".md", ""), "%Y-%m-%d")
            if file_date < cutoff:
                src = os.path.join(daily_dir, fname)
                if not dry_run:
                    # 记录快照（保存原位置的内容）
                    if snapshot_context is not None:
                        with open(src, "r", encoding="utf-8") as f:
                            content = f.read()
                        snapshot_context.append({
                            "path": src,
                            "content": content,
                            "mtime": os.path.getmtime(src),
                            "_archive_dest": os.path.join(archive_dir, fname),
                        })
                    os.makedirs(archive_dir, exist_ok=True)
                    shutil.move(src, os.path.join(archive_dir, fname))
                    print(f"  📦 归档: {fname} → archive/daily/")
                else:
                    print(f"  [DRY-RUN] 将归档: {fname}")
                archived += 1
        except ValueError:
            pass

    return archived


def main():
    parser = argparse.ArgumentParser(description="智能压缩记忆文件（支持快照+回滚）")
    parser.add_argument("--base-dir", default=".", help="记忆仓库根目录")
    parser.add_argument("--uid", help="用户ID（默认: MEMORY_USER_ID 或 'default'）")
    parser.add_argument("--level", default="ALL", choices=["L1", "L2", "L3", "ALL"],
                        help="压缩层级")
    parser.add_argument("--dry-run", action="store_true", help="仅分析，不实际压缩")
    parser.add_argument("--threshold-l1", type=int, default=150, help="L1 压缩触发行数")
    parser.add_argument("--threshold-l2", type=int, default=200, help="L2 压缩触发行数")
    parser.add_argument("--archive-days", type=int, default=30, help="归档超过N天的L1文件")
    parser.add_argument("--upgrade", action="store_true",
                        help="执行主题重复性升级（L1→L2）")
    parser.add_argument("--rebase-threshold", type=int, default=3,
                        help="同一主题出现N次触发升级")
    parser.add_argument("--undo", action="store_true", help="回滚到最近一次操作")
    parser.add_argument("--undo-id", help="回滚指定快照 ID")
    parser.add_argument("--list-snapshots", action="store_true", help="列出所有快照")
    parser.add_argument("--keep-snapshots", type=int, default=7, help="快照保留天数")
    args = parser.parse_args()

    base_dir = os.path.abspath(args.base_dir)
    uid = args.uid or os.environ.get("MEMORY_USER_ID", "default")

    # ── 快照列表 ──
    if args.list_snapshots:
        snapshots = list_snapshots(base_dir, uid, args.keep_snapshots)
        if not snapshots:
            print("📦 暂无快照")
            return
        print(f"📦 可用快照（共 {len(snapshots)} 个，保留 {args.keep_snapshots} 天）:\n")
        for s in snapshots[:20]:
            ts = s["created_at"][:19].replace("T", " ")
            print(f"  [{s['snapshot_id']}] {ts} | {s['operation_type']} | "
                  f"{len(s['files'])} 文件 | {s.get('description', '') or '无描述'}")
        return

    # ── 回滚 ──
    if args.undo:
        undo_snapshot(base_dir, args.undo_id)
        return

    # ── 普通压缩 ──
    user_dir = os.path.join(base_dir, "users", uid)
    if not os.path.exists(user_dir):
        print(f"❌ 用户目录不存在: users/{uid}/")
        print("   请先运行: python scripts/memory_init.py --uid " + uid)
        sys.exit(1)

    print(f"\n🗜️  记忆压缩工具（用户: {uid}）")
    print(f"📁 仓库根目录: {base_dir}")
    print(f"📅 快照保留: {args.keep_snapshots} 天")
    if args.dry_run:
        print("🔍 模式: DRY-RUN（仅分析）\n")
    else:
        print("⚡ 模式: 实际压缩\n")

    # 创建操作快照上下文
    snapshot_context = [] if not args.dry_run else None

    compressed_count = 0

    if args.level in ("L1", "ALL"):
        daily_dir = os.path.join(user_dir, "daily")
        print(f"📋 检查 L1 临时记忆（阈值: {args.threshold_l1}行）...")
        if os.path.exists(daily_dir):
            files = sorted([f for f in os.listdir(daily_dir) if f.endswith(".md")], reverse=True)
            if files:
                for fname in files:
                    fpath = os.path.join(daily_dir, fname)
                    if compress_file(fpath, args.threshold_l1, args.dry_run, "L1",
                                     snapshot_context):
                        compressed_count += 1
            else:
                print("  (无文件)")
        else:
            print("  (目录不存在)")

        if not args.dry_run:
            print(f"\n📦 归档超过 {args.archive_days} 天的 L1 文件...")
            archived = archive_old_l1(base_dir, uid, args.archive_days, args.dry_run,
                                      snapshot_context)
            if archived == 0:
                print("  (无需归档)")

    if args.level in ("L2", "ALL"):
        weekly_dir = os.path.join(user_dir, "weekly")
        print(f"\n📋 检查 L2 长期记忆（阈值: {args.threshold_l2}行）...")
        if os.path.exists(weekly_dir):
            files = sorted([f for f in os.listdir(weekly_dir) if f.endswith(".md")], reverse=True)
            if files:
                for fname in files:
                    fpath = os.path.join(weekly_dir, fname)
                    if compress_file(fpath, args.threshold_l2, args.dry_run, "L2",
                                     snapshot_context):
                        compressed_count += 1
            else:
                print("  (无文件)")
        else:
            print("  (目录不存在)")

    if args.upgrade:
        upgrade_repeated_themes(base_dir, uid, args.rebase_threshold, args.dry_run,
                               snapshot_context)

    # 创建快照（仅在实际操作且有文件变更时）
    if compressed_count > 0 and not args.dry_run and snapshot_context:
        create_snapshot(base_dir, uid, "compress",
                       f"压缩 {compressed_count} 个文件",
                       snapshot_context)

    print(f"\n{'✅' if compressed_count > 0 else '📊'} 完成！共压缩 {compressed_count} 个文件。")

    if compressed_count > 0 and not args.dry_run:
        print("\n💡 如需回滚: python scripts/memory_compress.py --uid " + uid + " --undo")
        print("💡 查看快照: python scripts/memory_compress.py --uid " + uid + " --list-snapshots")


if __name__ == "__main__":
    main()


path: scripts/memory_embed.py
content: #!/usr/bin/env python3
"""
memory_embed.py - 为记忆条目生成语义向量嵌入并存入 SQLite 向量库

用法:
  python scripts/memory_embed.py --uid pupper --rebuild        # 全量重建
  python scripts/memory_embed.py --uid pupper                  # 增量更新
  python scripts/memory_embed.py --uid pupper --scope shared   # 仅公共记忆
  python scripts/memory_embed.py --uid pupper --dry-run        # 预览不写入
  python scripts/memory_embed.py --uid pupper --delete         # 删除该用户所有向量
  python scripts/memory_embed.py --uid pupper --rebuild-index  # 重建 LanceDB 索引
  python scripts/memory_embed.py --list-backends              # 列出可用后端

后端配置:
  # 方式1：环境变量
  export EMBED_BACKEND=siliconflow        # openai / siliconflow / zhipu
  export SILICONFLOW_KEY=sk-xxx           # 对应后端的 API key

  # 方式2：配置文件（与 .memory_vectors.db 同目录）
  # .memory_config.json: {"backend": "siliconflow"}

依赖:
  pip install openai numpy lancedb  # lancedb 可选，增强搜索性能

优化特性:
  - 并行 embedding（多线程批量处理）
  - LanceDB 索引自动更新
"""

import os
import re
import sys
import json
import sqlite3
import hashlib
import argparse
import time
from datetime import datetime
from pathlib import Path
from concurrent.futures import ThreadPoolExecutor, as_completed

# 统一 embedding 后端（确保同目录下的模块可被导入）
import sys as _sys
_sys.path.insert(0, str(Path(__file__).parent))
from embed_backends import get_embeddings, get_embedding_dim, get_backend_info

DB_NAME = ".memory_vectors.db"
CFG_NAME = ".memory_config.json"


# ══════════════════════════════════════════════════════════════
#  并行 Embedding
# ══════════════════════════════════════════════════════════════

# 并行配置
PARALLEL_WORKERS = int(os.environ.get("PARALLEL_WORKERS", "4"))
EMBED_BATCH_SIZE = int(os.environ.get("EMBED_BATCH_SIZE", "20"))


def parallel_embed(texts: list, get_embeddings_func, workers: int = None, batch_size: int = None):
    """
    并行执行 embedding（多线程批量处理）
    
    Args:
        texts: 文本列表
        get_embeddings_func: embedding 函数 (接受 list 返回 list)
        workers: 并行线程数
        batch_size: 每批大小
        
    Returns:
        向量列表
        
    性能提升:
        - 4线程: ~2-3x 速度提升
        - 适合大批量数据 (>100 条)
    """
    if len(texts) < 10:
        # 数据量小，直接串行
        return get_embeddings_func(texts)
    
    workers = workers or PARALLEL_WORKERS
    batch_size = batch_size or EMBED_BATCH_SIZE
    
    # 分批
    batches = [texts[i:i+batch_size] for i in range(0, len(texts), batch_size)]
    
    print(f"  [Parallel] 分 {len(batches)} 批处理，每批 {batch_size} 条，{workers} 线程")
    
    results = [None] * len(batches)
    
    def process_batch(batch_idx, batch_texts):
        start = time.time()
        vectors = get_embeddings_func(batch_texts)
        elapsed = time.time() - start
        return batch_idx, vectors, elapsed
    
    batch_times = []
    with ThreadPoolExecutor(max_workers=workers) as executor:
        futures = {
            executor.submit(process_batch, i, batch): i 
            for i, batch in enumerate(batches)
        }
        
        for future in as_completed(futures):
            batch_idx, vectors, elapsed = future.result()
            results[batch_idx] = vectors
            batch_times.append(elapsed)
    
    # 合并结果
    all_vectors = []
    for vectors in results:
        all_vectors.extend(vectors)
    
    total_time = sum(batch_times)
    avg_time = total_time / len(batch_times) if batch_times else 0
    print(f"  [Parallel] 完成 {len(texts)} 条，总耗时 {total_time:.1f}s，平均每批 {avg_time:.2f}s")
    
    return all_vectors


# ══════════════════════════════════════════════════════════════
#  安全工具
# ══════════════════════════════════════════════════════════════

def safe_relpath(path, base):
    """安全获取相对路径，防止路径穿越攻击"""
    try:
        abspath = Path(path).resolve()
        basepath = Path(base).resolve()
        rel = abspath.relative_to(basepath)
        if str(rel).startswith(".."):
            return None
        return str(rel)
    except (ValueError, OSError):
        return None


# ══════════════════════════════════════════════════════════════
#  向量数据库层（SQLite + numpy）
# ══════════════════════════════════════════════════════════════

def get_db_path(base_dir):
    return os.path.join(os.path.abspath(base_dir), DB_NAME)


def get_cfg_path(base_dir):
    return os.path.join(os.path.abspath(base_dir), CFG_NAME)


def init_db(db_path):
    """初始化向量数据库表（含迁移：自动添加新列）"""
    conn = sqlite3.connect(db_path)
    c = conn.cursor()
    c.execute("PRAGMA journal_mode=WAL")

    # 新建表（首次）
    c.execute("""
        CREATE TABLE IF NOT EXISTS memory_embeddings (
            id             INTEGER PRIMARY KEY AUTOINCREMENT,
            chunk_id       TEXT    UNIQUE NOT NULL,
            uid            TEXT    NOT NULL,
            scope          TEXT    NOT NULL,
            level          TEXT    NOT NULL,
            filepath       TEXT    NOT NULL,
            section_idx    INTEGER NOT NULL,
            title          TEXT    NOT NULL,
            semantic_key   TEXT    NOT NULL,
            content        TEXT    NOT NULL,
            raw_content    TEXT    NOT NULL,
            embedding      BLOB    NOT NULL,
            content_hash   TEXT    NOT NULL,
            mtime          REAL    NOT NULL,
            created_at     TEXT    NOT NULL,
            importance_score REAL  DEFAULT 50.0,
            related_ids    TEXT    DEFAULT '[]'
        )
    """)
    c.execute("CREATE INDEX IF NOT EXISTS idx_uid ON memory_embeddings(uid)")
    c.execute("CREATE INDEX IF NOT EXISTS idx_scope ON memory_embeddings(scope)")
    c.execute("CREATE INDEX IF NOT EXISTS idx_level ON memory_embeddings(level)")
    c.execute("CREATE INDEX IF NOT EXISTS idx_content_hash ON memory_embeddings(content_hash)")

    # 迁移：如果表已存在，添加新列（v2.3 → v3.0）
    for col, col_type in [
        ("importance_score", "REAL DEFAULT 50.0"),
        ("related_ids", "TEXT DEFAULT '[]'"),
    ]:
        try:
            c.execute(f"ALTER TABLE memory_embeddings ADD COLUMN {col} {col_type}")
        except sqlite3.OperationalError:
            pass  # 列已存在

    conn.commit()
    return conn


def store_embedding(conn, row):
    """写入单条向量记录（upsert by chunk_id）"""
    c = conn.cursor()
    c.execute("""
        INSERT OR REPLACE INTO memory_embeddings
        (chunk_id, uid, scope, level, filepath, section_idx,
         title, semantic_key, content, raw_content, embedding, content_hash, mtime,
         created_at, importance_score, related_ids)
        VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
    """, row)
    conn.commit()


def delete_embeddings(conn, uid=None, scope=None):
    """删除向量记录"""
    c = conn.cursor()
    conditions = []
    params = []
    if uid:
        conditions.append("uid=?")
        params.append(uid)
    if scope:
        conditions.append("scope=?")
        params.append(scope)
    if conditions:
        c.execute(f"DELETE FROM memory_embeddings WHERE {' AND '.join(conditions)}", params)
    conn.commit()
    # VACUUM 释放空间
    c.execute("VACUUM")
    conn.commit()


def get_existing_records(conn, filepaths):
    """
    批量查询已有记录的 (content_hash, mtime_str) -> (chunk_id, filepath)
    用于增量判断。
    """
    if not filepaths:
        return {}
    c = conn.cursor()
    placeholders = ",".join("?" * len(filepaths))
    c.execute(
        f"SELECT content_hash, mtime, chunk_id, filepath FROM memory_embeddings WHERE filepath IN ({placeholders})",
        filepaths,
    )
    return {(row[0], str(row[1])): (row[2], row[3]) for row in c.fetchall()}


# ══════════════════════════════════════════════════════════════
#  记忆条目解析层
# ══════════════════════════════════════════════════════════════

def parse_sections(filepath):
    """解析 .md 文件，返回条目列表"""
    if not os.path.exists(filepath):
        return []
    with open(filepath, "r", encoding="utf-8") as f:
        content = f.read()
    sections = []
    parts = re.split(r"(?=^## )", content, flags=re.MULTILINE)
    for idx, part in enumerate(parts):
        part = part.strip()
        if not part or len(part) < 10:
            continue
        lines = part.split("\nfrom typing import List, Dict, Optional, Union, Any, Tuple\n")
        raw_title = lines[0].strip().lstrip("#").strip()
        title_clean = re.sub(r"^\[\d{2}:\d{2}\]\s*", "", raw_title)
        semantic_key = title_clean if len(title_clean) <= 20 else title_clean[:17] + "..."
        body = "\n".join(lines[1:]).strip() if len(lines) > 1 else ""
        if len(body) < 5:
            continue
        sections.append({
            "title": raw_title,
            "semantic_key": semantic_key,
            "content": body,
            "raw_content": part,
            "section_idx": idx,
        })
    return sections


def make_content_hash(content: str) -> str:
    """基于内容生成哈希（SHA-256前32位）"""
    return hashlib.sha256(content.encode("utf-8")).hexdigest()[:32]


def smart_embed_text(semantic_key: str, content: str, max_len: int = 800) -> str:
    """
    智能文本截断：头+中+尾，保留语义完整性。
    比简单截断更好地捕获长记忆的完整语义。
    """
    if len(content) <= max_len:
        return f"{semantic_key}: {content}"
    head = content[:400]
    mid_start = max(300, len(content) // 2 - 100)
    middle = content[mid_start:mid_start + 200]
    tail = content[-200:]
    return f"{semantic_key}: {head} ... {middle} ... {tail}"


def make_chunk_id(content_hash: str, mtime: float, section_idx: int) -> str:
    """生成唯一 chunk_id"""
    raw = f"{content_hash}|{mtime}|{section_idx}"
    return hashlib.sha256(raw.encode()).hexdigest()[:24]


def scan_memory_files(base_dir, uid, scope="all"):
    """
    扫描记忆文件，yield 待 embed 条目。
    base_dir: 仓库根目录（用于安全检查）
    """
    base_abs = os.path.abspath(base_dir)

    if scope in ("private", "all"):
        user_dir = os.path.join(base_abs, "users", uid)
        if os.path.exists(user_dir):
            for subdir, level in [
                (os.path.join(user_dir, "daily"), "L1"),
                (os.path.join(user_dir, "weekly"), "L2"),
                (os.path.join(user_dir, "permanent"), "L3"),
            ]:
                if not os.path.isdir(subdir):
                    continue
                for fname in sorted(os.listdir(subdir), reverse=True):
                    if not fname.endswith(".md"):
                        continue
                    fpath = os.path.join(subdir, fname)
                    rel = safe_relpath(fpath, base_abs)
                    if rel is None:
                        continue
                    mtime = os.path.getmtime(fpath)
                    sections = parse_sections(fpath)
                    if sections:
                        yield (fpath, rel, mtime, sections, level, uid, "private")

    if scope in ("shared", "all"):
        shared_dir = os.path.join(base_abs, "shared")
        if os.path.exists(shared_dir):
            for subdir, level in [
                (os.path.join(shared_dir, "daily"), "L1"),
                (os.path.join(shared_dir, "weekly"), "L2"),
                (os.path.join(shared_dir, "permanent"), "L3"),
            ]:
                if not os.path.isdir(subdir):
                    continue
                for fname in sorted(os.listdir(subdir), reverse=True):
                    if not fname.endswith(".md"):
                        continue
                    fpath = os.path.join(subdir, fname)
                    rel = safe_relpath(fpath, base_abs)
                    if rel is None:
                        continue
                    mtime = os.path.getmtime(fpath)
                    sections = parse_sections(fpath)
                    if sections:
                        yield (fpath, rel, mtime, sections, level, "shared", "shared")


# ══════════════════════════════════════════════════════════════
#  关联记忆 + 重要性计算
# ══════════════════════════════════════════════════════════════

def compute_related_memories(conn, uid, similarity_threshold=0.75):
    """
    计算所有记忆之间的语义关联。
    余弦相似度 > threshold 的两条记忆，互为 related_ids。

    关联存储在 related_ids 列（JSON 数组），每次 rebuild 时重新计算。
    关联数量无上限，但仅存 chunk_id，不存相似度分值（节省空间）。
    """
    try:
        import numpy as np
    except ImportError:
        print("  ⚠️  numpy 未安装，跳过关联计算")
        return

    c = conn.cursor()
    c.execute("""
        SELECT chunk_id, embedding, level, title
        FROM memory_embeddings
        WHERE uid = ? OR uid = 'shared'
    """, (uid,))
    rows = c.fetchall()

    if len(rows) < 2:
        return

    print(f"\n  🔗 计算 {len(rows)} 条记忆的关联关系（阈值: {similarity_threshold}）...")

    chunk_ids = [r[0] for r in rows]
    embeddings = [np.frombuffer(r[1], dtype=np.float32) for r in rows]

    # 预归一化，提升余弦相似度计算效率
    norms = [np.linalg.norm(e) for e in embeddings]
    normed = [e / n if n > 0 else e for e, n in zip(embeddings, norms)]

    # 建立 chunk_id → index 映射
    idx_map = {cid: i for i, cid in enumerate(chunk_ids)}

    # 计算关系（只存已存储的 chunk_id）
    related_map = {cid: [] for cid in chunk_ids}

    # 增量：只对本次 embed 的新 chunk 计算关系，已有关联保留
    existing_related = {}
    c.execute(f"SELECT chunk_id, related_ids FROM memory_embeddings WHERE uid = ?", (uid,))
    for cid, rids_json in c.fetchall():
        if rids_json and rids_json != "[]":
            try:
                existing_related[cid] = json.loads(rids_json)
            except Exception:
                existing_related[cid] = []

    for i, cid_i in enumerate(chunk_ids):
        if cid_i in existing_related:
            # 已有关联保留，新关联追加
            related_map[cid_i] = list(existing_related[cid_i])
        for j, cid_j in enumerate(chunk_ids):
            if i >= j:
                continue
            sim = float(np.dot(normed[i], normed[j]))
            if sim > similarity_threshold:
                for target_cid, source_cid in [(cid_i, cid_j), (cid_j, cid_i)]:
                    if target_cid not in related_map[target_cid]:
                        related_map[target_cid].append(source_cid)

    # 写回数据库
    updated = 0
    for cid, rids in related_map.items():
        if rids:  # 只更新有关系的
            c.execute(
                "UPDATE memory_embeddings SET related_ids = ? WHERE chunk_id = ?",
                (json.dumps(rids), cid)
            )
            updated += 1

    conn.commit()
    print(f"  ✅ 关联计算完成，{updated} 条记忆建立了关联关系")


def update_importance_scores(conn, uid):
    """
    从访问日志读取重要性评分，更新向量库。
    """
    try:
        sys.path.insert(0, os.path.dirname(__file__))
        from memory_access_log import get_all_importance_scores, calculate_importance_score
    except ImportError:
        return

    db_path = conn.execute("PRAGMA database_list").fetchone()[2]
    access_log_db = os.path.join(os.path.dirname(db_path), ".memory_access_log.db")
    if not os.path.exists(access_log_db):
        return

    stats = get_all_importance_scores(access_log_db, uid)
    if not stats:
        return

    c = conn.cursor()
    updated = 0
    for chunk_id, stat in stats.items():
        level_row = c.execute(
            "SELECT level FROM memory_embeddings WHERE chunk_id = ?", (chunk_id,)
        ).fetchone()
        if level_row:
            score = calculate_importance_score(
                access_log_db, uid, chunk_id, level_row[0]
            )
            c.execute(
                "UPDATE memory_embeddings SET importance_score = ? WHERE chunk_id = ?",
                (score, chunk_id)
            )
            updated += 1

    conn.commit()
    if updated > 0:
        print(f"  📊 重要性评分已更新（{updated} 条）")


# ══════════════════════════════════════════════════════════════
#  主流程
# ══════════════════════════════════════════════════════════════

def embed_all(base_dir, uid, scope, dry_run, rebuild):
    base_abs = os.path.abspath(base_dir)
    os.makedirs(base_abs, exist_ok=True)
    db_path = get_db_path(base_abs)
    conn = init_db(db_path)

    try:
        backend_info = get_backend_info(db_path)
        print(f"\n🔮 使用后端: {backend_info['name']} ({backend_info['dim']}维)")
    except Exception as e:
        print(f"\n⚠️  后端配置: {e}")
        print("   尝试使用默认配置...")

    if rebuild:
        print(f"🗑️  重建模式：清除 {uid} 的旧向量...")
        delete_embeddings(conn, uid=uid if scope != "all" else None,
                         scope=scope if scope == "shared" else None)
        existing_map = {}
    else:
        filepaths = [item[0] for item in scan_memory_files(base_abs, uid, scope)]
        existing_map = get_existing_records(conn, filepaths)

    print(f"\n📁 数据库: {db_path}")
    print(f"📦 范围: {scope} | 用户: {uid}\n")

    stats = {"total": 0, "new": 0, "skipped": 0, "errors": 0}
    batch_texts = []
    batch_rows = []

    for fpath, rel_path, mtime, sections, level, owner, sc in scan_memory_files(base_abs, uid, scope):
        for sec in sections:
            content_hash = make_content_hash(sec["content"])
            chunk_id = make_chunk_id(content_hash, mtime, sec["section_idx"])
            mtime_str = str(mtime)

            # 增量判断：content_hash + mtime 均相同才跳过
            if not rebuild and (content_hash, mtime_str) in existing_map:
                stats["skipped"] += 1
                continue

            embed_text = smart_embed_text(sec['semantic_key'], sec['content'])
            batch_texts.append(embed_text)
            batch_rows.append((
                chunk_id, uid, sc, level, rel_path, sec["section_idx"],
                sec["title"], sec["semantic_key"], sec["content"],
                sec["raw_content"], content_hash, mtime,
                # importance_score 和 related_ids 暂用默认值，写入后由 compute_related_memories 更新
                50.0, "[]",
            ))

    if not batch_texts:
        print("  ✅ 没有新内容需要 embed")
        print(f"  📊 统计: 跳过 {stats['skipped']} 条已有向量")
        conn.close()
        return

    print(f"  📦 准备 embed {len(batch_texts)} 条新条目...")

    try:
        start_time = time.time()
        # 使用并行 embedding（数据量大时自动启用）
        embeddings = parallel_embed(batch_texts, lambda texts: get_embeddings(texts, db_path=db_path))
        elapsed = time.time() - start_time
        print(f"  ⏱️  Embedding 完成，耗时 {elapsed:.2f}s ({len(batch_texts)/elapsed:.1f} 条/秒)")
    except Exception as e:
        print(f"  ❌ Embedding API 错误: {e}")
        conn.close()
        sys.exit(1)

    now = datetime.now().isoformat()
    for i, emb in enumerate(embeddings):
        (chunk_id, row_uid, sc, level, rel_path, section_idx,
         title, semantic_key, content, raw_content, content_hash, mtime) = batch_rows[i]

        store_row = (
            chunk_id, row_uid, sc, level, rel_path, section_idx,
            title, semantic_key, content, raw_content,
            emb.tobytes(), content_hash, mtime, now,
            50.0, "[]",
        )
        if not dry_run:
            store_embedding(conn, store_row)
        stats["new"] += 1

    conn.close()

    print(f"\n  ✅ 完成！新增向量: {stats['new']}")
    if stats["skipped"]:
        print(f"     跳过已有: {stats['skipped']}")
    if dry_run:
        print(f"     [DRY-RUN] 未实际写入数据库")

    # 关联记忆 + 重要性评分（写入完成后计算）
    if not dry_run and (stats["new"] > 0 or rebuild):
        conn2 = init_db(db_path)
        compute_related_memories(conn2, uid)
        update_importance_scores(conn2, uid)
        conn2.close()

    if not dry_run:
        print(f"\n💡 语义搜索: python scripts/memory_search.py --uid {uid} --scope private \"查询内容\"")
        print(f"💡 查看统计: python scripts/memory_search.py --uid {uid} --scope private --stats")


def main():
    parser = argparse.ArgumentParser(description="为记忆条目生成语义向量")
    parser.add_argument("--base-dir", default=".", help="记忆仓库根目录")
    parser.add_argument("--uid", required=True, help="用户ID")
    parser.add_argument("--scope", default="private", choices=["private", "shared", "all"],
                        help="嵌入范围")
    parser.add_argument("--rebuild", action="store_true", help="全量重建")
    parser.add_argument("--dry-run", action="store_true", help="预览，不写入数据库")
    parser.add_argument("--delete", action="store_true", help="删除该用户所有向量记录")
    parser.add_argument("--list-backends", action="store_true", help="列出可用后端")
    args = parser.parse_args()

    if args.list_backends:
        from embed_backends import list_backends
        print("\n可用 Embedding 后端:")
        for bid, info in list_backends().items():
            print(f"  {bid}: {info['name']} ({info['dim']}维)")
        return

    base_dir = os.path.abspath(args.base_dir)

    if args.delete:
        db_path = get_db_path(base_dir)
        if os.path.exists(db_path):
            conn = init_db(db_path)
            delete_embeddings(conn, uid=args.uid)
            conn.close()
            print(f"✅ 已删除用户 {args.uid} 的所有向量记录")
        else:
            print("数据库不存在，无需删除")
        return

    embed_all(base_dir, args.uid, args.scope, args.dry_run, args.rebuild)


if __name__ == "__main__":
    main()


path: scripts/memory_index.py
content: #!/usr/bin/env python3
"""
memory_index.py - 扫描记忆目录，自动生成/更新 INDEX.md（支持多用户 + 公共记忆）

用法:
  python scripts/memory_index.py [--base-dir PATH] [--uid UID] [--scope private|shared|all]
"""

import os
import re
import sys
import argparse
from datetime import datetime
from collections import Counter


def count_entries(filepath):
    if not os.path.exists(filepath):
        return 0
    with open(filepath, "r", encoding="utf-8") as f:
        content = f.read()
    return len(re.findall(r"^## ", content, re.MULTILINE))


def extract_first_summary(filepath):
    if not os.path.exists(filepath):
        return "—"
    with open(filepath, "r", encoding="utf-8") as f:
        content = f.read()
    match = re.search(r"\*\*(?:摘要|内容|描述)\*\*[:：]\s*(.+)", content)
    if match:
        return match.group(1)[:60].strip()
    headers = re.findall(r"^## (.+)$", content, re.MULTILINE)
    if headers:
        return "、".join(headers[:3])[:60]
    return "—"


def make_semantic_key(title):
    """
    为条目生成 semantic_key（20字以内的语义摘要）。
    用于向量搜索时作为高权重文本 + 人类快速浏览。
    """
    # 去掉时间前缀 [HH:MM]
    clean = re.sub(r"^\[\d{2}:\d{2}\]\s*", "", title.strip()).strip()
    if len(clean) <= 20:
        return clean
    return clean[:17] + "..."


def parse_sections_for_index(filepath):
    """
    解析文件，返回所有条目的 semantic_key 列表。
    [(section_idx, title, semantic_key, summary), ...]
    """
    if not os.path.exists(filepath):
        return []
    with open(filepath, "r", encoding="utf-8") as f:
        content = f.read()
    results = []
    parts = re.split(r"(?=^## )", content, flags=re.MULTILINE)
    for idx, part in enumerate(parts):
        part = part.strip()
        if not part or len(part) < 10:
            continue
        lines = part.split("\nfrom typing import List, Dict, Optional, Union, Any, Tuple\n")
        raw_title = lines[0].strip().lstrip("#").strip()
        body = "\n".join(lines[1:]).strip() if len(lines) > 1 else ""
        if not body:
            continue
        semantic_key = make_semantic_key(raw_title)
        summary = extract_first_summary_from_body(body)
        results.append((idx, raw_title, semantic_key, summary))
    return results


def extract_first_summary_from_body(body):
    """从 body 中提取摘要"""
    match = re.search(r"\*\*(?:摘要|内容|描述)\*\*[:：]\s*(.+)", body)
    if match:
        return match.group(1)[:60].strip()
    # 取非元数据行
    key_lines = []
    for line in body.split("\n"):
        stripped = line.strip()
        if not stripped or stripped.startswith("**"):
            continue
        key_lines.append(stripped)
        if len(key_lines) >= 2:
            break
    return " ".join(key_lines)[:60] if key_lines else body[:40]


def extract_tags(filepath):
    if not os.path.exists(filepath):
        return []
    with open(filepath, "r", encoding="utf-8") as f:
        content = f.read()
    return re.findall(r"#(\w[\w-]*)", content)


def extract_todos(filepath):
    if not os.path.exists(filepath):
        return []
    with open(filepath, "r", encoding="utf-8") as f:
        content = f.read()
    todos = re.findall(r"^- \[ \] (.+)$", content, re.MULTILINE)
    return [(os.path.basename(filepath).replace(".md", ""), t) for t in todos]


def _get_first_semantic_key(filepath):
    """提取文件第一条记录的 semantic key"""
    if not filepath or not os.path.exists(filepath):
        return "—"
    try:
        sections = parse_sections_for_index(filepath)
        if sections:
            return sections[0][2]  # semantic_key
    except Exception:
        pass
    return "—"

def scan_dir(dir_path, pattern, max_files=30):
    """扫描目录，返回文件摘要列表"""
    results = []
    if not os.path.exists(dir_path):
        return results
    for fname in sorted(os.listdir(dir_path), reverse=True)[:max_files]:
        if not fname.endswith(".md"):
            continue
        fpath = os.path.join(dir_path, fname)
        key = pattern.format(fname=fname.replace(".md", ""))
        results.append({
            "key": key,
            "_filepath": fpath,  # 用于提取 semantic key
            "entries": count_entries(fpath),
            "summary": extract_first_summary(fpath),
            "tags": extract_tags(fpath),
            "todos": extract_todos(fpath),
        })
    return results


def build_user_index(base_dir, uid):
    """构建单个用户的 INDEX.md"""
    user_dir = os.path.join(base_dir, "users", uid)
    daily_dir = os.path.join(user_dir, "daily")
    weekly_dir = os.path.join(user_dir, "weekly")
    permanent_dir = os.path.join(user_dir, "permanent")
    habits_path = os.path.join(user_dir, "HABITS.md")

    all_tags = Counter()
    all_todos = []
    now = datetime.now()

    # L1
    l1_data = scan_dir(daily_dir, "{fname}", max_files=30)
    for item in l1_data:
        all_tags.update(item["tags"])
        all_todos.extend(item["todos"])

    # L2
    l2_data = scan_dir(weekly_dir, "{fname}", max_files=12)
    for item in l2_data:
        all_tags.update(item["tags"])

    # L3
    l3_categories = ["decisions", "tech-stack", "projects", "people", "knowledge", "lessons", "preferences"]
    l3_rows = []
    for cat in l3_categories:
        fpath = os.path.join(permanent_dir, f"{cat}.md")
        if os.path.exists(fpath):
            mtime = datetime.fromtimestamp(os.path.getmtime(fpath)).strftime("%Y-%m-%d")
            entries = count_entries(fpath)
            summary = extract_first_summary(fpath)
            tags = extract_tags(fpath)
            all_tags.update(tags)
            l3_rows.append((cat, entries, summary[:50], mtime))

    # HABITS
    habits_count = count_entries(habits_path)

    total_l1 = len(l1_data)
    total_l2 = len(l2_data)
    total_l3 = sum(r[1] for r in l3_rows)

    content = f"""# Memory Index ({uid})
_最后更新: {now.strftime('%Y-%m-%d %H:%M')}_

## 快速摘要
| 层级 | 文件数 | 条目数 | 最近更新 |
|------|--------|--------|----------|
| L1 临时 | {total_l1} | {sum(e['entries'] for e in l1_data)} | {l1_data[0]['key'] if l1_data else '—'} |
| L2 长期 | {total_l2} | {sum(e['entries'] for e in l2_data)} | {l2_data[0]['key'] if l2_data else '—'} |
| L3 永久 | {len(l3_rows)} | {total_l3} | — |
| 习惯 | 1 | {habits_count} | — |

"""

    # 未完成任务
    if all_todos:
        content += "## 未完成任务追踪\n"
        for date_str, todo in all_todos[:20]:
            content += f"- [ ] {todo} _(来自 {date_str})_\n"
        content += "\n"
    else:
        content += "## 未完成任务追踪\n_暂无未完成任务_ ✅\n\n"

    # L1 - 增强版：每条记忆的 semantic key
    content += "## L1 临时记忆索引\n"
    if l1_data:
        content += "| 日期 | 条目 | Semantic Key | 标签 | 条目数 |\n"
        content += "|------|------|--------------|------|--------|\n"
        for item in l1_data[:15]:
            tag_str = " ".join(f"#{t}" for t in item["tags"][:4])
            # 从文件内容提取第一条 semantic key
            sk = _get_first_semantic_key(item.get("_filepath"))
            content += f"| {item['key']} | {item['summary'][:30]} | {sk} | {tag_str} | {item['entries']} |\n"
    else:
        content += "_暂无记录_\n"
    content += "\n"

    # L2
    content += "## L2 长期记忆索引\n"
    if l2_data:
        content += "| 周次 | 主要内容 | Semantic Key | 条目数 |\n"
        content += "|------|----------|--------------|--------|\n"
        for item in l2_data:
            sk = _get_first_semantic_key(item.get("_filepath"))
            content += f"| {item['key']} | {item['summary'][:35]} | {sk} | {item['entries']} |\n"
    else:
        content += "_暂无记录_\n"
    content += "\n"

    # L3 - 增强版
    content += "## L3 永久记忆索引\n"
    content += "| 分类 | 条目 | Semantic Key | 摘要 | 更新日期 |\n"
    content += "|------|------|--------------|------|----------|\n"
    if l3_rows:
        for cat, entries, summary, mtime in l3_rows:
            sk = _get_first_semantic_key(
                os.path.join(permanent_dir, f"{cat}.md")
            )
            content += f"| {cat} | {entries} | {sk} | {summary} | {mtime} |\n"
    else:
        content += "_暂无记录_\n"
    content += "\n"

    # 热门标签
    if all_tags:
        tags_str = " ".join(f"#{t}({c})" for t, c in all_tags.most_common(20))
        content += f"## 热门标签\n{tags_str}\n\n"

    content += """## 搜索提示
- 按日期: 查看 `users/{uid}/daily/YYYY-MM-DD.md`
- 按主题: 使用标签 `#tag`
- 按决策: 查看 `users/{uid}/permanent/decisions.md`
- 按技术: 查看 `users/{uid}/permanent/tech-stack.md`
- 按习惯: 查看 `users/{uid}/HABITS.md`
""".replace("{uid}", uid)

    return content


def build_shared_index(base_dir):
    """构建公共 INDEX.md"""
    shared_dir = os.path.join(base_dir, "shared")
    daily_dir = os.path.join(shared_dir, "daily")
    weekly_dir = os.path.join(shared_dir, "weekly")
    permanent_dir = os.path.join(shared_dir, "permanent")

    all_tags = Counter()
    now = datetime.now()

    # shared daily
    sd_data = scan_dir(daily_dir, "{fname}", max_files=14)
    for item in sd_data:
        all_tags.update(item["tags"])

    # shared weekly
    sw_data = scan_dir(weekly_dir, "{fname}", max_files=12)
    for item in sw_data:
        all_tags.update(item["tags"])

    # shared L3
    shared_l3_cats = ["decisions", "tech-stack", "projects", "knowledge"]
    l3_rows = []
    for cat in shared_l3_cats:
        fpath = os.path.join(permanent_dir, f"{cat}.md")
        if os.path.exists(fpath):
            mtime = datetime.fromtimestamp(os.path.getmtime(fpath)).strftime("%Y-%m-%d")
            entries = count_entries(fpath)
            summary = extract_first_summary(fpath)
            tags = extract_tags(fpath)
            all_tags.update(tags)
            l3_rows.append((cat, entries, summary[:50], mtime))

    content = f"""# Shared Memory Index
_最后更新: {now.strftime('%Y-%m-%d %H:%M')}_

## 快速摘要
| 层级 | 文件数 | 条目数 | 最近更新 |
|------|--------|--------|----------|
| 公共临时 | {len(sd_data)} | {sum(e['entries'] for e in sd_data)} | {sd_data[0]['key'] if sd_data else '—'} |
| 公共周报 | {len(sw_data)} | {sum(e['entries'] for e in sw_data)} | {sw_data[0]['key'] if sw_data else '—'} |
| 公共永久 | {len(l3_rows)} | {sum(r[1] for r in l3_rows)} | — |

"""

    # 公共 L1 - 增强版
    content += "## 公共临时记忆\n"
    if sd_data:
        content += "| 日期 | 条目 | Semantic Key | 标签 |\n"
        content += "|------|------|--------------|------|\n"
        for item in sd_data[:10]:
            tag_str = " ".join(f"#{t}" for t in item["tags"][:4])
            sk = _get_first_semantic_key(item.get("_filepath"))
            content += f"| {item['key']} | {item['summary'][:35]} | {sk} | {tag_str} |\n"
    else:
        content += "_暂无记录_\n"
    content += "\n"

    # 公共 L2
    content += "## 公共周报\n"
    if sw_data:
        content += "| 周次 | 主要内容 | 条目数 |\n"
        content += "|------|----------|--------|\n"
        for item in sw_data:
            content += f"| {item['key']} | {item['summary'][:50]} | {item['entries']} |\n"
    else:
        content += "_暂无记录_\n"
    content += "\n"

    # 公共 L3 - 增强版
    content += "## 公共永久记忆索引\n"
    content += "| 分类 | 条目 | Semantic Key | 摘要 | 更新日期 |\n"
    content += "|------|------|--------------|------|----------|\n"
    if l3_rows:
        for cat, entries, summary, mtime in l3_rows:
            sk = _get_first_semantic_key(
                os.path.join(permanent_dir, f"{cat}.md")
            )
            content += f"| {cat} | {entries} | {sk} | {summary} | {mtime} |\n"
    else:
        content += "_暂无记录_\n"
    content += "\n"

    if all_tags:
        tags_str = " ".join(f"#{t}({c})" for t, c in all_tags.most_common(15))
        content += f"## 公共热门标签\n{tags_str}\n\n"

    content += """## 搜索提示
- 团队当日进展: `shared/daily/YYYY-MM-DD.md`
- 团队周报: `shared/weekly/YYYY-WNN.md`
- 公共决策: `shared/permanent/decisions.md`
- 项目信息: `shared/permanent/projects.md`
"""

    return content


def main():
    parser = argparse.ArgumentParser(description="生成/更新记忆索引")
    parser.add_argument("--base-dir", default=".", help="记忆仓库根目录")
    parser.add_argument("--uid", help="用户ID（默认: MEMORY_USER_ID 或 'default'）")
    parser.add_argument("--scope", default="all", choices=["private", "shared", "all"],
                        help="索引范围")
    args = parser.parse_args()

    # 安全检查：确保 base_dir 在预期范围内
    base_dir = os.path.abspath(args.base_dir)
    # 禁止访问系统关键路径
    forbidden = {"/", "C:\\", "/home", "/Users", "/System"}
    if any(base_dir.startswith(p) for p in forbidden):
        print(f"❌ 禁止访问系统路径: {base_dir}")
        sys.exit(1)
    uid = args.uid or os.environ.get("MEMORY_USER_ID", "default")

    print(f"\n🔍 记忆索引构建")
    print(f"📁 仓库: {base_dir}")
    print(f"👤 用户: {uid}\n")

    # 私人索引
    if args.scope in ("private", "all"):
        user_dir = os.path.join(base_dir, "users", uid)
        if os.path.exists(user_dir):
            content = build_user_index(base_dir, uid)
            index_path = os.path.join(user_dir, "INDEX.md")
            with open(index_path, "w", encoding="utf-8") as f:
                f.write(content)
            print(f"✅ 个人索引已更新: users/{uid}/INDEX.md")
        else:
            print(f"⚠️  用户目录不存在: users/{uid}/")

    # 公共索引
    if args.scope in ("shared", "all"):
        shared_dir = os.path.join(base_dir, "shared")
        if os.path.exists(shared_dir):
            content = build_shared_index(base_dir)
            index_path = os.path.join(shared_dir, "INDEX.md")
            with open(index_path, "w", encoding="utf-8") as f:
                f.write(content)
            print(f"✅ 公共索引已更新: shared/INDEX.md")
        else:
            print(f"⚠️  公共目录不存在: shared/")


if __name__ == "__main__":
    main()


path: scripts/memory_init.py
content: #!/usr/bin/env python3
"""
memory_init.py - 初始化三层记忆管理系统（支持多用户 + GitHub）

用法:
  # 全新初始化（在现有空仓库中）
  python scripts/memory_init.py --uid pupper

  # 从 GitHub 克隆现有仓库并加入新用户
  python scripts/memory_init.py --clone --repo-url https://github.com/owner/memory-repo \
                                  --uid pupper --github-token TOKEN

  # 仅为当前仓库添加新用户
  python scripts/memory_init.py --add-user alice --repo-dir /path/to/repo
"""

import os
import sys
import re
import subprocess
import argparse
from datetime import datetime


# ──────────────────────────────────────────
# 工具函数
# ──────────────────────────────────────────

def write_file(path, content, overwrite=False):
    if os.path.exists(path) and not overwrite:
        print(f"  ⏭️  已存在，跳过: {os.path.relpath(path)}")
        return
    os.makedirs(os.path.dirname(path), exist_ok=True)
    with open(path, "w", encoding="utf-8") as f:
        f.write(content)
    print(f"  ✓ 创建: {os.path.relpath(path)}")


def is_git_repo(path):
    return os.path.exists(os.path.join(path, ".git"))


# ──────────────────────────────────────────
# 公共目录初始化
# ──────────────────────────────────────────

def init_shared(repo_dir):
    """初始化 shared/ 公共记忆目录"""
    print("\nfrom typing import List, Dict, Optional, Union, Any, Tuple\n📂 初始化公共记忆 (shared/)...")
    now = datetime.now().strftime("%Y-%m-%d")

    shared = os.path.join(repo_dir, "shared")
    permanent = os.path.join(shared, "permanent")
    os.makedirs(permanent, exist_ok=True)

    # 公共永久记忆分类
    categories = {
        "decisions": "公共决策记录",
        "projects": "项目信息与里程碑",
        "tech-stack": "技术选型与架构",
        "knowledge": "公共知识库",
    }
    for fname, desc in categories.items():
        write_file(
            os.path.join(permanent, f"{fname}.md"),
            f"# {desc}\n_最后更新: {now}_\n\n<!-- 团队共享记忆，所有成员均可读写 -->\n\n",
        )

    # 公共 INDEX
    write_file(
        os.path.join(shared, "INDEX.md"),
        f"""# Shared Memory Index
_最后更新: {now}_

## 分类目录
| 分类 | 条目数 | 最近更新 | 说明 |
|------|--------|----------|------|
| decisions | 0 | — | 公共决策 |
| projects | 0 | — | 项目信息 |
| tech-stack | 0 | — | 技术选型 |
| knowledge | 0 | — | 知识库 |

## 活跃用户
_（用户加入后自动更新）_

## 公共待办
_暂无_
""",
    )

    # SKILL_VERSION.md — 版本通知（存放在仓库根目录）
    create_skill_version(repo_dir)

    # 公共 STATS
    write_file(
        os.path.join(shared, "STATS.md"),
        f"""# Shared Memory Statistics
_最后更新: {now}_

## 概览
| 指标 | 数值 |
|------|------|
| 总条目数 | 0 |
| 活跃用户数 | 0 |
| 最后写入 | — |

## 分类统计
| 分类 | 条目数 | 最后写入者 |
|------|--------|------------|
| decisions | 0 | — |
| projects | 0 | — |
| tech-stack | 0 | — |
| knowledge | 0 | — |
""",
    )


# ──────────────────────────────────────────
# 用户目录初始化
# ──────────────────────────────────────────

def init_user(repo_dir, uid):
    """初始化 users/{uid}/ 私人目录"""
    print(f"\n👤 初始化用户私人记忆 (users/{uid}/)...")
    now = datetime.now().strftime("%Y-%m-%d")

    user_dir = os.path.join(repo_dir, "users", uid)
    dirs = ["daily", "weekly", os.path.join("permanent")]
    for d in dirs:
        os.makedirs(os.path.join(user_dir, d), exist_ok=True)

    # 私人永久记忆分类
    private_categories = {
        "decisions": "个人决策记录",
        "lessons": "经验教训与踩坑",
        "preferences": "个人偏好与工作风格",
        "people": "重要人物与关系",
    }
    for fname, desc in private_categories.items():
        write_file(
            os.path.join(user_dir, "permanent", f"{fname}.md"),
            f"# {desc}\n_最后更新: {now}_\n_用户: {uid}_\n\n",
        )

    # HABITS
    write_file(
        os.path.join(user_dir, "HABITS.md"),
        f"""# Habits & Preferences — {uid}
_最后更新: {now}_
_总观察次数: 0_

---

## 🕐 工作节奏
- **活跃时段**: _待观察_
- **会话平均时长**: _待观察_

## 💻 技术偏好
- **主要技术栈**: _待记录_
- **代码风格**: _待观察_

## 🗣️ 沟通风格
- **语言**: _待观察_
- **回答详细程度**: _待观察_
- **偏好格式**: _待观察_

## 🎯 决策模式
- **决策风格**: _待观察_

## ⚠️ 特别注意
_暂无_

---
| 日期 | 观察到的习惯 | 置信度 |
|------|-------------|--------|
""",
    )

    # INDEX
    write_file(
        os.path.join(user_dir, "INDEX.md"),
        f"""# Memory Index — {uid}
_最后更新: {now}_

## 快速摘要
| 层级 | 文件数 | 条目数 | 最近更新 |
|------|--------|--------|----------|
| L1 临时 | 0 | 0 | — |
| L2 长期 | 0 | 0 | — |
| L3 私人永久 | 0 | 0 | — |

## 未完成任务
_暂无_

## L1 临时记忆
| 日期 | 主要活动 | 条目数 |
|------|----------|--------|

## L2 长期记忆
| 周次 | 主要内容 | 条目数 |
|------|----------|--------|

## L3 私人永久记忆
| 分类 | 条目 | 更新日期 |
|------|------|----------|
| decisions | 0 | — |
| lessons | 0 | — |
| preferences | 0 | — |
| people | 0 | — |
""",
    )

    # STATS
    write_file(
        os.path.join(user_dir, "STATS.md"),
        f"""# Memory Statistics — {uid}
_最后更新: {now}_

## 总览
| 指标 | 数值 |
|------|------|
| L1 文件数 | 0 |
| L2 文件数 | 0 |
| L3 条目总数 | 0 |
| 未完成任务 | 0 |
| 最近7天活跃 | 0 条 |

## 健康度
- L1 状态: ✅ 正常
- L2 状态: ✅ 正常
- 索引状态: ✅ 最新

## 压缩历史
| 日期 | 层级 | 原行数 | 压缩后 | 压缩率 |
|------|------|--------|--------|--------|
""",
    )

    # config
    write_file(
        os.path.join(user_dir, "config.md"),
        f"""# Memory Config — {uid}
_最后修改: {now}_

## 压缩阈值
- L1 压缩触发: 150 行
- L2 压缩触发: 200 行

## 同步设置
- 自动同步: 开启
- 同步范围: all（私人+公共）

## 归档策略
- L1 超过 30 天 → 归档到 archive/
""",
    )

    # profile.md — 身份锚点文件（随 Git 同步，跨设备识别用户）
    write_file(
        os.path.join(user_dir, "profile.md"),
        f"""---
uid: {uid}
created: {now}
device_setup: local
---

# Profile — {uid}

_此文件由系统自动生成，用于跨设备识别用户身份。_
_任何设备 `git pull` 后，AI 可通过此文件确认 uid。_

## 用户信息
- **uid**: {uid}
- **加入时间**: {now}

## 设备记录
| 设备 | 初始化时间 | 说明 |
|------|------------|------|
| local | {now} | 首次初始化 |
""",
    )

    # .current_user 标记文件（本地快速读取，不入 Git）
    current_user_path = os.path.join(repo_dir, ".current_user")
    with open(current_user_path, "w", encoding="utf-8") as f:
        f.write(uid)
    print(f"  ✓ 设置当前用户: {uid}")

    # 更新 shared/INDEX.md 的活跃用户列表
    shared_index = os.path.join(repo_dir, "shared", "INDEX.md")
    if os.path.exists(shared_index):
        with open(shared_index, "r", encoding="utf-8") as f:
            content = f.read()
        if uid not in content:
            content = content.replace(
                "_（用户加入后自动更新）_",
                f"- {uid} (加入: {now})\n_（用户加入后自动更新）_",
            )
            with open(shared_index, "w", encoding="utf-8") as f:
                f.write(content)
            print(f"  ✓ 已将 {uid} 添加到公共索引")


# ──────────────────────────────────────────
# SKILL_VERSION.md — 版本通知文件
# ──────────────────────────────────────────

# 当前记忆系统所需的 Skill 最低版本（与 SKILL.md 的 version 字段同步）
REQUIRED_SKILL_VERSION = "3.5.0"

def create_skill_version(repo_dir):
    """创建/更新 SKILL_VERSION.md，用于提示 Skill 用户更新"""
    now = datetime.now().strftime("%Y-%m-%d")
    write_file(
        os.path.join(repo_dir, "SKILL_VERSION.md"),
        f"""---
required_skill_version: {REQUIRED_SKILL_VERSION}
last_updated: {now}
---

# Memory Manager Skill 版本要求

> 此文件由记忆系统自动维护，AI 在每次会话启动时读取并检查版本。

## 当前要求

本记忆仓库需要 **memory-manager skill >= {REQUIRED_SKILL_VERSION}**

## 如何检查本地版本

打开 `~/.workbuddy/skills/memory-manager/SKILL.md`，查看顶部 `version:` 字段。

## 如何更新 Skill

```bash
git -C ~/.workbuddy/skills/memory-manager pull
```

或访问 GitHub Release 页面手动下载：
https://github.com/Pupper0601/memory-manager/releases

## 更新历史

| 版本 | 日期 | 变更说明 |
|------|------|----------|
| 3.5.0 | 2026-04-05 | 修复跨设备身份识别；新增 profile.md 身份锚点；SKILL.md 加入 uid 自动解析 + 版本检查 |
| 3.4.0 | 2026-04-04 | 关联记忆网络；重要性评分；访问日志；向量搜索综合排序 |
| 3.0.0 | 2026-04-03 | 初始发布 |
""",
        overwrite=True,
    )
    print(f"  ✓ 更新 SKILL_VERSION.md (required >= {REQUIRED_SKILL_VERSION})")


# ──────────────────────────────────────────
# .gitignore
# ──────────────────────────────────────────

def create_gitignore(repo_dir):
    write_file(
        os.path.join(repo_dir, ".gitignore"),
        """# 备份文件
*.bak
.compress_backup/

# 系统文件
.DS_Store
Thumbs.db
desktop.ini

# Python
__pycache__/
*.pyc
*.pyo
.env

# 编辑器
.vscode/
.idea/
*.swp

# 本地身份文件（每台设备独立维护，不同步）
# 注意：users/{uid}/profile.md 才是跨设备身份锚点，会同步到 Git
.current_user
.mm.json

# 向量数据库（本地生成，不同步）
.memory_vectors.db
.memory_access_log.db
.lancedb/
""",
    )


# ──────────────────────────────────────────
# README
# ──────────────────────────────────────────

def create_readme(repo_dir):
    write_file(
        os.path.join(repo_dir, "README.md"),
        """# Memory Repository

AI 三层记忆管理系统 — 多用户 · 跨设备

## 目录结构

```
shared/                     ← 公共记忆（团队共享）
├── permanent/
│   ├── decisions.md
│   ├── projects.md
│   ├── tech-stack.md
│   └── knowledge.md
├── INDEX.md
└── STATS.md

users/                      ← 私人记忆（按用户隔离）
└── {uid}/
    ├── daily/              ← L1 临时记忆
    ├── weekly/             ← L2 长期记忆
    ├── permanent/          ← L3 私人永久记忆
    ├── HABITS.md
    ├── INDEX.md
    └── STATS.md
```

## 加入方式

```bash
# 克隆仓库
git clone https://github.com/owner/memory-repo
cd memory-repo

# 初始化你的用户空间
python scripts/memory_init.py --add-user YOUR_GITHUB_USERNAME
```

## 同步命令

```bash
python scripts/memory_sync.py pull --uid YOUR_UID
python scripts/memory_sync.py push --uid YOUR_UID
python scripts/memory_sync.py status
```
""",
        overwrite=True,
    )


# ──────────────────────────────────────────
# Git 初始化和远程设置
# ──────────────────────────────────────────

def setup_git(repo_dir, repo_url=None, token=None):
    """初始化 git 并安全设置远程（不暴露 token）"""
    if not is_git_repo(repo_dir):
        subprocess.run(["git", "init", "-b", "main"], cwd=repo_dir, check=True)
        subprocess.run(
            ["git", "config", "user.email", "memory@openclaw"],
            cwd=repo_dir, check=True,
        )
        subprocess.run(
            ["git", "config", "user.name", "Memory Manager Bot"],
            cwd=repo_dir, check=True,
        )
        print("  ✓ Git 仓库初始化完成")

    if repo_url:
        # 检查当前 remote
        rc = subprocess.run(
            ["git", "remote", "get-url", "origin"],
            cwd=repo_dir, capture_output=True
        ).returncode

        if rc == 0:
            print(f"  ✓ 远程已配置，跳过")
        else:
            # 去掉 URL 中的 token（如果带了的话）
            clean_url = repo_url
            if token and f"https://{token}@" in repo_url:
                clean_url = repo_url.replace(f"{token}@", "")

            # 用 credential helper 安全存储 token（不写进 remote URL）
            if [REDACTED_SECRET] = os.path.join(repo_dir, ".git", "credentials")
                os.makedirs(os.path.dirname(cred_file), exist_ok=True)
                with open(cred_file, "w", encoding="utf-8") as f:
                    f.write(f"https://{token}@github.com\n")
                try:
                    os.chmod(cred_file, 0o600)
                except Exception:
                    pass
                subprocess.run(
                    ["git", "config", "credential.helper",
                     f"store --file {cred_file}"],
                    cwd=repo_dir, check=True,
                )

            subprocess.run(
                ["git", "remote", "add", "origin", clean_url],
                cwd=repo_dir, check=True,
            )
            print(f"  ✓ 远程已配置（token 通过 credential helper 存储）")


def initial_commit_and_push(repo_dir, uid):
    """执行首次提交并推送"""
    try:
        subprocess.run(["git", "add", "-A"], cwd=repo_dir, check=True)
        result = subprocess.run(
            ["git", "diff", "--cached", "--quiet"],
            cwd=repo_dir, capture_output=True,
        )
        if result.returncode != 0:
            subprocess.run(
                ["git", "commit", "-m", f"memory(init): 初始化记忆系统 — 用户 {uid}"],
                cwd=repo_dir, check=True,
            )
            print("  ✓ 提交完成")

        # 尝试推送
        rc = subprocess.run(
            ["git", "push", "-u", "origin", "main"],
            cwd=repo_dir, capture_output=True,
        ).returncode
        if rc == 0:
            print("  ✓ 推送到 GitHub 成功")
        else:
            print("  ⚠️  推送失败（可能需要先在 GitHub 创建仓库）")
    except Exception as e:
        print(f"  ⚠️  Git 操作失败: {e}")


# ──────────────────────────────────────────
# 主流程
# ──────────────────────────────────────────

def main():
    parser = argparse.ArgumentParser(description="初始化三层记忆管理系统（多用户+GitHub）")
    parser.add_argument("--uid", help="当前用户ID（GitHub 用户名）")
    parser.add_argument("--add-user", help="仅为现有仓库添加新用户")
    parser.add_argument("--repo-dir", default=".", help="本地仓库目录")
    parser.add_argument("--repo-url", help="GitHub 仓库 URL（HTTPS）")
    parser.add_argument("--clone", action="store_true", help="克隆现有仓库后初始化")
    parser.add_argument("--github-token", help="GitHub Personal Access Token")
    args = parser.parse_args()

    repo_dir = os.path.abspath(args.repo_dir)
    [REDACTED_SECRET] or os.environ.get("GITHUB_TOKEN", "")

    print(f"\n🧠 三层记忆管理系统 — 初始化工具 v2.0")

    # ── 克隆模式 ──
    if args.clone and args.repo_url:
        if not os.path.exists(repo_dir) or not is_git_repo(repo_dir):
            print(f"\n⬇️  克隆仓库: {args.repo_url}")
            clone_url = args.repo_url
            if token and "https://github.com" in clone_url:
                clone_url = clone_url.replace("https://", f"https://{token}@")
            os.makedirs(os.path.dirname(repo_dir) or ".", exist_ok=True)
            subprocess.run(["git", "clone", clone_url, repo_dir], check=True)
            print(f"✅ 克隆完成: {repo_dir}")
        else:
            print(f"✅ 仓库已存在，跳过克隆: {repo_dir}")

    # ── 仅添加用户 ──
    uid = args.uid or args.add_user
    if not uid:
        # 尝试读取 .current_user
        current_user_file = os.path.join(repo_dir, ".current_user")
        if os.path.exists(current_user_file):
            with open(current_user_file, encoding="utf-8") as f:
                uid = f.read().strip()
            print(f"ℹ️  使用已有用户: {uid}")
        else:
            uid = input("请输入你的用户ID（建议使用 GitHub 用户名）: ").strip()
            if not uid:
                print("❌ 必须提供用户ID")
                sys.exit(1)

    print(f"👤 用户ID: {uid}")
    print(f"📁 仓库目录: {repo_dir}")

    os.makedirs(repo_dir, exist_ok=True)

    # ── Git 设置 ──
    if not args.clone:
        setup_git(repo_dir, args.repo_url, token)

    # ── 创建公共目录（若不存在） ──
    if not os.path.exists(os.path.join(repo_dir, "shared")):
        init_shared(repo_dir)
    else:
        print("\n📂 shared/ 目录已存在，跳过公共初始化")

    # ── 创建用户目录 ──
    user_dir = os.path.join(repo_dir, "users", uid)
    if os.path.exists(user_dir):
        print(f"\n👤 users/{uid}/ 已存在，仅更新 .current_user")
        current_user_path = os.path.join(repo_dir, ".current_user")
        with open(current_user_path, "w", encoding="utf-8") as f:
            f.write(uid)
    else:
        init_user(repo_dir, uid)

    # ── .gitignore 和 README ──
    create_gitignore(repo_dir)
    create_readme(repo_dir)

    # ── 首次提交 ──
    if not args.clone:
        initial_commit_and_push(repo_dir, uid)

    print(f"""
✅ 初始化完成！

记忆仓库结构:
{repo_dir}/
├── shared/                  ← 公共记忆（团队共享）
│   ├── permanent/
│   │   ├── decisions.md
│   │   ├── projects.md
│   │   ├── tech-stack.md
│   │   └── knowledge.md
│   ├── INDEX.md
│   └── STATS.md
└── users/
    └── {uid}/               ← 你的私人记忆空间
        ├── daily/           ← L1 临时记忆
        ├── weekly/          ← L2 长期记忆
        ├── permanent/       ← L3 私人永久记忆
        ├── HABITS.md
        ├── INDEX.md
        └── STATS.md

💡 快速上手:
  - 同步: python scripts/memory_sync.py pull --uid {uid}
  - 统计: python scripts/memory_stats.py --uid {uid} --update
  - 索引: python scripts/memory_index.py --uid {uid}

  或直接对AI说"记住这个" / "查找记忆 [关键词]"
""")


if __name__ == "__main__":
    main()


path: scripts/memory_insight.py
content: #!/usr/bin/env python3
"""
memory_insight.py - AI 记忆洞察与自我进化

功能:
  1. 分析记忆使用模式，发现规律和趋势
  2. 生成洞察报告，帮助理解自己
  3. 自动标记重要记忆，优化向量权重
  4. 发现知识盲点，建议补充

用法:
  python scripts/memory_insight.py --uid pupper                    # 综合洞察
  python scripts/memory_insight.py --uid pupper --daily          # 每日洞察
  python scripts/memory_insight.py --uid pupper --weekly           # 每周洞察
  python scripts/memory_insight.py --uid pupper --auto-mark       # 自动标记重要记忆
  python scripts/memory_insight.py --uid pupper --find-gaps        # 发现知识盲点
  python scripts/memory_insight.py --uid pupper --learn-patterns   # 学习记忆模式

依赖:
  pip install openai numpy
"""

import os
import sys
import json
import sqlite3
import argparse
import re
from datetime import datetime, timedelta, timezone
from pathlib import Path
from collections import Counter, defaultdict

# 添加 embed_backends 路径
SCRIPT_DIR = Path(__file__).parent
sys.path.insert(0, str(SCRIPT_DIR))


DB_NAME = ".memory_vectors.db"
ACCESS_DB = ".memory_access_log.db"


# ══════════════════════════════════════════════════════════════
#  数据提取
# ══════════════════════════════════════════════════════════════

def get_db_path(base_dir):
    return os.path.join(os.path.abspath(base_dir), DB_NAME)


def connect_db(db_path):
    if not os.path.exists(db_path):
        return None
    conn = sqlite3.connect(db_path)
    conn.execute("PRAGMA journal_mode=WAL")
    return conn


def extract_memories(conn, uid, days=None, scope="private"):
    """提取记忆数据"""
    c = conn.cursor()

    if days:
        since = (datetime.now() - timedelta(days=days)).timestamp()
        time_filter = f"AND mtime >= {since}"
    else:
        time_filter = ""

    scope_filter = f"AND scope = '{scope}'" if scope != "all" else ""

    c.execute(f"""
        SELECT chunk_id, title, content, level, importance_score,
               related_ids, mtime, created_at
        FROM memory_embeddings
        WHERE (uid = ? OR uid = 'shared') {scope_filter} {time_filter}
        ORDER BY mtime DESC
    """, (uid,))

    cols = [desc[0] for desc in c.description]
    return [dict(zip(cols, row)) for row in c.fetchall()]


def extract_access_patterns(base_dir, uid, days=7):
    """提取访问模式"""
    access_db = os.path.join(os.path.abspath(base_dir), ACCESS_DB)
    if not os.path.exists(access_db):
        return {}

    conn = sqlite3.connect(access_db)
    c = conn.cursor()

    since = (datetime.now() - timedelta(days=days)).timestamp()
    c.execute("""
        SELECT chunk_id, access_type, COUNT(*) as cnt,
               MAX(access_time) as last_access
        FROM memory_access_log
        WHERE uid = ? AND access_time >= ?
        GROUP BY chunk_id, access_type
    """, (uid, since))

    patterns = defaultdict(lambda: {"count": 0, "types": Counter(), "last": 0})
    for chunk_id, access_type, cnt, last in c.fetchall():
        patterns[chunk_id]["count"] += cnt
        patterns[chunk_id]["types"][access_type] = cnt
        patterns[chunk_id]["last"] = max(patterns[chunk_id]["last"], last)

    conn.close()
    return dict(patterns)


def analyze_topics(memories):
    """分析记忆主题"""
    # 简单的关键词提取
    stop_words = {"的", "是", "在", "了", "和", "与", "或", "为", "上", "下", "中", "后", "前", "我", "你", "他", "它", "我们", "你们", "他们"}

    all_text = " ".join(m.get("content", "") + " " + m.get("title", "") for m in memories)

    # 提取中文字符序列作为词
    words = re.findall(r'[\u4e00-\u9fff]{2,}', all_text)
    words = [w for w in words if w not in stop_words]

    # 词频统计
    word_freq = Counter(words)
    return word_freq.most_common(20)


def analyze_level_distribution(memories):
    """分析层级分布"""
    levels = Counter(m.get("level", "L1") for m in memories)
    return dict(levels)


def analyze_related_strength(memories):
    """分析关联强度"""
    strong_related = 0  # 有超过3条关联的记忆
    weak_related = 0    # 有关联但少于3条
    no_related = 0      # 无关联

    for m in memories:
        rids = m.get("related_ids", "[]")
        try:
            related = json.loads(rids) if rids else []
            if len(related) >= 3:
                strong_related += 1
            elif len(related) > 0:
                weak_related += 1
            else:
                no_related += 1
        except:
            no_related += 1

    return {
        "strong": strong_related,
        "weak": weak_related,
        "none": no_related
    }


def analyze_importance_distribution(memories):
    """分析重要性分布"""
    scores = [m.get("importance_score", 50) for m in memories]
    if not scores:
        return {"avg": 50, "high": 0, "medium": 0, "low": 0}

    avg = sum(scores) / len(scores)
    high = sum(1 for s in scores if s >= 75)
    medium = sum(1 for s in scores if 50 <= s < 75)
    low = sum(1 for s in scores if s < 50)

    return {
        "avg": avg,
        "high": high,
        "medium": medium,
        "low": low
    }


def analyze_recency(memories):
    """分析记忆新鲜度"""
    now = datetime.now().timestamp()
    recency = {
        "today": 0,
        "week": 0,
        "month": 0,
        "older": 0
    }

    for m in memories:
        mtime = m.get("mtime", 0)
        age_days = (now - mtime) / 86400

        if age_days < 1:
            recency["today"] += 1
        elif age_days < 7:
            recency["week"] += 1
        elif age_days < 30:
            recency["month"] += 1
        else:
            recency["older"] += 1

    return recency


def detect_patterns(memories, access_patterns):
    """检测记忆模式"""
    patterns = []

    # 模式1：高访问低重要性 -> 建议提升权重
    for m in memories:
        cid = m.get("chunk_id")
        if cid in access_patterns:
            access_cnt = access_patterns[cid]["count"]
            importance = m.get("importance_score", 50)
            if access_cnt > 10 and importance < 60:
                patterns.append({
                    "type": "high_access_low_importance",
                    "chunk_id": cid,
                    "title": m.get("title"),
                    "access_count": access_cnt,
                    "current_importance": importance,
                    "suggestion": f"该记忆被访问 {access_cnt} 次，建议提升重要性评分"
                })

    # 模式2：孤立记忆（无关联）-> 建议手动关联
    for m in memories:
        rids = m.get("related_ids", "[]")
        try:
            related = json.loads(rids) if rids else []
            if len(related) == 0 and m.get("importance_score", 50) > 70:
                patterns.append({
                    "type": "isolated_important",
                    "chunk_id": m.get("chunk_id"),
                    "title": m.get("title"),
                    "importance": m.get("importance_score"),
                    "suggestion": "重要记忆无关联，建议检查并建立关联"
                })
        except:
            pass

    # 模式3：同一主题频繁记录 -> 可能需要整合
    topics = Counter()
    for m in memories:
        content = m.get("content", "") + m.get("title", "")
        # 提取前5个字符作为简化的主题标识
        words = re.findall(r'[\u4e00-\u9fff]{3,}', content)
        if words:
            topics[words[0]] += 1

    for topic, cnt in topics.items():
        if cnt >= 5:
            patterns.append({
                "type": "frequent_topic",
                "topic": topic,
                "count": cnt,
                "suggestion": f"'{topic}' 主题已记录 {cnt} 次，建议整合"
            })

    return patterns


def find_knowledge_gaps(memories, topics):
    """发现知识盲点"""
    gaps = []

    # 基于已有主题，推测可能缺失的领域
    known_topics = set(t[0] for t in topics)

    # 常见技术领域（可以根据用户角色调整）
    common_domains = [
        ("架构设计", ["架构", "架构师", "设计模式", "系统设计"]),
        ("性能优化", ["性能", "优化", "benchmark", "profiling"]),
        ("安全", ["安全", "加密", "认证", "授权"]),
        ("测试", ["测试", "单元测试", "集成测试", "E2E"]),
        ("DevOps", ["部署", "CI/CD", "Docker", "K8s"]),
        ("数据库", ["数据库", "SQL", "索引", "优化"]),
        ("前端", ["React", "Vue", "CSS", "前端"]),
        ("后端", ["API", "微服务", "Node", "Python", "Go"]),
    ]

    for domain, keywords in common_domains:
        domain_mentioned = any(kw in " ".join(known_topics) for kw in keywords)
        if not domain_mentioned:
            gaps.append({
                "domain": domain,
                "suggestion": f"缺少{domain}相关记录，建议补充"
            })

    return gaps


# ══════════════════════════════════════════════════════════════
#  报告生成
# ══════════════════════════════════════════════════════════════

def generate_summary_report(memories, access_patterns, topics, uid):
    """生成综合洞察报告"""
    now = datetime.now().strftime("%Y-%m-%d %H:%M")

    level_dist = analyze_level_distribution(memories)
    related_stats = analyze_related_strength(memories)
    importance_dist = analyze_importance_distribution(memories)
    recency_dist = analyze_recency(memories)
    patterns = detect_patterns(memories, access_patterns)
    gaps = find_knowledge_gaps(memories, topics)

    # 洞察生成
    insights = []

    # 层级洞察
    total = len(memories)
    if total > 0:
        if level_dist.get("L3", 0) / total < 0.1:
            insights.append("📚 永久记忆较少，考虑将核心知识升级到 L3")
        if level_dist.get("L1", 0) / total > 0.7:
            insights.append("⚠️  临时记忆过多，建议定期整理，重要内容升级到 L2/L3")

    # 重要性洞察
    if importance_dist["avg"] < 50:
        insights.append("📊 平均重要性偏低，记忆可能缺乏足够的访问和关注")

    # 关联洞察
    if related_stats["none"] / total > 0.5:
        insights.append("🔗 大量记忆无关联，建议使用语义搜索重新整理")

    # 模式洞察
    high_access = [p for p in patterns if p["type"] == "high_access_low_importance"]
    if high_access:
        insights.append(f"🔥 发现 {len(high_access)} 条高访问低重要性记忆，建议提升权重")

    # 趋势洞察
    recent = sum(recency_dist.values())
    if recent < 5:
        insights.append("📝 最近记忆较少，建议保持每日记录习惯")

    report = f"""# 🧠 AI 记忆洞察报告

**生成时间**: {now}
**用户**: {uid}
**记忆总数**: {total}

---

## 📊 记忆概览

| 指标 | 数值 |
|------|------|
| 私人记忆 | {level_dist.get('private', 0)} 条 |
| 公共记忆 | {level_dist.get('shared', 0)} 条 |
| L1 临时 | {level_dist.get('L1', 0)} 条 ({level_dist.get('L1', 0)/max(total,1)*100:.0f}%) |
| L2 长期 | {level_dist.get('L2', 0)} 条 ({level_dist.get('L2', 0)/max(total,1)*100:.0f}%) |
| L3 永久 | {level_dist.get('L3', 0)} 条 ({level_dist.get('L3', 0)/max(total,1)*100:.0f}%) |

---

## ⭐ 重要性分布

- **高重要性** (≥75): {importance_dist['high']} 条
- **中等** (50-74): {importance_dist['medium']} 条
- **低** (<50): {importance_dist['low']} 条
- **平均分**: {importance_dist['avg']:.1f}

---

## 🔗 关联分析

| 类型 | 数量 |
|------|------|
| 强关联 (≥3条) | {related_stats['strong']} 条 |
| 弱关联 (1-2条) | {related_stats['weak']} 条 |
| 无关联 | {related_stats['none']} 条 |

---

## ⏰ 时间分布

| 时间范围 | 数量 |
|----------|------|
| 今天 | {recency_dist['today']} 条 |
| 近7天 | {recency_dist['week']} 条 |
| 近30天 | {recency_dist['month']} 条 |
| 更早 | {recency_dist['older']} 条 |

---

## 🔥 发现的问题

"""

    if patterns:
        for i, p in enumerate(patterns[:5], 1):
            report += f"{i}. **{p['suggestion']}**\nfrom typing import List, Dict, Optional, Union, Any, Tuple\n"
            if "title" in p:
                report += f"   - {p['title']}\n"
    else:
        report += "暂未发现问题 ✓\n"

    report += "\n---\n\n## 💡 AI 洞察\n\n"
    if insights:
        for insight in insights:
            report += f"- {insight}\n"
    else:
        report += "- 记忆系统运行良好，继续保持！\n"

    report += "\n---\n\n## 🎯 建议补充的知识领域\n\n"

    if gaps:
        for g in gaps[:5]:
            report += f"- {g['suggestion']}\n"
    else:
        report += "- 知识覆盖良好！\n"

    return report


def generate_daily_report(memories, access_patterns, uid):
    """生成每日洞察"""
    now = datetime.now().strftime("%Y-%m-%d")

    today_memories = [m for m in memories if (datetime.now().timestamp() - m.get("mtime", 0)) < 86400]
    today_access = {cid: p for cid, p in access_patterns.items() if p["last"] > (datetime.now() - timedelta(days=1)).timestamp()}

    report = f"""# 📅 每日洞察 - {now}

**用户**: {uid}

---

## 📝 今日记忆

- 新增记忆: {len(today_memories)} 条

"""

    if today_memories:
        report += "### 最新记录\n\n"
        for m in today_memories[:5]:
            report += f"- [{m.get('level', 'L1')}] {m.get('title', '无标题')}\n"
            report += f"  {m.get('content', '')[:80]}...\n\n"

    report += "\n## 🔍 今日搜索\n\n"
    search_access = [cid for cid, p in today_access.items() if "search" in p.get("types", {})]
    if search_access:
        report += f"搜索了 {len(search_access)} 条记忆\n"
    else:
        report += "今日暂无搜索记录\n"

    report += "\n---\n\n## 💡 每日建议\n\n"

    if len(today_memories) == 0:
        report += "- 今日尚未记录，养成每日总结的习惯很重要\n"
    elif len(today_memories) < 3:
        report += "- 今日记录较少，考虑补充更多细节\n"
    else:
        report += "- 今日记录良好！\n"

    return report


def generate_weekly_report(memories, access_patterns, uid):
    """生成每周洞察"""
    now = datetime.now().strftime("%Y-W%W")

    week_memories = [m for m in memories if (datetime.now().timestamp() - m.get("mtime", 0)) < 7 * 86400]
    topics = analyze_topics(week_memories)
    level_dist = analyze_level_distribution(week_memories)

    report = f"""# 📆 每周洞察 - {now}

**用户**: {uid}

---

## 📊 本周概况

| 指标 | 数值 |
|------|------|
| 本周新增 | {len(week_memories)} 条 |
| L1 临时 | {level_dist.get('L1', 0)} 条 |
| L2 长期 | {level_dist.get('L2', 0)} 条 |
| L3 永久 | {level_dist.get('L3', 0)} 条 |

---

## 🔥 本周热点主题

"""

    if topics:
        for i, (topic, cnt) in enumerate(topics[:10], 1):
            report += f"{i}. **{topic}** ({cnt} 次)\n"
    else:
        report += "本周暂无主题记录\n"

    report += "\n---\n\n## 📈 本周趋势\n\n"

    daily_counts = Counter()
    for m in week_memories:
        mtime = datetime.fromtimestamp(m.get("mtime", 0))
        daily_counts[mtime.strftime("%Y-%m-%d")] += 1

    for day in sorted(daily_counts.keys()):
        cnt = daily_counts[day]
        bar = "█" * cnt + "░" * max(0, 10 - cnt)
        report += f"{day}: {bar} {cnt}条\n"

    report += "\n---\n\n## 🧠 本周总结\n\n"

    if len(week_memories) < 7:
        report += f"- 本周记录较少（{len(week_memories)}条），建议增加记录频率\n"
    elif len(week_memories) > 30:
        report += "- 本周记录丰富，注意定期整理和归档\n"
    else:
        report += "- 记录节奏良好\n"

    if level_dist.get("L3", 0) == 0 and level_dist.get("L2", 0) > 5:
        report += "- 本周有内容可考虑升级到永久记忆\n"

    return report


# ══════════════════════════════════════════════════════════════
#  自动标记
# ══════════════════════════════════════════════════════════════

def auto_mark_importance(conn, memories, access_patterns):
    """自动标记高价值记忆"""
    c = conn.cursor()
    marked = 0

    for m in memories:
        cid = m.get("chunk_id")
        current_score = m.get("importance_score", 50)

        # 如果是高访问记忆，但重要性低 -> 提升
        if cid in access_patterns:
            access_cnt = access_patterns[cid]["count"]
            if access_cnt >= 10 and current_score < 70:
                new_score = min(100, current_score + 20)
                c.execute(
                    "UPDATE memory_embeddings SET importance_score = ? WHERE chunk_id = ?",
                    (new_score, cid)
                )
                marked += 1

        # 如果是永久记忆且无重要标记 -> 确保至少70分
        if m.get("level") == "L3" and current_score < 70:
            c.execute(
                "UPDATE memory_embeddings SET importance_score = ? WHERE chunk_id = ?",
                (70, cid)
            )
            marked += 1

    conn.commit()
    return marked


# ══════════════════════════════════════════════════════════════
#  主入口
# ══════════════════════════════════════════════════════════════

def generate_insight(base_dir, uid, report_type="summary", days=None):
    """生成洞察报告"""
    db_path = get_db_path(base_dir)
    conn = connect_db(db_path)

    if conn is None:
        raise Exception(f"向量数据库不存在: {db_path}")

    memories = extract_memories(conn, uid, days)
    access_patterns = extract_access_patterns(base_dir, uid, days or 7)
    topics = analyze_topics(memories)

    if report_type == "daily":
        report = generate_daily_report(memories, access_patterns, uid)
    elif report_type == "weekly":
        report = generate_weekly_report(memories, access_patterns, uid)
    else:
        report = generate_summary_report(memories, access_patterns, topics, uid)

    conn.close()
    return report


def main():
    parser = argparse.ArgumentParser(description="AI 记忆洞察与自我进化")
    parser.add_argument("--base-dir", default=".", help="记忆仓库根目录")
    parser.add_argument("--uid", required=True, help="用户ID")
    parser.add_argument("--days", type=int, help="分析最近N天")
    parser.add_argument("--daily", action="store_true", help="每日洞察")
    parser.add_argument("--weekly", action="store_true", help="每周洞察")
    parser.add_argument("--auto-mark", action="store_true", help="自动标记重要记忆")
    parser.add_argument("--find-gaps", action="store_true", help="发现知识盲点")
    parser.add_argument("--learn-patterns", action="store_true", help="学习记忆模式")
    parser.add_argument("--save", help="保存报告到文件")
    args = parser.parse_args()

    base_dir = os.path.abspath(args.base_dir)
    db_path = get_db_path(base_dir)

    if not os.path.exists(db_path):
        print(f"❌ 向量数据库不存在: {db_path}")
        print("   请先运行: python scripts/memory_embed.py --uid " + args.uid)
        return 1

    conn = connect_db(db_path)
    memories = extract_memories(conn, args.uid, args.days)
    access_patterns = extract_access_patterns(base_dir, args.uid, args.days or 7)

    if args.auto_mark:
        marked = auto_mark_importance(conn, memories, access_patterns)
        print(f"✅ 自动标记完成，{marked} 条记忆已更新")
        conn.close()
        return 0

    if args.find_gaps:
        topics = analyze_topics(memories)
        gaps = find_knowledge_gaps(memories, topics)
        print("\n🎯 知识盲点分析:\n")
        for g in gaps:
            print(f"  - {g['suggestion']}")
        conn.close()
        return 0

    if args.learn_patterns:
        patterns = detect_patterns(memories, access_patterns)
        print("\n🧠 记忆模式分析:\n")
        for p in patterns[:10]:
            print(f"  [{p['type']}] {p['suggestion']}")
        conn.close()
        return 0

    # 生成报告
    if args.daily:
        report = generate_daily_report(memories, access_patterns, args.uid)
    elif args.weekly:
        report = generate_weekly_report(memories, access_patterns, args.uid)
    else:
        topics = analyze_topics(memories)
        report = generate_summary_report(memories, access_patterns, topics, args.uid)

    print(report)

    if args.save:
        with open(args.save, "w", encoding="utf-8") as f:
            f.write(report)
        print(f"\n💾 已保存到: {args.save}")

    conn.close()
    return 0


if __name__ == "__main__":
    sys.exit(main())


path: scripts/memory_onboard.py
content: #!/usr/bin/env python3
"""
Memory Manager 首次使用引导脚本

用法:
    python memory_onboard.py              # 交互式引导
    python memory_onboard.py --silent     # 静默模式 (使用默认/参数)

首次运行会自动引导用户完成:
1. 配置用户名
2. 选择 Embedding 后端
3. 输入 API Key
4. 初始化记忆仓库
"""

import os
import sys
import json
import getpass
from pathlib import Path
from datetime import datetime

# 版本信息
VERSION = "3.5.0"

# ══════════════════════════════════════════════════════════════
#  颜色输出
# ══════════════════════════════════════════════════════════════

try:
    from colorama import init, Fore, Style
    init(autoreset=True)
    RESET = Style.RESET_ALL
    BOLD = Style.BRIGHT
    RED = Fore.RED
    GREEN = Fore.GREEN
    YELLOW = Fore.YELLOW
    CYAN = Fore.CYAN
    MAGENTA = Fore.MAGENTA
except ImportError:
    # Fallback: 无颜色
    RESET = BOLD = RED = GREEN = YELLOW = CYAN = MAGENTA = ""

def print_banner():
    """打印欢迎横幅"""
    print(f"""
{MAGENTA}╔════════════════════════════════════════════════════════════╗{RESET}
{MAGENTA}║                                                            ║{RESET}
{MAGENTA}║          🎉 Memory Manager 首次使用引导                    ║{RESET}
{MAGENTA}║                                                            ║{RESET}
{MAGENTA}║              版本: v{VERSION}{' ' * (37 - len(VERSION))}║{RESET}
{MAGENTA}║                                                            ║{RESET}
{MAGENTA}╚════════════════════════════════════════════════════════════╝{RESET}
    """)

def print_step(step, total, title):
    """打印步骤标题"""
    print(f"\nfrom typing import List, Dict, Optional, Union, Any, Tuple\n{CYAN}[{step}/{total}] {title}{RESET}\n")

def print_success(msg):
    print(f"  {GREEN}✅ {msg}{RESET}")

def print_info(msg):
    print(f"  {CYAN}ℹ️  {msg}{RESET}")

def print_warn(msg):
    print(f"  {YELLOW}⚠️  {msg}{RESET}")

def print_error(msg):
    print(f"  {RED}❌ {msg}{RESET}")

def print_box(title, lines=None):
    """打印带边框的文本块"""
    print(f"\n{CYAN}┌{'─' * 58}┐{RESET}")
    print(f"{CYAN}│{RESET}  {BOLD}{title}{RESET}{' ' * (54 - len(title))}{CYAN}│{RESET}")
    if lines:
        for line in lines:
            print(f"{CYAN}│{RESET}  {line}{' ' * (54 - len(line))}{CYAN}│{RESET}")
    print(f"{CYAN}└{'─' * 58}┘{RESET}")

# ══════════════════════════════════════════════════════════════
#  用户输入
# ══════════════════════════════════════════════════════════════

def input_text(prompt, default=None, required=False, password=False):
    """安全的用户输入"""
    if default:
        prompt_full = f"{prompt} [{default}]: "
    else:
        prompt_full = f"{prompt}: "

    while True:
        try:
            if password:
                value = getpass.getpass(prompt_full)
            else:
                value = input(prompt_full).strip()

            if not value and default:
                value = default

            if required and not value:
                print_warn("此项为必填项")
                continue

            return value
        except (KeyboardInterrupt, EOFError):
            print("\n\n已取消")
            sys.exit(0)

def input_choice(prompt, options, default=0):
    """选择输入"""
    print(f"\n{prompt}")
    for i, (key, desc) in enumerate(options):
        marker = "(推荐)" if i == default else ""
        print(f"  {CYAN}[{i + 1}]{RESET} {desc} {YELLOW}{marker}{RESET}")

    while True:
        try:
            choice = input(f"\n请选择 [{default + 1}]: ").strip()
            if not choice:
                choice = str(default + 1)
            idx = int(choice) - 1
            if 0 <= idx < len(options):
                return options[idx][0]
            print_warn(f"请输入 1-{len(options)} 之间的数字")
        except ValueError:
            print_warn("请输入有效数字")

def input_yes_no(prompt, default=True):
    """是/否输入"""
    default_str = "[Y/n]" if default else "[y/N]"
    while True:
        try:
            response = input(f"{prompt} {default_str}: ").strip().lower()
            if not response:
                return default
            if response in ('y', 'yes', '是'):
                return True
            if response in ('n', 'no', '否'):
                return False
            print_warn("请输入 y/n")
        except (KeyboardInterrupt, EOFError):
            print("\n已取消")
            sys.exit(0)

# ══════════════════════════════════════════════════════════════
#  配置步骤
# ══════════════════════════════════════════════════════════════

def step_user_info():
    """步骤 1: 用户信息"""
    print_step(1, 5, "配置用户信息")

    # 获取默认用户名
    default_user = os.environ.get("USER") or os.environ.get("USERNAME") or "user"

    user_name = input_text("👤 请输入用户名 (用于区分多用户)", default=default_user, required=True)
    print_success(f"用户 ID: {user_name}")

    return {"uid": user_name}

def step_backend():
    """步骤 2: 选择后端"""
    print_step(2, 5, "选择 Embedding 后端")

    backends = [
        ("openai", "OpenAI (推荐, 需要 API Key)"),
        ("siliconflow", "SiliconFlow (国内可用)"),
        ("zhipu", "Zhipu (国内可用)"),
    ]

    backend = input_choice("🌐 选择 Embedding 服务:", backends, default=0)
    print_success(f"后端: {backend}")

    return {"backend": backend}

def step_api_key(config):
    """步骤 3: 配置 API Key"""
    print_step(3, 5, "配置 API Key")

    backend = config["backend"]
    env_vars = {
        "openai": "OPENAI_API_KEY",
        "siliconflow": "SILICONFLOW_API_KEY",
        "zhipu": "ZHIPU_API_KEY",
    }
    prompts = {
        "openai": "OpenAI API Key (sk-...)",
        "siliconflow": "SiliconFlow API Key",
        "zhipu": "Zhipu API Key",
    }

    env_var = env_vars[backend]

    # 检查环境变量
    existing_key = os.environ.get(env_var, "")
    if existing_key and existing_key.startswith("sk-"):
        print_success(f"检测到已配置的环境变量 {env_var}")
        if input_yes_no("是否使用已保存的 Key?", default=True):
            return {"api_key": existing_key, "env_var": env_var}

    # 交互输入
    print_info(f"\n请前往以下网址获取 {backend} API Key:")
    urls = {
        "openai": "https://platform.openai.com/api-keys",
        "siliconflow": "https://siliconflow.cn/",
        "zhipu": "https://open.bigmodel.cn/",
    }
    print(f"  {CYAN}{urls.get(backend, '')}{RESET}\n")

    [REDACTED_SECRET], password=True)

    if [REDACTED_SECRET]"API Key 已记录")
        return {"api_key": api_key, "env_var": env_var}
    else:
        print_warn("未输入 API Key，尝试使用环境变量方式")
        return {"api_key": "", "env_var": env_var}

def step_memory_path():
    """步骤 4: 记忆仓库路径"""
    print_step(4, 5, "设置记忆仓库")

    script_dir = Path(__file__).parent.parent
    default_path = str(script_dir / "memory")

    memory_path = input_text("📁 记忆仓库路径", default=default_path)

    # 验证/创建目录
    path = Path(memory_path).resolve()
    if not path.exists():
        try:
            path.mkdir(parents=True, exist_ok=True)
            print_success(f"目录已创建: {path}")
        except Exception as e:
            print_error(f"无法创建目录: {e}")
            sys.exit(1)

    print_success(f"记忆仓库: {path}")
    return {"base_dir": str(path)}

def step_git_sync(config):
    """步骤 5: GitHub 同步"""
    print_step(5, 5, "GitHub 同步 (可选)")

    if not input_yes_no("是否配置 GitHub 同步?", default=False):
        print_info("跳过 GitHub 配置")
        return {"git_sync": False}

    repo_url = input_text("🔗 GitHub 仓库 URL ([REDACTED_SECRET]:user/repo.git)")
    if not repo_url:
        print_info("跳过 GitHub 配置")
        return {"git_sync": False}

    return {"git_sync": True, "repo_url": repo_url}

# ══════════════════════════════════════════════════════════════
#  保存配置
# ══════════════════════════════════════════════════════════════

def save_config(config):
    """保存配置到文件"""
    # 全局配置
    config_dir = Path.home() / ".memory-manager"
    config_dir.mkdir(parents=True, exist_ok=True)

    config_path = config_dir / "config.json"

    # 移除敏感信息
    save_data = {k: v for k, v in config.items() if k != "api_key"}
    save_data["version"] = VERSION
    save_data["created"] = datetime.now().isoformat()

    with open(config_path, "w", encoding="utf-8") as f:
        json.dump(save_data, f, indent=4, ensure_ascii=False)

    print_success(f"配置已保存: {config_path}")

    # 本地配置
    base_dir = config.get("base_dir", ".")
    local_config_path = Path(base_dir) / ".mm.json"

    local_config = {
        "default": {
            "uid": config.get("uid"),
            "base_dir": base_dir,
            "scope": "private"
        }
    }

    with open(local_config_path, "w", encoding="utf-8") as f:
        json.dump(local_config, f, indent=4, ensure_ascii=False)

    print_success(f"本地配置: {local_config_path}")

    # API Key 到环境变量
    if config.get("api_key") and config.get("env_var"):
        # 打印建议添加到 shell 配置
        print_info("\n建议将以下内容添加到 ~/.bashrc 或 ~/.zshrc:")
        print(f"  {CYAN}export {config['env_var']}='{config['api_key'][:10]}...'{RESET}")

def init_directories(config):
    """初始化目录结构"""
    base_dir = Path(config["base_dir"])
    uid = config["uid"]

    # 公共目录
    shared_dirs = [
        base_dir / "shared" / "daily",
        base_dir / "shared" / "weekly",
        base_dir / "shared" / "permanent",
    ]

    # 用户目录
    user_dirs = [
        base_dir / "users" / uid / "daily",
        base_dir / "users" / uid / "weekly",
        base_dir / "users" / uid / "permanent",
    ]

    for d in shared_dirs + user_dirs:
        d.mkdir(parents=True, exist_ok=True)

    # 创建 INDEX.md
    index_path = base_dir / "users" / uid / "INDEX.md"
    if not index_path.exists():
        index_content = f"""# 用户索引 - {uid}

## 基本信息
- 用户名: {uid}
- 创建时间: {datetime.now().strftime('%Y-%m-%d %H:%M')}
- 版本: v{VERSION}

## 记忆层级
| 层级 | 位置 | 说明 |
|------|------|------|
| L1 临时 | `daily/` | 当天记录，自动清理 |
| L2 长期 | `weekly/` | 本周总结 |
| L3 永久 | `permanent/` | 重要记忆，长期保存 |

## 快速命令
```bash
mm log "记录内容"    # 快速记录
mm search "关键词"   # 搜索
mm insight           # AI 总结
mm stats             # 统计
```

## 使用提示
- 重要记忆标记 `[IMPORTANT]` 可自动升级到 L2
- 永久记忆标记 `[PERMANENT]` 可升级到 L3
- 运行 `mm --help` 查看所有命令
"""
        with open(index_path, "w", encoding="utf-8") as f:
            f.write(index_content)

    print_success("目录结构初始化完成")

# ══════════════════════════════════════════════════════════════
#  完成
# ══════════════════════════════════════════════════════════════

def show_completion(config):
    """显示完成信息"""
    print(f"""
{GREEN}╔════════════════════════════════════════════════════════════╗{RESET}
{GREEN}║                                                            ║{RESET}
{GREEN}║  ✅ Memory Manager 配置完成！                              ║{RESET}
{GREEN}║                                                            ║{RESET}
{GREEN}╚════════════════════════════════════════════════════════════╝{RESET}

{BOLD}📖 快速开始:{RESET}

  {CYAN}mm log "这是我的第一条记忆"{RESET}   # 📝 记录记忆
  {CYAN}mm search "查找相关内容"{RESET}       # 🔍 语义搜索
  {CYAN}mm insight{RESET}                      # 🧠 AI 自我总结
  {CYAN}mm stats{RESET}                        # 📊 查看统计

{BOLD}📚 详细文档:{RESET}

  SKILL.md      - AI 使用的 Skill 文档
  README.md     - 完整使用说明
  reference.md  - 命令参考手册

{BOLD}⚠️  重要提示:{RESET}

  1. 请确保 API Key 已配置 (环境变量或 shell 配置)
  2. 运行 `mm embed` 生成向量索引
  3. 运行 `mm sync` 同步到 GitHub (如已配置)
  4. 查看 INDEX.md 了解记忆组织方式

""")

# ══════════════════════════════════════════════════════════════
#  主入口
# ══════════════════════════════════════════════════════════════

def main():
    import argparse

    parser = argparse.ArgumentParser(
        description="Memory Manager 首次使用引导",
        formatter_class=argparse.RawDescriptionHelpFormatter,
    )
    parser.add_argument("--silent", action="store_true", help="静默模式")
    parser.add_argument("--uid", help="用户名")
    parser.add_argument("--backend", choices=["openai", "siliconflow", "zhipu"], help="Embedding 后端")
    parser.add_argument("--path", dest="base_dir", help="记忆仓库路径")
    parser.add_argument("--skip-git", action="store_true", help="跳过 Git 配置")

    args = parser.parse_args()

    print_banner()

    config = {}

    # 静默模式
    if args.silent or args.uid:
        config["uid"] = args.uid or os.environ.get("USER", "user")
        config["backend"] = args.backend or "openai"
        config["base_dir"] = args.base_dir or "./memory"
        config["git_sync"] = False

        # 尝试从环境变量获取 API Key
        env_vars = {
            "openai": "OPENAI_API_KEY",
            "siliconflow": "SILICONFLOW_API_KEY",
            "zhipu": "ZHIPU_API_KEY",
        }
        config["api_key"] = os.environ.get(env_vars.get(config["backend"], ""), "")
        config["env_var"] = env_vars.get(config["backend"], "")
    else:
        # 交互模式
        config.update(step_user_info())
        config.update(step_backend())
        config.update(step_api_key(config))
        config.update(step_memory_path())

        if not args.skip_git:
            config.update(step_git_sync(config))
        else:
            config["git_sync"] = False

    # 保存配置
    print("\n" + "─" * 60)
    save_config(config)
    init_directories(config)

    # 完成
    show_completion(config)

if __name__ == "__main__":
    main()


path: scripts/memory_search.py
content: #!/usr/bin/env python3
"""
memory_search.py - 基于向量相似度的语义记忆搜索

用法:
  python scripts/memory_search.py --uid pupper "我上周做了什么关于memory的事"
  python scripts/memory_search.py --uid pupper "团队进展" --scope shared
  python scripts/memory_search.py --uid pupper "技术" --level L2 L3
  python scripts/memory_search.py --uid pupper "记忆" --top-k 5
  python scripts/memory_search.py --uid pupper "memory" --keyword-only
  python scripts/memory_search.py --uid pupper --stats
  python scripts/memory_search.py --uid pupper --rebuild-index  # 重建 LanceDB 索引
  python scripts/memory_search.py --list-backends

依赖:
  pip install openai numpy lancedb  # lancedb 可选，增强搜索性能

优化特性:
  - LanceDB HNSW 索引（需安装 lancedb）
  - 查询缓存（避免重复搜索）
  - 自动回退（无 LanceDB 时使用 SQLite）
"""

import os
import sys
import sqlite3
import argparse
import time
from datetime import datetime
from pathlib import Path
from typing import List, Dict, Optional, Union, Any, Tuple

try:
    import numpy as np
except ImportError:
    np = None

# 确保同目录模块可导入（跨平台兼容）
import sys as _sys
from pathlib import Path as _Path
_sys.path.insert(0, str(_Path(__file__).parent))

from embed_backends import get_embedding, get_backend_info

# LanceDB 懒加载
try:
    from lancedb_integration import LanceDBManager, SearchCache, smart_search
    LANCEDB_AVAILABLE = True
except ImportError:
    LANCEDB_AVAILABLE = False
    LanceDBManager = None
    SearchCache = None
    smart_search = None

DB_NAME = ".memory_vectors.db"


def get_db_path(base_dir):
    return os.path.join(os.path.abspath(base_dir), DB_NAME)


def connect_db(db_path):
    if not os.path.exists(db_path):
        return None
    conn = sqlite3.connect(db_path)
    conn.execute("PRAGMA journal_mode=WAL")
    return conn


def cosine_similarity(a: np.ndarray, b: np.ndarray) -> float:
    """计算余弦相似度（两个向量已归一化）"""
    norm_a = np.linalg.norm(a)
    norm_b = np.linalg.norm(b)
    if norm_a == 0 or norm_b == 0:
        return 0.0
    return float(np.dot(a, b) / (norm_a * norm_b))


def search_vectors_lancedb(query_emb, uid, scope, levels, top_k, db_path):
    """
    LanceDB HNSW 向量搜索
    
    优点：
    - O(log n) 复杂度 vs O(n) SQLite 扫描
    - 100x 速度提升（10k+ 向量时）
    """
    if not LANCEDB_AVAILABLE:
        return None
    
    manager = LanceDBManager(db_path)
    if not manager.connect():
        return None
    
    # 转换 numpy array 到 list
    query_vec = query_emb.tolist() if hasattr(query_emb, 'tolist') else list(query_emb)
    
    results = manager.search(
        query_vec, 
        top_k * 2,  # 多取一些，后面过滤
        uid=uid, 
        scope=scope if scope != "all" else None,
        level=levels[0] if levels and len(levels) == 1 else None
    )
    
    manager.close()
    
    if not results:
        return None
    
    return results[:top_k]


def search_vectors(conn, query_emb, uid, scope, levels, top_k, min_score, semantic_weight=0.6):
    """向量语义搜索（SQLite 全表扫描）"""
    c = conn.cursor()
    conditions = []
    params = []
    if uid != "*":
        conditions.append("uid = ?")
        params.append(uid)
    if scope != "*":
        conditions.append("scope = ?")
        params.append(scope)
    if levels:
        ph = ",".join("?" * len(levels))
        conditions.append(f"level IN ({ph})")
        params.extend(levels)
    where_clause = "WHERE " + " AND ".join(conditions) if conditions else ""

    c.execute(f"SELECT * FROM memory_embeddings {where_clause}", params)
    cols = [
        "id", "chunk_id", "uid", "scope", "level", "filepath",
        "section_idx", "title", "semantic_key", "content",
        "raw_content", "embedding", "content_hash", "mtime", "created_at",
        "importance_score", "related_ids"
    ]

    scored = []
    for row in c.fetchall():
        emb_bytes = row[cols.index("embedding")]
        emb = np.frombuffer(emb_bytes, dtype=np.float32)
        sim_score = cosine_similarity(query_emb, emb)
        if sim_score >= min_score:
            # 综合评分 = 语义相似度 × semantic_weight + 重要性 × (1-semantic_weight)
            imp_idx = cols.index("importance_score")
            importance = float(row[imp_idx]) if row[imp_idx] is not None else 50.0
            combined = sim_score * semantic_weight + (importance / 100.0) * (1 - semantic_weight)
            scored.append((combined, sim_score, importance, row))

    scored.sort(key=lambda x: -x[0])
    # 返回: (combined_score, sim_score, importance, row)
    return [(s[0], s[1], s[2], s[3]) for s in scored[:top_k]], cols


def keyword_search(conn, query, uid, scope, levels, top_k, semantic_weight=0.6):
    """纯关键词搜索（无 API 依赖）"""
    c = conn.cursor()
    conditions = []
    params = []
    if uid != "*":
        conditions.append("uid = ?")
        params.append(uid)
    if scope != "*":
        conditions.append("scope = ?")
        params.append(scope)
    if levels:
        ph = ",".join("?" * len(levels))
        conditions.append(f"level IN ({ph})")
        params.extend(levels)
    where_clause = "WHERE " + " AND ".join(conditions) if conditions else ""

    c.execute(f"SELECT * FROM memory_embeddings {where_clause}", params)
    cols = [
        "id", "chunk_id", "uid", "scope", "level", "filepath",
        "section_idx", "title", "semantic_key", "content",
        "raw_content", "embedding", "content_hash", "mtime", "created_at",
        "importance_score", "related_ids"
    ]

    keywords = query.lower().split()
    scored = []
    for row in c.fetchall():
        text = (row[cols.index("title")] + " " + row[cols.index("content")]).lower()
        sim_score = sum(1 for kw in keywords if kw in text) / len(keywords)
        if sim_score > 0:
            imp_idx = cols.index("importance_score")
            importance = float(row[imp_idx]) if row[imp_idx] is not None else 50.0
            combined = sim_score * semantic_weight + (importance / 100.0) * (1 - semantic_weight)
            scored.append((combined, sim_score, importance, row))

    scored.sort(key=lambda x: -x[0])
    return [(s[0], s[1], s[2], s[3]) for s in scored[:top_k]], cols


def format_row(combined, sim_score, importance, row, cols):
    idx = {col: i for i, col in enumerate(cols)}
    uid = row[idx["uid"]]
    level = row[idx["level"]]
    filepath = row[idx["filepath"]]
    title = row[idx["title"]]
    sk = row[idx["semantic_key"]]
    content = row[idx["content"]]
    mtime = datetime.fromtimestamp(row[idx["mtime"]]).strftime("%Y-%m-%d")
    owner = f"@{uid}" if uid != "shared" else "shared"

    # 重要性显示
    imp_bar = "█" * int(importance / 10) + "░" * (10 - int(importance / 10))

    # 关联记忆
    related_ids_raw = row[idx.get("related_ids", -1)]
    related_info = ""
    if related_ids_raw and related_ids_raw != "[]":
        try:
            import json as _json
            rids = _json.loads(related_ids_raw)
            if rids:
                related_info = f"\n   🔗 关联 {len(rids)} 条记忆: {', '.join(rids[:3])}"
                if len(rids) > 3:
                    related_info += f" (+{len(rids)-3} more)"
        except Exception:
            pass

    snippet = content[:150].replace("\n", " ").strip()
    if len(content) > 150:
        snippet += "..."

    star = "⭐⭐⭐" if sim_score >= 0.85 else ("⭐⭐" if sim_score >= 0.75 else ("⭐" if sim_score >= 0.65 else "  "))

    lines = [
        f"{star} [{combined:.2f}] {owner}/{level} — {title}  [{imp_bar}]",
        f"   📍 {filepath} | {mtime} | 重要性: {importance:.0f}",
        f"   🔑 {sk}",
        f"   💬 {snippet}{related_info}",
        "",
    ]
    return "\n".join(lines)


def format_raw_row(combined, row, cols):
    idx = {col: i for i, col in enumerate(cols)}
    return f"\n{'='*60}\n{row[idx['raw_content']]}\n{'='*60}\n"


def print_stats(conn, db_path=None):
    c = conn.cursor()
    print("\n📊 向量库统计")
    print("─" * 40)
    try:
        info = get_backend_info(db_path=db_path)
        print(f"  后端: {info['name']} ({info['dim']}维)")
    except Exception:
        pass
    c.execute("SELECT COUNT(*) FROM memory_embeddings")
    total = c.fetchone()[0]
    print(f"  总向量数: {total}")
    c.execute("SELECT uid, COUNT(*) FROM memory_embeddings GROUP BY uid ORDER BY COUNT(*) DESC")
    for uid, cnt in c.fetchall():
        print(f"    {uid}: {cnt} 条")
    c.execute("SELECT level, COUNT(*) FROM memory_embeddings GROUP BY level ORDER BY level")
    for level, cnt in c.fetchall():
        print(f"  {level}: {cnt} 条")

    # 关联记忆统计
    try:
        c.execute("SELECT COUNT(*) FROM memory_embeddings WHERE related_ids != '[]' AND related_ids IS NOT NULL")
        related_cnt = c.fetchone()[0]
        print(f"\n  🔗 有关联记忆: {related_cnt} 条")
    except Exception:
        pass

    # 平均重要性
    try:
        c.execute("SELECT AVG(importance_score) FROM memory_embeddings WHERE importance_score IS NOT NULL")
        avg_imp = c.fetchone()[0]
        if avg_imp:
            print(f"  📊 平均重要性: {avg_imp:.1f}")
    except Exception:
        pass
    c.execute("SELECT created_at FROM memory_embeddings ORDER BY created_at DESC LIMIT 1")
    row = c.fetchone()
    if row:
        print(f"\n  最新生成: {row[0][:16]}")


def main():
    parser = argparse.ArgumentParser(description="语义记忆搜索")
    parser.add_argument("--base-dir", default=".", help="记忆仓库根目录")
    parser.add_argument("--uid", required=True, help="用户ID")
    parser.add_argument("--query", help="搜索查询")
    parser.add_argument("--scope", default="private", choices=["private", "shared", "all", "*"],
                        help="搜索范围")
    parser.add_argument("--level", nargs="*", choices=["L1", "L2", "L3"],
                        help="限定层级")
    parser.add_argument("--top-k", type=int, default=10)
    parser.add_argument("--min-score", type=float, default=0.55)
    parser.add_argument("--semantic-weight", type=float, default=0.6,
                        help="语义权重 (0-1)，默认 0.6。0.6=语义优先，0.8=纯语义，0.3=重要性优先")
    parser.add_argument("--keyword-only", action="store_true", help="纯关键词模式")
    parser.add_argument("--stats", action="store_true", help="显示统计")
    parser.add_argument("--show-raw", action="store_true")
    parser.add_argument("--list-backends", action="store_true")
    # LanceDB 相关参数
    parser.add_argument("--no-lancedb", action="store_true", help="禁用 LanceDB HNSW 搜索")
    parser.add_argument("--no-cache", action="store_true", help="禁用查询缓存")
    parser.add_argument("--rebuild-index", action="store_true", help="重建 LanceDB 索引")
    parser.add_argument("--cache-stats", action="store_true", help="显示缓存统计")
    args = parser.parse_args()

    if args.list_backends:
        from embed_backends import list_backends
        print("\n可用 Embedding 后端:")
        for bid, info in list_backends().items():
            print(f"  {bid}: {info['name']} ({info['dim']}维)")
        return

    base_dir = os.path.abspath(args.base_dir)
    db_path = get_db_path(base_dir)

    if not os.path.exists(db_path):
        print(f"❌ 向量数据库不存在: {db_path}")
        print("   请先运行: python scripts/memory_embed.py --uid " + args.uid)
        sys.exit(1)

    # LanceDB 索引重建
    if args.rebuild_index:
        if not LANCEDB_AVAILABLE:
            print("❌ LanceDB 未安装。请运行: pip install lancedb")
            sys.exit(1)
        print("\n[1/2] 重建 LanceDB HNSW 索引...")
        manager = LanceDBManager(db_path)
        manager.connect()
        manager.rebuild_index(db_path)
        manager.close()
        print("[2/2] 缓存统计:")
        cache = SearchCache(db_path)
        print(f"   {cache.stats()}")
        print("\n✅ LanceDB 索引重建完成")
        return

    # 缓存统计
    if args.cache_stats:
        cache = SearchCache(db_path)
        stats = cache.stats()
        print("\n[Cache Statistics]")
        for k, v in stats.items():
            print(f"  {k}: {v}")
        return

    conn = connect_db(db_path)

    if args.stats:
        print_stats(conn, db_path=db_path)
        
        # 显示 LanceDB 状态
        if LANCEDB_AVAILABLE and not args.no_lancedb:
            print("\n[LanceDB Status]")
            manager = LanceDBManager(db_path)
            if manager.connect():
                print("  Status: Enabled")
                print("  Index Type: HNSW")
            else:
                print("  Status: Disabled")
            manager.close()
        elif not LANCEDB_AVAILABLE:
            print("\n[Hint] Install LanceDB for 100x faster search: pip install lancedb")
        
        conn.close()
        return

    if not args.query:
        print("❌ 请提供 --query 参数")
        sys.exit(1)

    use_lancedb = LANCEDB_AVAILABLE and not args.no_lancedb
    use_cache = not args.no_cache

    print(f"\n🔍 搜索: \"{args.query}\"")
    print(f"   范围: {args.scope} | 用户: {args.uid} | 层级: {args.level or '全部'} | top-k: {args.top_k}")
    print(f"   语义权重: {args.semantic_weight} | LanceDB: {'ON' if use_lancedb else 'OFF'} | Cache: {'ON' if use_cache else 'OFF'}\n")

    if args.keyword_only:
        print("  [关键词搜索]\n")
        results, cols = keyword_search(conn, args.query, args.uid, args.scope, args.level, args.top_k, args.semantic_weight)
    else:
        if np is None:
            print("⚠️  缺少 numpy，使用关键词模式...")
            results, cols = keyword_search(conn, args.query, args.uid, args.scope, args.level, args.top_k, args.semantic_weight)
        else:
            print("  [向量语义搜索]\n  🔮 生成查询向量...")
            try:
                query_emb = get_embedding(args.query, db_path=db_path)
            except Exception as e:
                print(f"  ❌ 向量生成失败: {e}")
                print("  → 回退到关键词搜索...\n")
                results, cols = keyword_search(conn, args.query, args.uid, args.scope, args.level, args.top_k, args.semantic_weight)
            else:
                # 尝试 LanceDB HNSW 搜索
                start_time = time.time()
                lancedb_results = None
                
                if use_lancedb:
                    print("  ⚡ 使用 LanceDB HNSW 搜索...")
                    lancedb_results = search_vectors_lancedb(
                        query_emb, args.uid, args.scope, args.level, args.top_k, db_path
                    )
                
                if lancedb_results:
                    # LanceDB 结果转换
                    elapsed = time.time() - start_time
                    print(f"  ⚡ LanceDB 搜索完成 ({elapsed*1000:.1f}ms)\n")
                    
                    # 转换 LanceDB 结果为标准格式
                    scored = []
                    cols = ["chunk_id", "uid", "scope", "level", "filepath", 
                           "title", "semantic_key", "content", "importance_score", "related_ids"]
                    for r in lancedb_results:
                        imp = r.get("importance_score", 50.0) or 50.0
                        # LanceDB 返回的是相似度分数
                        sim_score = r.get("_distance", 0.5)
                        combined = sim_score * args.semantic_weight + (imp / 100.0) * (1 - args.semantic_weight)
                        scored.append((combined, sim_score, imp, r))
                    scored.sort(key=lambda x: -x[0])
                    results = [(s[0], s[1], s[2], s[3]) for s in scored[:args.top_k]]
                else:
                    # 回退到 SQLite 全表扫描
                    if use_lancedb:
                        print("  ⚠️  LanceDB 不可用，回退到 SQLite...\n")
                    results, cols = search_vectors(
                        conn, query_emb, args.uid, args.scope, args.level, args.top_k, args.min_score, args.semantic_weight
                    )

    if not results:
        print("  😕 没有找到相关记忆")
        print(f"\n  💡 建议: 降低 --min-score / 用 --keyword-only / 先运行 memory_embed.py --rebuild")
        conn.close()
        return

    print(f"  ✅ 找到 {len(results)} 条相关记忆:\n")
    for i, (combined, sim_score, importance, row) in enumerate(results, 1):
        print(f"  ── 结果 {i}/{len(results)} ──")
        print(format_raw_row(combined, row, cols) if args.show_raw
              else format_row(combined, sim_score, importance, row, cols))

    if results[0][1] < 0.65:
        print("  💡 结果相似度偏低，建议: 用更具体的描述 / 运行 memory_embed.py --rebuild")

    # 记录搜索访问
    try:
        sys.path.insert(0, os.path.dirname(__file__))
        from memory_access_log import get_db_path as _al_path, log_access
        log_db = _al_path(base_dir)
        if os.path.exists(log_db):
            chunk_ids = [r[3][cols.index("chunk_id")] for r in results]
            for cid in chunk_ids:
                log_access(log_db, cid, args.uid, "search")
    except Exception:
        pass  # 访问日志不影响搜索功能

    conn.close()


if __name__ == "__main__":
    main()


path: tests/__init__.py
content: # Tests for memory-manager


path: tests/run_all_tests.py
content: #!/usr/bin/env python3
"""
run_all_tests.py - 运行所有单元测试
"""

import os
import sys
import io

# Windows GBK 环境支持
if sys.platform == 'win32':
    sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8', errors='replace')
    sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding='utf-8', errors='replace')

def run_test_file(filepath):
    """运行单个测试文件"""
    sep = "=" * 60
    print(f"\n{sep}")
    print(f"Running: {os.path.basename(filepath)}")
    print(sep)
    result = os.system(f'python "{filepath}"')
    return result == 0

def main():
    tests_dir = os.path.dirname(__file__)

    test_files = [
        os.path.join(tests_dir, "memory_embed_test.py"),
        os.path.join(tests_dir, "memory_search_test.py"),
        os.path.join(tests_dir, "memory_access_log_test.py"),
    ]

    print("[Rocket] Memory Manager - Unit Test Suite")
    print("=" * 60)

    results = []
    for tf in test_files:
        if os.path.exists(tf):
            results.append(run_test_file(tf))
        else:
            print(f"[Warning] Test file not found: {tf}")

    sep = "=" * 60
    print(f"\n{sep}")
    print("Stats - Final Results")
    print(sep)

    passed = sum(1 for r in results if r)
    total = len(results)

    if passed == total:
        print(f"[OK] All {total} test suites passed!")
        return 0
    else:
        print(f"[FAIL] {passed}/{total} test suites passed")
        return 1

if __name__ == "__main__":
    sys.exit(main())


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)