Skill: catplus-eric/nova-memory (version 1.0.0)

# nova-memory — Nova 工作空间自研记忆技能 ## 元信息 - **name**: nova-memory - **description**: Nova 工作空间自研记忆技能，提供本地语义记忆存储、检索、实体管理、自动反思能力。纯 Python 实现，无外部 API 依赖，安全可靠。 - **trigger words**: 记忆、remember、recall、实体、entity、反思、reflect、图谱、graph、搜索、search、标签、tag --- ## 功能说明 ### 核心能力 | 功能 | 说明 | |------|------| | `remember(content, tags, entity)` | 记忆一条信息，自动生成 ID，存储到本地 JSON | | `recall(query, top_k)` | 语义检索，基于 TF-IDF + Cosine Similarity 返回最相关记忆 | | `remember_entity(name, facts)` | 记住一个实体的多个属性 | | `get_entity(name)` | 获取实体完整信息 | | `search_by_tag(tag)` | 按标签精确搜索记忆 | | `get_memory_graph()` | 返回记忆图谱（实体关系可视化结构） | | `auto_reflect()` | 自动反思：基于近期记忆生成洞察摘要 | | `save()` / `load()` | 持久化保存与加载 | ### 技术方案 - **向量相似度**：TF-IDF + Cosine Similarity（sklearn，纯本地） - **存储格式**：JSON 文件，每条记忆独立一个 `.json` 文件 - **索引结构**： - 倒排索引：`tag → memory_id[]` - 向量索引：`memory_id → TF-IDF vector` - **实体图**：简单 JSON 图结构（`entity_name → {facts}`） - **存储路径**：`/workspace/memory/nova-memory/` ### 目录结构 ``` /workspace/skills/nova-memory/ ├── SKILL.md # 本文件 ├── src/ │ ├── __init__.py # 模块入口 │ └── memory_core.py # 核心引擎 └── scripts/ └── memory_cli.py # 命令行工具 ``` --- ## 使用示例 ### Python API ```python from nova_memory import NovaMemory # 初始化 mem = NovaMemory(storage_dir="/workspace/memory/nova-memory/") # 记忆一条信息 mid = mem.remember( content="Eric 喜欢在周末去重庆大学城附近骑行", tags=["eric", "偏好", "运动"], entity="Eric" ) # 语义检索 results = mem.recall("周末运动习惯", top_k=3) for r in results: print(r["content"], "→ 相似度:", round(r["score"], 3)) # 记住实体 mem.remember_entity("Eric", { "职业": "投资者", "时区": "UTC+8", "地点": "重庆" }) # 获取实体 info = mem.get_entity("Eric") print(info) # 按标签搜索 results = mem.search_by_tag("偏好") # 记忆图谱 graph = mem.get_memory_graph() print(graph) # 自动反思 insight = mem.auto_reflect() print(insight) # 持久化 mem.save() ``` ### CLI 工具 ```bash # 记忆一条信息 python -m nova_memory.scripts.memory_cli remember "Eric 今天讨论了投资组合优化" --tags 投资,组合 # 语义检索 python -m nova_memory.scripts.memory_cli recall "投资策略" # 实体操作 python -m nova_memory.scripts.memory_cli entity add "Eric" --facts 职业=投资者,地点=重庆 python -m nova_memory.scripts.memory_cli entity get "Eric" # 标签搜索 python -m nova_memory.scripts.memory_cli tags search "投资" # 记忆图谱 python -m nova_memory.scripts.memory_cli graph # 自动反思 python -m nova_memory.scripts.memory_cli reflect ``` --- ## OLAP 分析模块（memory_olap.py） 集成 SQLite 本地 OLAP 引擎，支持记忆数据的深度分析： ```python from nova_memory.memory_olap import MemoryOLAP olap = MemoryOLAP() # 生成完整分析报告 print(olap.full_report()) # 每日趋势 print(olap.trend_analysis(30)) # 标签热度 print(olap.tag_analysis(10)) # 实体统计 print(olap.entity_stats()) # 内容熵（多样性评分） print(olap.content_entropy()) ``` --- ## 与 Nova 记忆体系的整合 本技能与 Nova 四层记忆体系互补： | Nova 层级 | 文件 | 本技能可补充 | |-----------|------|------------| | 长期 | MEMORY.md | 结构化实体记忆 | | 项目 | memory/projects.md | 语义搜索相似项目 | | 日志 | memory/YYYY-MM-DD.md | 语义检索历史事件 | | 教训 | memory/lessons.md | 按标签回溯教训 | > `auto_reflect()` 的输出可写入 `MEMORY.md` 或 `memory/lessons.md`，形成闭环。 --- ## 安全与限制 - ✅ 纯本地运行，无外部 API 调用，无数据泄露风险 - ✅ 无 `eval()` / `exec()`，无代码注入风险 - ✅ 所有文件存储在 `/workspace/` 目录下 - ⚠️ 大量记忆（>10,000 条）建议定期清理旧记忆文件

path: scripts/memory_cli.py
content: #!/usr/bin/env python3
"""
NovaMemory CLI — 命令行记忆工具

用法示例:
    python memory_cli.py remember "内容" --tags tag1,tag2
    python memory_cli.py recall "查询"
    python memory_cli.py entity add "名字" --facts k=v,k2=v2
    python memory_cli.py entity get "名字"
    python memory_cli.py tags search "标签"
    python memory_cli.py graph
    python memory_cli.py reflect
    python memory_cli.py stats
"""

from __future__ import annotations

import argparse
import json
import sys
from pathlib import Path

# 添加 src 到路径，支持直接运行脚本
SRC_DIR = Path(__file__).resolve().parent.parent / "src"
sys.path.insert(0, str(SRC_DIR))

from memory_core import NovaMemory


# ---------------------------------------------------------------------------
# 默认存储路径
# ---------------------------------------------------------------------------
DEFAULT_STORAGE = "/workspace/memory/nova-memory"


# ---------------------------------------------------------------------------
# 命令处理器
# ---------------------------------------------------------------------------

def cmd_remember(args: argparse.Namespace) -> None:
    """记忆一条信息。"""
    mem = NovaMemory(storage_dir=args.storage or DEFAULT_STORAGE)

    tags = None
    if args.tags:
        tags = [t.strip() for t in args.tags.split(",") if t.strip()]

    mid = mem.remember(content=args.content, tags=tags, entity=args.entity)
    mem.save()

    print(f"✅ 已记忆，ID: {mid}")
    print(f"   内容: {args.content[:80]}{'…' if len(args.content) > 80 else ''}")
    if tags:
        print(f"   标签: {', '.join(tags)}")
    if args.entity:
        print(f"   实体: {args.entity}")


def cmd_recall(args: argparse.Namespace) -> None:
    """语义检索记忆。"""
    mem = NovaMemory(storage_dir=args.storage or DEFAULT_STORAGE)

    results = mem.recall(args.query, top_k=args.top_k)

    if not results:
        print("📭 未找到相关记忆。")
        return

    print(f"🔍 找到 {len(results)} 条相关记忆（top_k={args.top_k}）：\n")
    for i, r in enumerate(results, 1):
        tags_str = f" [🏷️ {', '.join(r['tags'])}]" if r["tags"] else ""
        entity_str = f" 👤 {r['entity']}" if r.get("entity") else ""
        print(f"  {i}. 【{r['score']:.4f}】{r['content'][:100]}{'…' if len(r['content']) > 100 else ''}{tags_str}{entity_str}")
        print(f"     ↳ ID: {r['id']} | 创建: {r.get('created_at', 'N/A')[:10]}\n")


def cmd_entity_add(args: argparse.Namespace) -> None:
    """添加/更新实体。"""
    mem = NovaMemory(storage_dir=args.storage or DEFAULT_STORAGE)

    facts: dict[str, str] = {}
    if args.facts:
        for item in args.facts.split(","):
            if "=" in item:
                k, v = item.split("=", 1)
                facts[k.strip()] = v.strip()

    mem.remember_entity(args.name, facts)
    mem.save()

    print(f"✅ 实体「{args.name}」已更新，属性：")
    for k, v in facts.items():
        print(f"   {k} = {v}")


def cmd_entity_get(args: argparse.Namespace) -> None:
    """获取实体信息。"""
    mem = NovaMemory(storage_dir=args.storage or DEFAULT_STORAGE)

    info = mem.get_entity(args.name)
    if not info:
        print(f"📭 未找到实体「{args.name}」")
        return

    print(f"👤 实体「{args.name}」：")
    for k, v in info.items():
        if not k.startswith("_"):
            print(f"   {k}: {v}")


def cmd_entity_list(args: argparse.Namespace) -> None:
    """列出所有实体。"""
    mem = NovaMemory(storage_dir=args.storage or DEFAULT_STORAGE)
    graph = mem.get_memory_graph()

    entities = [n for n in graph["nodes"] if n["type"] == "entity"]
    if not entities:
        print("📭 暂无实体记录。")
        return

    print(f"👥 共 {len(entities)} 个实体：\n")
    for e in entities:
        facts = e.get("facts", {})
        fact_preview = ", ".join(f"{k}={v}" for k, v in list(facts.items())[:3])
        print(f"  • {e['label']}")
        if fact_preview:
            print(f"    {fact_preview}")


def cmd_tags_search(args: argparse.Namespace) -> None:
    """按标签搜索记忆。"""
    mem = NovaMemory(storage_dir=args.storage or DEFAULT_STORAGE)

    results = mem.search_by_tag(args.tag)
    if not results:
        print(f"📭 没有找到标签为「{args.tag}」的记忆。")
        return

    print(f"🏷️ 标签「{args.tag}」共 {len(results)} 条记忆：\n")
    for r in results:
        entity_str = f" 👤 {r.get('entity', '')}" if r.get("entity") else ""
        print(f"  • {r['content'][:100]}{'…' if len(r['content']) > 100 else ''}{entity_str}")
        print(f"    ID: {r['id']} | 创建: {r.get('created_at', 'N/A')[:10]}\n")


def cmd_graph(args: argparse.Namespace) -> None:
    """打印记忆图谱。"""
    mem = NovaMemory(storage_dir=args.storage or DEFAULT_STORAGE)
    graph = mem.get_memory_graph()

    stats = graph.get("stats", {})
    print(f"🕸️ 记忆图谱概览")
    print(f"   记忆总数: {stats.get('total_memories', 0)}")
    print(f"   实体总数: {stats.get('total_entities', 0)}")
    print(f"   标签总数: {stats.get('total_tags', 0)}")
    print()

    # 实体节点
    entities = [n for n in graph["nodes"] if n["type"] == "entity"]
    if entities:
        print("👥 实体节点：")
        for e in entities:
            facts = e.get("facts", {})
            fact_str = " | ".join(f"{k}={v}" for k, v in list(facts.items())[:4])
            print(f"  ● {e['label']} {('(' + fact_str + ')') if fact_str else ''}")
        print()

    # 边
    edges = graph.get("edges", [])
    if edges:
        print(f"🔗 关系边（共 {len(edges)} 条）：")
        for edge in edges[:20]:  # 最多展示 20 条
            print(f"  {edge['from']} --[{edge.get('label', 'related')}]--> {edge.get('to', '')}")
        if len(edges) > 20:
            print(f"  … 还有 {len(edges) - 20} 条边")
        print()

    # 记忆节点（仅展示有实体的）
    memories = [n for n in graph["nodes"] if n["type"] == "memory"]
    if memories:
        print(f"💭 近期记忆节点（关联实体的前 10 条）：")
        for m in memories[:10]:
            tags_str = f"[{'|'.join(m.get('tags', [])[:3])}]"
            print(f"  ○ {m['label']} {tags_str} ← {m.get('entity', '')}")


def cmd_reflect(args: argparse.Namespace) -> None:
    """运行自动反思。"""
    mem = NovaMemory(storage_dir=args.storage or DEFAULT_STORAGE)
    report = mem.auto_reflect()
    print(report)


def cmd_stats(args: argparse.Namespace) -> None:
    """显示记忆库统计。"""
    mem = NovaMemory(storage_dir=args.storage or DEFAULT_STORAGE)
    stats = mem.stats

    print("📊 NovaMemory 统计：")
    print(f"   总记忆数 : {stats['total_memories']}")
    print(f"   总实体数 : {stats['total_entities']}")
    print(f"   总标签数 : {stats['total_tags']}")
    print(f"   存储路径 : {mem.storage_dir}")

    # top tags
    if mem._tag_index:
        top_tags = sorted(mem._tag_index.items(), key=lambda x: len(x[1]), reverse=True)[:5]
        print("\n🏷️ 标签 TOP5（按记忆数量）：")
        for tag, mids in top_tags:
            print(f"   {tag:<20} {len(mids)} 条记忆")


# ---------------------------------------------------------------------------
# CLI 入口
# ---------------------------------------------------------------------------

def build_parser() -> argparse.ArgumentParser:
    parser = argparse.ArgumentParser(
        prog="nova-memory",
        description="Nova 工作空间记忆技能 CLI",
        formatter_class=argparse.RawDescriptionHelpFormatter,
    )
    parser.add_argument(
        "--storage",
        default=None,
        help=f"存储目录（默认：{DEFAULT_STORAGE}）",
    )

    sub = parser.add_subparsers(dest="command", required=True)

    # remember
    p_remember = sub.add_parser("remember", help="记忆一条信息")
    p_remember.add_argument("content", help="记忆内容")
    p_remember.add_argument("--tags", help="逗号分隔的标签")
    p_remember.add_argument("--entity", help="关联实体名称")
    p_remember.set_defaults(func=cmd_remember)

    # recall
    p_recall = sub.add_parser("recall", help="语义检索记忆")
    p_recall.add_argument("query", help="查询内容")
    p_recall.add_argument("--top-k", type=int, default=5, help="返回数量（默认 5）")
    p_recall.set_defaults(func=cmd_recall)

    # entity add
    p_ent_add = sub.add_parser("entity", help="实体操作（子命令见 sub-add/sub-get/sub-list）")
    p_ent_subs = p_ent_add.add_subparsers(dest="entity_cmd", required=True)

    p_add = p_ent_subs.add_parser("add", help="添加/更新实体")
    p_add.add_argument("name", help="实体名称")
    p_add.add_argument("--facts", required=True, help="属性，格式 k=v,k2=v2")
    p_add.set_defaults(func=cmd_entity_add)

    p_get = p_ent_subs.add_parser("get", help="获取实体信息")
    p_get.add_argument("name", help="实体名称")
    p_get.set_defaults(func=cmd_entity_get)

    p_list = p_ent_subs.add_parser("list", help="列出所有实体")
    p_list.set_defaults(func=cmd_entity_list)

    # tags
    p_tags = sub.add_parser("tags", help="标签操作")
    p_tags_subs = p_tags.add_subparsers(dest="tags_cmd", required=True)

    p_search = p_tags_subs.add_parser("search", help="按标签搜索")
    p_search.add_argument("tag", help="标签名")
    p_search.set_defaults(func=cmd_tags_search)

    # graph
    p_graph = sub.add_parser("graph", help="打印记忆图谱")
    p_graph.set_defaults(func=cmd_graph)

    # reflect
    p_reflect = sub.add_parser("reflect", help="运行自动反思")
    p_reflect.set_defaults(func=cmd_reflect)

    # stats
    p_stats = sub.add_parser("stats", help="显示统计信息")
    p_stats.set_defaults(func=cmd_stats)

    return parser


def main() -> None:
    parser = build_parser()
    args = parser.parse_args()

    # 处理 entity 子命令分发
    if hasattr(args, "entity_cmd"):
        entity_dispatch = {
            "add": cmd_entity_add,
            "get": cmd_entity_get,
            "list": cmd_entity_list,
        }
        args.func = entity_dispatch.get(args.entity_cmd, cmd_entity_list)

    # 处理 tags 子命令分发
    if hasattr(args, "tags_cmd"):
        if args.tags_cmd == "search":
            args.func = cmd_tags_search

    try:
        args.func(args)
    except KeyboardInterrupt:
        print("\n⏹ 中断")
        sys.exit(0)
    except Exception as exc:
        print(f"❌ 错误：{exc}", file=sys.stderr)
        sys.exit(1)


if __name__ == "__main__":
    main()


path: src/__init__.py
content: """
nova-memory — Nova 工作空间自研记忆技能

纯本地语义记忆存储、检索、实体管理、自动反思引擎。
"""

from nova_memory.memory_core import NovaMemory

__all__ = ["NovaMemory"]


path: src/memory_core.py
content: """
NovaMemory Core Engine
Nova 工作空间自研记忆引擎 — 纯本地 TF-IDF + JSON 存储
"""

from __future__ import annotations

import json
import math
import os
import re
import uuid
from collections import defaultdict
from datetime import datetime, timezone
from pathlib import Path
from typing import Any

try:
    from sklearn.feature_extraction.text import TfidfVectorizer
    from sklearn.metrics.pairwise import cosine_similarity
except ImportError as exc:  # pragma: no cover
    raise ImportError(
        "nova_memory requires scikit-learn. Install with: pip install scikit-learn --break-system-packages"
    ) from exc


# ---------------------------------------------------------------------------
# 类型别名
# ---------------------------------------------------------------------------
MemoryRecord = dict[str, Any]  # 单条记忆的结构
EntityGraph = dict[str, dict[str, Any]]  # 实体图谱


# ---------------------------------------------------------------------------
# 工具函数
# ---------------------------------------------------------------------------

def _utc_now() -> str:
    """返回 UTC ISO 格式时间字符串。"""
    return datetime.now(timezone.utc).isoformat()


def _normalize_text(text: str) -> str:
    """小写 + 去除非字母数字，保留空格。"""
    return re.sub(r"[^a-z0-9\s]", " ", text.lower())


def _ensure_dir(path: Path) -> None:
    """确保目录存在。"""
    path.mkdir(parents=True, exist_ok=True)


# ---------------------------------------------------------------------------
# NovaMemory 主类
# ---------------------------------------------------------------------------

class NovaMemory:
    """
    Nova 工作空间自研记忆引擎。

    存储结构::

        storage_dir/
        ├── memories/          # 每条记忆一个 .json 文件
        │   └── <uuid>.json
        ├── entities.json      # 实体图谱
        ├── index.json         # tag 倒排索引
        └── meta.json          # 全局元数据

    参数:
        storage_dir: 存储根目录，默认为 /workspace/memory/nova-memory/
    """

    MEMORIES_DIR = "memories"
    ENTITIES_FILE = "entities.json"
    INDEX_FILE = "index.json"
    META_FILE = "meta.json"

    def __init__(self, storage_dir: str = "/workspace/memory/nova-memory/") -> None:
        self.storage_dir = Path(storage_dir)
        self.memories_dir = self.storage_dir / self.MEMORIES_DIR
        self.entities_path = self.storage_dir / self.ENTITIES_FILE
        self.index_path = self.storage_dir / self.INDEX_FILE
        self.meta_path = self.storage_dir / self.META_FILE

        # 内存状态
        self._memories: dict[str, MemoryRecord] = {}  # memory_id → record
        self._entities: EntityGraph = {}  # entity_name → facts
        self._tag_index: dict[str, list[str]] = defaultdict(list)  # tag → [memory_id, ...]
        self._vectorizer = TfidfVectorizer(
            analyzer="word",
            ngram_range=(1, 2),
            max_features=5000,
            stop_words="english",
        )
        self._vectors: dict[str, list[float]] = {}  # memory_id → vector

        _ensure_dir(self.memories_dir)
        self.load()

    # ------------------------------------------------------------------
    # 公共 API
    # ------------------------------------------------------------------

    def remember(
        self,
        content: str,
        tags: list[str] | None = None,
        entity: str | None = None,
    ) -> str:
        """
        记忆一条信息，返回 memory_id。

        参数:
            content: 记忆内容（任意文本）
            tags: 可选标签列表，用于分类和检索
            entity: 可选关联实体名称

        返回:
            新记忆的 UUID 字符串
        """
        memory_id = str(uuid.uuid4())
        now = _utc_now()

        record: MemoryRecord = {
            "id": memory_id,
            "content": content,
            "tags": [t.strip().lower() for t in (tags or [])],
            "entity": entity.strip() if entity else None,
            "created_at": now,
            "updated_at": now,
        }

        # 写入文件
        path = self.memories_dir / f"{memory_id}.json"
        path.write_text(json.dumps(record, ensure_ascii=False, indent=2), encoding="utf-8")

        # 更新内存状态
        self._memories[memory_id] = record

        # 更新 tag 倒排索引
        for tag in record["tags"]:
            if memory_id not in self._tag_index[tag]:
                self._tag_index[tag].append(memory_id)

        # 更新 TF-IDF 向量（追加到现有 vocab）
        self._refresh_vectors([record])

        return memory_id

    def recall(self, query: str, top_k: int = 5) -> list[dict[str, Any]]:
        """
        语义检索，返回最相关的记忆。

        使用 TF-IDF + Cosine Similarity 匹配，实体关联和标签可额外加权。

        参数:
            query: 自然语言查询
            top_k: 返回结果数量上限

        返回:
            按相似度降序排列的记忆列表，每项包含 id/content/tags/score/created_at
        """
        if not self._memories:
            return []

        # 构建语料库
        ids = list(self._memories.keys())
        texts = [
            self._memories[mid]["content"] for mid in ids
        ]

        try:
            tfidf_matrix = self._vectorizer.fit_transform(texts)
            query_vec = self._vectorizer.transform([_normalize_text(query)])
        except Exception:  # 词汇表为空或无法向量化
            return []

        # Cosine Similarity
        sims = cosine_similarity(query_vec, tfidf_matrix).flatten()

        # 组装结果（包含标签和实体加权）
        results = []
        for idx, score in enumerate(sims):
            mid = ids[idx]
            rec = self._memories[mid]

            # 标签/实体加权：查询词出现在 tags 中 +0.2
            q_lower = query.lower()
            tag_bonus = 0.0
            if any(q_lower in t or t in q_lower for t in rec.get("tags", [])):
                tag_bonus = 0.2
            entity_bonus = 0.1 if rec.get("entity") and rec["entity"].lower() in q_lower else 0.0

            final_score = min(score + tag_bonus + entity_bonus, 1.0)
            results.append(
                {
                    "id": mid,
                    "content": rec["content"],
                    "tags": rec.get("tags", []),
                    "entity": rec.get("entity"),
                    "score": round(float(final_score), 4),
                    "created_at": rec.get("created_at", ""),
                }
            )

        results.sort(key=lambda x: x["score"], reverse=True)
        return results[:top_k]

    def remember_entity(self, name: str, facts: dict[str, Any]) -> None:
        """
        记住一个实体的多个属性。

        参数:
            name: 实体名称（如 "Eric"）
            facts: 实体属性字典，如 {"职业": "投资者", "城市": "重庆"}
        """
        name_key = name.strip()
        if name_key not in self._entities:
            self._entities[name_key] = {"_meta": {"created_at": _utc_now()}}
        self._entities[name_key]["_meta"]["updated_at"] = _utc_now()
        for k, v in facts.items():
            self._entities[name_key][k.strip()] = v

    def get_entity(self, name: str) -> dict[str, Any] | None:
        """
        获取实体信息。

        参数:
            name: 实体名称

        返回:
            实体属性字典，不存在则返回 None
        """
        return self._entities.get(name.strip())

    def search_by_tag(self, tag: str) -> list[dict[str, Any]]:
        """
        按标签精确搜索记忆。

        参数:
            tag: 标签名（大小写不敏感）

        返回:
            匹配的记忆列表（按创建时间倒序）
        """
        tag_key = tag.strip().lower()
        memory_ids = self._tag_index.get(tag_key, [])
        results = []
        for mid in memory_ids:
            rec = self._memories.get(mid)
            if rec:
                results.append(
                    {
                        "id": mid,
                        "content": rec["content"],
                        "tags": rec.get("tags", []),
                        "entity": rec.get("entity"),
                        "created_at": rec.get("created_at", ""),
                    }
                )
        # 按时间倒序
        results.sort(key=lambda x: x.get("created_at", ""), reverse=True)
        return results

    def get_memory_graph(self) -> dict[str, Any]:
        """
        返回记忆图谱结构。

        返回:
            包含实体节点、记忆节点和边的完整图谱字典
        """
        nodes: list[dict[str, Any]] = []
        edges: list[dict[str, str]] = []

        # 实体节点
        for entity_name, facts in self._entities.items():
            nodes.append({
                "type": "entity",
                "id": entity_name,
                "label": entity_name,
                "facts": {k: v for k, v in facts.items() if not k.startswith("_")},
            })

        # 记忆节点（按 entity 聚合，最多展示最近 20 条）
        recent = sorted(
            [r for r in self._memories.values() if r.get("entity")],
            key=lambda x: x.get("created_at", ""),
            reverse=True,
        )[:20]

        for rec in recent:
            entity = rec.get("entity")
            nodes.append({
                "type": "memory",
                "id": rec["id"],
                "label": rec["content"][:60] + ("…" if len(rec["content"]) > 60 else ""),
                "tags": rec.get("tags", []),
                "entity": entity,
                "created_at": rec.get("created_at", ""),
            })
            if entity:
                edges.append({"from": entity, "to": rec["id"], "label": "recalled_by"})

        # 实体间关系（共享相同 tags 的实体视为相关）
        entity_names = list(self._entities.keys())
        seen_rels: set[tuple[str, str]] = set()
        for i, e1 in enumerate(entity_names):
            for e2 in entity_names[i + 1 :]:
                t1 = set(k for k in self._entities[e1].keys() if not k.startswith("_"))
                t2 = set(k for k in self._entities[e2].keys() if not k.startswith("_"))
                common = t1 & t2
                if common:
                    key = tuple(sorted([e1, e2]))
                    if key not in seen_rels:
                        seen_rels.add(key)
                        edges.append({
                            "from": e1,
                            "to": e2,
                            "label": f"shares: {', '.join(list(common)[:3])}",
                        })

        return {"nodes": nodes, "edges": edges, "stats": {
            "total_memories": len(self._memories),
            "total_entities": len(self._entities),
            "total_tags": len(self._tag_index),
        }}

    def auto_reflect(self) -> str:
        """
        自动反思：基于近期记忆生成洞察摘要。

        分析最近 50 条记忆，提取：
        - 高频标签
        - 关联最多的实体
        - 近期主题趋势

        返回:
            洞察摘要文本
        """
        if not self._memories:
            return "📭 记忆库为空，无法生成反思。"

        # 取最近 50 条
        recent = sorted(
            self._memories.values(),
            key=lambda x: x.get("created_at", ""),
            reverse=True,
        )[:50]

        # 统计标签频率
        tag_counts: dict[str, int] = defaultdict(int)
        for rec in recent:
            for tag in rec.get("tags", []):
                tag_counts[tag] += 1

        # 统计实体出现频率
        entity_counts: dict[str, int] = defaultdict(int)
        for rec in recent:
            if rec.get("entity"):
                entity_counts[rec["entity"]] += 1

        # 近期主题（recency 加权）
        all_tags = sorted(tag_counts.items(), key=lambda x: x[1], reverse=True)

        lines = [
            "🪞 **Nova 自动反思报告**",
            f"📊 分析样本：最近 {len(recent)} 条记忆（总记忆 {len(self._memories)} 条）",
            "",
        ]

        if all_tags[:5]:
            lines.append("**🏷️ 高频标签 TOP5**")
            for tag, cnt in all_tags[:5]:
                bar = "█" * cnt
                lines.append(f"  {tag:<20} {bar} ({cnt})")
            lines.append("")

        if entity_counts:
            top_entities = sorted(entity_counts.items(), key=lambda x: x[1], reverse=True)[:5]
            lines.append("**👤 高关联实体 TOP5**")
            for name, cnt in top_entities:
                lines.append(f"  {name:<20} 出现 {cnt} 次")
            lines.append("")

        # 洞察
        insights: list[str] = []
        if all_tags:
            top_tag = all_tags[0][0]
            insights.append(f"近期最活跃话题是「{top_tag}」")
        if entity_counts:
            top_entity = sorted(entity_counts.items(), key=lambda x: x[1], reverse=True)[0][0]
            insights.append(f"「{top_entity}」是记忆网络中最重要的实体")

        if len(self._memories) > 100:
            insights.append("记忆库规模较大，建议定期归档旧记忆")
        if tag_counts and len(all_tags) > 20:
            insights.append("标签体系丰富，可考虑合并相似标签")

        if insights:
            lines.append("**💡 洞察**")
            for ins in insights:
                lines.append(f"  • {ins}")

        lines.append("")
        lines.append(f"_生成时间：{_utc_now()}_")

        return "\n".join(lines)

    def save(self) -> None:
        """
        将内存状态持久化写入磁盘。
        """
        _ensure_dir(self.storage_dir)

        # entities
        with open(self.entities_path, "w", encoding="utf-8") as f:
            json.dump(self._entities, f, ensure_ascii=False, indent=2)

        # tag 倒排索引
        with open(self.index_path, "w", encoding="utf-8") as f:
            json.dump(dict(self._tag_index), f, ensure_ascii=False, indent=2)

        # meta
        meta = {
            "saved_at": _utc_now(),
            "total_memories": len(self._memories),
            "total_entities": len(self._entities),
            "total_tags": len(self._tag_index),
        }
        with open(self.meta_path, "w", encoding="utf-8") as f:
            json.dump(meta, f, ensure_ascii=False, indent=2)

    def load(self) -> None:
        """
        从磁盘加载持久化状态到内存。
        """
        # 加载 memories
        self._memories.clear()
        if self.memories_dir.exists():
            for fpath in self.memories_dir.glob("*.json"):
                try:
                    rec = json.loads(fpath.read_text(encoding="utf-8"))
                    self._memories[rec["id"]] = rec
                except Exception:
                    pass  # 跳过损坏的文件

        # 加载 entities
        self._entities.clear()
        if self.entities_path.exists():
            try:
                self._entities = json.loads(self.entities_path.read_text(encoding="utf-8"))
            except Exception:
                pass

        # 重建 tag 倒排索引
        self._tag_index.clear()
        for rec in self._memories.values():
            for tag in rec.get("tags", []):
                if rec["id"] not in self._tag_index[tag]:
                    self._tag_index[tag].append(rec["id"])

        # 重建 TF-IDF 向量
        self._refresh_vectors(list(self._memories.values()))

    # ------------------------------------------------------------------
    # 私有方法
    # ------------------------------------------------------------------

    def _refresh_vectors(self, records: list[MemoryRecord]) -> None:
        """
        基于所有当前记忆重建 TF-IDF 向量。
        """
        if not self._memories:
            self._vectors.clear()
            return

        ids = list(self._memories.keys())
        texts = [self._memories[mid]["content"] for mid in ids]

        try:
            tfidf_matrix = self._vectorizer.fit_transform(texts)
            for idx, mid in enumerate(ids):
                self._vectors[mid] = tfidf_matrix[idx].toarray().flatten().tolist()
        except Exception:
            self._vectors.clear()

    # ------------------------------------------------------------------
    # 便捷属性
    # ------------------------------------------------------------------

    @property
    def stats(self) -> dict[str, int]:
        """返回记忆库统计信息。"""
        return {
            "total_memories": len(self._memories),
            "total_entities": len(self._entities),
            "total_tags": len(self._tag_index),
        }

    def __repr__(self) -> str:
        return f"NovaMemory(storage_dir={str(self.storage_dir)!r}, memories={len(self._memories)})"


path: src/memory_olap.py
content: """
nova-memory-olap: DuckDB 驱动的记忆分析增强模块
将 nova-memory 的 JSON 记忆导入 DuckDB，支持 OLAP 分析
"""
import json, sqlite3
from pathlib import Path
from datetime import datetime, timedelta
from typing import Optional

class MemoryOLAP:
    """基于 SQLite（本地）+ DuckDB（分析）的记忆 OLAP 引擎"""

    def __init__(self, storage_dir: str = "/workspace/memory/nova-memory/"):
        self.storage_dir = Path(storage_dir)
        self.storage_dir.mkdir(parents=True, exist_ok=True)
        self.db_path = Path(storage_dir) / "memory_olap.db"
        self._init_db()

    def _init_db(self):
        """初始化 SQLite OLAP 数据库"""
        conn = sqlite3.connect(str(self.db_path))
        c = conn.cursor()
        c.execute("""
            CREATE TABLE IF NOT EXISTS memories (
                id          TEXT PRIMARY KEY,
                content     TEXT NOT NULL,
                tags        TEXT,          -- JSON list
                entity      TEXT,
                memory_id   TEXT,          -- 关联nova-memory ID
                created_at  TEXT,
                score       REAL DEFAULT 0  -- 检索相似度
            )
        """)
        c.execute("""
            CREATE TABLE IF NOT EXISTS entities (
                name       TEXT PRIMARY KEY,
                facts      TEXT,  -- JSON dict
                updated_at TEXT
            )
        """)
        c.execute("""
            CREATE TABLE IF NOT EXISTS memory_stats (
                date        TEXT PRIMARY KEY,
                count       INTEGER,
                top_tag     TEXT,
                top_entity  TEXT
            )
        """)
        # 时序索引
        c.execute("CREATE INDEX IF NOT EXISTS idx_created ON memories(created_at)")
        c.execute("CREATE INDEX IF NOT EXISTS idx_entity ON memories(entity)")
        conn.commit()
        conn.close()

    def ingest_memory(self, memory_id: str, content: str,
                      tags: list, entity: Optional[str],
                      created_at: str) -> None:
        """导入一条记忆到 OLAP 数据库"""
        conn = sqlite3.connect(str(self.db_path))
        c = conn.cursor()
        c.execute("""
            INSERT OR REPLACE INTO memories (id, content, tags, entity, created_at)
            VALUES (?, ?, ?, ?, ?)
        """, (memory_id, content, json.dumps(tags, ensure_ascii=False),
              entity, created_at))
        conn.commit()
        conn.close()

    def ingest_entity(self, name: str, facts: dict) -> None:
        """导入实体"""
        conn = sqlite3.connect(str(self.db_path))
        c = conn.cursor()
        c.execute("""
            INSERT OR REPLACE INTO entities (name, facts, updated_at)
            VALUES (?, ?, ?)
        """, (name, json.dumps(facts, ensure_ascii=False), datetime.utcnow().isoformat()))
        conn.commit()
        conn.close()

    def daily_stats(self, days: int = 30) -> list[dict]:
        """每日记忆增量（最近N天）"""
        conn = sqlite3.connect(str(self.db_path))
        c = conn.cursor()
        c.execute("""
            SELECT date(created_at) as day, COUNT(*) as count
            FROM memories
            WHERE created_at >= date('now', ?)
            GROUP BY date(created_at)
            ORDER BY day
        """, (f"-{days} days",))
        rows = c.fetchall()
        conn.close()
        return [{"date": r[0], "count": r[1]} for r in rows]

    def entity_stats(self) -> dict:
        """实体记忆统计"""
        conn = sqlite3.connect(str(self.db_path))
        c = conn.cursor()
        c.execute("SELECT entity, COUNT(*) FROM memories WHERE entity IS NOT NULL GROUP BY entity ORDER BY COUNT(*) DESC LIMIT 10")
        entity_rows = c.fetchall()
        c.execute("SELECT COUNT(DISTINCT entity) FROM memories")
        total_entities = c.fetchone()[0]
        conn.close()
        return {
            "total_entities": total_entities,
            "top_entities": [{"entity": r[0], "count": r[1]} for r in entity_rows]
        }

    def tag_analysis(self, top_n: int = 10) -> list[dict]:
        """标签热度分析"""
        conn = sqlite3.connect(str(self.db_path))
        c = conn.cursor()
        # 展开 tags JSON 数组，统计每个标签出现次数
        c.execute("SELECT tags FROM memories WHERE tags IS NOT NULL")
        tag_counter = {}
        for (tags_json,) in c.fetchall():
            try:
                tags = json.loads(tags_json)
                for tag in tags:
                    tag_counter[tag] = tag_counter.get(tag, 0) + 1
            except:
                pass
        conn.close()
        sorted_tags = sorted(tag_counter.items(), key=lambda x: -x[1])
        return [{"tag": t, "count": c} for t, c in sorted_tags[:top_n]]

    def trend_analysis(self, days: int = 30) -> dict:
        """趋势分析：记忆产生频率变化"""
        daily = self.daily_stats(days)
        if len(daily) < 2:
            return {"trend": "数据不足", "daily": daily}
        counts = [d["count"] for d in daily]
        avg = sum(counts) / len(counts)
        recent = sum(counts[-7:]) / min(7, len(counts))
        earlier = sum(counts[:7]) / min(7, len(counts))
        change = (recent - earlier) / earlier * 100 if earlier > 0 else 0
        return {
            "avg_daily": round(avg, 2),
            "recent_7d_avg": round(recent, 2),
            "earlier_7d_avg": round(earlier, 2),
            "change_pct": round(change, 1),
            "trend": "📈 上升" if change > 10 else "📉 下降" if change < -10 else "➡️ 平稳",
            "daily": daily
        }

    def content_entropy(self) -> dict:
        """内容熵分析：检测记忆多样性"""
        conn = sqlite3.connect(str(self.db_path))
        c = conn.cursor()
        c.execute("SELECT COUNT(*), COUNT(DISTINCT entity), COUNT(DISTINCT tags) FROM memories")
        total, distinct_entities, distinct_tags = c.fetchone()
        c.execute("SELECT AVG(LENGTH(content)) FROM memories")
        avg_len = c.fetchone()[0] or 0
        conn.close()
        return {
            "total_memories": total,
            "unique_entities": distinct_entities,
            "unique_tags": distinct_tags,
            "avg_content_length": round(avg_len, 1),
            "diversity_score": round(distinct_entities / total * distinct_tags / max(total, 1) * 100, 1)
        }

    def full_report(self) -> str:
        """生成完整 OLAP 报告"""
        trend = self.trend_analysis(30)
        entity_stats = self.entity_stats()
        top_tags = self.tag_analysis(10)
        entropy = self.content_entropy()

        report = f"""
╔══════════════════════════════════════════╗
║     NOVA MEMORY OLAP 报告                ║
║     {datetime.utcnow().strftime('%Y-%m-%d %H:%M UTC')}          ║
╚══════════════════════════════════════════╝

📊 记忆规模
  总记忆数    : {entropy['total_memories']}
  实体数量    : {entropy['unique_entities']}
  标签种类    : {entropy['unique_tags']}
  平均长度    : {entropy['avg_content_length']} 字符
  多样性得分  : {entropy['diversity_score']}/100

📈 趋势（近30天）
  日均记忆    : {trend['avg_daily']} 条/天
  近7天均值   : {trend['recent_7d_avg']} 条/天
  前7天均值   : {trend['earlier_7d_avg']} 条/天
  变化趋势    : {trend['trend']} ({trend['change_pct']:+.1f%})

🏆 热门标签 TOP10
"""
        for i, t in enumerate(top_tags, 1):
            report += f"  {i:2}. #{t['tag']:12s}  {t['count']}条\n"

        report += f"\n👤 实体分布 TOP10\n"
        for e in entity_stats["top_entities"][:10]:
            report += f"  • {e['entity']:12s}  {e['count']}条记忆\n"

        return report

    def __repr__(self):
        return f"<MemoryOLAP db={self.db_path}>"


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (MEDIUM)