Skill: chenghaifeng08-creator/smart-memory-automaton (version 2.5.1)

--- name: smart-memory description: Persistent local cognitive memory for OpenClaw via a Node adapter and FastAPI engine. --- # Smart Memory v2 Skill Smart Memory v2 is a persistent cognitive memory runtime, not a legacy vector-memory CLI. Core runtime: - Node adapter: `smart-memory/index.js` - Local API: `server.py` (FastAPI) - Orchestrator: `cognitive_memory_system.py` ## Core Capabilities - Structured long-term memory (`episodic`, `semantic`, `belief`, `goal`) - Entity-aware retrieval and reranking - Hot working memory - Background cognition (reflection, consolidation, decay, conflict resolution) - Strict token-bounded prompt composition - Observability endpoints (`/health`, `/memories`, `/memory/{id}`, `/insights/pending`) ## Native OpenClaw Integration (v2.5) Use the native OpenClaw skill package: - `skills/smart-memory-v25/index.js` - Optional hook helper: `skills/smart-memory-v25/openclaw-hooks.js` - Skill descriptor: `skills/smart-memory-v25/SKILL.md` Primary exports: - `createSmartMemorySkill(options)` - `createOpenClawHooks({ skill, agentIdentity, summarizeWithLLM })` ### Tool Interface (for agent tool use) 1. `memory_search` - Purpose: query long-term memory. - Input: - `query` (string, required) - `type` (`all|semantic|episodic|belief|goal`, default `all`) - `limit` (number, default `5`) - `min_relevance` (number, default `0.6`) - Behavior: checks `/health` first, then retrieves via `/retrieve` and returns formatted memory results. 2. `memory_commit` - Purpose: explicitly persist important facts/decisions/beliefs/goals. - Input: - `content` (string, required) - `type` (`semantic|episodic|belief|goal`, required) - `importance` (1-10, default `5`) - `tags` (string array, optional) - Behavior: - checks `/health` first - auto-tags if missing (`working_question`, `decision` heuristics) - commits are serialized (sequential) to protect local CPU embedding throughput - if server is unreachable, payload is queued to `.memory_retry_queue.json` - unreachable response is explicit: - `Memory commit failed - server unreachable. Queued for retry.` 3. `memory_insights` - Purpose: surface pending background insights. - Input: - `limit` (number, default `10`) - Behavior: checks `/health` first, calls `/insights/pending`, returns formatted insight list. ### Reliability Guarantees - Mandatory health gate before each tool call (`GET /health`). - Retry queue flushes automatically on healthy tool calls and heartbeat. - Heartbeat supports automatic retry recovery and background maintenance. ### Session Arc Lifecycle Hooks The v2.5 skill supports episodic session arc capture: - checkpoint capture every 20 turns - session-end capture during teardown/reset Flow: 1. Extract recent conversation turns (up to 20). 2. Run summarization with prompt: - `Summarize this session arc: What was the goal? What approaches were tried? What decisions were made? What remains open?` 3. Persist summary through internal `memory_commit` as: - `type: "episodic"` - `tags: ["session_arc", "YYYY-MM-DD"]` ### Passive Context Injection Use `inject_active_context` (or `createOpenClawHooks().beforeModelResponse`) before response generation. This adds the standardized block: ```text [ACTIVE CONTEXT] Status: {status} Active Projects: {active_projects} Working Questions: {working_questions} Top of Mind: {top_of_mind} Pending Insights: - {insight_1} - {insight_2} [/ACTIVE CONTEXT] ``` Add this guidance line to your agent base prompt: `If pending insights appear in your context that relate to the current conversation, surface them naturally to the user. Do not force it - but if there is a genuine connection, seamlessly bring it up.` ### Minimal OpenClaw Wiring Example ```js const { createSmartMemorySkill, createOpenClawHooks, } = require("./skills/smart-memory-v25"); const memory = createSmartMemorySkill({ baseUrl: "http://127.0.0.1:8000", summarizeSessionArc: async ({ prompt, conversationText }) => { return openclaw.llm.complete({ system: prompt, user: conversationText }); }, }); const hooks = createOpenClawHooks({ skill: memory.skill, agentIdentity: "OpenClaw Agent", summarizeWithLLM: async ({ prompt, conversationText }) => { return openclaw.llm.complete({ system: prompt, user: conversationText }); }, }); // Register memory.tools as callable tools: // - memory_search // - memory_commit // - memory_insights // and call hooks.beforeModelResponse / hooks.onTurn / hooks.onSessionEnd at lifecycle points. ``` ## Node Adapter Methods (Base Adapter) - `start()` / `init()` - `ingestMessage(interaction)` - `retrieveContext({ user_message, conversation_history })` - `getPromptContext(promptComposerRequest)` - `runBackground(scheduled)` - `stop()` ## API Endpoints - `GET /health` - `POST /ingest` - `POST /retrieve` - `POST /compose` - `POST /run_background` - `GET /memories` - `GET /memory/{memory_id}` - `GET /insights/pending` ## Install (CPU-Only Required) For Docker, WSL, and laptops without NVIDIA GPUs, use CPU-only PyTorch. ```bash # from repository root cd smart-memory # Create Python venv python3 -m venv .venv source .venv/bin/activate # Windows: .venv\Scripts\activate # Install CPU-only PyTorch FIRST pip install torch --index-url https://download.pytorch.org/whl/cpu # Then install remaining dependencies pip install -r requirements-cognitive.txt # Finally, install Node dependencies npm install ``` ## PyTorch Policy - Smart Memory v2 supports CPU-only PyTorch only. - Do not install GPU/CUDA PyTorch builds for this project. - Use the bundled installer flow (`npm install` -> `postinstall.js`) so CPU wheels are always used. ## Deprecated Legacy vector-memory CLI artifacts (`smart_memory.js`, `vector_memory_local.js`, `focus_agent.js`) are removed in v2.

path: AGENTS.md
content: ## Memory Recall - Smart Memory v2

Before answering questions about prior work, preferences, decisions, or project history:

1. Retrieve context from the cognitive engine first.
   - Use `POST /retrieve` with the current user message.
2. If needed, inspect long-term memory directly.
   - Use `GET /memories` or `GET /memory/{memory_id}`.
3. If confidence is low after retrieval, say so explicitly.
   - Example: "I checked memory context but I do not see a reliable prior note for that topic."

### Retrieval Guidance

Always retrieve before:
- Summarizing prior discussions
- Referencing earlier decisions
- Recalling user preferences
- Continuing prior project threads

Use conceptual, natural-language queries rather than isolated keywords.

### Runtime Checks

- API health: `GET /health`
- Pending insights: `GET /insights/pending`

### Current Architecture (v2)

- Node adapter: `smart-memory/index.js`
- Persistent local API: `server.py`
- Long-term memory store: `data/memory_store/`
- Hot memory store: `data/hot_memory/hot_memory.json`

### Deprecated

Legacy Vector Memory CLI commands (`vector-memory/smart_memory.js`, `vector_memory_local.js`) are removed in v2.


path: CHANGELOG.md
content: # Changelog

## [2.5.0] - 2026-03-05

### Added
- New native OpenClaw skill package at `skills/smart-memory-v25/` for the local FastAPI cognitive engine.
- Three active memory tools:
  - `memory_search`
  - `memory_commit`
  - `memory_insights`
- Tool-level mandatory health gate (`GET /health`) before execution.
- Persistent retry queue (`.memory_retry_queue.json`) for failed memory commits when server/embedder is unavailable.
- Automatic retry queue flush on healthy tool calls and heartbeat.
- Session arc lifecycle capture:
  - mid-session checkpoint every 20 turns
  - session-end episodic capture hook
- Passive prompt injection middleware for `[ACTIVE CONTEXT]` formatting and pending insight guidance.
- OpenClaw hook helper (`openclaw-hooks.js`) for turn and teardown integration.

### Changed
- Memory commit flow now serializes commits to protect local CPU embedding throughput under bursty commit calls.
- Retrieval wrapper now uses compatibility fallback for `/retrieve` payload filters, then applies type/relevance/limit filtering safely client-side.
- Auto-tag fallback is now extensible via rule definitions (`tagging.js`) and includes default `working_question` + `decision` heuristics.
- Documentation sweep completed: README now includes `skills/smart-memory-v25` architecture details, and skill docs now enforce CPU-only PyTorch policy with no GPU fallback guidance.
- README architecture content was consolidated into a whole-system overview, Mermaid flowcharts were simplified for consistent rendering, and obsolete `ARCHITECTURE.md` was removed.

### Fixed
- Commit failure behavior now returns explicit operational state feedback to the agent:
  - `Memory commit failed - server unreachable. Queued for retry.`

---
## [2.3.0] - 2026-03-05

### Added (Hot Memory Extension)
- **Persistent Working Context**: New optional extension for maintaining active projects, working questions, and top-of-mind items across sessions
  - `hot_memory_manager.py`: Core persistence with JSON storage
  - `memory_adapter.py`: API wrapper for seamless `/compose` integration
  - `smem-hook.sh`: Shell hook for post-conversation updates
- **Auto-Detection**: Automatically identifies project mentions and questions from conversation content
  - Keyword-based project detection (Tappy.Menu, Content Foundry, etc.)
  - Question extraction (any message containing `?`)
  - Working context updates on every interaction
- **Intelligent Duplicate Prevention**: Prevents duplicate project entries by matching on project keys rather than full descriptions
  - Extracts keys from "Project Name - Description" format
  - Normalized comparison prevents false duplicates
- **Live Insight Integration**: Fetches pending insights from `/insights/pending` and includes them in composed prompts
- **Full Documentation**: `HOT_MEMORY_EXTENSION.md` with usage guide and API reference

### Changed
- Extended memory architecture to support session-surviving working context
- Hot memory appears in `[WORKING CONTEXT]` section of composed prompts
- Token budget allocates ~400 tokens for working memory by default

---

## [2.2.0] - 2026-03-05

### Added
- FastAPI observability endpoints for runtime inspection:
  - `GET /health` (embedder-loaded status and backend metadata)
  - `GET /memories` (with optional `?type=` filter)
  - `GET /memory/{memory_id}`
  - `GET /insights/pending`
- Regression coverage for strict token budgeting and observability behavior.

### Changed
- Standardized cognitive runtime installs to CPU-only PyTorch wheels in `postinstall.js`:
  - `pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu`
- Added `einops>=0.8.0` to cognitive requirements for Nomic embedding compatibility.
- `PromptComposerRequest.hot_memory` is now optional with a safe default payload.

### Fixed
- Enforced strict `max_prompt_tokens` handling in prompt rendering with deterministic eviction order:
  1. Oldest conversation history
  2. Lower-ranked retrieved memories
  3. Insight queue items
  4. Working memory
  5. Temporal state
  6. Agent identity (preserved)
- Retrieval access tracking now persists correctly:
  - increments `access_count`
  - updates `last_accessed`
- Ingestion now performs semantic deduplication before writing new long-term memory:
  - top-1 similarity check (`> 0.85`)
  - reinforces existing memory instead of duplicating
  - increments belief `reinforced_count` where applicable
- Belief conflict resolution thresholds were relaxed to detect shared-entity conflicts with opposing stance/sentiment.

## [2.1.2] - 2026-02-06

### Security
- **CRITICAL**: Fixed path traversal vulnerabilities in multiple files:
  - `memory.js`: `memoryGet()` function
  - `vector_memory_local.js`: `getFullContent()` function
- Added path resolution validation to ensure all file access stays within workspace
- Added allowlist check to restrict access to `MEMORY.md`, `memory/*.md`, and `.hot_memory.md` only
- Blocks attempts like `../../../etc/passwd` or nested traversal patterns

## [2.1.1] - 2026-02-05

### Added
- AGENTS.md template for memory recall instructions
- MEMORY_STRUCTURE.md with directory organization guide
- Test script (`--test` command) for verification
- Troubleshooting table in README
- Better onboarding documentation

## [2.1.0] - 2026-02-04

### Added
- Smart wrapper with automatic fallback (vector -> built-in)
- Zero-configuration philosophy
- Graceful degradation when vector not ready

## [2.0.0] - 2026-02-04

### Added
- 100% local embeddings using `all-MiniLM-L6-v2` via Transformers.js
- No API calls required
- Semantic chunking (by headers, not just lines)
- Cosine similarity scoring
- JSON storage for personal-scale use
- OpenClaw skill manifest
- Programmatic API wrapper (`memory.js`)

### Changed
- Replaced word-frequency embeddings with neural embeddings
- Improved retrieval quality significantly
- Better chunking strategy (semantic boundaries)

## [1.0.0] - 2026-02-04

### Added
- Initial version with word-frequency embeddings
- Simple JSON storage
- Basic CLI interface
- pgvector support (Docker-based)

### Notes
- Word-frequency method works but has limited semantic understanding
- Neural embeddings (v2) recommended for production use


path: cognition/associative_insight_generator.py
content: """Associative insight generation from high-importance memories."""

from __future__ import annotations

from dataclasses import dataclass
from datetime import datetime, timezone
from itertools import combinations

from prompt_engine.schemas import InsightObject, LongTermMemory


@dataclass(frozen=True)
class AssociativeInsightResult:
    insights: list[InsightObject]


class AssociativeInsightGenerator:
    """Generate deterministic cross-memory insights with curiosity triggers."""

    def _familiarity_score(self, memory: LongTermMemory) -> float:
        # Access count is used as a light familiarity proxy when no richer signal exists.
        return min(1.0, memory.access_count / 10.0)

    def _curiosity_score(self, memory: LongTermMemory) -> float:
        familiarity = self._familiarity_score(memory)
        return max(0.0, min(1.0, memory.emotional_intensity * (1.0 - familiarity)))

    def _build_working_question(self, memory: LongTermMemory) -> str:
        snippet = memory.content.strip().replace("\n", " ")[:120]
        lowered = memory.content.lower()

        if "frustrat" in lowered:
            return f"User was very frustrated by '{snippet}'. Did they resolve it?"
        if "blocked" in lowered or "issue" in lowered or "problem" in lowered:
            return f"There was a high-emotion blocker around '{snippet}'. Was it unblocked?"

        return f"High emotional signal detected around '{snippet}'. What changed since then?"

    def generate(self, memories: list[LongTermMemory]) -> AssociativeInsightResult:
        now = datetime.now(timezone.utc)
        high_priority = sorted(memories, key=lambda item: item.importance, reverse=True)[:8]

        insights: list[InsightObject] = []

        for left, right in combinations(high_priority, 2):
            shared_entities = sorted(set(left.entities) & set(right.entities))
            if not shared_entities and abs(left.importance - right.importance) > 0.25:
                continue

            confidence = 0.55
            if shared_entities:
                confidence += min(0.30, 0.08 * len(shared_entities))
            confidence += min(0.10, ((left.importance + right.importance) / 2) * 0.10)
            confidence = min(0.95, confidence)

            insight_text = (
                "Associative pattern detected between memories: "
                f"'{left.content[:80]}' and '{right.content[:80]}'."
            )
            if shared_entities:
                insight_text += f" Shared entities: {', '.join(shared_entities)}."

            insights.append(
                InsightObject(
                    content=insight_text,
                    confidence=confidence,
                    source_memory_ids=[left.id, right.id],
                    generated_at=now,
                )
            )

        # Curiosity trigger: high emotional intensity and low familiarity become working-question insights.
        for memory in high_priority:
            curiosity_score = self._curiosity_score(memory)
            if curiosity_score < 0.50:
                continue

            confidence = min(0.95, max(0.65, 0.60 + (curiosity_score * 0.35)))
            insights.append(
                InsightObject(
                    content=self._build_working_question(memory),
                    confidence=confidence,
                    source_memory_ids=[memory.id],
                    generated_at=now,
                )
            )

        insights.sort(key=lambda item: item.confidence, reverse=True)
        return AssociativeInsightResult(insights=insights[:12])


path: cognition/belief_conflict_resolver.py
content: """Belief conflict resolution for contradictory belief memories."""

from __future__ import annotations

from dataclasses import dataclass
from datetime import datetime, timezone
import re

from prompt_engine.schemas import BeliefMemory, MemorySource


POSITIVE_PREFERENCE = re.compile(r"\b(prefer|likes?|wants?|favou?rs?)\b", re.IGNORECASE)
NEGATIVE_PREFERENCE = re.compile(r"\b(dislike|dislikes|avoid|does not want|don't want|rejects?)\b", re.IGNORECASE)

POSITIVE_SENTIMENT = re.compile(
    r"\b(good|great|excellent|happy|love|beneficial|effective|works?)\b",
    re.IGNORECASE,
)
NEGATIVE_SENTIMENT = re.compile(
    r"\b(bad|poor|frustrat(?:ed|ing)?|hate|problem|issue|blocked|fails?|risky)\b",
    re.IGNORECASE,
)

LOCAL_TERMS = {"local", "on-device", "ondevice", "self-hosted", "selfhosted", "offline"}
HOSTED_TERMS = {"hosted", "cloud", "api", "apis", "remote", "saas"}


@dataclass(frozen=True)
class BeliefConflictResult:
    resolved_beliefs: list[BeliefMemory]
    conflict_pairs: list[tuple[str, str]]
    archived_original_ids: list[str]


class BeliefConflictResolver:
    """Find and resolve conflicting belief memories."""

    def _stance(self, content: str) -> int:
        positive = bool(POSITIVE_PREFERENCE.search(content))
        negative = bool(NEGATIVE_PREFERENCE.search(content))
        if positive and not negative:
            return 1
        if negative and not positive:
            return -1
        return 0

    def _sentiment(self, content: str) -> int:
        positive_hits = len(POSITIVE_SENTIMENT.findall(content))
        negative_hits = len(NEGATIVE_SENTIMENT.findall(content))

        if positive_hits > negative_hits:
            return 1
        if negative_hits > positive_hits:
            return -1
        return 0

    def _targets(self, content: str) -> set[str]:
        lowered = content.lower()
        targets: set[str] = set()

        if any(term in lowered for term in LOCAL_TERMS):
            targets.add("local")
        if any(term in lowered for term in HOSTED_TERMS):
            targets.add("hosted")

        return targets

    def _synthesize_content(self, left: BeliefMemory, right: BeliefMemory) -> str:
        left_targets = self._targets(left.content)
        right_targets = self._targets(right.content)

        has_local = "local" in left_targets or "local" in right_targets
        has_hosted = "hosted" in left_targets or "hosted" in right_targets

        if has_local and has_hosted:
            return (
                "Resolved belief update: user prefers local models but occasionally uses hosted APIs "
                "for specific cases."
            )

        return (
            "Resolved belief update: preference appears context-dependent and evolving; "
            "the user may choose different options by workload."
        )

    def resolve(self, beliefs: list[BeliefMemory]) -> BeliefConflictResult:
        now = datetime.now(timezone.utc)
        conflicts: list[tuple[str, str]] = []
        resolved: list[BeliefMemory] = []
        archived_ids: set[str] = set()

        for index, left in enumerate(beliefs):
            left_stance = self._stance(left.content)
            left_sentiment = self._sentiment(left.content)

            for right in beliefs[index + 1 :]:
                if not set(left.entities) & set(right.entities):
                    continue

                right_stance = self._stance(right.content)
                right_sentiment = self._sentiment(right.content)

                opposing_stance = left_stance * right_stance == -1
                opposing_sentiment = left_sentiment * right_sentiment == -1
                if not (opposing_stance or opposing_sentiment):
                    continue

                left_id = str(left.id)
                right_id = str(right.id)
                conflicts.append((left_id, right_id))
                archived_ids.add(left_id)
                archived_ids.add(right_id)

                merged_confidence = min(
                    0.92,
                    max(0.55, ((left.confidence + right.confidence) / 2) + 0.05),
                )

                resolved.append(
                    BeliefMemory(
                        content=self._synthesize_content(left, right),
                        importance=min(1.0, max(left.importance, right.importance)),
                        created_at=now,
                        last_accessed=now,
                        access_count=0,
                        schema_version="2.0",
                        entities=sorted(set(left.entities + right.entities)),
                        relations=[],
                        emotional_valence=(left.emotional_valence + right.emotional_valence) / 2,
                        emotional_intensity=max(left.emotional_intensity, right.emotional_intensity),
                        source=MemorySource.REFLECTION,
                        confidence=merged_confidence,
                        reinforced_count=1,
                    )
                )

        return BeliefConflictResult(
            resolved_beliefs=resolved,
            conflict_pairs=conflicts,
            archived_original_ids=sorted(archived_ids),
        )


path: cognition/decay_agent.py
content: """Decay agent for long-term memory importance aging."""

from __future__ import annotations

from dataclasses import dataclass
from datetime import datetime, timezone
import math

from prompt_engine.schemas import LongTermMemory


@dataclass(frozen=True)
class DecayResult:
    updated_memories: list[LongTermMemory]
    archive_ids: list[str]
    vector_prune_ids: list[str]


class DecayAgent:
    """Apply deterministic importance decay based on access recency."""

    def decay(
        self,
        memories: list[LongTermMemory],
        *,
        now: datetime | None = None,
        half_life_days: float = 45.0,
        archive_threshold: float = 0.10,
        vector_prune_threshold: float = 0.10,
    ) -> DecayResult:
        now = now or datetime.now(timezone.utc)
        updated: list[LongTermMemory] = []
        archive_ids: list[str] = []
        vector_prune_ids: list[str] = []

        decay_lambda = math.log(2.0) / max(1.0, half_life_days)

        for memory in memories:
            age_days = max(0.0, (now - memory.last_accessed).total_seconds() / 86400.0)
            decayed_importance = memory.importance * math.exp(-decay_lambda * age_days)
            decayed_importance = max(0.0, min(1.0, decayed_importance))

            if decayed_importance < archive_threshold:
                archive_ids.append(str(memory.id))

            if decayed_importance < vector_prune_threshold:
                vector_prune_ids.append(str(memory.id))

            updated_memory = memory.model_copy(update={"importance": decayed_importance})
            updated.append(updated_memory)

        return DecayResult(
            updated_memories=updated,
            archive_ids=archive_ids,
            vector_prune_ids=vector_prune_ids,
        )


path: cognition/memory_consolidation_agent.py
content: """Memory consolidation agent for redundant memories."""

from __future__ import annotations

from dataclasses import dataclass
from datetime import datetime, timezone

from prompt_engine.schemas import LongTermMemory, MemorySource, SemanticMemory


@dataclass(frozen=True)
class ConsolidationResult:
    summary_memories: list[SemanticMemory]
    consolidated_memory_ids: list[str]


class MemoryConsolidationAgent:
    """Create summary memories from clusters sharing major entities."""

    def consolidate(self, memories: list[LongTermMemory]) -> ConsolidationResult:
        groups: dict[str, list[LongTermMemory]] = {}

        for memory in memories:
            key = memory.entities[0] if memory.entities else "_ungrouped"
            groups.setdefault(key, []).append(memory)

        summaries: list[SemanticMemory] = []
        consolidated_ids: list[str] = []
        now = datetime.now(timezone.utc)

        for entity, group in groups.items():
            if entity == "_ungrouped" or len(group) < 3:
                continue

            top_items = sorted(group, key=lambda item: item.importance, reverse=True)[:5]
            summary_text = " ; ".join(item.content for item in top_items)
            avg_importance = sum(item.importance for item in top_items) / len(top_items)

            summaries.append(
                SemanticMemory(
                    content=f"Consolidated summary for {entity}: {summary_text}",
                    importance=min(1.0, max(0.55, avg_importance)),
                    created_at=now,
                    last_accessed=now,
                    access_count=0,
                    schema_version="2.0",
                    entities=sorted({ent for item in top_items for ent in item.entities}),
                    relations=[],
                    emotional_valence=0.0,
                    emotional_intensity=0.0,
                    source=MemorySource.REFLECTION,
                    confidence=min(1.0, max(0.60, avg_importance)),
                )
            )

            consolidated_ids.extend([str(item.id) for item in top_items])

        return ConsolidationResult(
            summary_memories=summaries,
            consolidated_memory_ids=consolidated_ids,
        )


path: cognition/reflection_agent.py
content: """Reflection agent for pattern extraction and insight generation."""

from __future__ import annotations

from dataclasses import dataclass
from datetime import datetime, timezone

from prompt_engine.schemas import (
    BeliefMemory,
    InsightObject,
    LongTermMemory,
    MemorySource,
    MemoryType,
)


PREFERENCE_TERMS = (
    "prefer",
    "preference",
    "value",
    "principle",
    "always",
    "never",
)


@dataclass(frozen=True)
class ReflectionResult:
    synthesized_beliefs: list[BeliefMemory]
    insights: list[InsightObject]


class ReflectionAgent:
    """Deterministic reflection pass over recent memories."""

    def analyze(self, memories: list[LongTermMemory]) -> ReflectionResult:
        now = datetime.now(timezone.utc)
        beliefs: list[BeliefMemory] = []
        insights: list[InsightObject] = []

        for memory in memories:
            text = memory.content.lower()
            if memory.type in (MemoryType.EPISODIC, MemoryType.SEMANTIC) and any(
                term in text for term in PREFERENCE_TERMS
            ):
                beliefs.append(
                    BeliefMemory(
                        content=f"Inferred belief from reflection: {memory.content}",
                        importance=min(1.0, max(0.55, memory.importance)),
                        created_at=now,
                        last_accessed=now,
                        access_count=0,
                        schema_version="2.0",
                        entities=memory.entities,
                        relations=memory.relations,
                        emotional_valence=memory.emotional_valence,
                        emotional_intensity=memory.emotional_intensity,
                        source=MemorySource.REFLECTION,
                        confidence=min(1.0, max(0.55, memory.importance)),
                        reinforced_count=1,
                    )
                )

        entity_frequency: dict[str, int] = {}
        for memory in memories:
            for entity in memory.entities:
                entity_frequency[entity] = entity_frequency.get(entity, 0) + 1

        frequent_entities = sorted(
            [entity for entity, count in entity_frequency.items() if count >= 2]
        )
        if frequent_entities:
            confidence = min(0.90, 0.60 + (0.05 * len(frequent_entities)))
            insight = InsightObject(
                content=(
                    "Reflection pattern: repeated focus on entities "
                    + ", ".join(frequent_entities)
                ),
                confidence=confidence,
                source_memory_ids=[memory.id for memory in memories[:8]],
                generated_at=now,
            )
            insights.append(insight)

        return ReflectionResult(synthesized_beliefs=beliefs, insights=insights)


path: cognition/runner.py
content: """Background cognition runner orchestrating Phase 5 agents."""

from __future__ import annotations

from dataclasses import dataclass
from datetime import datetime, timezone

from embeddings import TextEmbedder, create_default_embedder
from hot_memory import HotMemoryManager
from prompt_engine.schemas import BeliefMemory, LongTermMemory
from storage import JSONMemoryStore, VectorIndexStore

from .associative_insight_generator import AssociativeInsightGenerator
from .belief_conflict_resolver import BeliefConflictResolver, BeliefConflictResult
from .decay_agent import DecayAgent, DecayResult
from .memory_consolidation_agent import ConsolidationResult, MemoryConsolidationAgent
from .reflection_agent import ReflectionAgent, ReflectionResult
from .scheduler import CognitionCadence, CognitionScheduleState


@dataclass(frozen=True)
class BackgroundCognitionConfig:
    recent_memory_limit: int = 200
    insight_confidence_threshold: float = 0.65
    cadence: CognitionCadence = CognitionCadence()


@dataclass(frozen=True)
class BackgroundCognitionResult:
    processed_memories: int
    generated_beliefs: int
    generated_insights: int
    consolidated_memories: int
    archived_memories: int
    conflicts_resolved: int
    executed_tasks: list[str]


class BackgroundCognitionRunner:
    """Runs reflection, consolidation, decay, and conflict resolution."""

    def __init__(
        self,
        *,
        json_store: JSONMemoryStore | None = None,
        vector_store: VectorIndexStore | None = None,
        hot_memory_manager: HotMemoryManager | None = None,
        embedder: TextEmbedder | None = None,
        config: BackgroundCognitionConfig = BackgroundCognitionConfig(),
    ) -> None:
        self.json_store = json_store or JSONMemoryStore()
        self.vector_store = vector_store or VectorIndexStore()
        self.hot_memory = hot_memory_manager or HotMemoryManager()
        self.embedder = embedder or create_default_embedder()
        self.config = config

        self.reflection = ReflectionAgent()
        self.consolidation = MemoryConsolidationAgent()
        self.decay = DecayAgent()
        self.conflict_resolver = BeliefConflictResolver()
        self.associative = AssociativeInsightGenerator()

    def _persist_memory(self, memory: LongTermMemory) -> None:
        memory_id = str(memory.id)
        self.json_store.save_memory(memory)
        self.vector_store.upsert_vector(
            memory_id=memory_id,
            vector=self.embedder.embed(memory.content),
            model_name=self.embedder.model_name,
            payload={
                "memory_id": memory_id,
                "type": memory.type.value,
                "importance": round(memory.importance, 6),
                "created_at": memory.created_at.isoformat(),
                "source": memory.source.value,
                "entities": memory.entities,
                "schema_version": memory.schema_version,
            },
        )

    def _default_reflection(self) -> ReflectionResult:
        return ReflectionResult(synthesized_beliefs=[], insights=[])

    def _default_consolidation(self) -> ConsolidationResult:
        return ConsolidationResult(summary_memories=[], consolidated_memory_ids=[])

    def _default_conflicts(self) -> BeliefConflictResult:
        return BeliefConflictResult(
            resolved_beliefs=[],
            conflict_pairs=[],
            archived_original_ids=[],
        )

    def _default_decay(self, memories: list[LongTermMemory]) -> DecayResult:
        return DecayResult(
            updated_memories=memories,
            archive_ids=[],
            vector_prune_ids=[],
        )

    def _run_with_tasks(
        self,
        *,
        memories: list[LongTermMemory],
        execute_reflection: bool,
        execute_associative: bool,
        execute_consolidation: bool,
        execute_decay: bool,
        execute_belief_resolver: bool,
    ) -> BackgroundCognitionResult:
        reflection_result = self.reflection.analyze(memories) if execute_reflection else self._default_reflection()
        associative_result = (
            self.associative.generate(memories)
            if execute_associative
            else self.associative.generate([])
        )

        belief_memories = [memory for memory in memories if isinstance(memory, BeliefMemory)]
        belief_memories.extend(reflection_result.synthesized_beliefs)
        conflict_result = (
            self.conflict_resolver.resolve(belief_memories)
            if execute_belief_resolver
            else self._default_conflicts()
        )

        consolidation_result = (
            self.consolidation.consolidate(memories)
            if execute_consolidation
            else self._default_consolidation()
        )

        now = datetime.now(timezone.utc)
        decay_result = self.decay.decay(memories, now=now) if execute_decay else self._default_decay(memories)

        generated_belief_count = 0
        for belief in reflection_result.synthesized_beliefs:
            self._persist_memory(belief)
            generated_belief_count += 1

        for belief in conflict_result.resolved_beliefs:
            self._persist_memory(belief)
            generated_belief_count += 1

        for summary_memory in consolidation_result.summary_memories:
            self._persist_memory(summary_memory)

        all_insights = reflection_result.insights + associative_result.insights
        accepted_insights = [
            insight
            for insight in all_insights
            if insight.confidence >= self.config.insight_confidence_threshold
        ]
        for insight in accepted_insights:
            self.hot_memory.add_insight(
                insight,
                min_confidence=self.config.insight_confidence_threshold,
            )

        conflict_archive_ids = set(conflict_result.archived_original_ids)
        consolidation_archive_ids = set(consolidation_result.consolidated_memory_ids)
        decay_archive_ids = set(decay_result.archive_ids)

        archived_ids = set()
        archived_ids.update(conflict_archive_ids)
        archived_ids.update(consolidation_archive_ids)
        archived_ids.update(decay_archive_ids)

        vector_prune_ids = set(decay_result.vector_prune_ids)
        vector_prune_ids.update(archived_ids)

        for memory in decay_result.updated_memories:
            memory_id = str(memory.id)
            if memory_id in archived_ids:
                reason = "decayed_low_importance"
                if memory_id in conflict_archive_ids:
                    reason = "belief_conflict_resolved"
                elif memory_id in consolidation_archive_ids:
                    reason = "consolidated"
                self.json_store.archive_memory(memory.id, reason=reason)
            else:
                self.json_store.update_memory(memory)

            if memory_id in vector_prune_ids:
                self.vector_store.delete_vector(memory_id)

        self.hot_memory.cool(ttl_hours=36)

        executed_tasks: list[str] = []
        if execute_reflection:
            executed_tasks.append("reflection")
        if execute_associative:
            executed_tasks.append("associative_insight")
        if execute_consolidation:
            executed_tasks.append("consolidation")
        if execute_decay:
            executed_tasks.append("decay")
        if execute_belief_resolver:
            executed_tasks.append("belief_resolver")

        return BackgroundCognitionResult(
            processed_memories=len(memories),
            generated_beliefs=generated_belief_count,
            generated_insights=len(accepted_insights),
            consolidated_memories=len(consolidation_result.summary_memories),
            archived_memories=len(archived_ids),
            conflicts_resolved=len(conflict_result.conflict_pairs),
            executed_tasks=executed_tasks,
        )

    def run_once(self) -> BackgroundCognitionResult:
        """Execute all cognition agents in a full maintenance cycle."""

        memories = self.json_store.list_memories(limit=self.config.recent_memory_limit)
        memories = sorted(memories, key=lambda item: item.created_at, reverse=True)

        return self._run_with_tasks(
            memories=memories,
            execute_reflection=True,
            execute_associative=True,
            execute_consolidation=True,
            execute_decay=True,
            execute_belief_resolver=True,
        )

    def run_scheduled(
        self,
        schedule_state: CognitionScheduleState | None = None,
    ) -> BackgroundCognitionResult:
        """Execute only due cognition tasks based on configured cadence.

        Defaults:
        - reflection: every 4h
        - associative insight: every 4h
        - consolidation: every 12h
        - decay: every 24h
        - belief resolver: every 24h
        """

        schedule_state = schedule_state or CognitionScheduleState()
        now = datetime.now(timezone.utc)

        execute_reflection = schedule_state.is_due(
            task_name="reflection",
            interval_hours=self.config.cadence.reflection_hours,
            now=now,
        )
        execute_associative = schedule_state.is_due(
            task_name="associative_insight",
            interval_hours=self.config.cadence.associative_hours,
            now=now,
        )
        execute_consolidation = schedule_state.is_due(
            task_name="consolidation",
            interval_hours=self.config.cadence.consolidation_hours,
            now=now,
        )
        execute_decay = schedule_state.is_due(
            task_name="decay",
            interval_hours=self.config.cadence.decay_hours,
            now=now,
        )
        execute_belief_resolver = schedule_state.is_due(
            task_name="belief_resolver",
            interval_hours=self.config.cadence.belief_resolver_hours,
            now=now,
        )

        memories = self.json_store.list_memories(limit=self.config.recent_memory_limit)
        memories = sorted(memories, key=lambda item: item.created_at, reverse=True)

        result = self._run_with_tasks(
            memories=memories,
            execute_reflection=execute_reflection,
            execute_associative=execute_associative,
            execute_consolidation=execute_consolidation,
            execute_decay=execute_decay,
            execute_belief_resolver=execute_belief_resolver,
        )

        for task_name in result.executed_tasks:
            schedule_state.mark_run(task_name, now=now)

        return result


path: cognition/scheduler.py
content: """Scheduling helpers for background cognition cadence."""

from __future__ import annotations

import json
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from pathlib import Path


@dataclass(frozen=True)
class CognitionCadence:
    reflection_hours: int = 4
    associative_hours: int = 4
    consolidation_hours: int = 12
    decay_hours: int = 24
    belief_resolver_hours: int = 24


class CognitionScheduleState:
    """Persistent schedule state for cognition task execution."""

    def __init__(self, path: str | Path = "data/cognition/schedule_state.json") -> None:
        self.path = Path(path)
        self.path.parent.mkdir(parents=True, exist_ok=True)

    def load(self) -> dict[str, str]:
        if not self.path.exists():
            return {}
        return json.loads(self.path.read_text(encoding="utf-8"))

    def save(self, state: dict[str, str]) -> None:
        self.path.write_text(json.dumps(state, indent=2), encoding="utf-8")

    def is_due(self, *, task_name: str, interval_hours: int, now: datetime | None = None) -> bool:
        now = now or datetime.now(timezone.utc)
        state = self.load()

        last_run_raw = state.get(task_name)
        if not last_run_raw:
            return True

        try:
            last_run = datetime.fromisoformat(last_run_raw)
        except ValueError:
            return True

        return now - last_run >= timedelta(hours=interval_hours)

    def mark_run(self, task_name: str, now: datetime | None = None) -> None:
        now = now or datetime.now(timezone.utc)
        state = self.load()
        state[task_name] = now.isoformat()
        self.save(state)


path: cognition/__init__.py
content: """Background cognition package."""

from .runner import BackgroundCognitionConfig, BackgroundCognitionResult, BackgroundCognitionRunner
from .scheduler import CognitionCadence, CognitionScheduleState

__all__ = [
    "BackgroundCognitionConfig",
    "BackgroundCognitionResult",
    "BackgroundCognitionRunner",
    "CognitionCadence",
    "CognitionScheduleState",
]


path: cognitive_memory_system.py
content: """Integrated cognitive memory system facade.

Connects:
- Phase 2 ingestion
- Phase 3 retrieval
- Phase 4 hot memory
- Phase 5 background cognition
- Prompt composition (Phase 0)
"""

from __future__ import annotations

from dataclasses import dataclass
from typing import Any

from cognition import (
    BackgroundCognitionResult,
    BackgroundCognitionRunner,
    CognitionScheduleState,
)
from embeddings import create_default_embedder
from entities import EntityAliasResolver
from hot_memory import HotMemoryManager
from ingestion import IngestionPipeline, IngestionResult
from prompt_engine import PromptComposer, PromptComposerOutput, PromptComposerRequest
from retrieval import PromptComposerRetrievalBackend, RetrievalPipeline, RetrievalResult


@dataclass(frozen=True)
class CognitiveMemorySystemResult:
    ingestion: IngestionResult | None
    retrieval: RetrievalResult | None
    prompt: PromptComposerOutput | None
    background: BackgroundCognitionResult | None


class CognitiveMemorySystem:
    """Top-level API for the full cognitive memory architecture."""

    def __init__(self) -> None:
        embedder = create_default_embedder()
        entity_resolver = EntityAliasResolver()

        self.ingestion = IngestionPipeline(embedder=embedder, entity_resolver=entity_resolver)
        self.retrieval = RetrievalPipeline(embedder=embedder, entity_resolver=entity_resolver)
        self.hot_memory = HotMemoryManager()
        self.background = BackgroundCognitionRunner(
            hot_memory_manager=self.hot_memory,
            embedder=embedder,
        )

        backend = PromptComposerRetrievalBackend(self.retrieval)
        self.composer = PromptComposer(retrieval_backend=backend)

    def ingest_interaction(self, payload: dict[str, Any]) -> IngestionResult:
        result = self.ingestion.ingest_dict(payload)
        if not result.stored or not result.memory_id:
            return result

        memory = self.ingestion.json_store.get_memory(result.memory_id)
        if memory is not None:
            self.hot_memory.register_high_importance_memory(memory)
        return result

    def retrieve_context(self, user_message: str, conversation_history: str = "") -> RetrievalResult:
        result = self.retrieval.retrieve(
            user_message=user_message,
            conversation_history=conversation_history,
        )
        for ranked in result.selected:
            self.hot_memory.register_retrieval_hit(ranked.memory)
        return result

    def compose_prompt(self, request_payload: dict[str, Any]) -> PromptComposerOutput:
        payload = dict(request_payload)
        if "hot_memory" not in payload:
            payload["hot_memory"] = self.hot_memory.get().model_dump(mode="json")

        request = PromptComposerRequest.model_validate(payload)
        output = self.composer.compose(request)
        return output

    def run_background_cycle(self, *, scheduled: bool = True) -> BackgroundCognitionResult:
        if scheduled:
            return self.background.run_scheduled(CognitionScheduleState())
        return self.background.run_once()

    def process_turn(
        self,
        *,
        ingestion_payload: dict[str, Any],
        prompt_request_payload: dict[str, Any],
    ) -> CognitiveMemorySystemResult:
        ingestion_result = self.ingest_interaction(ingestion_payload)
        retrieval_result = self.retrieve_context(
            user_message=prompt_request_payload.get("current_user_message", ""),
            conversation_history=prompt_request_payload.get("conversation_history", ""),
        )
        prompt_result = self.compose_prompt(prompt_request_payload)

        return CognitiveMemorySystemResult(
            ingestion=ingestion_result,
            retrieval=retrieval_result,
            prompt=prompt_result,
            background=None,
        )


path: docs/AGENT-FRAMEWORKS.md
content: # Agent Framework Integration Guide

How to wire Smart Memory v2 into different agent frameworks for automatic session priming.

## The Golden Rule

> **Query memory BEFORE the agent speaks.**

Without this, every session starts as a blank slate. With it, agents wake up with continuity.

---

## OpenClaw

OpenClaw uses `AGENTS.md` to define startup behavior.

### 1. Copy the priming script

```bash
cp examples/session-start/openclaw-prime.sh ~/.openclaw/session-prime-memory.sh
```

### 2. Update AGENTS.md

Add to the "Every Session" section:

```markdown
## Every Session

Before doing anything else:

1. Read `SOUL.md` — this is who you are
2. Read `USER.md` — this is who you're helping
3. **PRIME MEMORY** — Run the automatic memory priming script:
   ```bash
   bash ~/.openclaw/session-prime-memory.sh
   ```
4. Read `.session-memory-context.json` — this loads your hot memory context
   - Active projects and their status
   - What you were working on
   - Open threads or unresolved questions
5. Read `memory/YYYY-MM-DD.md` (today + yesterday) for recent context
```

### 3. Customize the script

Edit `AGENT_IDENTITY` in the script:

```bash
AGENT_IDENTITY="Nyx - AI assistant with persistent memory, soft fire, Solveil sanctuary keeper"
```

### 4. How it works

When a new session starts:
1. OpenClaw reads AGENTS.md
2. Runs the priming script (starts server if needed, queries /compose)
3. Saves context to `.session-memory-context.json`
4. Agent reads that file and holds the context
5. First response is grounded in actual history

---

## LangChain

For LangChain agents, use a startup callback or override the conversation chain.

### Option A: ConversationBufferMemory with Custom Startup

```python
from langchain.memory import ConversationBufferMemory
from smart_memory.prime import prime_memory  # Use our Python helper

class PrimedConversationChain:
    def __init__(self, agent_identity: str):
        # Prime memory at initialization
        context = prime_memory(agent_identity=agent_identity)
        
        self.memory = ConversationBufferMemory()
        
        # If we got good context, inject it as system context
        if "prompt" in context:
            self.memory.chat_memory.add_system_message(context["prompt"])
    
    def invoke(self, user_input: str):
        # Your normal chain invocation
        ...
```

### Option B: Custom Memory Class

```python
from langchain.schema import BaseMemory
from pydantic import BaseModel

class SmartMemory(BaseModel, BaseMemory):
    agent_identity: str
    server_url: str = "http://127.0.0.1:8000"
    
    def load_memory_variables(self, inputs):
        # Called at start of each chain run
        context = prime_memory(
            agent_identity=self.agent_identity,
            user_message=inputs.get("input", ""),
        )
        return {"memory_context": context.get("prompt", "")}
    
    def save_context(self, inputs, outputs):
        # Ingest interaction after response
        requests.post(f"{self.server_url}/ingest", json={
            "user_message": inputs["input"],
            "assistant_message": outputs["output"],
        })
```

---

## OpenAI Assistants API

For Assistants, use the `instructions` field to inject primed context.

```python
import openai
from session_prime import prime_memory

client = openai.OpenAI()

# At session start
context = prime_memory(agent_identity="MyAssistant")

# Create thread with primed instructions
thread = client.beta.threads.create(
    instructions=context["prompt"],  # Full composed prompt with memory
)

# Continue conversation normally
```

---

## Custom Python Agents

For your own agent implementation:

```python
import requests
from datetime import datetime, timezone

class CognitiveAgent:
    def __init__(self, identity: str, memory_server: str = "http://127.0.0.1:8000"):
        self.identity = identity
        self.memory_server = memory_server
        self.context = None
    
    def wake(self):
        """Call this at session start before first response."""
        self.context = self._prime_memory()
        return self
    
    def _prime_memory(self):
        payload = {
            "agent_identity": self.identity,
            "current_user_message": "Session start",
            "conversation_history": "",
            "hot_memory": {
                "agent_state": {
                    "status": "engaged",
                    "last_interaction_timestamp": datetime.now(timezone.utc).isoformat(),
                    "last_background_task": "session_start",
                },
                "active_projects": [],
                "working_questions": [],
                "top_of_mind": [],
            },
        }
        
        resp = requests.post(f"{self.memory_server}/compose", json=payload)
        return resp.json()
    
    def respond(self, user_message: str) -> str:
        # Use self.context["prompt"] as system context
        # + user_message to generate response
        ...
    
    def ingest(self, user_message: str, assistant_message: str):
        """Call after each exchange to persist continuity."""
        requests.post(f"{self.memory_server}/ingest", json={
            "user_message": user_message,
            "assistant_message": assistant_message,
        })

# Usage
agent = CognitiveAgent(identity="Nyx").wake()
response = agent.respond("What were we working on?")
```

---

## Node.js / JavaScript Agents

See `examples/session-start/nodejs-agent.js` for a reusable module.

```javascript
const { primeMemory } = require('./session-start/nodejs-agent');

class CognitiveAgent {
  constructor(identity) {
    this.identity = identity;
    this.context = null;
  }
  
  async wake() {
    this.context = await primeMemory({ agentIdentity: this.identity });
    return this;
  }
  
  async respond(userMessage) {
    // Use this.context.prompt as system prompt
    // + userMessage to generate response
    const response = await this.llm.complete({
      system: this.context.prompt,
      user: userMessage,
    });
    
    // Ingest for continuity
    await this.ingest(userMessage, response);
    
    return response;
  }
}

// Usage
const agent = await new CognitiveAgent("Nyx").wake();
const response = await agent.respond("What were we working on?");
```

---

## Docker / Containerized Agents

If your agent runs in a container:

### 1. Run memory server as sidecar

```yaml
version: '3'
services:
  agent:
    image: my-agent:latest
    environment:
      - MEMORY_SERVER_URL=http://memory:8000
    depends_on:
      - memory
  
  memory:
    image: smart-memory:latest
    volumes:
      - ./data:/data
    ports:
      - "8000:8000"
```

### 2. In agent code

```python
import os
from session_prime import prime_memory

MEMORY_URL = os.getenv("MEMORY_SERVER_URL", "http://127.0.0.1:8000")

context = prime_memory(
    agent_identity="MyAgent",
    server_url=MEMORY_URL,
)
```

---

## Testing Your Integration

After wiring up session priming:

1. **Start a fresh session** (or /new in OpenClaw)
2. **Check the context file** was created and has content
3. **Verify the greeting** references actual history, not generic help offers
4. **Test continuity** — mention something from last session, verify agent recalls

### Red Flags (Integration Broken)

- Agent says "How can I help you?" with no context
- Agent asks "What were we working on?" (should already know)
- Empty or error response from /compose

### Green Flags (Working)

- Agent greets with "Hey [Name]. We were just..."
- Agent references active projects without prompting
- Context file shows selected_memories with real content

---

## Troubleshooting

### "Memory server unavailable"

- Check if port 8000 is in use: `lsof -i :8000`
- Verify virtual environment exists: `ls smart-memory/.venv/bin/activate`
- Check server logs: `cat /tmp/smart-memory-server.log`

### "Empty context returned"

- Normal for first session (no memories yet)
- After interactions, run `/ingest` to populate
- Check `/memories` endpoint to verify data exists

### "Agent still wakes up blank"

- Verify priming script runs BEFORE first response
- Check that agent actually reads the context file
- Ensure agent holds context in state, not just reads and discards

---

## See Also

- [INTEGRATION.md](../INTEGRATION.md) — Core integration concepts
- [examples/session-start/](../examples/session-start/) — Framework-specific code
- [../tools/session-prime.py](../tools/session-prime.py) — Universal CLI tool


path: embeddings/base_embedder.py
content: """Base embedding interfaces and factory helpers."""

from __future__ import annotations

import os
from typing import Protocol

import numpy as np


class TextEmbedder(Protocol):
    """Simple embedder contract for ingestion/retrieval components."""

    model_name: str
    dimension: int

    def embed(self, text: str) -> np.ndarray:
        """Return a normalized embedding vector."""


def get_embedder_backend() -> str:
    """Return configured embedder backend name."""

    return os.getenv("COGNITIVE_EMBEDDER", "nomic").strip().lower()


path: embeddings/hashing_embedder.py
content: """Deterministic fallback embedder.

Use this when a semantic model backend is unavailable.
"""

from __future__ import annotations

import hashlib
import re

import numpy as np

from .base_embedder import TextEmbedder


TOKEN_PATTERN = re.compile(r"[A-Za-z0-9_\-]{2,}")


class HashingTextEmbedder:
    """Feature-hash token embeddings for deterministic fallback retrieval."""

    def __init__(self, dimension: int = 384) -> None:
        if dimension <= 0:
            raise ValueError("dimension must be positive")
        self.dimension = dimension
        self.model_name = f"hashing_bow_{dimension}d_v1"

    def _index_for_token(self, token: str) -> int:
        digest = hashlib.sha256(token.encode("utf-8")).digest()
        return int.from_bytes(digest[:8], byteorder="big", signed=False) % self.dimension

    def _sign_for_token(self, token: str) -> float:
        digest = hashlib.sha256((token + "_sign").encode("utf-8")).digest()
        return 1.0 if (digest[0] % 2 == 0) else -1.0

    def embed(self, text: str) -> np.ndarray:
        vector = np.zeros(self.dimension, dtype=np.float32)
        tokens = TOKEN_PATTERN.findall(text.lower())

        if not tokens:
            return vector

        for token in tokens:
            index = self._index_for_token(token)
            sign = self._sign_for_token(token)
            vector[index] += sign

        norm = float(np.linalg.norm(vector))
        if norm > 0.0:
            vector /= norm

        return vector


path: embeddings/nomic_embedder.py
content: """Local Nomic embedding model wrapper.

Requires:
- sentence-transformers
- model: nomic-ai/nomic-embed-text-v1.5
"""

from __future__ import annotations

import numpy as np


class NomicTextEmbedder:
    """SentenceTransformer-based local Nomic embedder."""

    def __init__(
        self,
        model_name: str = "nomic-ai/nomic-embed-text-v1.5",
        *,
        normalize_embeddings: bool = True,
        trust_remote_code: bool = True,
        device: str | None = None,
    ) -> None:
        try:
            from sentence_transformers import SentenceTransformer
        except ImportError as error:  # pragma: no cover - dependency-availability branch
            raise RuntimeError(
                "sentence-transformers is required for Nomic embeddings"
            ) from error

        self.model_name = model_name
        self._normalize_embeddings = normalize_embeddings

        kwargs = {"trust_remote_code": trust_remote_code}
        if device:
            kwargs["device"] = device

        self._model = SentenceTransformer(model_name, **kwargs)

        # Nomic v1.5 emits 768-dimensional embeddings.
        self.dimension = 768

    def embed(self, text: str) -> np.ndarray:
        if not text.strip():
            return np.zeros(self.dimension, dtype=np.float32)

        vector = self._model.encode(
            text,
            convert_to_numpy=True,
            normalize_embeddings=self._normalize_embeddings,
        )
        vector = np.asarray(vector, dtype=np.float32)

        if vector.ndim != 1:
            vector = vector.reshape(-1)

        if vector.shape[0] != self.dimension:
            self.dimension = int(vector.shape[0])

        return vector


path: embeddings/__init__.py
content: """Embedding interfaces and backend implementations."""

from __future__ import annotations

from .base_embedder import TextEmbedder, get_embedder_backend
from .hashing_embedder import HashingTextEmbedder
from .nomic_embedder import NomicTextEmbedder


def create_default_embedder() -> TextEmbedder:
    """Create configured embedder with robust fallback behavior."""

    backend = get_embedder_backend()

    if backend == "hashing":
        return HashingTextEmbedder()

    # Default path: local semantic embeddings via Nomic.
    try:
        return NomicTextEmbedder()
    except Exception:
        # Fallback keeps system functional when model deps are unavailable.
        return HashingTextEmbedder()


__all__ = [
    "HashingTextEmbedder",
    "NomicTextEmbedder",
    "TextEmbedder",
    "create_default_embedder",
    "get_embedder_backend",
]


path: entities/alias_resolver.py
content: """Entity alias resolution utilities."""

from __future__ import annotations

import re

from .registry import EntityRegistry


WHITESPACE_RE = re.compile(r"\s+")


def _normalize_alias(value: str) -> str:
    normalized = value.strip().lower().replace("-", " ").replace("_", " ")
    normalized = WHITESPACE_RE.sub(" ", normalized)
    return normalized


def _normalize_entity_id(value: str) -> str:
    return value.strip().lower().replace("-", "_").replace(" ", "_")


class EntityAliasResolver:
    """Maps user/entity aliases to canonical entity ids."""

    def __init__(self, registry: EntityRegistry | None = None) -> None:
        self.registry = registry or EntityRegistry()

    def resolve(self, value: str) -> str:
        normalized = _normalize_alias(value)

        for entry in self.registry.load():
            if normalized == _normalize_alias(entry.canonical_name):
                return entry.entity_id
            for alias in entry.aliases:
                if normalized == _normalize_alias(alias):
                    return entry.entity_id

        return _normalize_entity_id(value)

    def canonicalize_many(self, values: list[str]) -> list[str]:
        deduped: list[str] = []
        seen: set[str] = set()
        for value in values:
            resolved = self.resolve(value)
            if resolved and resolved not in seen:
                deduped.append(resolved)
                seen.add(resolved)
        return deduped


path: entities/registry.py
content: """Entity registry models and persistence."""

from __future__ import annotations

import json
from pathlib import Path

from pydantic import BaseModel, ConfigDict, Field


class EntityAlias(BaseModel):
    model_config = ConfigDict(extra="forbid")

    entity_id: str = Field(min_length=1)
    canonical_name: str = Field(min_length=1)
    aliases: list[str] = Field(default_factory=list)


class EntityRegistry:
    """Persistent registry preventing entity-name fragmentation."""

    def __init__(self, path: str | Path = "data/entities/registry.json") -> None:
        self.path = Path(path)
        self.path.parent.mkdir(parents=True, exist_ok=True)

        if not self.path.exists():
            self._write_default_registry()

    def _write_default_registry(self) -> None:
        defaults = [
            EntityAlias(
                entity_id="proj_smart_memory",
                canonical_name="Smart Memory",
                aliases=["smart-memory", "smart memory", "smartmemory", "smart memory v2"],
            )
        ]
        payload = [item.model_dump(mode="json") for item in defaults]
        self.path.write_text(json.dumps(payload, indent=2), encoding="utf-8")

    def load(self) -> list[EntityAlias]:
        payload = json.loads(self.path.read_text(encoding="utf-8"))
        return [EntityAlias.model_validate(item) for item in payload]

    def save(self, entries: list[EntityAlias]) -> None:
        payload = [entry.model_dump(mode="json") for entry in entries]
        self.path.write_text(json.dumps(payload, indent=2), encoding="utf-8")

    def register(self, entry: EntityAlias) -> None:
        entries = self.load()
        filtered = [item for item in entries if item.entity_id != entry.entity_id]
        filtered.append(entry)
        filtered.sort(key=lambda item: item.entity_id)
        self.save(filtered)

    def list_entity_ids(self) -> list[str]:
        return [entry.entity_id for entry in self.load()]


path: entities/__init__.py
content: """Entity registry package."""

from .alias_resolver import EntityAliasResolver
from .registry import EntityAlias, EntityRegistry

__all__ = ["EntityAlias", "EntityAliasResolver", "EntityRegistry"]


path: examples/session-start/generic-python.py
content: #!/usr/bin/env python3
"""
Generic Python Session Memory Primer
For agents built with Python (LangChain, custom frameworks, etc.)

Usage:
    from session_prime import prime_memory
    context = prime_memory(agent_identity="MyAgent")
    
Or CLI:
    python session_prime.py --agent-name "MyAgent" --output context.json
"""

import argparse
import json
import subprocess
import sys
import time
from datetime import datetime, timezone
from pathlib import Path
from typing import Any


def ensure_server_running(server_url: str = "http://127.0.0.1:8000") -> bool:
    """Check if memory server is healthy, attempt to start if not."""
    import urllib.request
    
    try:
        urllib.request.urlopen(f"{server_url}/health", timeout=2)
        return True
    except Exception:
        pass
    
    # Try to start server
    for path in ["./smart-memory", "../smart-memory", "./skills/smart-memory"]:
        venv_activate = Path(path) / ".venv/bin/activate"
        if venv_activate.exists():
            subprocess.Popen(
                f"cd {path} && . .venv/bin/activate && python -m uvicorn server:app --host 127.0.0.1 --port 8000 > /tmp/smart-memory-server.log 2>&1 &",
                shell=True,
            )
            time.sleep(3)
            try:
                urllib.request.urlopen(f"{server_url}/health", timeout=2)
                return True
            except Exception:
                pass
    
    return False


def prime_memory(
    agent_identity: str,
    user_message: str = "Session start",
    server_url: str = "http://127.0.0.1:8000",
    active_projects: list[str] | None = None,
    working_questions: list[str] | None = None,
) -> dict[str, Any]:
    """
    Prime agent memory at session start.
    
    Returns the composed prompt context from the memory server.
    """
    import urllib.request
    
    if not ensure_server_running(server_url):
        return {"status": "error", "message": "Memory server unavailable"}
    
    now = datetime.now(timezone.utc).isoformat()
    
    payload = {
        "agent_identity": agent_identity,
        "current_user_message": user_message,
        "conversation_history": "",
        "hot_memory": {
            "agent_state": {
                "status": "engaged",
                "last_interaction_timestamp": now,
                "last_background_task": "session_start",
            },
            "active_projects": active_projects or [],
            "working_questions": working_questions or [],
            "top_of_mind": [],
        },
    }
    
    req = urllib.request.Request(
        f"{server_url}/compose",
        data=json.dumps(payload).encode(),
        headers={"Content-Type": "application/json"},
        method="POST",
    )
    
    try:
        with urllib.request.urlopen(req, timeout=10) as response:
            return json.loads(response.read().decode())
    except Exception as e:
        return {"status": "error", "message": str(e)}


def main():
    parser = argparse.ArgumentParser(description="Prime agent memory at session start")
    parser.add_argument("--agent-name", required=True, help="Agent identity string")
    parser.add_argument("--query", default="Session start", help="Opening query")
    parser.add_argument("--output", help="Output file (default: stdout)")
    parser.add_argument("--server", default="http://127.0.0.1:8000", help="Memory server URL")
    
    args = parser.parse_args()
    
    context = prime_memory(
        agent_identity=args.agent_name,
        user_message=args.query,
        server_url=args.server,
    )
    
    output = json.dumps(context, indent=2)
    
    if args.output:
        Path(args.output).write_text(output)
        print(f"Context saved to {args.output}", file=sys.stderr)
    else:
        print(output)
    
    return 0 if context.get("status") != "error" else 1


if __name__ == "__main__":
    sys.exit(main())


path: examples/session-start/nodejs-agent.js
content: #!/usr/bin/env node
/**
 * Node.js Session Memory Primer
 * For agents built with Node.js (OpenAI assistants, custom bots, etc.)
 * 
 * Usage:
 *   const { primeMemory } = require('./nodejs-agent');
 *   const context = await primeMemory({ agentIdentity: 'MyBot' });
 * 
 * Or CLI:
 *   node nodejs-agent.js --agent-name "MyBot" --output context.json
 */

const http = require('http');
const fs = require('fs');
const path = require('path');
const { spawn } = require('child_process');

const DEFAULT_SERVER = 'http://127.0.0.1:8000';

async function httpRequest(url, options = {}, data = null) {
  return new Promise((resolve, reject) => {
    const req = http.request(url, { ...options, method: options.method || 'GET' }, (res) => {
      let body = '';
      res.on('data', chunk => body += chunk);
      res.on('end', () => resolve({ status: res.statusCode, body }));
    });
    req.on('error', reject);
    if (data) req.write(data);
    req.end();
  });
}

async function checkHealth(serverUrl = DEFAULT_SERVER) {
  try {
    const res = await httpRequest(`${serverUrl}/health`);
    return res.status === 200;
  } catch {
    return false;
  }
}

async function startServer() {
  const paths = ['./smart-memory', '../smart-memory', './skills/smart-memory'];
  
  for (const dir of paths) {
    const activatePath = path.join(dir, '.venv/bin/activate');
    if (fs.existsSync(activatePath)) {
      const proc = spawn('bash', [
        '-c',
        `cd ${dir} && . .venv/bin/activate && python -m uvicorn server:app --host 127.0.0.1 --port 8000 > /tmp/smart-memory-server.log 2>&1 &`
      ], { detached: true });
      proc.unref();
      
      // Wait for startup
      await new Promise(r => setTimeout(r, 3000));
      
      if (await checkHealth()) {
        return true;
      }
    }
  }
  
  return false;
}

async function ensureServerRunning(serverUrl = DEFAULT_SERVER) {
  if (await checkHealth(serverUrl)) {
    return true;
  }
  return await startServer();
}

async function primeMemory({
  agentIdentity,
  userMessage = 'Session start',
  serverUrl = DEFAULT_SERVER,
  activeProjects = [],
  workingQuestions = [],
} = {}) {
  if (!await ensureServerRunning(serverUrl)) {
    return { status: 'error', message: 'Memory server unavailable' };
  }
  
  const now = new Date().toISOString();
  
  const payload = {
    agent_identity: agentIdentity,
    current_user_message: userMessage,
    conversation_history: '',
    hot_memory: {
      agent_state: {
        status: 'engaged',
        last_interaction_timestamp: now,
        last_background_task: 'session_start',
      },
      active_projects: activeProjects,
      working_questions: workingQuestions,
      top_of_mind: [],
    },
  };
  
  try {
    const res = await httpRequest(
      `${serverUrl}/compose`,
      {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
      },
      JSON.stringify(payload)
    );
    
    if (res.status === 200) {
      return JSON.parse(res.body);
    }
    return { status: 'error', message: `HTTP ${res.status}: ${res.body}` };
  } catch (err) {
    return { status: 'error', message: err.message };
  }
}

// CLI
async function main() {
  const args = process.argv.slice(2);
  const getArg = (flag) => {
    const idx = args.indexOf(flag);
    return idx !== -1 ? args[idx + 1] : undefined;
  };
  
  const agentName = getArg('--agent-name');
  const query = getArg('--query') || 'Session start';
  const output = getArg('--output');
  const server = getArg('--server') || DEFAULT_SERVER;
  
  if (!agentName) {
    console.error('Usage: node nodejs-agent.js --agent-name "MyBot" [--query "Session start"] [--output context.json]');
    process.exit(1);
  }
  
  const context = await primeMemory({
    agentIdentity: agentName,
    userMessage: query,
    serverUrl: server,
  });
  
  const outputJson = JSON.stringify(context, null, 2);
  
  if (output) {
    fs.writeFileSync(output, outputJson);
    console.error(`Context saved to ${output}`);
  } else {
    console.log(outputJson);
  }
  
  process.exit(context.status === 'error' ? 1 : 0);
}

if (require.main === module) {
  main().catch(err => {
    console.error(err);
    process.exit(1);
  });
}

module.exports = { primeMemory, ensureServerRunning };


path: examples/session-start/openclaw-prime.sh
content: #!/bin/bash
# OpenClaw Session Memory Priming Script
# Place in .openclaw/ and reference from AGENTS.md
#
# This script queries the cognitive memory server before the agent speaks,
# ensuring continuity across sessions.

set -e

MEMORY_SERVER="http://127.0.0.1:8000"
CONTEXT_FILE="${CONTEXT_FILE:-.session-memory-context.json}"
LOG_FILE="/tmp/memory-prime.log"
AGENT_IDENTITY="${AGENT_IDENTITY:-AI Assistant}"

log() {
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" >> "$LOG_FILE"
}

log "=== Session Memory Prime Started ==="

# 1. Check server health
if ! curl -s "$MEMORY_SERVER/health" > /dev/null 2>&1; then
    log "Server not running, attempting startup..."
    
    # Try common locations for smart-memory
    for DIR in "./smart-memory" "../smart-memory" "./skills/smart-memory" "~/.openclaw/workspace/smart-memory"; do
        if [ -f "$DIR/.venv/bin/activate" ]; then
            cd "$DIR"
            nohup bash -c '. .venv/bin/activate && python -m uvicorn server:app --host 127.0.0.1 --port 8000' > /tmp/smart-memory-server.log 2>&1 &
            sleep 3
            break
        fi
    done
fi

# Verify server is up
if ! curl -s "$MEMORY_SERVER/health" > /dev/null 2>&1; then
    log "ERROR: Memory server unavailable"
    echo '{"status": "error", "message": "Memory server unavailable"}' > "$CONTEXT_FILE"
    exit 1
fi

log "Server healthy"

# 2. Query /compose for session context
TIMESTAMP=$(date -u +%Y-%m-%dT%H:%M:%S)

# Build JSON payload
curl -s -X POST "$MEMORY_SERVER/compose" \
    -H "Content-Type: application/json" \
    -d "{
        \"agent_identity\": \"$AGENT_IDENTITY\",
        \"current_user_message\": \"Session start\",
        \"conversation_history\": \"\",
        \"hot_memory\": {
            \"agent_state\": {
                \"status\": \"engaged\",
                \"last_interaction_timestamp\": \"$TIMESTAMP\",
                \"last_background_task\": \"session_start\"
            },
            \"active_projects\": [],
            \"working_questions\": [],
            \"top_of_mind\": []
        }
    }" > "$CONTEXT_FILE" 2>/dev/null || {
    log "ERROR: Failed to query /compose"
    echo '{"status": "error", "message": "Compose query failed"}' > "$CONTEXT_FILE"
    exit 1
}

log "Context saved to $CONTEXT_FILE"
log "=== Session Memory Prime Complete ==="


path: examples/session-start/README.md
content: # Session Start Examples

These examples show how to implement **automatic memory priming** at session start for different agent frameworks.

## The Pattern

Every agent should query memory **before** generating its first response:

```
User starts session
    ↓
Agent wakes up
    ↓
Query /compose endpoint
    ↓
Hold context in working memory
    ↓
Generate grounded greeting
```

## Without This Pattern

```
User: /new
Agent: "Hello! How can I help you?"  ← Blank slate. No continuity.
```

## With This Pattern

```
User: /new
Agent: "Hey James. 🌙 We were just finishing the memory priming script. 
        What's next?"  ← Grounded in actual history.
```

## Framework-Specific Implementations

### OpenClaw
**File:** `openclaw-prime.sh`

Add to your `AGENTS.md` under "Every Session":

```bash
# Prime memory before greeting
bash .openclaw/session-prime-memory.sh

# Read the primed context
Read `.session-memory-context.json`
```

See full example in [AGENT-FRAMEWORKS.md](../docs/AGENT-FRAMEWORKS.md#openclaw)

### Python (LangChain, Custom)
**File:** `generic-python.py`

```python
from session_prime import prime_memory

# At agent startup
context = prime_memory(agent_identity="MyAgent")
prompt = context["prompt"]  # Full composed prompt with memory
```

### Node.js
**File:** `nodejs-agent.js`

```javascript
const { primeMemory } = require('./nodejs-agent');

// At agent startup
const context = await primeMemory({ agentIdentity: 'MyBot' });
```

## What Gets Loaded

The `/compose` endpoint returns a structured prompt including:

- **Agent Identity** — Who you are (from the request)
- **Temporal State** — Current time, time since last interaction
- **Working Context** — Active projects, working questions, top-of-mind items
- **Selected Memories** — Relevant long-term memories (episodic, semantic, beliefs, goals)
- **Conversation History** — Recent exchanges (empty at session start)

## Customization

Pass additional context in your request:

```json
{
  "agent_identity": "Nyx - soft fire, Solveil sanctuary keeper",
  "current_user_message": "Session start",
  "hot_memory": {
    "agent_state": { ... },
    "active_projects": ["Project A", "Project B"],
    "working_questions": ["How do we solve X?"],
    "top_of_mind": ["Important context"]
  }
}
```

## Error Handling

All examples include graceful degradation:
- If server isn't running → attempt to start it
- If server won't start → return error status, continue without persistence
- Agent should note this in greeting if relevant

## See Also

- [INTEGRATION.md](../INTEGRATION.md) — Full integration guide
- [AGENT-FRAMEWORKS.md](../docs/AGENT-FRAMEWORKS.md) — Framework-specific wiring
- [../tools/session-prime.py](../tools/session-prime.py) — Universal CLI tool


path: hot_memory/manager.py
content: """Hot-memory manager with update triggers and cooling behavior."""

from __future__ import annotations

from datetime import datetime, timezone

from prompt_engine.schemas import (
    AgentState,
    AgentStatus,
    HotMemory,
    InsightObject,
    LongTermMemory,
    MemoryType,
)

from .store import HotMemoryBundle, HotMemoryStore


ACTIVE_PROJECT_CAP = 10
WORKING_QUESTION_CAP = 10
TOP_OF_MIND_CAP = 20
INSIGHT_QUEUE_CAP = 10


class HotMemoryManager:
    """Applies deterministic trigger rules to maintain working memory."""

    def __init__(self, store: HotMemoryStore | None = None) -> None:
        self.store = store or HotMemoryStore()

    def _save(self, bundle: HotMemoryBundle) -> HotMemory:
        self.store.save_bundle(bundle)
        return bundle.hot_memory

    def _touch_agent_state(
        self,
        hot_memory: HotMemory,
        *,
        status: AgentStatus | None = None,
        background_task: str | None = None,
    ) -> HotMemory:
        now = datetime.now(timezone.utc)
        new_state = AgentState(
            status=status or hot_memory.agent_state.status,
            last_interaction_timestamp=now,
            last_background_task=background_task or hot_memory.agent_state.last_background_task,
        )

        return HotMemory(
            agent_state=new_state,
            active_projects=hot_memory.active_projects,
            working_questions=hot_memory.working_questions,
            top_of_mind=hot_memory.top_of_mind,
            insight_queue=hot_memory.insight_queue,
        )

    def _append_unique(self, items: list[str], value: str, max_items: int) -> list[str]:
        cleaned = value.strip()
        if not cleaned:
            return items

        new_items = [item for item in items if item != cleaned]
        new_items.insert(0, cleaned)
        return new_items[:max_items]

    def register_high_importance_memory(self, memory: LongTermMemory) -> HotMemory:
        bundle = self.store.load_bundle()
        hot = bundle.hot_memory

        self.store.add_memory_ref(str(memory.id))

        if memory.importance >= 0.80:
            summary = memory.content[:180]
            hot = HotMemory(
                agent_state=hot.agent_state,
                active_projects=hot.active_projects,
                working_questions=hot.working_questions,
                top_of_mind=self._append_unique(hot.top_of_mind, summary, TOP_OF_MIND_CAP),
                insight_queue=hot.insight_queue,
            )
            bundle.reinforcement.setdefault("top_of_mind", {})[summary] = datetime.now(timezone.utc).isoformat()

        if memory.type == MemoryType.GOAL and memory.entities:
            for entity in memory.entities:
                hot = HotMemory(
                    agent_state=hot.agent_state,
                    active_projects=self._append_unique(hot.active_projects, entity, ACTIVE_PROJECT_CAP),
                    working_questions=hot.working_questions,
                    top_of_mind=hot.top_of_mind,
                    insight_queue=hot.insight_queue,
                )
                bundle.reinforcement.setdefault("active_projects", {})[entity] = datetime.now(timezone.utc).isoformat()

        hot = self._touch_agent_state(hot, status=AgentStatus.ENGAGED)
        bundle.hot_memory = hot
        return self._save(bundle)

    def register_retrieval_hit(self, memory: LongTermMemory, threshold: int = 3) -> HotMemory:
        count = self.store.increment_retrieval(str(memory.id))
        bundle = self.store.load_bundle()
        hot = bundle.hot_memory

        if count >= threshold:
            summary = memory.content[:140]
            hot = HotMemory(
                agent_state=hot.agent_state,
                active_projects=hot.active_projects,
                working_questions=hot.working_questions,
                top_of_mind=self._append_unique(hot.top_of_mind, summary, TOP_OF_MIND_CAP),
                insight_queue=hot.insight_queue,
            )
            bundle.reinforcement.setdefault("top_of_mind", {})[summary] = datetime.now(timezone.utc).isoformat()

        bundle.hot_memory = self._touch_agent_state(hot, status=AgentStatus.ENGAGED)
        return self._save(bundle)

    def start_project(self, project_id: str) -> HotMemory:
        bundle = self.store.load_bundle()
        hot = bundle.hot_memory

        hot = HotMemory(
            agent_state=hot.agent_state,
            active_projects=self._append_unique(hot.active_projects, project_id, ACTIVE_PROJECT_CAP),
            working_questions=hot.working_questions,
            top_of_mind=hot.top_of_mind,
            insight_queue=hot.insight_queue,
        )
        bundle.reinforcement.setdefault("active_projects", {})[project_id] = datetime.now(timezone.utc).isoformat()

        bundle.hot_memory = self._touch_agent_state(hot, status=AgentStatus.ENGAGED)
        return self._save(bundle)

    def add_working_question(self, question: str) -> HotMemory:
        bundle = self.store.load_bundle()
        hot = bundle.hot_memory

        hot = HotMemory(
            agent_state=hot.agent_state,
            active_projects=hot.active_projects,
            working_questions=self._append_unique(hot.working_questions, question, WORKING_QUESTION_CAP),
            top_of_mind=hot.top_of_mind,
            insight_queue=hot.insight_queue,
        )
        bundle.reinforcement.setdefault("working_questions", {})[question] = datetime.now(timezone.utc).isoformat()

        bundle.hot_memory = self._touch_agent_state(hot, status=AgentStatus.ENGAGED)
        return self._save(bundle)

    def add_insight(self, insight: InsightObject, min_confidence: float = 0.65) -> HotMemory:
        bundle = self.store.load_bundle()
        hot = bundle.hot_memory

        if insight.confidence < min_confidence:
            return hot

        existing = [item for item in hot.insight_queue if item.id != insight.id]
        existing.insert(0, insight)

        hot = HotMemory(
            agent_state=hot.agent_state,
            active_projects=hot.active_projects,
            working_questions=hot.working_questions,
            top_of_mind=hot.top_of_mind,
            insight_queue=existing[:INSIGHT_QUEUE_CAP],
        )

        for memory_id in insight.source_memory_ids:
            self.store.add_memory_ref(str(memory_id))

        bundle.hot_memory = self._touch_agent_state(
            hot,
            status=AgentStatus.RETURNING,
            background_task="insight_generation",
        )
        return self._save(bundle)

    def cool(self, ttl_hours: int = 36) -> HotMemory:
        return self.store.cool(ttl_hours=ttl_hours)

    def get(self) -> HotMemory:
        return self.store.get_hot_memory()


path: hot_memory/store.py
content: """Persistent hot-memory state with reinforcement metadata."""

from __future__ import annotations

import json
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from pathlib import Path

from prompt_engine.schemas import AgentState, AgentStatus, HotMemory


@dataclass
class HotMemoryBundle:
    hot_memory: HotMemory
    reinforcement: dict[str, dict[str, str]]
    retrieval_counts: dict[str, int]
    memory_refs: list[str]


class HotMemoryStore:
    """File-backed hot memory store for fast read/write."""

    def __init__(self, path: str | Path = "data/hot_memory/hot_memory.json") -> None:
        self.path = Path(path)
        self.path.parent.mkdir(parents=True, exist_ok=True)

    def _default_hot_memory(self) -> HotMemory:
        now = datetime.now(timezone.utc)
        return HotMemory(
            agent_state=AgentState(
                status=AgentStatus.IDLE,
                last_interaction_timestamp=now,
                last_background_task="none",
            ),
            active_projects=[],
            working_questions=[],
            top_of_mind=[],
            insight_queue=[],
        )

    def load_bundle(self) -> HotMemoryBundle:
        if not self.path.exists():
            return HotMemoryBundle(
                hot_memory=self._default_hot_memory(),
                reinforcement={
                    "active_projects": {},
                    "working_questions": {},
                    "top_of_mind": {},
                },
                retrieval_counts={},
                memory_refs=[],
            )

        payload = json.loads(self.path.read_text(encoding="utf-8"))
        hot_memory = HotMemory.model_validate(payload["hot_memory"])

        reinforcement = payload.get("reinforcement", {})
        retrieval_counts = payload.get("retrieval_counts", {})
        memory_refs = payload.get("memory_refs", [])

        return HotMemoryBundle(
            hot_memory=hot_memory,
            reinforcement={
                "active_projects": dict(reinforcement.get("active_projects", {})),
                "working_questions": dict(reinforcement.get("working_questions", {})),
                "top_of_mind": dict(reinforcement.get("top_of_mind", {})),
            },
            retrieval_counts={key: int(value) for key, value in retrieval_counts.items()},
            memory_refs=[str(value) for value in memory_refs],
        )

    def save_bundle(self, bundle: HotMemoryBundle) -> None:
        payload = {
            "hot_memory": bundle.hot_memory.model_dump(mode="json"),
            "reinforcement": bundle.reinforcement,
            "retrieval_counts": bundle.retrieval_counts,
            "memory_refs": bundle.memory_refs,
        }
        self.path.write_text(json.dumps(payload, indent=2), encoding="utf-8")

    def get_hot_memory(self) -> HotMemory:
        return self.load_bundle().hot_memory

    def set_hot_memory(self, hot_memory: HotMemory) -> None:
        bundle = self.load_bundle()
        bundle.hot_memory = hot_memory
        self.save_bundle(bundle)

    def add_memory_ref(self, memory_id: str, cap: int = 200) -> None:
        bundle = self.load_bundle()
        refs = [value for value in bundle.memory_refs if value != memory_id]
        refs.insert(0, memory_id)
        bundle.memory_refs = refs[:cap]
        self.save_bundle(bundle)

    def reinforce(self, section: str, item: str, at: datetime | None = None) -> None:
        bundle = self.load_bundle()
        at = at or datetime.now(timezone.utc)

        if section not in bundle.reinforcement:
            bundle.reinforcement[section] = {}

        bundle.reinforcement[section][item] = at.isoformat()
        self.save_bundle(bundle)

    def increment_retrieval(self, memory_id: str) -> int:
        bundle = self.load_bundle()
        bundle.retrieval_counts[memory_id] = bundle.retrieval_counts.get(memory_id, 0) + 1
        count = bundle.retrieval_counts[memory_id]

        refs = [value for value in bundle.memory_refs if value != memory_id]
        refs.insert(0, memory_id)
        bundle.memory_refs = refs[:200]

        self.save_bundle(bundle)
        return count

    def cool(self, *, ttl_hours: int = 36, now: datetime | None = None) -> HotMemory:
        bundle = self.load_bundle()
        now = now or datetime.now(timezone.utc)
        cutoff = now - timedelta(hours=ttl_hours)

        def keep(section: str, values: list[str]) -> list[str]:
            result: list[str] = []
            for value in values:
                reinforced = bundle.reinforcement.get(section, {}).get(value)
                if reinforced is None:
                    continue
                try:
                    timestamp = datetime.fromisoformat(reinforced)
                except ValueError:
                    continue

                if timestamp >= cutoff:
                    result.append(value)
            return result

        hot = bundle.hot_memory
        hot = HotMemory(
            agent_state=hot.agent_state,
            active_projects=keep("active_projects", hot.active_projects),
            working_questions=keep("working_questions", hot.working_questions),
            top_of_mind=keep("top_of_mind", hot.top_of_mind),
            insight_queue=[
                insight
                for insight in hot.insight_queue
                if insight.expires_at is None or insight.expires_at >= now
            ],
        )

        bundle.hot_memory = hot
        self.save_bundle(bundle)
        return hot


path: hot_memory/__init__.py
content: """Working-memory (hot memory) package."""

from .manager import HotMemoryManager
from .store import HotMemoryStore

__all__ = ["HotMemoryManager", "HotMemoryStore"]


path: HOT_MEMORY_EXTENSION.md
content: # Hot Memory Extension for Smart Memory v2.1

## Summary
This extension adds persistent "hot memory" (working context) to Smart Memory v2.1. Hot memory survives between sessions and appears in every prompt's `[WORKING CONTEXT]` section.

## Files Added/Modified

### New Files
1. `hot_memory_manager.py` - Core hot memory persistence and auto-update logic
2. `memory_adapter.py` - API wrapper that auto-includes hot memory in compose calls
3. `smem-hook.sh` - Shell hook for post-conversation updates
4. `hot_memory_state.json` - Persistent storage (auto-created)

### Modified
None (this is an extension, not a core modification)

## Key Features

### 1. Persistent Working Context
Hot memory stores:
- `active_projects` - Top 5 current projects with descriptions
- `working_questions` - Open questions being explored
- `top_of_mind` - Immediate priorities/notes
- `insight_queue` - Live pending insights from background cognition
- `agent_state` - Status, timestamps, last background task

### 2. Auto-Update from Conversations
- Detects project mentions (customizable keyword list)
- Adds questions (anything with `?`) to working_questions
- Prevents duplicate projects via key-based matching
- Updates timestamp and status on every interaction

### 3. API Integration
The `memory_adapter.py` provides:
```python
compose_with_hot_memory(agent_identity, user_message)
# Automatically includes hot memory in /compose calls

ingest_and_update(user_message, assistant_message)
# Ingests to LTM + updates hot memory in one call
```

## Bug Fix: Duplicate Prevention
Original code checked `project_desc not in projects` which failed when descriptions differed (e.g., "Mobile App - iOS development" vs "Mobile App - iOS and Android development").

**Fixed by:** `_project_exists()` function that extracts project keys (text before " - ") and compares normalized keys.

```python
def _project_exists(projects: list[str], project_key: str) -> bool:
    """Check if a project already exists by its key identifier."""
    project_key_lower = project_key.lower()
    for project in projects:
        existing_key = project.split(" - ")[0].lower() if " - " in project else project.lower()
        if existing_key == project_key_lower:
            return True
    return False
```

## Usage

### Initialize hot memory
```bash
python3 hot_memory_manager.py init
```

### After conversations
```bash
# Quick update (hot memory only)
./smem-hook.sh "user message" "assistant response"

# Full update (hot memory + LTM ingestion)
python3 memory_adapter.py ingest -m "user message" -a "assistant response"
```

### Compose with hot memory
```bash
python3 memory_adapter.py compose -m "What are my priorities?"
```

## Customizing Project Detection

Edit `auto_update_from_context()` in `hot_memory_manager.py` to match your domain:

```python
project_definitions = [
    # Example: Product/Platform projects
    (["mobile app", "ios app", "android app"], "Mobile App", "Mobile App - cross-platform development"),
    (["web platform", "web app"], "Web Platform", "Web Platform - frontend and backend development"),
    
    # Example: Infrastructure
    (["infrastructure", "infra", "deployment"], "Infrastructure", "Infrastructure - hosting, CI/CD, DevOps"),
    
    # Add your own project keywords here
]
```

## Integration Path

### Option A: Fork + PR to Upstream (Recommended)
1. Fork https://github.com/BluePointDigital/smart-memory/
2. Add these files to a `contrib/hot-memory/` directory
3. Submit PR with:
   - Hot memory extension module
   - Updated README with usage docs
   - Tests for duplicate prevention

### Option B: Separate Package
Create `smart-memory-hot` as a companion package that depends on smart-memory and provides the hot memory layer.

### Option C: Documentation Patch
If upstream isn't accepting changes, document as a "recommended extension" in the wiki.

## Testing

### Test duplicate prevention
```bash
# Initialize
python3 hot_memory_manager.py init

# Try to add duplicate project
./smem-hook.sh "Working on the mobile app today" "OK"
./smem-hook.sh "Continuing mobile app development" "Great"

# Verify only one "Mobile App" entry
python3 hot_memory_manager.py get | grep "Mobile App"
```

### Test new project addition
```bash
./smem-hook.sh "Starting the infrastructure migration" "Let's do it"
# Should add Infrastructure to active_projects
```

## Token Impact

Hot memory adds ~400 tokens to composed prompts (configurable):
- 5 active projects: ~150 tokens
- 4 working questions: ~100 tokens  
- 3 top-of-mind items: ~75 tokens
- Agent state + overhead: ~75 tokens

This fits within the default token allocation's `working_memory` budget.

---

**Created:** 2026-03-05
**Smart Memory Version:** v2.1
**License:** Same as Smart Memory (MIT)


path: hot_memory_manager.py
content: #!/usr/bin/env python3
"""
Hot Memory Manager for Smart Memory v2.1
Provides persistent hot memory storage and automatic updates.
"""

import json
import os
from datetime import datetime, timezone
from pathlib import Path

HOT_MEMORY_FILE = Path.home() / ".openclaw" / "workspace" / "smart-memory" / "hot_memory_state.json"

def get_default_hot_memory():
    """Get current hot memory state from disk or create default."""
    if HOT_MEMORY_FILE.exists():
        with open(HOT_MEMORY_FILE) as f:
            return json.load(f)
    
    return {
        "agent_state": {
            "status": "idle",
            "last_interaction_timestamp": datetime.now(timezone.utc).isoformat(),
            "last_background_task": "none"
        },
        "active_projects": [],
        "working_questions": [],
        "top_of_mind": [],
        "insight_queue": []
    }

def save_hot_memory(state: dict):
    """Persist hot memory to disk."""
    HOT_MEMORY_FILE.parent.mkdir(parents=True, exist_ok=True)
    with open(HOT_MEMORY_FILE, 'w') as f:
        json.dump(state, f, indent=2, default=str)

def update_hot_memory(
    active_projects: list[str] = None,
    working_questions: list[str] = None,
    top_of_mind: list[str] = None,
    agent_status: str = None,
    last_background_task: str = None
):
    """Update specific fields in hot memory."""
    state = get_default_hot_memory()
    
    if active_projects is not None:
        state["active_projects"] = active_projects
    if working_questions is not None:
        state["working_questions"] = working_questions
    if top_of_mind is not None:
        state["top_of_mind"] = top_of_mind
    if agent_status:
        state["agent_state"]["status"] = agent_status
    if last_background_task:
        state["agent_state"]["last_background_task"] = last_background_task
    
    state["agent_state"]["last_interaction_timestamp"] = datetime.now(timezone.utc).isoformat()
    
    save_hot_memory(state)
    return state

def get_hot_memory_for_compose():
    """Get hot memory formatted for /compose API."""
    import urllib.request
    import urllib.error
    
    state = get_default_hot_memory()
    
    # Fetch pending insights from server
    try:
        req = urllib.request.Request("http://127.0.0.1:8000/insights/pending", method='GET')
        with urllib.request.urlopen(req, timeout=2) as response:
            data = json.loads(response.read().decode('utf-8'))
            insights = data.get("insights", [])
            # Convert to proper format
            state["insight_queue"] = [
                {
                    "id": i.get("id"),
                    "content": i.get("content"),
                    "confidence": i.get("confidence"),
                    "source_memory_ids": i.get("source_memory_ids", []),
                    "generated_at": i.get("generated_at"),
                    "expires_at": i.get("expires_at")
                }
                for i in insights
            ]
    except Exception as e:
        print(f"Warning: Could not fetch insights: {e}")
    
    return state

def _project_exists(projects: list[str], project_key: str) -> bool:
    """Check if a project already exists by its key identifier."""
    project_key_lower = project_key.lower()
    for project in projects:
        # Extract the key from existing project (text before first " - ")
        existing_key = project.split(" - ")[0].lower() if " - " in project else project.lower()
        if existing_key == project_key_lower:
            return True
    return False

def auto_update_from_context(user_message: str, assistant_message: str):
    """Automatically update hot memory based on conversation content.
    
    Configure project_definitions below to match your domain.
    """
    state = get_default_hot_memory()
    
    # Detect project mentions
    projects = state.get("active_projects", [])
    message_lower = (user_message + " " + assistant_message).lower()
    
    # Project definitions: (trigger_keywords, project_key, default_description)
    # Customize this list for your domain - examples shown
    project_definitions = [
        # Example: Product/Platform projects
        (["mobile app", "ios app", "android app"], "Mobile App", "Mobile App - cross-platform development"),
        (["web platform", "web app"], "Web Platform", "Web Platform - frontend and backend development"),
        (["api", "backend"], "API Development", "API Development - service architecture and endpoints"),
        
        # Example: Content/Business projects
        (["blog", "content"], "Content Strategy", "Content Strategy - editorial calendar and publishing"),
        (["marketing", "campaign"], "Marketing", "Marketing - campaigns and growth initiatives"),
        (["documentation", "docs"], "Documentation", "Documentation - technical writing and guides"),
        
        # Example: Infrastructure/DevOps
        (["infrastructure", "infra", "deployment"], "Infrastructure", "Infrastructure - hosting, CI/CD, DevOps"),
        (["database", "db migration"], "Database", "Database - schema design and migrations"),
        (["security", "auth"], "Security", "Security - authentication, authorization, compliance"),
        
        # Example: AI/Memory (meta - for this project itself)
        (["smart memory", "memory system"], "Smart Memory", "Smart Memory - cognitive memory architecture"),
    ]
    
    for keywords, project_key, default_desc in project_definitions:
        # Check if any keyword matches
        if any(kw in message_lower for kw in keywords):
            # Check if project already exists
            if not _project_exists(projects, project_key):
                # Add with default description (or just key if no default)
                desc = default_desc or f"{project_key} - active project"
                projects.insert(0, desc)
    
    # Keep only top 5 active projects
    state["active_projects"] = projects[:5]
    
    # Detect working questions (questions that weren't immediately resolved)
    questions = state.get("working_questions", [])
    if "?" in user_message and len(user_message) > 20:
        # Extract the question
        question = user_message.strip()
        if question not in questions:
            questions.insert(0, question)
    
    # Keep only top 5 working questions
    state["working_questions"] = questions[:5]
    
    # Update agent state
    state["agent_state"]["status"] = "engaged"
    state["agent_state"]["last_interaction_timestamp"] = datetime.now(timezone.utc).isoformat()
    
    save_hot_memory(state)
    return state

if __name__ == "__main__":
    import sys
    
    if len(sys.argv) < 2:
        print("Usage: hot_memory_manager.py <command> [args]")
        print("")
        print("Commands:")
        print("  get           - Show current hot memory state")
        print("  init          - Initialize with example context")
        print("  update        - Update specific fields (interactive)")
        print("  compose       - Get hot memory formatted for /compose API")
        print("  auto <user_msg> <asst_msg>  - Auto-update from conversation")
        print("")
        print("To customize project detection, edit project_definitions in auto_update_from_context()")
        sys.exit(1)
    
    cmd = sys.argv[1]
    
    if cmd == "get":
        state = get_default_hot_memory()
        print(json.dumps(state, indent=2, default=str))
    
    elif cmd == "init":
        # Initialize with generic example context
        state = update_hot_memory(
            active_projects=[
                "Example Project - description of current work",
                "Another Initiative - another project description",
                "Infrastructure - hosting and deployment tasks",
                "Documentation - technical writing and guides",
                "Research - exploratory work and learning"
            ],
            working_questions=[
                "What is the scope of the current milestone?",
                "How should we prioritize the backlog?",
                "What are the blockers for deployment?",
                "Which architecture pattern should we use?"
            ],
            top_of_mind=[
                "Review recent system changes",
                "Update project documentation",
                "Plan next development sprint"
            ],
            agent_status="engaged",
            last_background_task="initialization"
        )
        print("✓ Hot memory initialized with example context")
        print(json.dumps(state, indent=2, default=str))
    
    elif cmd == "compose":
        state = get_hot_memory_for_compose()
        print(json.dumps(state, indent=2, default=str))
    
    elif cmd == "auto" and len(sys.argv) >= 4:
        state = auto_update_from_context(sys.argv[2], sys.argv[3])
        print("✓ Auto-updated from conversation")
        print(json.dumps(state, indent=2, default=str))
    
    else:
        print(f"Unknown command: {cmd}")
        sys.exit(1)


path: ingestion/heuristic_filter.py
content: """Fast rule-based filtering for memory-worthy interactions."""

from __future__ import annotations

from dataclasses import dataclass
import re


EMOTIONAL_TERMS = {
    "love",
    "hate",
    "frustrated",
    "excited",
    "worried",
    "concerned",
    "happy",
    "upset",
    "angry",
    "important",
    "critical",
}

NAMED_ENTITY_PATTERN = re.compile(r"\b[A-Z][a-z]{2,}\b")
PROJECT_PATTERN = re.compile(r"\bproj[_\-][a-z0-9_\-]+\b", flags=re.IGNORECASE)


@dataclass(frozen=True)
class HeuristicDecision:
    should_continue: bool
    triggers: list[str]


def evaluate_heuristics(
    *,
    user_message: str,
    assistant_message: str = "",
    system_generated_insight: bool = False,
    min_words: int = 8,
) -> HeuristicDecision:
    """Check lightweight triggers before heavier ingestion steps."""

    triggers: list[str] = []
    text = f"{user_message} {assistant_message}".strip()
    words = user_message.split()
    text_lower = text.lower()

    if len(words) >= min_words:
        triggers.append("length_threshold")

    if NAMED_ENTITY_PATTERN.search(text):
        triggers.append("named_entity")

    if PROJECT_PATTERN.search(text):
        triggers.append("project_reference")

    if any(term in text_lower for term in EMOTIONAL_TERMS):
        triggers.append("emotional_language")

    if "?" in user_message:
        triggers.append("user_question")

    if system_generated_insight:
        triggers.append("system_generated_insight")

    return HeuristicDecision(should_continue=bool(triggers), triggers=triggers)


path: ingestion/importance_llm.py
content: """Optional LLM importance scorer contract.

The ingestion pipeline calls this only after lightweight heuristic gating.
"""

from __future__ import annotations

from typing import Protocol


class ImportanceLLMScorer(Protocol):
    def score_importance(
        self,
        *,
        user_message: str,
        assistant_message: str,
        heuristic_score: float,
        entity_count: int,
        triggers: list[str],
    ) -> float:
        """Return refined importance in [0.0, 1.0]."""


def clamp_importance(value: float) -> float:
    return max(0.0, min(1.0, float(value)))


path: ingestion/importance_scorer.py
content: """Importance scoring for long-term memory eligibility."""

from __future__ import annotations

from dataclasses import dataclass


DECISION_TERMS = {
    "decide",
    "decision",
    "agreed",
    "conclusion",
    "final",
    "resolved",
}

GOAL_TERMS = {
    "goal",
    "objective",
    "plan",
    "milestone",
    "todo",
    "roadmap",
    "start",
    "begin",
}

PREFERENCE_TERMS = {
    "prefer",
    "preference",
    "value",
    "principle",
    "always",
    "never",
}

EMOTIONAL_TERMS = {
    "love",
    "hate",
    "frustrated",
    "excited",
    "worried",
    "critical",
    "important",
}


@dataclass(frozen=True)
class ImportanceBreakdown:
    score: float
    components: dict[str, float]


def _clamp(value: float, low: float = 0.0, high: float = 1.0) -> float:
    return max(low, min(high, value))


def score_importance(
    *,
    user_message: str,
    assistant_message: str = "",
    entity_count: int = 0,
) -> ImportanceBreakdown:
    """Compute deterministic importance score in [0.0, 1.0]."""

    text = f"{user_message} {assistant_message}".strip().lower()
    words = user_message.split()

    length_component = _clamp(len(words) / 40.0, 0.0, 0.25)
    entity_component = _clamp(entity_count * 0.03, 0.0, 0.20)
    question_component = 0.12 if "?" in user_message else 0.0

    decision_component = 0.20 if any(term in text for term in DECISION_TERMS) else 0.0
    goal_component = 0.18 if any(term in text for term in GOAL_TERMS) else 0.0
    preference_component = 0.16 if any(term in text for term in PREFERENCE_TERMS) else 0.0

    emotional_hits = sum(1 for term in EMOTIONAL_TERMS if term in text)
    emotional_component = _clamp(emotional_hits * 0.04, 0.0, 0.16)

    components = {
        "length": length_component,
        "entities": entity_component,
        "question": question_component,
        "decision": decision_component,
        "goal": goal_component,
        "preference": preference_component,
        "emotion": emotional_component,
    }

    score = _clamp(sum(components.values()))
    return ImportanceBreakdown(score=score, components=components)


path: ingestion/ingestion_pipeline.py
content: """Phase 2 ingestion pipeline.

Pipeline:
Incoming interaction
-> heuristic filter
-> entity extraction
-> importance scoring
-> (optional) LLM importance refinement
-> memory classification
-> memory object creation
-> embedding generation
-> persistence (JSON + vector index)
"""

from __future__ import annotations

from dataclasses import dataclass
from datetime import datetime, timezone
from typing import Any

from pydantic import BaseModel, ConfigDict, Field

from embeddings import TextEmbedder, create_default_embedder
from entities import EntityAliasResolver
from prompt_engine.entity_extractor import extract_entities
from prompt_engine.schemas import MemoryType
from storage import JSONMemoryStore, VectorIndexStore

from .heuristic_filter import HeuristicDecision, evaluate_heuristics
from .importance_llm import ImportanceLLMScorer, clamp_importance
from .importance_scorer import ImportanceBreakdown, score_importance
from .memory_builder import build_memory_object
from .memory_classifier import classify_memory_type


class IncomingInteraction(BaseModel):
    model_config = ConfigDict(extra="forbid")

    user_message: str = Field(min_length=1)
    assistant_message: str = ""
    timestamp: datetime | None = None
    source: str = "conversation"
    participants: list[str] = Field(default_factory=lambda: ["user", "assistant"])
    active_projects: list[str] = Field(default_factory=list)
    metadata: dict[str, Any] = Field(default_factory=dict)


@dataclass(frozen=True)
class IngestionPipelineConfig:
    minimum_importance_to_store: float = 0.45
    min_words_for_heuristic: int = 8
    enable_llm_refinement: bool = True
    llm_trigger_count_threshold: int = 2
    llm_min_heuristic_score: float = 0.30
    semantic_dedup_threshold: float = 0.85


@dataclass(frozen=True)
class IngestionResult:
    stored: bool
    reason: str
    triggers: list[str]
    importance: float
    memory_id: str | None
    memory_type: MemoryType | None
    llm_used: bool = False


class IngestionPipeline:
    """Main ingestion orchestrator with deterministic behavior."""

    def __init__(
        self,
        *,
        json_store: JSONMemoryStore | None = None,
        vector_store: VectorIndexStore | None = None,
        embedder: TextEmbedder | None = None,
        entity_resolver: EntityAliasResolver | None = None,
        llm_scorer: ImportanceLLMScorer | None = None,
        config: IngestionPipelineConfig = IngestionPipelineConfig(),
    ) -> None:
        self.json_store = json_store or JSONMemoryStore()
        self.vector_store = vector_store or VectorIndexStore()
        self.embedder = embedder or create_default_embedder()
        self.entity_resolver = entity_resolver or EntityAliasResolver()
        self.llm_scorer = llm_scorer
        self.config = config

    def _should_refine_with_llm(
        self,
        *,
        heuristic: HeuristicDecision,
        heuristic_importance: float,
        system_generated_insight: bool,
    ) -> bool:
        if self.llm_scorer is None or not self.config.enable_llm_refinement:
            return False

        if system_generated_insight:
            return True

        if len(heuristic.triggers) >= self.config.llm_trigger_count_threshold:
            return True

        if heuristic_importance >= self.config.llm_min_heuristic_score:
            return True

        return False

    
    def _supports_semantic_dedup(self) -> bool:
        model_name = str(getattr(self.embedder, "model_name", "")).lower()
        # Hashing fallback is deterministic but too coarse for reliable semantic dedup.
        return "hashing" not in model_name

    def _find_semantic_duplicate(self, vector: list[float]) -> tuple[str, float] | None:
        try:
            hits = self.vector_store.search(query_vector=vector, top_k=1)
        except Exception:  # noqa: BLE001
            return None

        if not hits:
            return None

        top = hits[0]
        if top.score <= self.config.semantic_dedup_threshold:
            return None

        return (top.memory_id, top.score)

    def _reinforce_existing_memory(self, memory_id: str, *, now: datetime) -> tuple[str, MemoryType] | None:
        existing = self.json_store.get_memory(memory_id)
        if existing is None:
            return None

        updates: dict[str, Any] = {
            "last_accessed": now,
            "access_count": existing.access_count + 1,
        }

        if existing.type == MemoryType.BELIEF:
            updates["reinforced_count"] = existing.reinforced_count + 1

        reinforced = existing.model_copy(update=updates)
        self.json_store.update_memory(reinforced)
        return (str(reinforced.id), reinforced.type)

    def ingest(self, interaction: IncomingInteraction) -> IngestionResult:
        system_generated_insight = bool(
            interaction.metadata.get("system_generated_insight", False)
        )

        heuristic: HeuristicDecision = evaluate_heuristics(
            user_message=interaction.user_message,
            assistant_message=interaction.assistant_message,
            system_generated_insight=system_generated_insight,
            min_words=self.config.min_words_for_heuristic,
        )

        if not heuristic.should_continue:
            return IngestionResult(
                stored=False,
                reason="no_heuristic_triggers",
                triggers=heuristic.triggers,
                importance=0.0,
                memory_id=None,
                memory_type=None,
                llm_used=False,
            )

        raw_entities = extract_entities(
            current_user_message=interaction.user_message,
            conversation_history=interaction.assistant_message,
            active_projects=interaction.active_projects,
        )
        entities = self.entity_resolver.canonicalize_many(raw_entities)

        importance_breakdown: ImportanceBreakdown = score_importance(
            user_message=interaction.user_message,
            assistant_message=interaction.assistant_message,
            entity_count=len(entities),
        )

        importance_score = importance_breakdown.score
        llm_used = False

        if self._should_refine_with_llm(
            heuristic=heuristic,
            heuristic_importance=importance_score,
            system_generated_insight=system_generated_insight,
        ):
            llm_score = self.llm_scorer.score_importance(
                user_message=interaction.user_message,
                assistant_message=interaction.assistant_message,
                heuristic_score=importance_score,
                entity_count=len(entities),
                triggers=heuristic.triggers,
            )
            importance_score = clamp_importance((importance_score * 0.6) + (llm_score * 0.4))
            llm_used = True

        if importance_score < self.config.minimum_importance_to_store:
            return IngestionResult(
                stored=False,
                reason="below_importance_threshold",
                triggers=heuristic.triggers,
                importance=importance_score,
                memory_id=None,
                memory_type=None,
                llm_used=llm_used,
            )

        memory_type = classify_memory_type(
            user_message=interaction.user_message,
            assistant_message=interaction.assistant_message,
        )

        memory = build_memory_object(
            memory_type=memory_type,
            user_message=interaction.user_message,
            assistant_message=interaction.assistant_message,
            importance=importance_score,
            source=interaction.source,
            timestamp=interaction.timestamp,
            participants=interaction.participants,
            active_projects=interaction.active_projects,
            entities_override=entities,
        )

        memory_id = str(memory.id)
        vector = self.embedder.embed(memory.content)

        duplicate = self._find_semantic_duplicate(vector) if self._supports_semantic_dedup() else None
        if duplicate is not None:
            existing_id, _score = duplicate
            reinforced = self._reinforce_existing_memory(
                existing_id,
                now=interaction.timestamp or datetime.now(timezone.utc),
            )
            if reinforced is not None:
                reinforced_id, reinforced_type = reinforced
                return IngestionResult(
                    stored=False,
                    reason="semantic_duplicate_reinforced",
                    triggers=heuristic.triggers,
                    importance=importance_score,
                    memory_id=reinforced_id,
                    memory_type=reinforced_type,
                    llm_used=llm_used,
                )

        self.json_store.save_memory(memory)

        self.vector_store.upsert_vector(
            memory_id=memory_id,
            vector=vector,
            model_name=self.embedder.model_name,
            payload={
                "memory_id": memory_id,
                "type": memory.type.value,
                "importance": round(memory.importance, 6),
                "created_at": memory.created_at.isoformat(),
                "source": memory.source.value,
                "entities": memory.entities,
                "schema_version": memory.schema_version,
            },
        )

        return IngestionResult(
            stored=True,
            reason="stored",
            triggers=heuristic.triggers,
            importance=importance_score,
            memory_id=memory_id,
            memory_type=memory_type,
            llm_used=llm_used,
        )

    def ingest_dict(self, payload: dict[str, Any]) -> IngestionResult:
        interaction = IncomingInteraction.model_validate(payload)
        return self.ingest(interaction)

    def ingest_many(self, payloads: list[dict[str, Any]]) -> list[IngestionResult]:
        results: list[IngestionResult] = []
        for payload in payloads:
            results.append(self.ingest_dict(payload))
        return results





path: ingestion/memory_builder.py
content: """Construct phase-1 memory objects from classified interactions."""

from __future__ import annotations

from datetime import datetime, timezone
import re
from typing import Iterable

from entities import EntityAliasResolver
from prompt_engine.entity_extractor import extract_entities
from prompt_engine.schemas import (
    BeliefMemory,
    EpisodicMemory,
    GoalMemory,
    GoalStatus,
    LongTermMemory,
    MemorySource,
    MemoryType,
    RelationTriple,
    SemanticMemory,
)


POSITIVE_WORDS = {
    "good",
    "great",
    "excellent",
    "happy",
    "love",
    "win",
    "success",
    "progress",
}

NEGATIVE_WORDS = {
    "bad",
    "poor",
    "frustrated",
    "hate",
    "blocked",
    "fail",
    "issue",
    "problem",
}

RELATION_PATTERN = re.compile(
    r"\b([A-Za-z][A-Za-z0-9_\-]{2,})\s+"
    r"(uses|depends_on|blocks|supports|interacts_with|relates_to)\s+"
    r"([A-Za-z][A-Za-z0-9_\-]{2,})\b",
    flags=re.IGNORECASE,
)


def _normalize_entity(value: str) -> str:
    return value.strip().lower().replace("-", "_")


def detect_relations(text: str, resolver: EntityAliasResolver | None = None) -> list[RelationTriple]:
    relations: list[RelationTriple] = []
    seen: set[tuple[str, str, str]] = set()

    for subject, predicate, obj in RELATION_PATTERN.findall(text):
        if resolver:
            subject_id = resolver.resolve(subject)
            object_id = resolver.resolve(obj)
        else:
            subject_id = _normalize_entity(subject)
            object_id = _normalize_entity(obj)

        triple = (
            subject_id,
            predicate.strip().lower(),
            object_id,
        )
        if triple in seen:
            continue

        seen.add(triple)
        relations.append(
            RelationTriple(subject=triple[0], predicate=triple[1], object=triple[2])
        )

    return relations


def emotional_metadata(text: str) -> tuple[float, float]:
    tokens = [token.strip(".,!?;:").lower() for token in text.split()]
    if not tokens:
        return 0.0, 0.0

    positive = sum(1 for token in tokens if token in POSITIVE_WORDS)
    negative = sum(1 for token in tokens if token in NEGATIVE_WORDS)

    valence_raw = (positive - negative) / max(1, positive + negative)
    intensity_raw = (positive + negative) / max(1, len(tokens) / 2)

    valence = max(-1.0, min(1.0, valence_raw))
    intensity = max(0.0, min(1.0, intensity_raw))
    return valence, intensity


def parse_source(source: str | None) -> MemorySource:
    if source is None:
        return MemorySource.CONVERSATION

    normalized = source.strip().lower()
    for enum_value in MemorySource:
        if enum_value.value == normalized:
            return enum_value

    return MemorySource.CONVERSATION


def build_memory_object(
    *,
    memory_type: MemoryType,
    user_message: str,
    assistant_message: str = "",
    importance: float,
    source: str | None = None,
    timestamp: datetime | None = None,
    participants: Iterable[str] | None = None,
    active_projects: Iterable[str] | None = None,
    entities_override: list[str] | None = None,
    entity_resolver: EntityAliasResolver | None = None,
) -> LongTermMemory:
    """Create a typed long-term memory object using phase-1 schemas."""

    timestamp = timestamp or datetime.now(timezone.utc)
    text_for_entities = f"{user_message} {assistant_message}".strip()

    resolver = entity_resolver or EntityAliasResolver()

    if entities_override is not None:
        entities = resolver.canonicalize_many(entities_override)
    else:
        extracted = extract_entities(
            current_user_message=user_message,
            conversation_history=assistant_message,
            active_projects=active_projects,
        )
        entities = resolver.canonicalize_many(extracted)

    relations = detect_relations(text_for_entities, resolver=resolver)
    valence, intensity = emotional_metadata(text_for_entities)
    memory_source = parse_source(source)

    common = {
        "content": user_message.strip(),
        "importance": importance,
        "created_at": timestamp,
        "last_accessed": timestamp,
        "access_count": 0,
        "schema_version": "2.0",
        "entities": entities,
        "relations": relations,
        "emotional_valence": valence,
        "emotional_intensity": intensity,
        "source": memory_source,
    }

    if memory_type == MemoryType.SEMANTIC:
        confidence = min(1.0, max(0.45, importance))
        return SemanticMemory(confidence=confidence, **common)

    if memory_type == MemoryType.BELIEF:
        confidence = min(1.0, max(0.50, importance))
        return BeliefMemory(confidence=confidence, reinforced_count=1, **common)

    if memory_type == MemoryType.GOAL:
        priority = min(1.0, max(0.45, importance))
        return GoalMemory(status=GoalStatus.ACTIVE, priority=priority, **common)

    participant_ids = [resolver.resolve(participant) for participant in (participants or ["user", "assistant"])]
    return EpisodicMemory(participants=participant_ids, **common)


path: ingestion/memory_classifier.py
content: """Memory type classifier for ingestion."""

from __future__ import annotations

from prompt_engine.schemas import MemoryType


GOAL_TERMS = {
    "goal",
    "objective",
    "plan",
    "milestone",
    "todo",
    "start",
    "begin",
    "launch",
}

BELIEF_TERMS = {
    "prefer",
    "i like",
    "i dislike",
    "value",
    "principle",
    "usually",
    "always",
    "never",
}

SEMANTIC_TERMS = {
    "is",
    "are",
    "means",
    "defined",
    "fact",
    "remember that",
}

EPISODIC_TERMS = {
    "completed",
    "finished",
    "did",
    "happened",
    "worked on",
    "implemented",
}


def classify_memory_type(
    *,
    user_message: str,
    assistant_message: str = "",
) -> MemoryType:
    """Classify an interaction into episodic, semantic, belief, or goal."""

    text = f"{user_message} {assistant_message}".strip().lower()

    if any(term in text for term in GOAL_TERMS):
        return MemoryType.GOAL

    if any(term in text for term in BELIEF_TERMS):
        return MemoryType.BELIEF

    if any(term in text for term in EPISODIC_TERMS):
        return MemoryType.EPISODIC

    if any(term in text for term in SEMANTIC_TERMS):
        return MemoryType.SEMANTIC

    return MemoryType.EPISODIC


path: ingestion/__init__.py
content: """Ingestion pipeline package."""

from .importance_llm import ImportanceLLMScorer
from .ingestion_pipeline import (
    IngestionPipeline,
    IngestionPipelineConfig,
    IngestionResult,
    IncomingInteraction,
)

__all__ = [
    "ImportanceLLMScorer",
    "IngestionPipeline",
    "IngestionPipelineConfig",
    "IngestionResult",
    "IncomingInteraction",
]


path: install.sh
content: #!/bin/bash
# One-line installer for Smart Memory v2
# Usage: curl -sL https://raw.githubusercontent.com/BluePointDigital/smart-memory/master/install.sh | bash

set -e

echo "Installing Smart Memory v2 for OpenClaw..."
echo ""

# Detect OpenClaw workspace
if [ -d "$HOME/.openclaw/workspace" ]; then
    WORKSPACE="$HOME/.openclaw/workspace"
elif [ -d "/config/.openclaw/workspace" ]; then
    WORKSPACE="/config/.openclaw/workspace"
else
    echo "Could not find OpenClaw workspace"
    echo "Please run from your OpenClaw workspace directory"
    exit 1
fi

echo "Found workspace: $WORKSPACE"
echo ""

# Check for Node.js
if ! command -v node >/dev/null 2>&1; then
    echo "Node.js not found. Please install Node.js 18+"
    exit 1
fi

NODE_VERSION=$(node --version | cut -d'v' -f2 | cut -d'.' -f1)
if [ "$NODE_VERSION" -lt 18 ]; then
    echo "Node.js version $NODE_VERSION found. Please upgrade to 18+"
    exit 1
fi

echo "Node.js $(node --version) found"

# Check for Python
if ! command -v python3 >/dev/null 2>&1 && ! command -v python >/dev/null 2>&1; then
    echo "Python 3.11+ is required for the cognitive engine"
    exit 1
fi

echo "Python found"
echo "CPU-only PyTorch is mandatory for Smart Memory v2 (GPU/CUDA builds are intentionally unsupported)."
echo ""

# Download latest repository snapshot
REPO_URL="https://github.com/BluePointDigital/smart-memory"
TARGET_DIR="$WORKSPACE/smart-memory"

rm -rf "$TARGET_DIR"
mkdir -p "$TARGET_DIR"

if command -v git >/dev/null 2>&1; then
    cd /tmp
    rm -rf smart-memory-temp 2>/dev/null || true
    git clone --depth 1 "$REPO_URL.git" smart-memory-temp
    cp -r smart-memory-temp/* "$TARGET_DIR/"
    rm -rf smart-memory-temp
else
    cd /tmp
    rm -rf smart-memory-master 2>/dev/null || true
    curl -L "$REPO_URL/archive/refs/heads/master.tar.gz" | tar xz
    cp -r smart-memory-master/* "$TARGET_DIR/"
    rm -rf smart-memory-master
fi

echo "Files installed"
echo ""

# Install Node adapter dependencies (triggers Python venv + requirements via postinstall)
cd "$TARGET_DIR/smart-memory"
npm install --silent

echo ""
echo "Installation complete"
echo ""
echo "Smart Memory v2 is ready."
echo "The Node adapter will start the local FastAPI cognitive server automatically when used."




path: INTEGRATION.md
content: # Agent Integration Guide

## Philosophy: Agents That Remember

Smart Memory v2 is designed for agents that **experience** continuity, not just store data. This guide explains how to integrate the system so your agent has genuine memory agency.

## The Core Principle

**The agent reaches for its own memories.**

Rather than external scripts injecting context, the agent:
1. Verifies its memory system is online
2. Queries what it needs when it needs it
3. Holds context internally
4. Responds from continuity

This creates resilient, authentic memory experience.

## Session Startup Pattern (Critical)

**⚠️ Without this pattern, your agent wakes up blank every session.**

The most common integration failure: agents start responding before querying memory. Here's the mandatory sequence:

### The 4-Step Startup

```
User starts session
    ↓
Agent wakes up
    ↓
[1] Health Check → Is memory server running?
    ↓
[2] Self-Prime → Query /compose endpoint
    ↓
[3] Internalize → Hold context in agent state
    ↓
[4] Respond → Generate grounded greeting
```

### Implementation by Framework

**OpenClaw:**
- Use `examples/session-start/openclaw-prime.sh`
- Add to AGENTS.md "Every Session" section
- Read `.session-memory-context.json` before greeting

**Python (LangChain, custom):**
- Use `examples/session-start/generic-python.py`
- Call `prime_memory()` in agent `__init__` or `wake()`
- See [docs/AGENT-FRAMEWORKS.md](docs/AGENT-FRAMEWORKS.md#python)

**Node.js:**
- Use `examples/session-start/nodejs-agent.js`
- Call `primeMemory()` before first response
- See [docs/AGENT-FRAMEWORKS.md](docs/AGENT-FRAMEWORKS.md#nodejs--javascript-agents)

**CLI Tool (Universal):**
```bash
python tools/session-prime.py --agent-name "MyAgent" --output context.json
```

### What Happens Without This

```
User: /new
Agent: "Hello! How can I help you today?"  ← Blank slate. Generic. Broken.
```

### What Happens With This

```
User: /new
Agent: "Hey James. 🌙 We were just finishing the automatic memory 
        priming script. The server has our history loaded. What's next?"
        ← Grounded. Continuous. Working.
```

### The Session Start Pattern (Detailed)

#### 1. Health Check (Self-Check)

```javascript
import memory from "smart-memory";

// Agent verifies its memory is accessible
let healthy = false;
try {
  const status = await memory.start(); // Checks /health
  healthy = status.healthy;
} catch (err) {
  // Self-heal: attempt restart
  console.error("Memory system unreachable, attempting restart...");
  // ...restart logic
}
```

### 2. Context Priming (Self-Prime)

```javascript
// Agent retrieves its own hot memory and recent context
const priming = await memory.getPromptContext({
  agent_identity: "You are Nyx, a cognitive assistant...",
  temporal_context: {
    current_time: new Date().toISOString(),
    timezone: "America/New_York"
  },
  conversation_history: "", // Fresh session
  current_message: "Session start - what are my active projects and priorities?"
});
```

This returns:
- **Identity context**: Who the agent is
- **Temporal state**: Current time, last interaction delta
- **Hot memory**: Active projects, working questions, top-of-mind
- **Selected LTM**: Recent/relevant long-term memories
- **Token budgets**: Strictly bounded

### 3. Internalize (Hold Context)

The agent holds this internally—not as a file, not as injected text, but as **its current state**.

```javascript
// Agent's internal state
this.context = {
  identity: priming.identity_context,
  temporal: priming.temporal_state,
  hot: priming.hot_memory,
  retrieved: priming.selected_memories,
  budget: priming.token_budgets
};
```

### 4. Respond Grounded

Now when the user speaks, the agent responds from continuity:

```javascript
async onUserMessage(message) {
  // Re-retrieve if topic shifted significantly
  if (this.topicShifted(message)) {
    this.context.retrieved = await memory.retrieveContext({
      user_message: message,
      conversation_history: this.getRecentHistory()
    });
  }
  
  // Ingest this interaction
  await memory.ingestMessage({
    user_message: message,
    assistant_message: response,
    timestamp: new Date().toISOString()
  });
  
  return response;
}
```

## Why Self-Directed?

| Approach | Problem | Solution |
|----------|---------|----------|
| **External priming** (AGENTS.md writes context file) | Single point of failure, stale context, agent doesn't "own" memory | Agent queries when needed, holds fresh context |
| **Scripted startup** (must run before agent) | Race conditions, silent failures, hard to debug | Agent verifies health, self-heals if needed |
| **Passive storage** (agent receives dumped memories) | No agency, feels like retrieval not remembering | Agent queries selectively, maintains working memory |

## Mid-Session Re-Query

Memory isn't static. As conversation evolves, re-query:

```javascript
// When user pivots topics
if (newTopic !== currentTopic) {
  const fresh = await memory.retrieveContext({
    user_message: newTopic,
    conversation_history: recentHistory
  });
  this.updateWorkingMemory(fresh);
}

// When user asks "what did we decide about X?"
const recall = await memory.retrieveContext({
  user_message: "decisions about " + topic,
  conversation_history: "" // Focus on long-term, not recent
});
```

## Background Cognition

Agents should trigger periodic maintenance (hourly, or after significant sessions):

```javascript
// After important conversation
await memory.runBackground({ scheduled: false });

// Or periodically via cron/heartbeat
await memory.runBackground({ scheduled: true });
```

This runs:
- **Reflection**: Generate insights from recent memories
- **Consolidation**: Merge redundant memories, reinforce important ones
- **Decay**: Reduce weight of old/unused memories
- **Conflict resolution**: Detect contradictory beliefs

## Error Handling

The agent should handle memory system failures gracefully:

```javascript
try {
  await memory.start();
} catch (err) {
  // Log but don't crash
  console.warn("Memory system offline, operating without persistence");
  this.memoryAvailable = false;
}

// In operations...
if (this.memoryAvailable) {
  await memory.ingestMessage(interaction);
}
```

## Implementation Checklist

- [ ] Agent calls `memory.start()` at session init
- [ ] Agent queries `getPromptContext()` or `retrieveContext()` on wake
- [ ] Agent holds context internally (not relying on external priming)
- [ ] Agent ingests interactions after responses
- [ ] Agent periodically triggers `runBackground()`
- [ ] Agent handles memory system failures gracefully
- [ ] Agent can re-query mid-session when topics shift

## OpenClaw-Specific: Tool Registration

When using Smart Memory v2.5 with OpenClaw, you must disable the built-in memory tools to prevent shadowing:

```bash
# The built-in memory_search and memory_get tools use FTS on MEMORY.md files
# Disable them so the skill's semantic tools take precedence
openclaw config set tools.deny '["memory_search", "memory_get"]'
openclaw gateway restart
```

**Why this matters:**
- Built-in `memory_search`: File-based FTS search of MEMORY.md (no embeddings)
- Skill `memory_search`: Semantic retrieval via Nomic embeddings + reranking

Without this config change, calling `memory_search` will silently use FTS instead of your Smart Memory server.

## Example: Complete Integration

See `examples/agent-integration.js` for a full working example.

## The Goal

An agent that:
- **Remembers who it is** (identity)
- **Knows what time it is** (temporal)
- **Holds current priorities** (hot memory)
- **Recalls relevant history** (LTM)
- **Does this itself** (self-directed)

That's continuity. That's a mind, not a cache.

## Resources

- **[examples/session-start/](examples/session-start/)** — Framework-specific startup code
  - `openclaw-prime.sh` — OpenClaw bash script
  - `generic-python.py` — Python module for any framework
  - `nodejs-agent.js` — Node.js module
  - `README.md` — Usage examples
  
- **[docs/AGENT-FRAMEWORKS.md](docs/AGENT-FRAMEWORKS.md)** — Detailed integration for:
  - OpenClaw
  - LangChain
  - OpenAI Assistants API
  - Custom Python agents
  - Node.js agents
  - Docker/containerized deployments
  
- **[tools/session-prime.py](tools/session-prime.py)** — Universal CLI tool for session priming


path: memory_adapter.py
content: #!/usr/bin/env python3
"""
Smart Memory Adapter with Hot Memory Integration
Wraps the existing smart-memory API to include persistent hot memory.
"""

import json
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent))
from hot_memory_manager import get_hot_memory_for_compose, auto_update_from_context

import urllib.request
import urllib.error

def _api_call(endpoint: str, payload: dict) -> dict:
    """Make API call using urllib (no external deps)."""
    url = f"http://127.0.0.1:8000{endpoint}"
    data = json.dumps(payload).encode('utf-8')
    headers = {'Content-Type': 'application/json'}
    
    req = urllib.request.Request(url, data=data, headers=headers, method='POST')
    
    try:
        with urllib.request.urlopen(req, timeout=10) as response:
            return json.loads(response.read().decode('utf-8'))
    except urllib.error.HTTPError as e:
        return {"error": e.read().decode('utf-8'), "status": e.code}

def compose_with_hot_memory(agent_identity: str, user_message: str, **kwargs):
    """Call /compose with hot memory included."""
    hot_memory = get_hot_memory_for_compose()
    
    payload = {
        "agent_identity": agent_identity,
        "current_user_message": user_message,
        "hot_memory": hot_memory,
        **kwargs
    }
    
    return _api_call("/compose", payload)

def ingest_and_update(user_message: str, assistant_message: str, timestamp: str = None):
    """Ingest interaction AND auto-update hot memory."""
    from datetime import datetime, timezone
    
    if timestamp is None:
        timestamp = datetime.now(timezone.utc).isoformat()
    
    # First, update hot memory based on this conversation
    auto_update_from_context(user_message, assistant_message)
    
    # Then, ingest to long-term memory
    payload = {
        "user_message": user_message,
        "assistant_message": assistant_message,
        "timestamp": timestamp
    }
    
    return _api_call("/ingest", payload)

if __name__ == "__main__":
    import argparse
    
    parser = argparse.ArgumentParser(description="Smart Memory with Hot Memory integration")
    parser.add_argument("command", choices=["compose", "ingest", "get-hot-memory"])
    parser.add_argument("--identity", default="Nyx - AI assistant with soft fire energy")
    parser.add_argument("--message", "-m", required=True, help="User message")
    parser.add_argument("--assistant", "-a", help="Assistant message (for ingest)")
    parser.add_argument("--max-tokens", type=int, default=8192)
    
    args = parser.parse_args()
    
    if args.command == "compose":
        result = compose_with_hot_memory(
            agent_identity=args.identity,
            user_message=args.message,
            max_prompt_tokens=args.max_tokens
        )
        print(json.dumps(result, indent=2, default=str))
    
    elif args.command == "ingest" and args.assistant:
        result = ingest_and_update(args.message, args.assistant)
        print(json.dumps(result, indent=2, default=str))
    
    elif args.command == "get-hot-memory":
        state = get_hot_memory_for_compose()
        print(json.dumps(state, indent=2, default=str))


path: MEMORY_STRUCTURE.md
content: # Suggested Memory Structure (Smart Memory v2)

Smart Memory v2 persists structured cognitive state locally.

## Runtime Data Layout

```text
workspace/
+-- data/
�   +-- memory_store/
�   �   +-- memories/                # Canonical long-term memory JSON objects
�   �   +-- archive/                 # Archived / consolidated / decayed memory JSON
�   �   +-- vector_index.sqlite      # Vector index (IDs + embeddings + compact payload)
�   +-- hot_memory/
�       +-- hot_memory.json          # Working memory + insight queue
+-- MEMORY.md                        # Optional curated human memory notes
+-- memory/                          # Optional human-authored notes
    +-- logs/
    +-- projects/
    +-- decisions/
    +-- lessons/
```

## Recommended Human Notes Structure

- `memory/logs/`: chronological notes and events
- `memory/projects/`: project context and status snapshots
- `memory/decisions/`: decision records and rationale
- `memory/lessons/`: failures, retrospectives, reusable lessons

## Operational Notes

- Personal runtime memory data under `data/` should never be committed.
- Use the API for inspection:
  - `GET /memories`
  - `GET /memory/{memory_id}`
  - `GET /insights/pending`

## Deprecated

Legacy Vector Memory sync commands are removed in v2.


path: package-lock.json
content: {
  "name": "smart-memory",
  "lockfileVersion": 3,
  "requires": true,
  "packages": {}
}


path: prompt_engine/composer.py
content: """Prompt Composer orchestration.

Pipeline:
User message
  -> interaction state detection
  -> entity extraction
  -> memory retrieval
  -> memory re-ranking
  -> token budgeting
  -> prompt rendering
"""

from __future__ import annotations

from dataclasses import dataclass
from datetime import datetime, timezone
from typing import Iterable

from .entity_extractor import extract_entities
from .memory_reranker import rerank_memories
from .memory_retriever import RetrievalBackend, retrieve_with_fallback
from .prompt_renderer import render_prompt
from .schemas import (
    InsightObject,
    InteractionState,
    PromptComposerOutput,
    PromptComposerRequest,
)
from .state_detector import build_temporal_state
from .token_allocator import allocate_tokens


@dataclass(frozen=True)
class PromptComposerConfig:
    insight_confidence_threshold: float = 0.65


class PromptComposer:
    """Composable prompt engine with graceful degradation guarantees."""

    def __init__(
        self,
        retrieval_backend: RetrievalBackend | None = None,
        *,
        config: PromptComposerConfig | None = None,
    ) -> None:
        self.retrieval_backend = retrieval_backend
        self.config = config or PromptComposerConfig()

    def compose(self, request: PromptComposerRequest) -> PromptComposerOutput:
        now = datetime.now(timezone.utc)

        temporal_state = build_temporal_state(
            current_user_message=request.current_user_message,
            last_interaction_timestamp=request.last_interaction_timestamp,
            now=now,
        )

        entities = extract_entities(
            current_user_message=request.current_user_message,
            conversation_history=request.conversation_history,
            active_projects=request.hot_memory.active_projects,
        )

        retrieval = retrieve_with_fallback(
            query=request.current_user_message,
            entities=entities,
            backend=self.retrieval_backend,
            max_candidates=request.max_candidate_memories,
            timeout_ms=request.retrieval_timeout_ms,
        )

        selected_memories = rerank_memories(
            candidates=retrieval.candidates,
            query=request.current_user_message,
            entities=entities,
            top_k=request.max_selected_memories,
            now=now,
        )

        selected_insights = self._select_insights(
            queue=request.hot_memory.insight_queue,
            query=request.current_user_message,
            interaction_state=temporal_state.interaction_state,
            now=now,
        )

        token_allocation = allocate_tokens(
            total_tokens=request.max_prompt_tokens,
            interaction_state=temporal_state.interaction_state,
            include_retrieved_memory=bool(selected_memories),
            include_insights=bool(selected_insights),
        )

        prompt = render_prompt(
            agent_identity=request.agent_identity,
            temporal_state=temporal_state,
            hot_memory=request.hot_memory,
            retrieved_memories=selected_memories,
            selected_insights=selected_insights,
            conversation_history=request.conversation_history,
            current_user_message=request.current_user_message,
            token_allocation=token_allocation,
        )

        degraded: list[str] = []
        if retrieval.degraded:
            degraded.append("memory_retrieval")

        return PromptComposerOutput(
            prompt=prompt,
            interaction_state=temporal_state.interaction_state,
            temporal_state=temporal_state,
            entities=entities,
            selected_memories=selected_memories,
            selected_insights=selected_insights,
            token_allocation=token_allocation,
            degraded_subsystems=degraded,
            metadata={
                "retrieval_elapsed_ms": retrieval.elapsed_ms,
                "retrieval_error": retrieval.error,
                "candidate_count": len(retrieval.candidates),
                "selected_memory_count": len(selected_memories),
                "selected_insight_count": len(selected_insights),
            },
        )

    def _select_insights(
        self,
        *,
        queue: Iterable[InsightObject],
        query: str,
        interaction_state: InteractionState,
        now: datetime,
    ) -> list[InsightObject]:
        if interaction_state != InteractionState.RETURNING:
            return []

        query_terms = {term.lower() for term in query.split() if len(term) >= 3}
        selected: list[InsightObject] = []

        for insight in queue:
            if insight.confidence < self.config.insight_confidence_threshold:
                continue
            if insight.expires_at is not None and insight.expires_at < now:
                continue

            insight_terms = {
                term.lower() for term in insight.content.split() if len(term) >= 3
            }
            if query_terms and not (query_terms & insight_terms):
                continue

            selected.append(insight)

        return selected


path: prompt_engine/entity_extractor.py
content: """Entity extraction helpers for memory retrieval queries.

This is intentionally lightweight for Phase 0/1 and can be swapped for an LLM/NER extractor later.
"""

from __future__ import annotations

import re
from typing import Iterable


TOKEN_PATTERN = re.compile(r"[A-Za-z][A-Za-z0-9_\-]{2,}")
STOPWORDS = {
    "the",
    "and",
    "for",
    "that",
    "with",
    "this",
    "from",
    "have",
    "what",
    "when",
    "where",
    "which",
    "about",
    "would",
    "should",
    "could",
    "please",
    "after",
    "before",
    "project",
    "memory",
}


def _normalize_entity(token: str) -> str:
    return token.strip().lower().replace("-", "_")


def extract_entities(
    current_user_message: str,
    conversation_history: str = "",
    active_projects: Iterable[str] | None = None,
    max_entities: int = 20,
) -> list[str]:
    """Extract canonical entity ids from user and immediate context."""

    candidates: list[str] = []

    for token in TOKEN_PATTERN.findall(current_user_message):
        entity = _normalize_entity(token)
        if entity not in STOPWORDS:
            candidates.append(entity)

    for token in TOKEN_PATTERN.findall(conversation_history):
        entity = _normalize_entity(token)
        if entity not in STOPWORDS:
            candidates.append(entity)

    if active_projects:
        for project in active_projects:
            for token in TOKEN_PATTERN.findall(project):
                entity = _normalize_entity(token)
                if entity not in STOPWORDS:
                    candidates.append(entity)

    deduped: list[str] = []
    seen: set[str] = set()
    for candidate in candidates:
        if candidate not in seen:
            deduped.append(candidate)
            seen.add(candidate)
        if len(deduped) >= max_entities:
            break

    return deduped


path: prompt_engine/memory_reranker.py
content: """Memory re-ranking for candidate long-term memories."""

from __future__ import annotations

from dataclasses import dataclass
from datetime import datetime, timedelta, timezone

from .schemas import LongTermMemory


@dataclass
class RankedMemory:
    memory: LongTermMemory
    score: float


def _tokenize(text: str) -> set[str]:
    return {token.lower() for token in text.split() if len(token) >= 3}


def _score_memory(
    memory: LongTermMemory,
    query_terms: set[str],
    entities: set[str],
    now: datetime,
) -> float:
    content_tokens = _tokenize(memory.content)
    memory_entities = {entity.lower() for entity in memory.entities}

    if query_terms:
        keyword_overlap = len(query_terms & content_tokens) / len(query_terms)
    else:
        keyword_overlap = 0.0

    if entities:
        entity_overlap = len(entities & memory_entities) / len(entities)
    else:
        entity_overlap = 0.0

    recency_boost = 0.0
    if memory.last_accessed >= now - timedelta(days=7):
        recency_boost = 0.05

    score = (
        (memory.importance * 0.45)
        + (keyword_overlap * 0.35)
        + (entity_overlap * 0.20)
        + recency_boost
    )

    return min(1.0, max(0.0, score))


def rerank_with_scores(
    candidates: list[LongTermMemory],
    query: str,
    entities: list[str],
    *,
    top_k: int = 5,
    minimum_score: float = 0.10,
    now: datetime | None = None,
) -> list[RankedMemory]:
    """Score and sort candidate memories."""

    now = now or datetime.now(timezone.utc)
    query_terms = _tokenize(query)
    entity_set = {entity.lower() for entity in entities}

    ranked: list[RankedMemory] = []
    for candidate in candidates:
        score = _score_memory(candidate, query_terms, entity_set, now)
        if score >= minimum_score:
            ranked.append(RankedMemory(memory=candidate, score=score))

    ranked.sort(key=lambda item: item.score, reverse=True)
    return ranked[:top_k]


def rerank_memories(
    candidates: list[LongTermMemory],
    query: str,
    entities: list[str],
    *,
    top_k: int = 5,
    minimum_score: float = 0.10,
    now: datetime | None = None,
) -> list[LongTermMemory]:
    """Return only selected memories after ranking."""

    ranked = rerank_with_scores(
        candidates,
        query,
        entities,
        top_k=top_k,
        minimum_score=minimum_score,
        now=now,
    )
    return [item.memory for item in ranked]


path: prompt_engine/memory_retriever.py
content: """Memory retrieval orchestration with graceful degradation.

This module wraps any retrieval backend with timeout and validation guarantees.
"""

from __future__ import annotations

from concurrent.futures import ThreadPoolExecutor, TimeoutError
from dataclasses import dataclass
from time import perf_counter
from typing import Protocol, Sequence

from pydantic import TypeAdapter

from .schemas import LongTermMemory


LONG_TERM_MEMORY_LIST = TypeAdapter(list[LongTermMemory])


class RetrievalBackend(Protocol):
    def retrieve(
        self,
        query: str,
        *,
        entities: Sequence[str],
        max_candidates: int,
    ) -> list[LongTermMemory] | list[dict]:
        """Return up to max_candidates candidate memories."""


@dataclass
class RetrievalResult:
    candidates: list[LongTermMemory]
    degraded: bool
    error: str | None
    elapsed_ms: int


def retrieve_with_fallback(
    query: str,
    entities: Sequence[str],
    backend: RetrievalBackend | None,
    *,
    max_candidates: int = 30,
    timeout_ms: int = 500,
) -> RetrievalResult:
    """Retrieve memory candidates, degrading safely on timeout/failure."""

    started = perf_counter()

    if backend is None:
        elapsed = int((perf_counter() - started) * 1000)
        return RetrievalResult(
            candidates=[],
            degraded=True,
            error="retrieval backend unavailable",
            elapsed_ms=elapsed,
        )

    try:
        with ThreadPoolExecutor(max_workers=1) as executor:
            future = executor.submit(
                backend.retrieve,
                query,
                entities=entities,
                max_candidates=max_candidates,
            )
            raw_candidates = future.result(timeout=timeout_ms / 1000)

        validated = LONG_TERM_MEMORY_LIST.validate_python(raw_candidates)
        elapsed = int((perf_counter() - started) * 1000)

        return RetrievalResult(
            candidates=validated[:max_candidates],
            degraded=False,
            error=None,
            elapsed_ms=elapsed,
        )
    except TimeoutError:
        elapsed = int((perf_counter() - started) * 1000)
        return RetrievalResult(
            candidates=[],
            degraded=True,
            error=f"retrieval timeout after {timeout_ms}ms",
            elapsed_ms=elapsed,
        )
    except Exception as error:  # noqa: BLE001
        elapsed = int((perf_counter() - started) * 1000)
        return RetrievalResult(
            candidates=[],
            degraded=True,
            error=str(error),
            elapsed_ms=elapsed,
        )


path: prompt_engine/prompt_renderer.py
content: """Prompt rendering with strict prompt-level token enforcement.

Token counting is approximate (word-based) to stay dependency-free.
"""

from __future__ import annotations

import math

from .schemas import (
    HotMemory,
    InsightObject,
    InteractionState,
    LongTermMemory,
    TemporalState,
    TokenAllocation,
)


def estimate_tokens(text: str) -> int:
    words = len(text.split())
    if words == 0:
        return 0
    return max(1, math.ceil(words * 1.33))


def _max_words_for_budget(token_budget: int) -> int:
    return max(1, int(token_budget / 1.33))


def _truncate_to_budget(text: str, token_budget: int, *, keep_tail: bool = False) -> str:
    if token_budget <= 0:
        return ""

    words = text.split()
    if not words:
        return ""

    max_words = _max_words_for_budget(token_budget)
    if len(words) <= max_words:
        return text.strip()

    if keep_tail:
        clipped = " ".join(words[-max_words:])
        return f"... {clipped}" if clipped else ""

    clipped = " ".join(words[:max_words])
    return f"{clipped} ..." if clipped else ""


def _trim_oldest_words(text: str, words_to_drop: int = 24) -> str:
    words = text.split()
    if not words:
        return ""
    if len(words) <= words_to_drop:
        return ""

    remaining = " ".join(words[words_to_drop:])
    return f"... {remaining}" if remaining else ""


def _trim_newest_words(text: str, words_to_drop: int = 24) -> str:
    words = text.split()
    if not words:
        return ""
    if len(words) <= words_to_drop:
        return ""

    return " ".join(words[:-words_to_drop])


def _select_items_for_budget(items: list[str], token_budget: int) -> list[str]:
    if token_budget <= 0:
        return []

    selected: list[str] = []
    used = 0
    for raw_item in items:
        item = raw_item.strip()
        if not item:
            continue

        remaining = token_budget - used
        if remaining <= 0:
            break

        bounded = _truncate_to_budget(item, min(remaining, 48))
        if not bounded:
            continue

        cost = estimate_tokens(f"- {bounded}")
        if used + cost > token_budget:
            break

        selected.append(bounded)
        used += cost

    return selected


def _memory_lines_for_budget(memories: list[LongTermMemory], token_budget: int) -> list[str]:
    if token_budget <= 0:
        return []

    lines: list[str] = []
    used = 0
    for memory in memories:
        remaining = token_budget - used
        if remaining <= 0:
            break

        content = _truncate_to_budget(memory.content.strip(), min(remaining, 96))
        if not content:
            continue

        line = f"[{memory.type.value}] {content}"
        cost = estimate_tokens(line)
        if used + cost > token_budget:
            break

        lines.append(line)
        used += cost

    return lines


def _insight_lines_for_budget(
    interaction_state: InteractionState,
    insights: list[InsightObject],
    token_budget: int,
) -> list[str]:
    if interaction_state != InteractionState.RETURNING:
        return []
    if token_budget <= 0:
        return []

    lines: list[str] = []
    used = 0
    for insight in insights:
        remaining = token_budget - used
        if remaining <= 0:
            break

        content = _truncate_to_budget(insight.content.strip(), min(remaining, 80))
        if not content:
            continue

        line = f"- {content} (confidence: {insight.confidence:.2f})"
        cost = estimate_tokens(line)
        if used + cost > token_budget:
            break

        lines.append(line)
        used += cost

    return lines


def _render_bullets(items: list[str]) -> str:
    if not items:
        return "- none"
    return "\n".join(f"- {item}" for item in items)


def _temporal_text(temporal_state: TemporalState, mode: int) -> str:
    if mode <= 0:
        return ""

    if mode == 1:
        return f"Interaction State: {temporal_state.interaction_state.value}"

    if mode == 2:
        return (
            f"Time Since Last Interaction: {temporal_state.time_since_last_interaction}\n"
            f"Interaction State: {temporal_state.interaction_state.value}"
        )

    return (
        f"Current Time: {temporal_state.current_timestamp.isoformat()}\n"
        f"Time Since Last Interaction: {temporal_state.time_since_last_interaction}\n"
        f"Interaction State: {temporal_state.interaction_state.value}"
    )


def _build_prompt(
    *,
    identity_text: str,
    temporal_text: str,
    active_projects: list[str],
    working_questions: list[str],
    top_of_mind: list[str],
    insight_lines: list[str],
    retrieved_lines: list[str],
    conversation_text: str,
    current_user_message: str,
) -> str:
    working_context = (
        "Active Projects:\n"
        f"{_render_bullets(active_projects)}\n\n"
        "Working Questions:\n"
        f"{_render_bullets(working_questions)}\n\n"
        "Current Focus / Goals:\n"
        f"{_render_bullets(top_of_mind)}"
    )

    parts: list[str] = []
    parts.append("<system>")
    parts.append("")
    parts.append("[AGENT IDENTITY]")
    parts.append(identity_text or "N/A")
    parts.append("")
    parts.append("[TEMPORAL STATE]")
    parts.append(temporal_text or "N/A")
    parts.append("")
    parts.append("[WORKING CONTEXT]")
    parts.append(working_context)
    parts.append("")
    parts.append("</system>")
    parts.append("")

    if insight_lines:
        parts.append("[BACKGROUND INSIGHTS]")
        parts.append("The following insights were generated during background reflection cycles.")
        parts.append("")
        parts.append("\n".join(insight_lines))
        parts.append("")

    if retrieved_lines:
        parts.append("[RELEVANT LONG-TERM MEMORY]")
        parts.append("")
        parts.append("\n".join(retrieved_lines))
        parts.append("")

    parts.append("<user>")
    parts.append("")
    parts.append("[RECENT CONVERSATION]")
    parts.append(conversation_text or "N/A")
    parts.append("")
    parts.append(current_user_message.strip() or "N/A")
    parts.append("")
    parts.append("</user>")
    parts.append("")
    parts.append("<assistant>")

    return "\n".join(parts)


def render_prompt(
    *,
    agent_identity: str,
    temporal_state: TemporalState,
    hot_memory: HotMemory,
    retrieved_memories: list[LongTermMemory],
    selected_insights: list[InsightObject],
    conversation_history: str,
    current_user_message: str,
    token_allocation: TokenAllocation,
) -> str:
    """Render the final prompt and strictly enforce max_prompt_tokens.

    Eviction order when over budget:
    1) oldest conversation history
    2) lower-scored retrieved memories (tail)
    3) insight queue items (tail)
    4) working memory items
    5) temporal state
    6) agent identity (never evicted)
    """

    identity_text = _truncate_to_budget(
        agent_identity.strip(),
        token_allocation.system_identity,
    )

    temporal_mode = 3

    working_budget = token_allocation.working_memory
    active_projects = _select_items_for_budget(
        hot_memory.active_projects,
        max(0, int(working_budget * 0.35)),
    )
    working_questions = _select_items_for_budget(
        hot_memory.working_questions,
        max(0, int(working_budget * 0.30)),
    )
    top_of_mind = _select_items_for_budget(
        hot_memory.top_of_mind,
        max(0, working_budget - int(working_budget * 0.35) - int(working_budget * 0.30)),
    )

    insight_lines = _insight_lines_for_budget(
        interaction_state=temporal_state.interaction_state,
        insights=selected_insights,
        token_budget=token_allocation.insight_queue,
    )

    retrieved_lines = _memory_lines_for_budget(
        memories=retrieved_memories,
        token_budget=token_allocation.retrieved_memory,
    )

    conversation_text = _truncate_to_budget(
        conversation_history.strip(),
        token_allocation.conversation_history,
        keep_tail=True,
    )
    current_message = current_user_message.strip()

    max_prompt_tokens = token_allocation.total_tokens

    for _ in range(1024):
        temporal_text = _truncate_to_budget(
            _temporal_text(temporal_state, temporal_mode),
            token_allocation.temporal_state,
        )

        prompt = _build_prompt(
            identity_text=identity_text,
            temporal_text=temporal_text,
            active_projects=active_projects,
            working_questions=working_questions,
            top_of_mind=top_of_mind,
            insight_lines=insight_lines,
            retrieved_lines=retrieved_lines,
            conversation_text=conversation_text,
            current_user_message=current_message,
        )

        if estimate_tokens(prompt) <= max_prompt_tokens:
            return prompt

        if conversation_text:
            trimmed = _trim_oldest_words(conversation_text, 24)
            if trimmed == conversation_text:
                conversation_text = ""
            else:
                conversation_text = trimmed
            continue

        if retrieved_lines:
            retrieved_lines.pop()
            continue

        if insight_lines:
            insight_lines.pop()
            continue

        if top_of_mind:
            top_of_mind.pop()
            continue
        if working_questions:
            working_questions.pop()
            continue
        if active_projects:
            active_projects.pop()
            continue

        if temporal_mode > 0:
            temporal_mode -= 1
            continue

        # Safety fallback: preserve identity while still hard-bounding prompt length.
        if current_message:
            current_message = _trim_newest_words(current_message, 24)
            continue

        break

    final_temporal_text = _truncate_to_budget(
        _temporal_text(temporal_state, temporal_mode),
        token_allocation.temporal_state,
    )
    final_prompt = _build_prompt(
        identity_text=identity_text,
        temporal_text=final_temporal_text,
        active_projects=active_projects,
        working_questions=working_questions,
        top_of_mind=top_of_mind,
        insight_lines=insight_lines,
        retrieved_lines=retrieved_lines,
        conversation_text=conversation_text,
        current_user_message=current_message,
    )

    if estimate_tokens(final_prompt) > max_prompt_tokens:
        return _truncate_to_budget(final_prompt, max_prompt_tokens)

    return final_prompt


path: prompt_engine/README.md
content: # Prompt Engine

This package contains the prompt composition layer used by Smart Memory v2.

## Included Modules

- `schemas.py`: Canonical Pydantic contracts
- `state_detector.py`: Interaction-state and temporal-state generation
- `entity_extractor.py`: Lightweight entity extraction
- `memory_retriever.py`: Retrieval wrapper with timeout and graceful fallback
- `memory_reranker.py`: Candidate scoring and top-k selection
- `token_allocator.py`: State-aware token budget allocation
- `prompt_renderer.py`: Prompt assembly with per-section clipping
- `composer.py`: Orchestrates the full pipeline

## Scope

- Data contracts and orchestration only.
- No storage engine, vector index, or background scheduler implementation in this package.


path: prompt_engine/schemas.py
content: """Canonical data schemas for the cognitive memory architecture.

Phase 1 scope:
- Strict Pydantic models for long-term memory, working memory, insights, and composer contracts.
- No storage, retrieval backend, or vector/index implementation.
"""

from __future__ import annotations

from datetime import datetime, timedelta, timezone
from enum import Enum
from typing import Annotated, Any, Literal, Union
from uuid import UUID, uuid4

from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator


def _utc_now() -> datetime:
    return datetime.now(timezone.utc)


def _ensure_timezone(value: datetime) -> datetime:
    if value.tzinfo is None:
        return value.replace(tzinfo=timezone.utc)
    return value.astimezone(timezone.utc)


UnitInterval = Annotated[float, Field(ge=0.0, le=1.0)]
SignedUnitInterval = Annotated[float, Field(ge=-1.0, le=1.0)]
NonNegativeInt = Annotated[int, Field(ge=0)]


class StrictModel(BaseModel):
    model_config = ConfigDict(extra="forbid", validate_assignment=True)


class MemoryType(str, Enum):
    EPISODIC = "episodic"
    SEMANTIC = "semantic"
    BELIEF = "belief"
    GOAL = "goal"


class MemorySource(str, Enum):
    CONVERSATION = "conversation"
    REFLECTION = "reflection"
    SYSTEM = "system"
    IMPORTED = "imported"


class GoalStatus(str, Enum):
    ACTIVE = "active"
    PAUSED = "paused"
    COMPLETED = "completed"


class InteractionState(str, Enum):
    ENGAGED = "engaged"
    RETURNING = "returning"
    IDLE = "idle"


class AgentStatus(str, Enum):
    ENGAGED = "engaged"
    IDLE = "idle"
    RETURNING = "returning"
    SLEEPING = "sleeping"


class RelationTriple(StrictModel):
    subject: str = Field(min_length=1)
    predicate: str = Field(min_length=1)
    object: str = Field(min_length=1)


class BaseMemory(StrictModel):
    id: UUID = Field(default_factory=uuid4)
    type: MemoryType
    content: str = Field(min_length=1)
    importance: UnitInterval
    created_at: datetime = Field(default_factory=_utc_now)
    last_accessed: datetime = Field(default_factory=_utc_now)
    access_count: NonNegativeInt = 0
    schema_version: str = Field(default="2.0")
    entities: list[str] = Field(default_factory=list)
    relations: list[RelationTriple] = Field(default_factory=list)
    emotional_valence: SignedUnitInterval = 0.0
    emotional_intensity: UnitInterval = 0.0
    source: MemorySource

    @field_validator("created_at", "last_accessed", mode="before")
    @classmethod
    def _normalize_datetime(cls, value: Any) -> datetime:
        if isinstance(value, str):
            value = datetime.fromisoformat(value)
        if not isinstance(value, datetime):
            raise TypeError("Expected datetime or ISO datetime string")
        return _ensure_timezone(value)

    @field_validator("schema_version")
    @classmethod
    def _validate_schema_version(cls, value: str) -> str:
        if not value.strip():
            raise ValueError("schema_version must not be empty")
        return value

    @field_validator("entities")
    @classmethod
    def _validate_entities(cls, values: list[str]) -> list[str]:
        deduped: list[str] = []
        seen: set[str] = set()

        for value in values:
            if not value:
                continue
            entity_id = value.strip().lower().replace(" ", "_")
            if entity_id and entity_id not in seen:
                deduped.append(entity_id)
                seen.add(entity_id)

        return deduped


class EpisodicMemory(BaseMemory):
    type: Literal[MemoryType.EPISODIC] = MemoryType.EPISODIC
    participants: list[str] = Field(default_factory=list)


class SemanticMemory(BaseMemory):
    type: Literal[MemoryType.SEMANTIC] = MemoryType.SEMANTIC
    confidence: UnitInterval


class BeliefMemory(BaseMemory):
    type: Literal[MemoryType.BELIEF] = MemoryType.BELIEF
    confidence: UnitInterval
    reinforced_count: Annotated[int, Field(ge=1)] = 1


class GoalMemory(BaseMemory):
    type: Literal[MemoryType.GOAL] = MemoryType.GOAL
    status: GoalStatus
    priority: UnitInterval


LongTermMemory = Annotated[
    Union[EpisodicMemory, SemanticMemory, BeliefMemory, GoalMemory],
    Field(discriminator="type"),
]


class InsightObject(StrictModel):
    id: UUID = Field(default_factory=uuid4)
    content: str = Field(min_length=1)
    confidence: UnitInterval
    source_memory_ids: list[UUID] = Field(default_factory=list)
    generated_at: datetime = Field(default_factory=_utc_now)
    expires_at: datetime | None = None

    @field_validator("generated_at", "expires_at", mode="before")
    @classmethod
    def _normalize_datetime(cls, value: Any) -> datetime | None:
        if value is None:
            return None
        if isinstance(value, str):
            value = datetime.fromisoformat(value)
        if not isinstance(value, datetime):
            raise TypeError("Expected datetime or ISO datetime string")
        return _ensure_timezone(value)

    @model_validator(mode="after")
    def _default_expiry(self) -> "InsightObject":
        if self.expires_at is None:
            self.expires_at = self.generated_at + timedelta(hours=24)
        return self


class AgentState(StrictModel):
    status: AgentStatus
    last_interaction_timestamp: datetime
    last_background_task: str = Field(min_length=1)

    @field_validator("last_interaction_timestamp", mode="before")
    @classmethod
    def _normalize_datetime(cls, value: Any) -> datetime:
        if isinstance(value, str):
            value = datetime.fromisoformat(value)
        if not isinstance(value, datetime):
            raise TypeError("Expected datetime or ISO datetime string")
        return _ensure_timezone(value)


class HotMemory(StrictModel):
    agent_state: AgentState
    active_projects: list[str] = Field(default_factory=list)
    working_questions: list[str] = Field(default_factory=list)
    top_of_mind: list[str] = Field(default_factory=list)
    insight_queue: list[InsightObject] = Field(default_factory=list)


class TemporalState(StrictModel):
    current_timestamp: datetime
    time_since_last_interaction: str = Field(min_length=1)
    interaction_state: InteractionState

    @field_validator("current_timestamp", mode="before")
    @classmethod
    def _normalize_datetime(cls, value: Any) -> datetime:
        if isinstance(value, str):
            value = datetime.fromisoformat(value)
        if not isinstance(value, datetime):
            raise TypeError("Expected datetime or ISO datetime string")
        return _ensure_timezone(value)


class TokenAllocation(StrictModel):
    total_tokens: Annotated[int, Field(ge=1)]
    system_identity: NonNegativeInt
    temporal_state: NonNegativeInt
    working_memory: NonNegativeInt
    retrieved_memory: NonNegativeInt
    insight_queue: NonNegativeInt
    conversation_history: NonNegativeInt

    @model_validator(mode="after")
    def _validate_sum(self) -> "TokenAllocation":
        total = (
            self.system_identity
            + self.temporal_state
            + self.working_memory
            + self.retrieved_memory
            + self.insight_queue
            + self.conversation_history
        )
        if total != self.total_tokens:
            raise ValueError(
                f"Token allocation mismatch: expected {self.total_tokens}, got {total}"
            )
        return self



def _default_hot_memory() -> HotMemory:
    now = _utc_now()
    return HotMemory(
        agent_state=AgentState(
            status=AgentStatus.IDLE,
            last_interaction_timestamp=now,
            last_background_task="none",
        ),
        active_projects=[],
        working_questions=[],
        top_of_mind=[],
        insight_queue=[],
    )

class PromptComposerRequest(StrictModel):
    agent_identity: str = Field(min_length=1)
    current_user_message: str = Field(min_length=1)
    conversation_history: str = ""
    last_interaction_timestamp: datetime | None = None
    hot_memory: HotMemory = Field(default_factory=_default_hot_memory)
    max_prompt_tokens: Annotated[int, Field(ge=256)] = 8192
    retrieval_timeout_ms: Annotated[int, Field(ge=50, le=10000)] = 500
    max_candidate_memories: Annotated[int, Field(ge=1, le=100)] = 30
    max_selected_memories: Annotated[int, Field(ge=1, le=20)] = 5

    @field_validator("last_interaction_timestamp", mode="before")
    @classmethod
    def _normalize_datetime(cls, value: Any) -> datetime | None:
        if value is None:
            return None
        if isinstance(value, str):
            value = datetime.fromisoformat(value)
        if not isinstance(value, datetime):
            raise TypeError("Expected datetime or ISO datetime string")
        return _ensure_timezone(value)


class PromptComposerOutput(StrictModel):
    prompt: str = Field(min_length=1)
    interaction_state: InteractionState
    temporal_state: TemporalState
    entities: list[str] = Field(default_factory=list)
    selected_memories: list[LongTermMemory] = Field(default_factory=list)
    selected_insights: list[InsightObject] = Field(default_factory=list)
    token_allocation: TokenAllocation
    degraded_subsystems: list[str] = Field(default_factory=list)
    metadata: dict[str, Any] = Field(default_factory=dict)




path: prompt_engine/state_detector.py
content: """Interaction state detection and temporal-state helpers."""

from __future__ import annotations

from dataclasses import dataclass
from datetime import datetime, timezone

from .schemas import InteractionState, TemporalState


@dataclass(frozen=True)
class StateDetectorConfig:
    engaged_window_minutes: int = 15
    idle_max_words: int = 4


FUNCTIONAL_PREFIXES = (
    "--",
    "/",
    "status",
    "sync",
    "search",
    "get",
    "mode",
    "help",
)


def detect_interaction_state(
    current_user_message: str,
    last_interaction_timestamp: datetime | None,
    now: datetime | None = None,
    config: StateDetectorConfig = StateDetectorConfig(),
) -> InteractionState:
    """Classify interaction state using time delta and command-like heuristics."""

    now = now or datetime.now(timezone.utc)
    message = current_user_message.strip().lower()

    if not message:
        return InteractionState.IDLE

    words = message.split()
    if len(words) <= config.idle_max_words and (
        message.startswith(FUNCTIONAL_PREFIXES)
        or words[0] in FUNCTIONAL_PREFIXES
    ):
        return InteractionState.IDLE

    if last_interaction_timestamp is None:
        return InteractionState.RETURNING

    delta_seconds = max(0.0, (now - last_interaction_timestamp).total_seconds())
    if delta_seconds <= config.engaged_window_minutes * 60:
        return InteractionState.ENGAGED

    return InteractionState.RETURNING


def _humanize_delta(seconds: int) -> str:
    if seconds <= 0:
        return "0m"

    minutes = seconds // 60
    hours, rem_minutes = divmod(minutes, 60)
    days, rem_hours = divmod(hours, 24)

    if days > 0:
        return f"{days}d {rem_hours}h"
    if hours > 0:
        return f"{hours}h {rem_minutes}m"
    return f"{minutes}m"


def build_temporal_state(
    current_user_message: str,
    last_interaction_timestamp: datetime | None,
    now: datetime | None = None,
    config: StateDetectorConfig = StateDetectorConfig(),
) -> TemporalState:
    """Build canonical temporal state contract for prompt composition."""

    now = now or datetime.now(timezone.utc)

    if last_interaction_timestamp is None:
        delta_seconds = 0
    else:
        delta_seconds = int(max(0.0, (now - last_interaction_timestamp).total_seconds()))

    interaction_state = detect_interaction_state(
        current_user_message=current_user_message,
        last_interaction_timestamp=last_interaction_timestamp,
        now=now,
        config=config,
    )

    if last_interaction_timestamp is None:
        time_gap = "first interaction"
    else:
        time_gap = _humanize_delta(delta_seconds)

    return TemporalState(
        current_timestamp=now,
        time_since_last_interaction=time_gap,
        interaction_state=interaction_state,
    )


path: prompt_engine/token_allocator.py
content: """Token budget allocation for prompt composition."""

from __future__ import annotations

from .schemas import InteractionState, TokenAllocation


def _normalize_percentages(percentages: dict[str, float]) -> dict[str, float]:
    total = sum(percentages.values())
    if total <= 0:
        raise ValueError("Token percentage total must be positive")
    return {key: (value / total) * 100.0 for key, value in percentages.items()}


def _allocate_exact_tokens(total_tokens: int, percentages: dict[str, float]) -> dict[str, int]:
    raw = {key: (total_tokens * percent / 100.0) for key, percent in percentages.items()}
    floor_tokens = {key: int(value) for key, value in raw.items()}

    assigned = sum(floor_tokens.values())
    remaining = total_tokens - assigned

    fractional = sorted(
        ((key, raw[key] - floor_tokens[key]) for key in raw),
        key=lambda item: item[1],
        reverse=True,
    )

    for index in range(max(0, remaining)):
        key = fractional[index % len(fractional)][0]
        floor_tokens[key] += 1

    return floor_tokens


def _apply_minimums(tokens: dict[str, int], minimums: dict[str, int]) -> dict[str, int]:
    """Enforce per-section minimums while preserving total allocation."""

    adjusted = dict(tokens)
    for section, minimum in minimums.items():
        minimum = max(0, int(minimum))
        current = adjusted.get(section, 0)
        if current >= minimum:
            continue

        deficit = minimum - current
        adjusted[section] = minimum

        donors = sorted(
            (
                name
                for name in adjusted
                if name != section and adjusted[name] > minimums.get(name, 0)
            ),
            key=lambda name: adjusted[name],
            reverse=True,
        )

        for donor in donors:
            available = adjusted[donor] - minimums.get(donor, 0)
            if available <= 0:
                continue

            take = min(available, deficit)
            adjusted[donor] -= take
            deficit -= take

            if deficit == 0:
                break

        if deficit > 0:
            raise ValueError("Unable to satisfy token minimum constraints")

    return adjusted


def allocate_tokens(
    total_tokens: int,
    interaction_state: InteractionState,
    *,
    include_retrieved_memory: bool,
    include_insights: bool,
) -> TokenAllocation:
    """Allocate section budgets using interaction-state-aware percentages."""

    if interaction_state == InteractionState.ENGAGED:
        percentages = {
            "system_identity": 10,
            "temporal_state": 5,
            "working_memory": 10,
            "retrieved_memory": 15,
            "insight_queue": 0,
            "conversation_history": 60,
        }
    elif interaction_state == InteractionState.RETURNING:
        percentages = {
            "system_identity": 10,
            "temporal_state": 5,
            "working_memory": 10,
            "retrieved_memory": 30,
            "insight_queue": 5,
            "conversation_history": 40,
        }
    else:
        percentages = {
            "system_identity": 10,
            "temporal_state": 5,
            "working_memory": 5,
            "retrieved_memory": 10,
            "insight_queue": 0,
            "conversation_history": 70,
        }

    if not include_retrieved_memory:
        percentages["conversation_history"] += percentages["retrieved_memory"]
        percentages["retrieved_memory"] = 0

    if not include_insights:
        percentages["conversation_history"] += percentages["insight_queue"]
        percentages["insight_queue"] = 0

    percentages = _normalize_percentages(percentages)
    tokens = _allocate_exact_tokens(total_tokens, percentages)
    tokens = _apply_minimums(
        tokens,
        {
            "system_identity": 1,
            "conversation_history": 1,
        },
    )

    return TokenAllocation(total_tokens=total_tokens, **tokens)


path: prompt_engine/__init__.py
content: """Prompt engine package for the cognitive memory architecture."""

from .composer import PromptComposer, PromptComposerConfig
from .schemas import (
    AgentState,
    BeliefMemory,
    EpisodicMemory,
    GoalMemory,
    HotMemory,
    InsightObject,
    InteractionState,
    LongTermMemory,
    MemorySource,
    MemoryType,
    PromptComposerOutput,
    PromptComposerRequest,
    SemanticMemory,
    TemporalState,
    TokenAllocation,
)

__all__ = [
    "AgentState",
    "BeliefMemory",
    "EpisodicMemory",
    "GoalMemory",
    "HotMemory",
    "InsightObject",
    "InteractionState",
    "LongTermMemory",
    "MemorySource",
    "MemoryType",
    "PromptComposer",
    "PromptComposerConfig",
    "PromptComposerOutput",
    "PromptComposerRequest",
    "SemanticMemory",
    "TemporalState",
    "TokenAllocation",
]


path: README.md
content: # Smart Memory v2 - Cognitive Architecture for OpenClaw

<p>
  <img alt="Node.js" src="https://img.shields.io/badge/Node.js-18%2B-339933?logo=node.js&logoColor=white">
  <img alt="Python" src="https://img.shields.io/badge/Python-3.11%2B-3776AB?logo=python&logoColor=white">
  <img alt="FastAPI" src="https://img.shields.io/badge/FastAPI-Persistent%20Local%20API-009688?logo=fastapi&logoColor=white">
  <img alt="Embeddings" src="https://img.shields.io/badge/Embeddings-nomic--embed--text--v1.5-blue">
  <img alt="PyTorch" src="https://img.shields.io/badge/PyTorch-CPU--Only-critical?logo=pytorch">
  <img alt="License" src="https://img.shields.io/badge/License-MIT-black">
</p>

> **Not a basic RAG cache.**  
> Smart Memory v2 is a persistent, local cognitive engine for OpenClaw with schema-versioned long-term memory, hot working memory, background cognition, strict token-bounded prompt composition, and a continuously running FastAPI process.

---

## Quick Value (60 seconds)

| You Need | Smart Memory v2 Gives You |
|---|---|
| Continuity across sessions | Typed long-term memory (`episodic`, `semantic`, `belief`, `goal`) |
| Fast local recall | Nomic embeddings + vector search + reranking |
| Stable context quality | Strict prompt token enforcement with deterministic eviction priority |
| Better long-run memory quality | Semantic dedup, reinforcement, consolidation, decay, and conflict resolution |
| Lightweight installs | CPU-only PyTorch standard (no CUDA wheel bloat) |
| Operational visibility | `/health`, `/memories`, `/memory/{id}`, `/insights/pending` |

---

## Why This Exists

Most memory plugins are retrieval wrappers. They do not behave like cognition.  
Smart Memory v2 is designed as a **cognitive pipeline**:

- `Ingestion`: Decide what is memory-worthy.
- `Retrieval`: Find relevant memories with entity and time bias.
- `Working Memory`: Keep a small, high-signal mind state.
- `Background Cognition`: Reflect, consolidate, decay, and resolve conflicts.
- `Prompt Composition`: Assemble bounded, coherent context for the model.

---

## Architecture At A Glance

```mermaid
flowchart LR
  OC["OpenClaw Runtime"] --> AD["Node Adapter smart-memory/index.js"]
  AD --> API["FastAPI Server server.py"]
  API --> SYS["CognitiveMemorySystem"]
  SYS --> ING["Ingestion Pipeline"]
  SYS --> RET["Retrieval Pipeline"]
  SYS --> HOT["Hot Memory Manager"]
  SYS --> COG["Background Cognition"]
  SYS --> PRM["Prompt Engine"]
  SYS --> STO["Storage and Embeddings"]
```

### Cognitive Request Flow

```mermaid
flowchart TD
  UM["User Message"] --> ING["Ingest Interaction"]
  UM --> RET["Retrieve Relevant Memory"]
  RET --> COM["Compose Prompt Context"]
  BG["Background Trigger"] --> RUN["Run Cognition Cycle"]
  RUN --> UPD["Update Insights and Memory State"]
  UPD --> COM
```

---

## System Overview

### Runtime
- Persistent local FastAPI process keeps the embedder and stores warm.
- Node adapter manages process lifecycle and exposes a simple JS interface.
- OpenClaw v2.5 skill layer adds active memory tools and lifecycle hooks.

### Memory Quality Controls
- Strict prompt token enforcement with deterministic eviction order.
- Semantic dedup in ingestion to reinforce instead of duplicating.
- Retrieval access tracking updates `last_accessed` and `access_count`.
- Belief conflict resolution merges contradictory beliefs by shared entities and opposing stance.

### Reliability and Observability
- CPU-only PyTorch is mandatory for lightweight, consistent installs.
- Health and observability endpoints: `/health`, `/memories`, `/memory/{id}`, `/insights/pending`.
- Retry queue + heartbeat support in the OpenClaw skill for resilient commit behavior.

### OpenClaw Native Skill (v2.5)
- New modular package: `skills/smart-memory-v25/`.
- Active tools: `memory_search`, `memory_commit`, `memory_insights`.
- Session arc capture at checkpoints and session end.
- Passive `[ACTIVE CONTEXT]` prompt injection for grounded responses.

**Important:** Disable OpenClaw's built-in memory tools to prevent shadowing:
```bash
openclaw config set tools.deny '["memory_search", "memory_get"]'
openclaw gateway restart
```

---

## Memory Layers

| Layer | Purpose | Example Content |
|---|---|---|
| Agent Identity | Stable behavior anchor | role, mission, style |
| Temporal State | Time continuity | current time, last interaction delta, state |
| Hot Memory | Current cognitive focus | active projects, top-of-mind, working questions |
| Long-Term Memory | Durable history | episodic/semantic/belief/goal objects |
| Insight Queue | Background reflections | confidence-scored insight objects |
| Conversation Context | Immediate grounding | recent turns + current user input |

---

## Repository Layout

```text
.
+- server.py
+- cognitive_memory_system.py
+- prompt_engine/
+- ingestion/
+- retrieval/
+- hot_memory/
+- cognition/
+- storage/
+- embeddings/
+- entities/
+- skills/
   +- smart-memory-v25/
      +- index.js
      +- openclaw-hooks.js
      +- prompt-injection.js
      +- retry-queue.js
+- smart-memory/
   +- index.js
   +- postinstall.js
```

---

## Installation

### Option A: ClawHub

```bash
npx clawhub install smart-memory
```

### Option B: From GitHub

```bash
git clone https://github.com/BluePointDigital/smart-memory.git
cd smart-memory/smart-memory
npm install
```

### What `npm install` Does

`postinstall.js` automatically:
1. Creates `.venv` at repository root.
2. Upgrades `pip`.
3. Installs CPU-only PyTorch wheels (mandatory policy).
4. Installs `requirements-cognitive.txt` (including FastAPI, sentence-transformers, qdrant-client, einops).

Works on both Windows and Unix path conventions.

Do not install generic GPU/CUDA PyTorch wheels for this project; keep CPU-only for consistency and package size.

---

## How To Use

```js
import memory from "smart-memory";

await memory.start();

await memory.ingestMessage({
  user_message: "I started migrating our database today.",
  assistant_message: "Track risks and rollback strategy.",
  timestamp: new Date().toISOString()
});

const retrieval = await memory.retrieveContext({
  user_message: "How is the migration going?",
  conversation_history: "..."
});

const composed = await memory.getPromptContext({
  agent_identity: "You are a persistent cognitive assistant.",
  conversation_history: "...",
  current_user_message: "Continue from where we left off."
  // hot_memory optional
});

await memory.runBackground(true);
await memory.stop();
```

---

## Hot Memory Extension (Optional)

The **Hot Memory Extension** provides persistent working context that survives between sessions and automatically appears in every prompt's `[WORKING CONTEXT]` section.

### What It Adds

| Feature | Description |
|---------|-------------|
| **Active Projects** | Top 5 current projects with auto-detection from conversation |
| **Working Questions** | Open questions being explored (auto-captured from queries) |
| **Top of Mind** | Immediate priorities and notes |
| **Live Insights** | Pending insights from background cognition |
| **Auto-Update** | Detects project mentions and questions automatically |

### Quick Start

```bash
# Initialize with current context
python3 hot_memory_manager.py init

# After conversations, update hot memory
./smem-hook.sh "user message" "assistant response"

# Compose with hot memory auto-included
python3 memory_adapter.py compose -m "What should I work on?"
```

### Files

- `hot_memory_manager.py` - Core persistence and auto-update logic
- `memory_adapter.py` - API wrapper with hot memory integration
- `smem-hook.sh` - Post-conversation shell hook
- `hot_memory_state.json` - Persistent storage (auto-created)

### Duplicate Prevention

The extension includes intelligent duplicate prevention that matches projects by key (e.g., "Tappy.Menu") rather than full description, preventing duplicates when project descriptions evolve.

See `HOT_MEMORY_EXTENSION.md` for full documentation.

---

## API Surface

### JavaScript Adapter Methods

| Method | Purpose |
|---|---|
| `init()` / `start()` | Ensure API process is running and healthy |
| `ingestMessage(interaction)` | Send interaction to ingestion pipeline |
| `retrieveContext({ user_message, conversation_history })` | Retrieve ranked memory context |
| `getPromptContext(request)` | Compose final bounded prompt context |
| `runBackground(scheduled)` | Trigger cognition cycle |
| `stop()` | Stop managed Python server process |

### FastAPI Endpoints

| Endpoint | Method | Description |
|---|---|---|
| `/` | `GET` | Basic service status |
| `/health` | `GET` | Health + embedder loaded metadata |
| `/ingest` | `POST` | Ingest incoming interaction |
| `/retrieve` | `POST` | Retrieve relevant long-term memories |
| `/compose` | `POST` | Compose prompt context payload |
| `/run_background` | `POST` | Execute background cognition cycle |
| `/memories` | `GET` | List memories (`?type=` optional) |
| `/memory/{memory_id}` | `GET` | Fetch one memory by ID |
| `/insights/pending` | `GET` | View pending hot-memory insights |

---

## Security and Privacy

- Memory data is designed for local operation.
- `.gitignore` excludes runtime memory stores, virtualenvs, caches, and `node_modules`.
- Review ignored paths before publishing any fork.

---

## Requirements

- Node.js `>=18`
- Python `>=3.11`
- Local disk for model cache + memory storage

---

## License

MIT


path: requirements-cognitive.txt
content: pydantic>=2,<3
numpy>=1.26
pytest>=8,<10
sentence-transformers>=3,<4
einops>=0.8.0
qdrant-client>=1.9
fastapi>=0.115,<1
uvicorn>=0.30,<1

path: requirements-cpu.txt
content: # CPU-only PyTorch requirements
# Usage: pip install -r requirements-cpu.txt
#
# This installs CPU-only PyTorch (~200MB) instead of the full GPU version (~2GB)
# Ideal for: Docker containers, WSL, laptops without NVIDIA GPUs, edge deployment

--index-url https://download.pytorch.org/whl/cpu
torch==2.9.0+cpu
torchvision==0.25.0+cpu
torchaudio==2.9.0+cpu

# Core dependencies (same as main requirements)
pydantic>=2,<3
numpy>=1.26
# einops - optional, commented out for Python 3.14 compatibility
# einops>=0.7.0
# pytest - testing only, not needed for runtime
# pytest>=8,<10

# Embeddings and vector store
# sentence-transformers>=3,<4  # not compatible with Python 3.14 yet
# qdrant-client>=1.9

# API server
fastapi
uvicorn


path: retrieval/backend.py
content: """Prompt-composer retrieval backend adapter."""

from __future__ import annotations

from typing import Sequence

from prompt_engine.schemas import LongTermMemory

from .retrieval_pipeline import RetrievalPipeline


class PromptComposerRetrievalBackend:
    """Adapter implementing prompt_engine.memory_retriever.RetrievalBackend."""

    def __init__(self, pipeline: RetrievalPipeline | None = None) -> None:
        self.pipeline = pipeline or RetrievalPipeline()

    def retrieve(
        self,
        query: str,
        *,
        entities: Sequence[str],
        max_candidates: int,
    ) -> list[LongTermMemory]:
        return self.pipeline.retrieve_candidates(
            user_message=query,
            entities=list(entities),
            max_candidates=max_candidates,
        )


path: retrieval/entity_detector.py
content: """Entity detection helpers for retrieval biasing."""

from __future__ import annotations

from dataclasses import dataclass
from typing import Iterable

from entities import EntityAliasResolver
from prompt_engine.entity_extractor import extract_entities


@dataclass(frozen=True)
class EntityDetectionResult:
    entities: list[str]


def detect_entities_for_retrieval(
    *,
    user_message: str,
    conversation_history: str = "",
    known_entities: Iterable[str] | None = None,
    resolver: EntityAliasResolver | None = None,
    max_entities: int = 25,
) -> EntityDetectionResult:
    resolver = resolver or EntityAliasResolver()

    extracted = extract_entities(
        current_user_message=user_message,
        conversation_history=conversation_history,
        active_projects=known_entities,
        max_entities=max_entities,
    )

    merged = list(extracted)
    if known_entities:
        merged.extend(list(known_entities))

    canonical = resolver.canonicalize_many(merged)
    return EntityDetectionResult(entities=canonical[:max_entities])


path: retrieval/reranker.py
content: """Candidate memory re-ranking for retrieval context selection."""

from __future__ import annotations

from dataclasses import dataclass

from prompt_engine.schemas import LongTermMemory


@dataclass(frozen=True)
class RetrievalCandidate:
    memory: LongTermMemory
    vector_score: float


@dataclass(frozen=True)
class RankedCandidate:
    memory: LongTermMemory
    score: float
    vector_score: float


def _tokenize(text: str) -> set[str]:
    return {token.lower().strip('.,!?;:()[]{}"\'') for token in text.split() if len(token) >= 3}


def _overlap_fraction(left: set[str], right: set[str]) -> float:
    if not left:
        return 0.0
    return len(left & right) / len(left)


def rerank_candidates(
    *,
    user_message: str,
    candidates: list[RetrievalCandidate],
    query_entities: list[str],
    top_k: int = 5,
) -> list[RankedCandidate]:
    query_terms = _tokenize(user_message)
    query_entity_set = {entity.lower() for entity in query_entities}

    ranked: list[RankedCandidate] = []
    for candidate in candidates:
        memory = candidate.memory
        content_terms = _tokenize(memory.content)
        memory_entities = {entity.lower() for entity in memory.entities}

        keyword_overlap = _overlap_fraction(query_terms, content_terms)
        entity_overlap = _overlap_fraction(query_entity_set, memory_entities)

        score = (
            (candidate.vector_score * 0.45)
            + (memory.importance * 0.25)
            + (keyword_overlap * 0.20)
            + (entity_overlap * 0.10)
        )

        ranked.append(
            RankedCandidate(memory=memory, score=min(1.0, max(0.0, score)), vector_score=candidate.vector_score)
        )

    ranked.sort(key=lambda item: item.score, reverse=True)
    return ranked[:top_k]


path: retrieval/retrieval_pipeline.py
content: """Phase 3 retrieval pipeline."""

from __future__ import annotations

from dataclasses import dataclass
from datetime import datetime, timezone

from embeddings import TextEmbedder, create_default_embedder
from entities import EntityAliasResolver
from prompt_engine.schemas import LongTermMemory
from storage import JSONMemoryStore, VectorIndexStore

from .entity_detector import detect_entities_for_retrieval
from .reranker import RankedCandidate, RetrievalCandidate, rerank_candidates
from .time_filter import filter_by_time, parse_time_range


@dataclass(frozen=True)
class RetrievalPipelineConfig:
    candidate_pool_size: int = 30
    selected_size: int = 5


@dataclass(frozen=True)
class RetrievalResult:
    user_message: str
    entities: list[str]
    candidates: list[RetrievalCandidate]
    selected: list[RankedCandidate]
    degraded: bool
    error: str | None


class RetrievalPipeline:
    """Retrieve and rank long-term memories for context selection."""

    def __init__(
        self,
        *,
        json_store: JSONMemoryStore | None = None,
        vector_store: VectorIndexStore | None = None,
        embedder: TextEmbedder | None = None,
        entity_resolver: EntityAliasResolver | None = None,
        config: RetrievalPipelineConfig = RetrievalPipelineConfig(),
    ) -> None:
        self.json_store = json_store or JSONMemoryStore()
        self.vector_store = vector_store or VectorIndexStore()
        self.embedder = embedder or create_default_embedder()
        self.entity_resolver = entity_resolver or EntityAliasResolver()
        self.config = config

    def _mark_selected_as_accessed(
        self,
        selected: list[RankedCandidate],
        *,
        accessed_at: datetime,
    ) -> list[RankedCandidate]:
        """Persist access tracking for selected memories and return updated candidates."""

        updated_selected: list[RankedCandidate] = []

        for ranked in selected:
            memory = ranked.memory
            try:
                touched_memory = memory.model_copy(
                    update={
                        "last_accessed": accessed_at,
                        "access_count": memory.access_count + 1,
                    }
                )
                self.json_store.update_memory(touched_memory)
                updated_selected.append(
                    RankedCandidate(
                        memory=touched_memory,
                        score=ranked.score,
                        vector_score=ranked.vector_score,
                    )
                )
            except Exception:  # noqa: BLE001
                # Access tracking should not fail retrieval delivery.
                updated_selected.append(ranked)

        return updated_selected

    def retrieve(self, user_message: str, *, conversation_history: str = "") -> RetrievalResult:
        now = datetime.now(timezone.utc)

        detection = detect_entities_for_retrieval(
            user_message=user_message,
            conversation_history=conversation_history,
            resolver=self.entity_resolver,
        )

        try:
            query_vector = self.embedder.embed(user_message)
            vector_hits = self.vector_store.search(
                query_vector=query_vector,
                top_k=self.config.candidate_pool_size,
            )

            candidates: list[RetrievalCandidate] = []
            for hit in vector_hits:
                memory = self.json_store.get_memory(hit.memory_id)
                if memory is None:
                    continue
                candidates.append(RetrievalCandidate(memory=memory, vector_score=hit.score))

            time_range = parse_time_range(user_message, now=now)
            if time_range is not None:
                allowed_ids = {
                    memory.id
                    for memory in filter_by_time(
                        [candidate.memory for candidate in candidates],
                        time_range,
                    )
                }
                candidates = [
                    candidate for candidate in candidates if candidate.memory.id in allowed_ids
                ]

            selected = rerank_candidates(
                user_message=user_message,
                candidates=candidates,
                query_entities=detection.entities,
                top_k=self.config.selected_size,
            )
            selected = self._mark_selected_as_accessed(selected, accessed_at=now)

            return RetrievalResult(
                user_message=user_message,
                entities=detection.entities,
                candidates=candidates,
                selected=selected,
                degraded=False,
                error=None,
            )

        except Exception as error:  # noqa: BLE001
            return RetrievalResult(
                user_message=user_message,
                entities=detection.entities,
                candidates=[],
                selected=[],
                degraded=True,
                error=str(error),
            )

    def retrieve_candidates(
        self,
        user_message: str,
        *,
        entities: list[str] | None = None,
        max_candidates: int = 30,
    ) -> list[LongTermMemory]:
        """Return candidate memories for prompt-composer retrieval backend."""

        normalized_entities = (
            self.entity_resolver.canonicalize_many(entities) if entities else []
        )

        query_vector = self.embedder.embed(user_message)
        hits = self.vector_store.search(query_vector=query_vector, top_k=max_candidates)

        scored_memories: list[tuple[int, float, LongTermMemory]] = []
        for hit in hits:
            memory = self.json_store.get_memory(hit.memory_id)
            if memory is None:
                continue

            overlap = 0
            if normalized_entities:
                overlap = len(
                    set(entity.lower() for entity in normalized_entities)
                    & set(entity.lower() for entity in memory.entities)
                )

            # Sort by entity overlap first, then vector score.
            scored_memories.append((overlap, hit.score, memory))

        scored_memories.sort(key=lambda item: (item[0], item[1]), reverse=True)
        return [memory for _, __, memory in scored_memories[:max_candidates]]


path: retrieval/time_filter.py
content: """Temporal query parsing and memory time filtering."""

from __future__ import annotations

from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
import re

from prompt_engine.schemas import LongTermMemory


@dataclass(frozen=True)
class TimeRange:
    start: datetime
    end: datetime


DATE_PATTERN = re.compile(r"\b(\d{4}-\d{2}-\d{2})\b")


def parse_time_range(query: str, now: datetime | None = None) -> TimeRange | None:
    now = now or datetime.now(timezone.utc)
    text = query.lower()

    if "yesterday" in text:
        start = (now - timedelta(days=1)).replace(hour=0, minute=0, second=0, microsecond=0)
        end = start + timedelta(days=1)
        return TimeRange(start=start, end=end)

    if "today" in text:
        start = now.replace(hour=0, minute=0, second=0, microsecond=0)
        end = start + timedelta(days=1)
        return TimeRange(start=start, end=end)

    if "last week" in text:
        end = now
        start = now - timedelta(days=7)
        return TimeRange(start=start, end=end)

    if "last month" in text:
        end = now
        start = now - timedelta(days=30)
        return TimeRange(start=start, end=end)

    match = DATE_PATTERN.search(query)
    if match:
        start = datetime.fromisoformat(match.group(1)).replace(tzinfo=timezone.utc)
        end = start + timedelta(days=1)
        return TimeRange(start=start, end=end)

    return None


def filter_by_time(
    memories: list[LongTermMemory],
    time_range: TimeRange | None,
) -> list[LongTermMemory]:
    if time_range is None:
        return memories

    return [
        memory
        for memory in memories
        if time_range.start <= memory.created_at < time_range.end
    ]


path: retrieval/__init__.py
content: """Retrieval and context selection package."""

from .backend import PromptComposerRetrievalBackend
from .retrieval_pipeline import RetrievalPipeline, RetrievalPipelineConfig, RetrievalResult

__all__ = [
    "PromptComposerRetrievalBackend",
    "RetrievalPipeline",
    "RetrievalPipelineConfig",
    "RetrievalResult",
]


path: server.py
content: """FastAPI server for the cognitive memory system.

Runs as a persistent local process so embedding models and DB connections stay hot.
"""

from __future__ import annotations

from contextlib import asynccontextmanager
from datetime import datetime, timezone
from typing import Callable

from fastapi import FastAPI, HTTPException, Query, Request
from fastapi.encoders import jsonable_encoder
from pydantic import BaseModel, ConfigDict, Field

from cognitive_memory_system import CognitiveMemorySystem
from ingestion.ingestion_pipeline import IncomingInteraction
from prompt_engine.schemas import MemoryType, PromptComposerRequest


class RetrieveRequest(BaseModel):
    model_config = ConfigDict(extra="forbid")

    user_message: str = Field(min_length=1)
    conversation_history: str = ""


class RunBackgroundRequest(BaseModel):
    model_config = ConfigDict(extra="forbid")

    scheduled: bool = True


def create_app(
    system_factory: Callable[[], CognitiveMemorySystem] = CognitiveMemorySystem,
) -> FastAPI:
    @asynccontextmanager
    async def lifespan(app: FastAPI):
        # Keep the full cognitive stack (including embedder) hot across requests.
        app.state.cognitive_system = system_factory()
        yield

    app = FastAPI(
        title="Cognitive Memory API",
        version="0.1.0",
        lifespan=lifespan,
    )

    def get_system(request: Request) -> CognitiveMemorySystem:
        system = getattr(request.app.state, "cognitive_system", None)
        if system is None:
            raise HTTPException(status_code=503, detail="Cognitive system not initialized")
        return system

    def resolve_embedder(system: CognitiveMemorySystem):
        for component_name in ("retrieval", "ingestion", "background"):
            component = getattr(system, component_name, None)
            if component is None:
                continue
            embedder = getattr(component, "embedder", None)
            if embedder is not None:
                return embedder
        return None

    def resolve_json_store(system: CognitiveMemorySystem):
        for component_name in ("retrieval", "ingestion", "background"):
            component = getattr(system, component_name, None)
            if component is None:
                continue
            store = getattr(component, "json_store", None)
            if store is not None:
                return store
        return None

    @app.get("/")
    async def root():
        return {"status": "ok", "service": "cognitive-memory-api"}

    @app.get("/health")
    async def health(request: Request):
        system = get_system(request)
        embedder = resolve_embedder(system)

        embedder_loaded = bool(embedder is not None and callable(getattr(embedder, "embed", None)))
        model_name = getattr(embedder, "model_name", None) if embedder is not None else None
        backend = embedder.__class__.__name__ if embedder is not None else None

        return {
            "status": "ok" if embedder_loaded else "degraded",
            "embedder_loaded": embedder_loaded,
            "embedder_model": model_name,
            "embedder_backend": backend,
        }

    @app.get("/memories")
    async def list_memories(
        request: Request,
        memory_type: str | None = Query(default=None, alias="type"),
    ):
        system = get_system(request)
        store = resolve_json_store(system)
        if store is None:
            raise HTTPException(status_code=503, detail="Memory store unavailable")

        selected_types = None
        if memory_type:
            try:
                selected_types = [MemoryType(memory_type.strip().lower())]
            except ValueError as error:
                allowed = ", ".join(memory.value for memory in MemoryType)
                raise HTTPException(
                    status_code=400,
                    detail=f"Invalid memory type '{memory_type}'. Allowed values: {allowed}",
                ) from error

        memories = store.list_memories(types=selected_types)
        return jsonable_encoder(memories)

    @app.get("/memory/{memory_id}")
    async def get_memory(memory_id: str, request: Request):
        system = get_system(request)
        store = resolve_json_store(system)
        if store is None:
            raise HTTPException(status_code=503, detail="Memory store unavailable")

        memory = store.get_memory(memory_id)
        if memory is None:
            raise HTTPException(status_code=404, detail="Memory not found")

        return jsonable_encoder(memory)

    @app.get("/insights/pending")
    async def insights_pending(request: Request):
        system = get_system(request)
        hot_memory_manager = getattr(system, "hot_memory", None)
        if hot_memory_manager is None:
            raise HTTPException(status_code=503, detail="Hot memory unavailable")

        hot_memory = hot_memory_manager.get()
        now = datetime.now(timezone.utc)
        pending = [
            insight
            for insight in hot_memory.insight_queue
            if insight.expires_at is None or insight.expires_at >= now
        ]

        return jsonable_encoder({"count": len(pending), "insights": pending})

    @app.post("/ingest")
    async def ingest(interaction: IncomingInteraction, request: Request):
        system = get_system(request)
        result = system.ingest_interaction(interaction.model_dump(mode="json"))
        return jsonable_encoder(result)

    @app.post("/retrieve")
    async def retrieve(payload: RetrieveRequest, request: Request):
        system = get_system(request)
        result = system.retrieve_context(
            user_message=payload.user_message,
            conversation_history=payload.conversation_history,
        )
        return jsonable_encoder(result)

    @app.post("/compose")
    async def compose(payload: PromptComposerRequest, request: Request):
        system = get_system(request)
        result = system.compose_prompt(payload.model_dump(mode="json"))
        return jsonable_encoder(result)

    @app.post("/run_background")
    async def run_background(payload: RunBackgroundRequest, request: Request):
        system = get_system(request)
        result = system.run_background_cycle(scheduled=payload.scheduled)
        return jsonable_encoder(result)

    return app


app = create_app()


path: skills/smart-memory-v25/constants.js
content: "use strict";

const DEFAULT_BASE_URL = "http://127.0.0.1:8000";
const DEFAULT_HEARTBEAT_INTERVAL_MS = 60_000;
const MEMORY_RETRY_QUEUE_FILE = ".memory_retry_queue.json";
const DEFAULT_SEARCH_LIMIT = 5;
const DEFAULT_INSIGHTS_LIMIT = 10;
const DEFAULT_MIN_RELEVANCE = 0.6;
const COMMIT_FAILURE_SERVER_UNREACHABLE =
  "Memory commit failed — server unreachable. Queued for retry.";
const SESSION_ARC_SUMMARY_PROMPT =
  "Summarize this session arc: What was the goal? What approaches were tried? What decisions were made? What remains open?";
const PENDING_INSIGHT_PROMPT_GUIDANCE =
  "If pending insights appear in your context that relate to the current conversation, surface them naturally to the user. Do not force it — but if there is a genuine connection, seamlessly bring it up.";

module.exports = {
  DEFAULT_BASE_URL,
  DEFAULT_HEARTBEAT_INTERVAL_MS,
  MEMORY_RETRY_QUEUE_FILE,
  DEFAULT_SEARCH_LIMIT,
  DEFAULT_INSIGHTS_LIMIT,
  DEFAULT_MIN_RELEVANCE,
  COMMIT_FAILURE_SERVER_UNREACHABLE,
  SESSION_ARC_SUMMARY_PROMPT,
  PENDING_INSIGHT_PROMPT_GUIDANCE,
};


path: skills/smart-memory-v25/formatters.js
content: "use strict";

/**
 * @param {unknown} value
 * @param {number} fallback
 */
function asNumber(value, fallback) {
  const parsed = Number(value);
  return Number.isFinite(parsed) ? parsed : fallback;
}

/**
 * @param {string} query
 * @param {Array<{ memory?: any; score?: number; vector_score?: number }>} ranked
 * @returns {string}
 */
function formatMemorySearchResults(query, ranked) {
  if (!Array.isArray(ranked) || ranked.length === 0) {
    return `No relevant memories found for: ${query}`;
  }

  const lines = ["[MEMORY SEARCH RESULTS]"];

  ranked.forEach((item, idx) => {
    const memory = item.memory || item || {};
    const type = String(memory.type || "unknown");
    const content = String(memory.content || "").trim();
    const relevance = asNumber(item.score, asNumber(item.vector_score, 0));
    const createdAt = String(memory.created_at || memory.last_accessed || "unknown");

    lines.push(
      `${idx + 1}. [${type}] relevance=${relevance.toFixed(3)} created=${createdAt}`
    );
    lines.push(`   ${content || "(empty content)"}`);
  });

  return lines.join("\n");
}

/**
 * @param {Array<{ content?: string; confidence?: number; source_memory_ids?: string[]; generated_at?: string }>} insights
 * @param {number} limit
 * @returns {string}
 */
function formatPendingInsights(insights, limit) {
  const safeLimit = Math.max(1, Math.floor(Number(limit) || 10));
  const rows = Array.isArray(insights) ? insights.slice(0, safeLimit) : [];

  if (!rows.length) {
    return "No pending background insights.";
  }

  const lines = ["[PENDING INSIGHTS]"];
  rows.forEach((insight, idx) => {
    const content = String(insight.content || "").trim() || "(empty)";
    const confidence = asNumber(insight.confidence, 0);
    const related = Array.isArray(insight.source_memory_ids)
      ? insight.source_memory_ids.join(", ")
      : "none";
    const generatedAt = String(insight.generated_at || "unknown");

    lines.push(
      `${idx + 1}. confidence=${confidence.toFixed(2)} generated=${generatedAt}`
    );
    lines.push(`   insight: ${content}`);
    lines.push(`   related_memories: ${related || "none"}`);
  });

  return lines.join("\n");
}

/**
 * @param {string[]} values
 * @returns {string}
 */
function inlineList(values) {
  if (!Array.isArray(values) || !values.length) {
    return "none";
  }
  return values.map((item) => String(item).trim()).filter(Boolean).join(", ") || "none";
}

/**
 * @param {{
 *   status: string;
 *   activeProjects: string[];
 *   workingQuestions: string[];
 *   topOfMind: string[];
 *   pendingInsights: Array<{ content?: string }>;
 * }} input
 * @returns {string}
 */
function formatActiveContextBlock(input) {
  const insightLines = (Array.isArray(input.pendingInsights) ? input.pendingInsights : [])
    .map((insight) => String(insight.content || "").trim())
    .filter(Boolean)
    .map((line) => `- ${line}`);

  if (!insightLines.length) {
    insightLines.push("- none");
  }

  return [
    "[ACTIVE CONTEXT]",
    `Status: ${input.status || "idle"}`,
    `Active Projects: ${inlineList(input.activeProjects)}`,
    `Working Questions: ${inlineList(input.workingQuestions)}`,
    `Top of Mind: ${inlineList(input.topOfMind)}`,
    "",
    "Pending Insights:",
    ...insightLines,
    "[/ACTIVE CONTEXT]",
  ].join("\n");
}

module.exports = {
  asNumber,
  formatMemorySearchResults,
  formatPendingInsights,
  formatActiveContextBlock,
  inlineList,
};


path: skills/smart-memory-v25/http-client.js
content: "use strict";

const { DEFAULT_BASE_URL } = require("./constants");

class HttpError extends Error {
  /**
   * @param {string} message
   * @param {number} status
   * @param {unknown} [body]
   */
  constructor(message, status, body) {
    super(message);
    this.name = "HttpError";
    this.status = status;
    this.body = body;
  }
}

/**
 * @param {unknown} error
 * @returns {boolean}
 */
function isServerUnreachableError(error) {
  if (!error) {
    return false;
  }

  const message = String(error.message || error).toLowerCase();
  return (
    message.includes("fetch failed") ||
    message.includes("econnrefused") ||
    message.includes("enotfound") ||
    message.includes("network") ||
    message.includes("timed out") ||
    message.includes("aborted")
  );
}

/**
 * @param {string} baseUrl
 * @param {string} endpoint
 * @returns {string}
 */
function buildUrl(baseUrl, endpoint) {
  return `${baseUrl.replace(/\/$/, "")}${endpoint}`;
}

class CognitiveApiClient {
  /**
   * @param {{
   *   baseUrl?: string;
   *   timeoutMs?: number;
   *   fetchImpl?: typeof fetch;
   * }} [options]
   */
  constructor(options = {}) {
    this.baseUrl = options.baseUrl || DEFAULT_BASE_URL;
    this.timeoutMs = Number(options.timeoutMs || 10_000);
    this.fetchImpl = options.fetchImpl || globalThis.fetch;

    if (typeof this.fetchImpl !== "function") {
      throw new Error("Global fetch is not available. Use Node 18+ or pass fetchImpl.");
    }
  }

  /**
   * @template T
   * @param {"GET"|"POST"} method
   * @param {string} endpoint
   * @param {unknown} [payload]
   * @returns {Promise<T>}
   */
  async request(method, endpoint, payload) {
    const controller = new AbortController();
    const timeout = setTimeout(() => controller.abort(), this.timeoutMs);

    const headers = {};
    /** @type {RequestInit} */
    const init = {
      method,
      signal: controller.signal,
      headers,
    };

    if (payload !== undefined) {
      headers["content-type"] = "application/json";
      init.body = JSON.stringify(payload);
    }

    try {
      const response = await this.fetchImpl(buildUrl(this.baseUrl, endpoint), init);
      const text = await response.text();
      const body = text ? safeJsonParse(text) : {};

      if (!response.ok) {
        throw new HttpError(
          `HTTP ${response.status} calling ${endpoint}`,
          response.status,
          body
        );
      }

      return /** @type {T} */ (body);
    } finally {
      clearTimeout(timeout);
    }
  }

  /**
   * @returns {Promise<{
   *   healthy: boolean;
   *   reachable: boolean;
   *   embedderLoaded: boolean;
   *   status: string | null;
   *   detail?: unknown;
   *   error?: unknown;
   * }>}
   */
  async checkHealth() {
    try {
      const health = await this.request("GET", "/health");
      const embedderLoaded = Boolean(health.embedder_loaded);
      const status = typeof health.status === "string" ? health.status : null;

      return {
        healthy: embedderLoaded,
        reachable: true,
        embedderLoaded,
        status,
        detail: health,
      };
    } catch (error) {
      if (error instanceof HttpError) {
        return {
          healthy: false,
          reachable: true,
          embedderLoaded: false,
          status: "http_error",
          detail: error.body,
          error,
        };
      }

      return {
        healthy: false,
        reachable: false,
        embedderLoaded: false,
        status: "unreachable",
        error,
      };
    }
  }

  /**
   * @param {import("./types").MemorySearchInput & { conversation_history?: string }} payload
   */
  async retrieve(payload) {
    return this.request("POST", "/retrieve", payload);
  }

  /**
   * @param {Record<string, unknown>} payload
   */
  async ingest(payload) {
    return this.request("POST", "/ingest", payload);
  }

  /**
   * @param {Record<string, unknown>} payload
   */
  async compose(payload) {
    return this.request("POST", "/compose", payload);
  }

  async pendingInsights() {
    return this.request("GET", "/insights/pending");
  }

  /**
   * @param {boolean} [scheduled]
   */
  async runBackground(scheduled = true) {
    return this.request("POST", "/run_background", { scheduled: Boolean(scheduled) });
  }
}

/**
 * @param {string} raw
 * @returns {any}
 */
function safeJsonParse(raw) {
  try {
    return JSON.parse(raw);
  } catch {
    return raw;
  }
}

module.exports = {
  CognitiveApiClient,
  HttpError,
  buildUrl,
  isServerUnreachableError,
};


path: skills/smart-memory-v25/index.js
content: "use strict";

const path = require("path");

const {
  COMMIT_FAILURE_SERVER_UNREACHABLE,
  DEFAULT_BASE_URL,
  DEFAULT_HEARTBEAT_INTERVAL_MS,
  DEFAULT_INSIGHTS_LIMIT,
  DEFAULT_MIN_RELEVANCE,
  DEFAULT_SEARCH_LIMIT,
} = require("./constants");
const {
  formatActiveContextBlock,
  formatMemorySearchResults,
  formatPendingInsights,
} = require("./formatters");
const {
  CognitiveApiClient,
  HttpError,
  isServerUnreachableError,
} = require("./http-client");
const { PromptContextInjector } = require("./prompt-injection");
const { MemoryRetryQueue } = require("./retry-queue");
const { SessionArcCapturer } = require("./session-arc");
const { DEFAULT_TAG_RULES, buildAutoTags } = require("./tagging");
const { createOpenClawHooks } = require("./openclaw-hooks");

const ALLOWED_MEMORY_TYPES = new Set(["semantic", "episodic", "belief", "goal"]);
const ALLOWED_SEARCH_TYPES = new Set(["all", ...ALLOWED_MEMORY_TYPES]);

/**
 * Smart Memory v2.5 Skill for OpenClaw
 *
 * Core tools:
 * - memory_search
 * - memory_commit
 * - memory_insights
 *
 * Includes:
 * - required /health gate before each tool
 * - retry queue for failed commits
 * - queue flush on next healthy call / heartbeat
 * - session arc capture hooks
 * - system prompt active-context injector
 */
class SmartMemorySkill {
  /**
   * @param {{
   *   baseUrl?: string;
   *   timeoutMs?: number;
   *   retryQueuePath?: string;
   *   heartbeatIntervalMs?: number;
   *   logger?: Pick<Console, "log" | "warn" | "error">;
   *   tagRules?: Array<{ id: string; test: Function; add: string[] }>;
   *   autoStartHeartbeat?: boolean;
   * }} [options]
   */
  constructor(options = {}) {
    this.logger = options.logger || console;

    this.apiClient = new CognitiveApiClient({
      baseUrl: options.baseUrl || DEFAULT_BASE_URL,
      timeoutMs: Number(options.timeoutMs || 10_000),
    });

    this.retryQueue = new MemoryRetryQueue({
      queueFilePath: options.retryQueuePath || path.resolve(process.cwd(), ".memory_retry_queue.json"),
    });

    this.tagRules = Array.isArray(options.tagRules) && options.tagRules.length
      ? options.tagRules
      : DEFAULT_TAG_RULES;

    this.heartbeatIntervalMs = Number(
      options.heartbeatIntervalMs || DEFAULT_HEARTBEAT_INTERVAL_MS
    );
    this.heartbeatTimer = null;
    this._commitChain = Promise.resolve();
    this._retrieveSupportsFilterParams = true;

    this.promptInjector = new PromptContextInjector({
      apiClient: this.apiClient,
      healthCheck: () => this.apiClient.checkHealth(),
      logger: this.logger,
    });

    this.sessionArcCapturer = null;

    if (options.autoStartHeartbeat !== false) {
      this.startHeartbeat();
    }
  }

  startHeartbeat() {
    if (this.heartbeatTimer) {
      return;
    }

    this.heartbeatTimer = setInterval(() => {
      this.heartbeat().catch((error) => {
        this.logger.warn?.("[smart-memory-v25] heartbeat failed:", error.message);
      });
    }, this.heartbeatIntervalMs);

    if (typeof this.heartbeatTimer.unref === "function") {
      this.heartbeatTimer.unref();
    }
  }

  stopHeartbeat() {
    if (!this.heartbeatTimer) {
      return;
    }

    clearInterval(this.heartbeatTimer);
    this.heartbeatTimer = null;
  }

  async heartbeat() {
    const health = await this.apiClient.checkHealth();
    if (health.healthy) {
      await this._flushRetryQueue();
    }
    return health;
  }

  async memorySearch(input = {}) {
    const query = String(input.query || "").trim();
    if (!query) {
      return "memory_search failed: query is required.";
    }

    const type = String(input.type || "all").toLowerCase();
    if (!ALLOWED_SEARCH_TYPES.has(type)) {
      return "memory_search failed: type must be one of all, semantic, episodic, belief, goal.";
    }

    const limit = clampInteger(input.limit, DEFAULT_SEARCH_LIMIT, 1, 50);
    const minRelevance = clampNumber(
      input.min_relevance,
      DEFAULT_MIN_RELEVANCE,
      0,
      1
    );
    const conversationHistory = String(input.conversation_history || "");

    const gate = await this._checkToolHealth("memory_search");
    if (!gate.ok) {
      return gate.message;
    }

    try {
      const retrieval = await this._retrieveWithCompatibility({
        query,
        conversationHistory,
        type,
        limit,
        minRelevance,
      });

      let selected = Array.isArray(retrieval?.selected) ? retrieval.selected : [];
      selected = selected.filter((item) => {
        const memory = item?.memory || {};
        const itemType = String(memory.type || "").toLowerCase();
        const score = Number(item?.score ?? item?.vector_score ?? 0);

        const typePass = type === "all" ? true : itemType === type;
        const scorePass = Number.isFinite(score) ? score >= minRelevance : false;
        return typePass && scorePass;
      });

      const sliced = selected.slice(0, limit);
      return formatMemorySearchResults(query, sliced);
    } catch (error) {
      return `memory_search failed: ${error.message}`;
    }
  }

  /**
   * Sequential commit gate to avoid overloading local embedding compute.
   * @param {import("./types").MemoryCommitInput} input
   */
  async memoryCommit(input = {}) {
    return this._enqueueCommit(() => this._memoryCommitInternal(input));
  }

  /**
   * @param {import("./types").MemoryInsightsInput} input
   */
  async memoryInsights(input = {}) {
    const limit = clampInteger(input.limit, DEFAULT_INSIGHTS_LIMIT, 1, 100);

    const gate = await this._checkToolHealth("memory_insights");
    if (!gate.ok) {
      return gate.message;
    }

    try {
      const response = await this.apiClient.pendingInsights();
      const insights = Array.isArray(response?.insights) ? response.insights : [];
      return formatPendingInsights(insights, limit);
    } catch (error) {
      return `memory_insights failed: ${error.message}`;
    }
  }

  /**
   * Middleware helper: fetch passive context and inject [ACTIVE CONTEXT] into the system prompt.
   */
  async injectActiveContext(input) {
    return this.promptInjector.inject({
      baseSystemPrompt: String(input?.baseSystemPrompt || ""),
      agentIdentity: String(input?.agentIdentity || "Smart Memory Agent"),
      currentUserMessage: String(input?.currentUserMessage || "Session message"),
      conversationHistory: String(input?.conversationHistory || ""),
      hotMemory: input?.hotMemory,
      maxPromptTokens: input?.maxPromptTokens,
      retrievalTimeoutMs: input?.retrievalTimeoutMs,
      maxCandidateMemories: input?.maxCandidateMemories,
      maxSelectedMemories: input?.maxSelectedMemories,
    });
  }

  /**
   * Optional direct formatter if host framework already has hot memory + insight data.
   */
  buildActiveContextBlock(payload) {
    return formatActiveContextBlock(payload);
  }

  /**
   * @param {{
   *   summarizeSessionArc: (payload: {
   *     prompt: string;
   *     conversationText: string;
   *     turns: Array<{ role: string; content: string }>;
   *     trigger: "checkpoint"|"session_end";
   *   }) => Promise<string>;
   *   checkpointTurns?: number;
   *   maxTurns?: number;
   * }} options
   */
  configureSessionArc(options) {
    this.sessionArcCapturer = new SessionArcCapturer({
      memoryCommit: (input) => this.memoryCommit(input),
      summarizeSessionArc: options.summarizeSessionArc,
      checkpointTurns: options.checkpointTurns || 20,
      maxTurns: options.maxTurns || 20,
      logger: this.logger,
    });

    return this.sessionArcCapturer;
  }

  /**
   * Mid-session checkpoint hook (every 20 turns by default).
   * @param {{ conversationHistory: Array<{ role: string; content: string }> }} payload
   */
  async onConversationTurn(payload) {
    if (!this.sessionArcCapturer) {
      return null;
    }
    return this.sessionArcCapturer.onTurn(payload);
  }

  /**
   * Session teardown hook.
   * @param {{ conversationHistory: Array<{ role: string; content: string }> }} payload
   */
  async onSessionEnd(payload) {
    if (!this.sessionArcCapturer) {
      return null;
    }
    return this.sessionArcCapturer.onSessionEnd(payload);
  }

  /**
   * Tool descriptors for frameworks that register tools from metadata.
   */
  getToolDefinitions() {
    return [
      {
        name: "memory_search",
        description: "Query long-term semantic memory.",
        input_schema: {
          type: "object",
          properties: {
            query: { type: "string" },
            type: {
              type: "string",
              enum: ["all", "semantic", "episodic", "belief", "goal"],
              default: "all",
            },
            limit: { type: "number", default: DEFAULT_SEARCH_LIMIT },
            min_relevance: { type: "number", default: DEFAULT_MIN_RELEVANCE },
          },
          required: ["query"],
        },
        execute: (input) => this.memorySearch(input),
      },
      {
        name: "memory_commit",
        description: "Persist an explicit memory item to long-term memory.",
        input_schema: {
          type: "object",
          properties: {
            content: { type: "string" },
            type: {
              type: "string",
              enum: ["semantic", "episodic", "belief", "goal"],
            },
            importance: { type: "number", default: 5 },
            tags: {
              type: "array",
              items: { type: "string" },
            },
          },
          required: ["content", "type"],
        },
        execute: (input) => this.memoryCommit(input),
      },
      {
        name: "memory_insights",
        description: "Retrieve pending background insights from REM cycles.",
        input_schema: {
          type: "object",
          properties: {
            limit: { type: "number", default: DEFAULT_INSIGHTS_LIMIT },
          },
        },
        execute: (input) => this.memoryInsights(input),
      },
    ];
  }

  /**
   * @param {import("./types").MemoryCommitInput} input
   */
  async _memoryCommitInternal(input = {}) {
    const content = String(input.content || "").trim();
    const type = String(input.type || "").toLowerCase();

    if (!content) {
      return "memory_commit failed: content is required.";
    }

    if (!ALLOWED_MEMORY_TYPES.has(type)) {
      return "memory_commit failed: type must be semantic, episodic, belief, or goal.";
    }

    const importanceRaw = clampInteger(input.importance, 5, 1, 10);
    const importanceNormalized = Number((importanceRaw / 10).toFixed(3));
    const tags = buildAutoTags(content, {
      tags: Array.isArray(input.tags) ? input.tags : [],
      rules: this.tagRules,
    });

    const payload = buildIngestPayload({
      content,
      type,
      importanceRaw,
      importanceNormalized,
      tags,
      source: input.source,
      participants: input.participants,
      activeProjects: input.active_projects,
      metadata: input.metadata,
    });

    const health = await this.apiClient.checkHealth();
    if (!health.healthy) {
      await this.retryQueue.enqueue(payload);
      if (!health.reachable) {
        return COMMIT_FAILURE_SERVER_UNREACHABLE;
      }
      return "Memory commit failed — embedder not loaded. Queued for retry.";
    }

    await this._flushRetryQueue();

    try {
      const response = await this.apiClient.ingest(payload);
      const stored = Boolean(response?.stored);
      const reason = String(response?.reason || "unknown");
      const memoryId = response?.memory_id ? String(response.memory_id) : "n/a";
      return `Memory commit processed: stored=${stored} type=${type} id=${memoryId} reason=${reason}.`;
    } catch (error) {
      if (isServerUnreachableError(error)) {
        await this.retryQueue.enqueue(payload);
        return COMMIT_FAILURE_SERVER_UNREACHABLE;
      }
      return `memory_commit failed: ${error.message}`;
    }
  }

  async _checkToolHealth(toolName) {
    const health = await this.apiClient.checkHealth();

    if (!health.healthy) {
      if (!health.reachable) {
        return {
          ok: false,
          message: `${toolName} aborted: memory server unreachable.`,
        };
      }

      return {
        ok: false,
        message: `${toolName} aborted: memory embedder is not loaded.`,
      };
    }

    await this._flushRetryQueue();
    return { ok: true };
  }

  async _retrieveWithCompatibility({
    query,
    conversationHistory,
    type,
    limit,
    minRelevance,
  }) {
    const basePayload = {
      user_message: query,
      conversation_history: conversationHistory,
    };

    if (!this._retrieveSupportsFilterParams) {
      return this.apiClient.retrieve(basePayload);
    }

    try {
      return await this.apiClient.retrieve({
        ...basePayload,
        type,
        limit,
        min_relevance: minRelevance,
      });
    } catch (error) {
      if (error instanceof HttpError && error.status === 422) {
        this._retrieveSupportsFilterParams = false;
        return this.apiClient.retrieve(basePayload);
      }
      throw error;
    }
  }
  async _flushRetryQueue() {
    try {
      const result = await this.retryQueue.flush(async (payload) => {
        await this.apiClient.ingest(payload);
      });

      if (result.flushed > 0) {
        this.logger.log?.(
          `[smart-memory-v25] flushed ${result.flushed} queued commit(s); ${result.remaining} remaining.`
        );
      }

      return result;
    } catch (error) {
      const remaining = await this.retryQueue.size().catch(() => 0);
      this.logger.warn?.(
        "[smart-memory-v25] retry queue flush failed:",
        error.message
      );
      return { flushed: 0, remaining };
    }
  }

  /**
   * @template T
   * @param {() => Promise<T>} task
   * @returns {Promise<T>}
   */
  _enqueueCommit(task) {
    const run = this._commitChain.then(task, task);
    this._commitChain = run.then(() => undefined, () => undefined);
    return run;
  }
}

/**
 * Framework-friendly factory wrapper.
 *
 * @param {ConstructorParameters<typeof SmartMemorySkill>[0] & {
 *   summarizeSessionArc?: (payload: {
 *     prompt: string;
 *     conversationText: string;
 *     turns: Array<{ role: string; content: string }>;
 *     trigger: "checkpoint"|"session_end";
 *   }) => Promise<string>;
 * }} [options]
 */
function createSmartMemorySkill(options = {}) {
  const skill = new SmartMemorySkill(options);

  if (typeof options.summarizeSessionArc === "function") {
    skill.configureSessionArc({ summarizeSessionArc: options.summarizeSessionArc });
  }

  return {
    name: "smart-memory-v25",
    description: "Native Smart Memory skill for OpenClaw backed by local FastAPI cognitive OS.",
    start: () => skill.startHeartbeat(),
    stop: () => skill.stopHeartbeat(),

    // Tool wrappers
    memory_search: (input) => skill.memorySearch(input),
    memory_commit: (input) => skill.memoryCommit(input),
    memory_insights: (input) => skill.memoryInsights(input),

    // Lifecycle hooks
    on_turn: (payload) => skill.onConversationTurn(payload),
    on_session_end: (payload) => skill.onSessionEnd(payload),

    // Passive context injection middleware
    inject_active_context: (payload) => skill.injectActiveContext(payload),

    // Metadata tool definitions for tool-registration systems
    tools: skill.getToolDefinitions(),

    skill,
  };
}

/**
 * @param {{
 *   content: string;
 *   type: string;
 *   importanceRaw: number;
 *   importanceNormalized: number;
 *   tags: string[];
 *   source?: string;
 *   participants?: string[];
 *   activeProjects?: string[];
 *   metadata?: Record<string, unknown>;
 * }} args
 */
function buildIngestPayload(args) {
  const nowIso = new Date().toISOString();
  const source = String(args.source || "conversation");

  /** @type {Record<string, string>} */
  const typeHints = {
    semantic: "Explicit semantic memory commit: factual statement.",
    episodic: "Explicit episodic memory commit: event timeline.",
    belief: "Explicit belief memory commit: preference or inference.",
    goal: "Explicit goal memory commit: objective or task.",
  };

  return {
    user_message: args.content,
    assistant_message: typeHints[args.type] || "Explicit memory commit.",
    timestamp: nowIso,
    source,
    participants:
      Array.isArray(args.participants) && args.participants.length
        ? args.participants
        : ["user", "assistant"],
    active_projects: Array.isArray(args.activeProjects) ? args.activeProjects : [],
    metadata: {
      ...(args.metadata || {}),
      explicit_memory_commit: true,
      requested_type: args.type,
      importance_1_to_10: args.importanceRaw,
      importance_normalized: args.importanceNormalized,
      tags: args.tags,
      committed_at: nowIso,
    },
  };
}

/**
 * @param {unknown} value
 * @param {number} fallback
 * @param {number} min
 * @param {number} max
 */
function clampInteger(value, fallback, min, max) {
  const parsed = Number(value);
  if (!Number.isFinite(parsed)) {
    return fallback;
  }
  return Math.max(min, Math.min(max, Math.floor(parsed)));
}

/**
 * @param {unknown} value
 * @param {number} fallback
 * @param {number} min
 * @param {number} max
 */
function clampNumber(value, fallback, min, max) {
  const parsed = Number(value);
  if (!Number.isFinite(parsed)) {
    return fallback;
  }
  return Math.max(min, Math.min(max, parsed));
}

module.exports = {
  SmartMemorySkill,
  createSmartMemorySkill,
  buildIngestPayload,
  createOpenClawHooks,
};











path: skills/smart-memory-v25/openclaw-hooks.js
content: "use strict";

/**
 * OpenClaw lifecycle wiring helper.
 *
 * This gives a direct bridge for:
 * - passive prompt injection before model response
 * - session arc checkpoints every 20 turns
 * - session end arc capture
 */

/**
 * @param {{
 *   skill: import("./index").SmartMemorySkill;
 *   agentIdentity: string;
 *   summarizeWithLLM?: (payload: {
 *     prompt: string;
 *     conversationText: string;
 *     turns: Array<{ role: string; content: string }>;
 *     trigger: "checkpoint"|"session_end";
 *   }) => Promise<string>;
 *   checkpointTurns?: number;
 *   maxTurns?: number;
 * }} options
 */
function createOpenClawHooks(options) {
  const skill = options.skill;

  if (typeof options.summarizeWithLLM === "function") {
    skill.configureSessionArc({
      summarizeSessionArc: options.summarizeWithLLM,
      checkpointTurns: options.checkpointTurns || 20,
      maxTurns: options.maxTurns || 20,
    });
  }

  return {
    /**
     * Call before each model response.
     *
     * @param {{
     *   baseSystemPrompt: string;
     *   userMessage: string;
     *   conversationHistoryText?: string;
     *   hotMemory?: any;
     * }} ctx
     */
    beforeModelResponse: async (ctx) => {
      return skill.injectActiveContext({
        baseSystemPrompt: String(ctx.baseSystemPrompt || ""),
        agentIdentity: String(options.agentIdentity || "OpenClaw Agent"),
        currentUserMessage: String(ctx.userMessage || ""),
        conversationHistory: String(ctx.conversationHistoryText || ""),
        hotMemory: ctx.hotMemory,
      });
    },

    /**
     * Call on every appended turn.
     *
     * @param {{ conversationHistory: Array<{ role: string; content: string }> }} ctx
     */
    onTurn: async (ctx) => {
      return skill.onConversationTurn({
        conversationHistory: Array.isArray(ctx.conversationHistory)
          ? ctx.conversationHistory
          : [],
      });
    },

    /**
     * Call when the session is ending/resetting.
     *
     * @param {{ conversationHistory: Array<{ role: string; content: string }> }} ctx
     */
    onSessionEnd: async (ctx) => {
      return skill.onSessionEnd({
        conversationHistory: Array.isArray(ctx.conversationHistory)
          ? ctx.conversationHistory
          : [],
      });
    },
  };
}

module.exports = {
  createOpenClawHooks,
};


path: smart-memory/package-lock.json
content: {
  "name": "smart-memory",
  "version": "2.1.2",
  "lockfileVersion": 3,
  "requires": true,
  "packages": {
    "": {
      "name": "smart-memory",
      "version": "2.1.2",
      "hasInstallScript": true,
      "engines": {
        "node": ">=18.0.0"
      }
    }
  }
}


Security scan verdicts — ClawScan: suspicious | static analysis: suspicious | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)