Skill: nieto42/openclaw-memoria (version 3.34.0)

--- name: Memoria for OpenClaw version: 3.34.0 description: "The most advanced memory system for AI agents. 24 cognitive layers, knowledge graph, procedural learning, dialectic queries, AI self-observation, auto skill creation, crash-safe WAL, async prefetch. Works with Claude, Cursor, Copilot, ChatGPT & any OpenClaw agent. 100% local-first (SQLite + Ollama), zero cloud cost, zero API keys required." author: Primo Studio (@Nieto42) license: Apache-2.0 homepage: https://github.com/Primo-Studio/openclaw-memoria repository: https://github.com/Primo-Studio/openclaw-memoria feedback: https://x.com/Nitix_ keywords: - memory - ai-agent - persistent-memory - long-term-memory - knowledge-graph - procedural-learning - vector-search - sqlite - ollama - lm-studio - local-first - cognitive - context - claude - cursor - copilot - chatgpt - openclaw - wal - vibe-coding - developer-tools - devtools - typescript - llm - automation - brain-inspired - multi-layer - fact-extraction - self-improving tags: - memory - ai-agent - persistent-memory - long-term-memory - knowledge-graph - procedural-learning - vector-search - sqlite - ollama - lm-studio - local-first - cognitive - context - claude - cursor - copilot - chatgpt - openclaw - wal - vibe-coding - developer-tools - devtools - typescript - llm - automation - brain-inspired - multi-layer - fact-extraction - self-improving - self-observation - dialectic - auto-skill - prefetch - crash-recovery - async env: - name: OPENAI_API_KEY required: false description: Optional — used as fallback for LLM extraction and embeddings when local models are unavailable. Never required for default operation. - name: OPENROUTER_API_KEY required: false description: Optional — used as fallback for remote LLM provider. Never required for default operation. - name: OPENCLAW_WORKSPACE required: false description: Auto-set by OpenClaw runtime — workspace path for memory files. Do not set manually. security: | ## Data & Privacy - Memoria runs 100% locally by default. No data leaves your machine unless you explicitly configure a remote LLM. - All memory is stored in a local SQLite database in your OpenClaw workspace. - API keys (OPENAI_API_KEY, OPENROUTER_API_KEY) are optional fallbacks — never required. ## What Memoria reads - Workspace files: USER.md, COMPANY.md, projects/* — used for identity-aware relevance scoring. These files may contain personal or business information. Review them before enabling Memoria. - Conversation content: assistant messages and tool call results, to extract durable facts. - No files outside the OpenClaw workspace are read. ## What Memoria writes - A single SQLite database (memoria.db) in your workspace/memory/ folder. - Optional markdown summaries in workspace/memory/ folder. - No network requests unless a remote LLM fallback is configured. ## Hooks used - before_prompt_build: injects recalled facts into context - after_tool_call: captures procedural knowledge (how-to steps) - agent_end: extracts facts from completed conversations - after_compaction: extracts facts from compacted summaries These are standard OpenClaw plugin hooks. They run locally within the plugin process. entrypoint: index.ts --- # 🧠 Memoria — Multi-Layer Persistent Memory for OpenClaw **The most complete memory system for OpenClaw.** 20 layers of memory that work together, powered by YOUR choice of LLM. ## Why Memoria? ### 🏗️ 20 Memory Layers (not just a fact store) 1. **Facts** — Durable knowledge extracted from every conversation 2. **Procedures** — HOW to do things, improves with repetition, learns from failures 3. **Knowledge Graph** — Entities + relations connecting your facts 4. **Topics & Expertise** — Tracks what you talk about most, specializes over time 5. **Observations** — Short-term working memory for active context 6. **Error Detection** 🔥 — Touch fire once, remember forever. Dangers captured on first occurrence 7. **Lifecycle** — Fresh → Settled → Dormant. Nothing deleted, priority shifts naturally ### 🔌 Bring Your Own LLM Configure each layer independently. Mix and match: - **Ollama** — Run gemma3, qwen3.5, llama, or any model locally (recommended) - **LM Studio** — Use any GGUF model from your local server - **Remote APIs** — OpenAI, Anthropic, OpenRouter as primary or fallback - **Fallback chains** — Ollama → LM Studio → API. If one fails, the next takes over automatically ### 🏠 100% Local-First - **SQLite + FTS5** — No external database needed - **Local embeddings** — nomic-embed-text via Ollama (zero API cost) - **Zero cloud dependency** — Works offline, your data stays on your machine - **Fallback chain** — Degrades gracefully if a provider goes down ### 🧬 What Makes Memoria Different | Feature | Memoria | Basic memory plugins | |---------|---------|---------------------| | Memory layers | 20 specialized layers | Single fact store | | LLM choice | Any local or remote model | Usually hardcoded | | Per-layer LLM config | ✅ Different model per layer | ❌ | | Procedural learning | ✅ Learns HOW, not just WHAT | ❌ | | Error detection | ✅ Auto-captures dangers | ❌ | | Knowledge graph | ✅ Entities + relations | ❌ | | Lifecycle management | ✅ Smart aging, never forgets | ❌ or simple TTL | | Cost | $0 with local models | Varies | ## Installation ### As Plugin (recommended — one command) ```bash openclaw plugins install clawhub:memoria-plugin ``` This installs Memoria from the ClawHub registry. No manual steps needed. ### From source (for contributors / advanced users) If you prefer to inspect the code first: 1. Browse the repository: [github.com/Primo-Studio/openclaw-memoria](https://github.com/Primo-Studio/openclaw-memoria) 2. Review the source code, especially `index.ts` (main entrypoint) and `openclaw.plugin.json` (config schema) 3. Clone and install: ```bash cd ~/.openclaw/extensions git clone https://github.com/Primo-Studio/openclaw-memoria.git memoria cd memoria && npm install ``` Then add to your `openclaw.json` under `plugins.entries`: ```json { "memoria": { "enabled": true }, "memory-convex": { "enabled": false } } ``` ## Configuration ### Minimal (works out of the box with Ollama) Just install and restart. Defaults: Ollama + gemma3:4b for extraction, nomic for embeddings. ### Custom LLM per layer ```json "memoria": { "enabled": true, "config": { "llm": { "default": { "provider": "ollama", "model": "qwen3.5:4b" }, "procedural": { "provider": "lmstudio", "model": "your-model" }, "graph": { "provider": "openai", "model": "gpt-4o-mini" } } } } ``` ## Source Code The full source is available on GitHub: [Primo-Studio/openclaw-memoria](https://github.com/Primo-Studio/openclaw-memoria) Key files: - `index.ts` — Main plugin entrypoint (hooks, extraction, recall pipeline) - `procedural.ts` — Procedural memory (how-to learning) - `lifecycle.ts` — Lifecycle management (fresh/settled/dormant) - `scoring.ts` — Temporal scoring and relevance ranking - `selective.ts` — Dedup, contradiction detection, fact quality - `openclaw.plugin.json` — Configuration schema ## Feedback & Community **We'd love your feedback!** Tell us how Memoria works for you: - 🐦 **Tweet us** [@Nitix_](https://x.com/Nitix_) — share your setup, results, or ideas - ⭐ **Star the repo**: [github.com/Primo-Studio/openclaw-memoria](https://github.com/Primo-Studio/openclaw-memoria) - 🐛 **Issues**: [GitHub Issues](https://github.com/Primo-Studio/openclaw-memoria/issues) Built with ❤️ by [Primo Studio](https://primo-studio.fr) 🇬🇫 — AI tooling from French Guiana.

path: CHANGELOG.md
content: ## 3.22.2 — Layer 21: Continuous Learning + 6 Bug Fixes (2026-03-28)
## [3.22.3] — 2026-03-28

### Fixed
- **CRITICAL: CONTINUOUS_ENABLED TDZ crash on Node 24.x** — `const` variable accessed before declaration at register time, causing plugin to fail silently on gateways running Node 24.x (embedded binary). Inlined config read for boot log.
- **better-sqlite3 cross-Node-version build guide** — documented that `npx node-gyp rebuild --target=24.13.1` is needed when shell Node differs from gateway's embedded Node.


### New: Layer 21 — Continuous Learning
Real-time fact capture via `message_received` + `llm_output` hooks, independent of session end or compaction.
- **3 extraction modes**: periodic (every N turns), urgent (on user frustration/error), self-error (on assistant self-admission)
- **Cross-layer integration**: facts go through selective dedup → full postProcess pipeline (embed, graph, topics, observations, clusters, sync)
- **Smart dedup with agent_end**: when continuous already captured during the session, agent_end reduces its extraction scope to avoid double LLM calls
- **Configurable**: `continuous.interval` (default 4), `continuous.cooldownMs` (default 45s), `continuous.enabled` (default true)

### Bug Fixes (3 rounds of audit)
**v3.22.0** — Initial implementation
**v3.22.1** — Audit round 1:
- `cfg.continuous` not typed in `MemoriaConfig` → added full interface
- No `enabled` guard → both hooks now check `CONTINUOUS_ENABLED` before running
- `cooldownMs` hardcoded → now reads from config
- agent_end double capture → reduces scope when continuous already ran

**v3.22.2** — Audit round 2:
- **Concurrent extraction risk** — urgent trigger during periodic extraction ran 2 extractions in parallel → added `continuousExtractionInProgress` lock with `finally` release
- **Buffer never cleared** — same messages re-analyzed at each extraction → snapshot + clear before extraction

### Documentation
- `docs/ARCHITECTURE.md`: full Layer 21 section with config, hooks, extraction modes, cross-layer integration
- Boot log now shows continuous learning status

## 3.21.0 — Deep Audit: 10 Bugs Found & Fixed (2026-03-28)

### Critical Fixes
- **Hebbian learning was 100% dead** — `hebbian.ts` used wrong column names (`from_entity`/`to_entity`/`relation_type`/`updated_at`) but DB has `source_id`/`target_id`/`relation`/`last_accessed_at`). All queries silently returned nothing since creation.
- **Proactive revision never triggered** — searched for `lifecycle_state = 'mature'` but DB only has `fresh`/`settled`/`dormant`
- **storeFact() lost 6 columns** on INSERT — `usefulness`, `recall_count`, `used_count`, `synced_to_md`, `relevance_weight`, `lifecycle_state` silently dropped
- **Fact interface missing 4 DB columns** — 8 `as any` casts removed
- **4 SQL queries filtered wrong lifecycle state** — `!= 'archived'` → `!= 'dormant'`
- **Cross-layer 9b used wrong column names** — same `from_entity`/`to_entity` bug as hebbian
- **revision.ts imported non-existent file** — `./llm-provider.js` → `./providers/types.js`
- **procedural.ts wrong import path** — aligned to `./providers/types.js`
- **Fact type unions incomplete** — added `"cluster"` and `"pattern"` to `fact_type`

## 3.20.1 — Audit: 6 Bugs Found & Fixed (2026-03-28)
- Type alignment: `fact_type` and `lifecycle_state` unions updated
- SQL queries: 4 instances of `archived` → `dormant`
- Import fixes: `revision.ts` and `procedural.ts`

## 3.20.0 — Cross-Layer Connections (2026-03-28)
- **Feedback → Lifecycle**: facts recalled 5+ times with usefulness ≥ 2 → auto-promoted to "settled"
- **Hebbian → Topics**: strong relations (weight ≥ 1.0) auto-organize topic hierarchy (smaller becomes child)
- **Lifecycle → Patterns**: patterns with 5+ occurrences → settled
- Pattern detection step added to postProcessNewFacts

## 3.19.0 — Behavioral Pattern Detection, Layer 20 (2026-03-28)
- New module `patterns.ts` (~477 LOC) — detects repeated similar facts and consolidates
- Patterns stored as `fact_type = "pattern"` with occurrence metadata in tags
- Wired into capture pipeline (postProcessNewFacts) and recall scoring
- ARCHITECTURE.md updated to reflect Layer 20

## 3.18.0 — Fix Existing Layers, Phase 1 (2026-03-28)
- New DB table `cluster_members` with backfill (~407 links)
- Topics parent inference improved (composite-name strategy)
- Boot-time reparenting of existing topics (17 reparented)
- ARCHITECTURE.md reviewed/updated (19→20 layers documented)

## 3.17.0 — ClawHub Security Fixes (2026-03-28)
- SKILL.md: explicit `entrypoint: index.ts`, env vars declared optional, expanded security section
- Install instructions: plugin install (one command) + "review code first" for source
- Bundle includes all .ts modules + index.ts
- License: Apache-2.0 in YAML frontmatter

## 3.16.0 — ClawHub Suspicious Fix (2026-03-28)
- Created `.clawignore` to exclude dev/audit artifacts
- Declared env vars (OPENAI_API_KEY, OPENROUTER_API_KEY, OPENCLAW_WORKSPACE) as optional
- Added security posture section to SKILL.md

## 3.14.1 — Error Detection: Touch Fire Once (2026-03-27)
- Automatic error signal detection (crashes, frustration keywords, rollbacks)
- Each error captures: what happened, why dangerous, what to never repeat, safe alternative

## 3.14.0 — Smarter Extraction + Consolidation (2026-03-27)
- Extraction prompt rewritten: demands concrete details (who/what/when/why)
- Cluster-aware recall: member facts get -40% score penalty
- Procedures: first success = worth remembering (was: needed 3+ steps)
- Added `failure_reasons` column for contextual alternative selection

## 3.12.0 — Capture Quality & Contradiction Detection

### Fix 1: Capture Filter
- Only store **reusable** procedures (≥3 meaningful steps + at least one "action" command)
- Skip noise: health checks, diagnostics, log inspections, status checks
- Double-check LLM-assigned names against noise patterns

### Fix 2: Duplicate Detection
- `findSimilarProcedure()` — word-overlap matching on name + goal (threshold 50%)
- Before creating a new procedure, check if a similar one exists → reinforce instead of duplicate
- Applied at both `extractProcedure()` and `after_tool_call` hook levels

### Fix 3: Contradiction Check on Facts
- Widened entity search from 5 to 10 candidates
- Version-containing facts prioritized in contradiction search
- Enhanced contradiction prompt: version changes, status changes, quantity changes = explicit contradictions
- Prevents stale facts (e.g., "Sol = v2.7.0") from persisting when newer facts arrive ("Sol = v3.11.0")

## 3.10.0 (2026-03-27)

### Features
- **FTS5 procedural search**: `procedures_fts` virtual table with LIKE fallback — fast full-text search on procedures (name, goal, context, gotchas, steps)
- **Configurable thresholds**: `ProceduralConfig` interface with `qualityWeights`, `degradationStep`, `healingStep`, `reflectEvery`, `degradedThreshold`, `defaultSafety`
- **FTS auto-sync**: index created at boot, rebuilt if empty, kept in sync on every `storeProcedure` call
- **Plugin schema**: procedural config exposed in `openclaw.plugin.json` for wizard/UI configuration

### Fixes
- `kg` → `graph` variable reference (runtime crash)
- Feedback proc IDs removed (was querying wrong table)
- Procedure objects fully typed (no more `as any` partial objects)

# Changelog

## [3.9.0] - 2026-03-27
### Added — Reflective Procedural Learning
- **Quality dimensions** — each procedure scored on speed, reliability, elegance, safety
  - Weighted composite: reliability (35%) > safety (25%) > speed (25%) > elegance (15%)
  - Quality evolves with each execution, not static
- **Post-execution reflection** — every 3rd success triggers LLM review
  - "Was this the best approach?" → suggestions, quality reassessment
  - Blends new assessment (70%) with accumulated wisdom (30%)
  - Tracks gotchas/workarounds learned
- **Alternatives** — same goal, different approaches
  - `getAlternatives()` finds competing procedures
  - `setPreferred()` marks the best approach
  - Search prioritizes preferred procedures
- **Version tracking** — procedures evolve: version increments on each improvement
- **Personal best** — tracks fastest execution, speed quality improves when beaten
- **Schema auto-migration** — new quality columns added seamlessly on boot

### Why
"Un humain n'enregistre pas un savoir en rentrant chez lui le soir — 
il apprend sur le tas, il améliore en direct. La qualité passe par 
une meilleure réflexion, et c'est en améliorant la qualité qu'on 
gagne en vitesse d'exécution car on la reproduit plus souvent."

## [3.8.0] - 2026-03-27
### Added — Real-time Procedural Learning
- **`after_tool_call` hook** — captures procedures in real-time, not at end of session
  - Buffers tool calls during conversation (last 30)
  - On success signal (Published, ✅, deployed, committed, etc.) → immediately assembles procedure via LLM
  - If similar procedure exists → reinforces it (success_count++) and adds improvements
  - If new → creates new procedure with steps, goal, trigger patterns, gotchas
  - 60s cooldown between assemblies to avoid spam
  - Fingerprint dedup to avoid duplicate captures
- `agent_end` remains as safety net for any uncaptured sequences

### Why this change
- Humans learn on-the-fly, not at the end of the day
- `agent_end` only fires at conversation end → in long-running sessions, procedures were never captured
- Real-time learning means knowledge is available immediately for the next similar task

## [3.7.2] - 2026-03-27
### Fixed — 3 Critical Memory Issues
- **ProceduralMemory DB fix**: was receiving MemoriaDB wrapper instead of raw better-sqlite3 Database, causing `this.db.prepare is not a function` — procedures were "captured" in logs but never persisted (0 in DB)
- **Recall query pollution fix**: FTS5 search was matching on OpenClaw envelope metadata (`"Conversation info (untrusted metadata)..."`) instead of actual user message — causing 89% of facts to never be recalled. Now strips envelope before search
- **DB cleanup**: 22 vague/meta facts superseded (e.g., "Le nouveau fait complète l'ancien"), cortex.db archived

### Impact
- Procedures now persist to SQLite correctly
- Recall will match on what the user actually says, not Telegram metadata
- 436 active facts (was 450, 14 were noise)

## [3.7.1] - 2026-03-27
### Fixed — Phase 3 Procedural Capture
- **Dual-strategy extraction** for better reliability:
  - Strategy A: extract from `event.toolCalls` when available (original path)
  - Strategy B: parse assistant messages for command patterns (fallback)
  - Patterns detected: bash code blocks, inline commands, shell prompts (`$ ...`)
  - Success detection: ✅|success|published|deployed|completed keywords
  - Deduplication of consecutive identical commands
- **Debug logging** added to diagnose capture behavior in production
- New method: `ProceduralMemory.extractFromMessages(messages, context)`

### Why this fix
- v3.7.0 captured 0 procedures because `event.toolCalls` was empty/unavailable
- Message parsing ensures capture works even when toolCalls not exposed by OpenClaw
- Enables real-world validation of Phase 3 procedural learning

## [3.7.0] - 2026-03-27
### Added — Procedural Memory (Phase 3)
- **How-to knowledge that improves with repetition**
  - New `procedures` table: stores sequences of successful actions (exec/tool calls)
  - Captures steps, success/failure counts, degradation score, alternatives
  - Hook `agent_end`: detects successful command sequences → extracts procedure
  - Hook `before_prompt_build`: searches matching procedures → injects steps
  - Dynamic improvement: success_count++ reduces degradation, failure++ increases it
  - Alternative paths: when degradation > 0.5, searches for better alternative procedure
  - Example: "Publish to ClawHub" captured as 4-step procedure with success rate

- **Stats at boot**: `procedures: 0✓/0⚠` (healthy/degraded)

### Why this matters
- Memoria now learns "how to do things" (not just "what happened")
- Procedures improve over time as they're repeated successfully
- Failed attempts trigger degradation → search for alternative approach
- Solves: "I published v3.5.0 but don't remember HOW" → now it's stored & recalled

## [3.6.0] - 2026-03-27
### Added — Human-Like Memory Architecture
- **Identity-aware memory** (Phase 0)
  - New `relevance_weight` column (0.0-1.0, default 0.5) on facts
  - Parses `USER.md`, `COMPANY.md`, `projects/objectifs.md` to extract identity/priorities
  - Boosts facts about Bureau, Polymarket, Primask (core work) vs Memoria internals (meta)
  - Scoring integrates relevance: Bureau facts rise, config/plugin facts sink
  - New `identity_cache` table stores parsed identity for fast lookup

- **Lifecycle states** (Phase 1.1)
  - Facts evolve through 4 states: `fresh` → `mature` → `aged` → `archived`
  - Automatic transitions based on time + usage ratio + recall count
  - `archived` facts excluded from recall (forgotten, not deleted)
  - Stats displayed at boot: `338f/0m/0a/0⚰` (fresh/mature/aged/archived)

- **Proactive revision** (Phase 1.2)
  - Mature facts with 10+ recalls trigger LLM revision proposal
  - If improved → new fact created + old superseded
  - Revision runs in background (non-blocking)

- **Hebbian reinforcement** (Phase 2)
  - Relations now have `weight` (0.0-2.0, default 1.0)
  - Co-occurrence → weight++ (entities seen together strengthen)
  - Time decay → weight-- (unused relations fade)
  - Weak relations pruned automatically (<0.3)
  - Stats: `21 strong, 0 weak` relations

- **Expertise specialization** (Phase 2)
  - Topics gain "expertise levels": novice/familiar/experienced/expert
  - Based on `access_count` (interaction frequency)
  - Expert topics boost recall score (1.3x for expert, 1.1x for experienced)
  - Stats: `8★★★/6★★/4★` (expert/experienced/familiar)

### Fixed
- Added try/catch to lifecycle, hebbian, expertise modules (prevent crash on SQL errors)
- Expertise module: fixed schema mismatch (`topic` → `name`, `interaction_count` → `access_count`)

## [3.5.1] - 2026-03-26
### Fixed
- TypeScript parse error in `feedback.ts` (class closing brace misplaced) — plugin was crashing silently for 7h
- Plugin now loads correctly after restart

## [3.5.0] - 2026-03-26
### Added — Feedback Loop & Adaptive Learning
- **Usefulness tracking** — each recalled fact now has `usefulness`, `recall_count`, `used_count` scores
  - Facts referenced in the assistant's response → usefulness++ (boost)
  - Facts ignored repeatedly → usefulness-- (sink naturally)
  - Scoring integrates usefulness: high-use facts rise, never-used facts decay faster
- **User correction detection** — detects patterns like "non c'est", "en fait", "actually", "that's wrong" (FR+EN)
  - Penalizes the last-recalled facts that may have caused the error (-1.5 penalty)
- **User frustration detection** — detects "putain", "bordel", "wtf", repeated questions
  - Mild penalty (-0.5) on last-recalled facts
- **Adaptive budget** — budget now learns from compactions:
  - If recall → compaction within 5 min → penalty increases (injected too many facts)
  - Penalty reduces limit by 1-3 facts (minimum always respected)
  - Penalty decays naturally when compactions stop (self-correcting)

### Added — Cross-Layer Supersede Cascade
- When a fact is superseded, ALL layers are notified:
  - **Observations**: superseded fact removed from evidence lists; empty observations deleted
  - **Graph**: fact removed from relation contexts; orphaned relations weakened (-0.15) or pruned
  - **Topics**: fact↔topic links removed; empty topics deleted; fact_count updated
  - **Embeddings**: stale embedding vector deleted (no more ghost matches in semantic search)
- Before: layers were disconnected. A superseded fact's ghost persisted in graph, topics, embeddings.

### Added — Smart md-regen
- Auto-triggers on 3 conditions (replaces dumb "lines > 200" check):
  - `captures_since_regen >= 20` — enough new facts accumulated
  - `last_regen_at > 7 days` — stale files even with few captures
  - Any `.md file > 200 lines` — backward-compatible safety net
- Tracks `captures_since_regen` and `last_regen_at` in meta table

### Improved — Extraction Quality
- **Anti-meta prompt** — blocks vague/meta-facts ("Le nouveau fait fournit des informations...")
  - Requires at least one proper noun, number, or concrete command per fact
- **Tighter dedup** — combined threshold lowered to 0.75 + new "8 first words identical" → instant duplicate
- **Dynamic entity matching** — `SelectiveMemory` now loads entities from the Knowledge Graph DB (373+ entities)
  instead of a hardcoded regex list. Refreshes every 5 min.

### Fixed
- DB cleanup: 307→294 active facts (13 superseded, 5 duplicate clusters purged, 3 meta-facts removed)

## [3.4.1] - 2026-03-26
### Improved — Install Wizard UX
- **Clearer prompts**: "Tapez 1, 2 ou 3" on all choices (not just "Choix [1]")
- **Cloud providers**: choose between OpenAI, OpenRouter, or Anthropic (was OpenAI-only)
- **Modifiable after install**: all prompts now mention `configure.sh` for post-install changes
- **Update mode**: `--update` flag for quick silent updates; auto-detection of existing install
- **Existing install detection**: if Memoria is already installed, proposes Update / Reinstall / Cancel
- **Thank-you message**: links to @Nitix_ (X), GitHub star, Primo Studio credit
- **Auto-cleanup**: `memory-convex` entry automatically removed from `openclaw.json` if present
- **Fallback info**: warns user that crash notifications appear in logs
- **Embeddings note**: displayed during install with "changeable later" mention

## [3.4.0] - 2026-03-26
### Added — Fact Clusters
- **Entity-grouped "dossier" summaries**: groups 3+ facts sharing the same entity into one dense paragraph
- Clusters stored as `fact_type = "cluster"` — searchable via FTS5 + embeddings like regular facts
- 15% scoring boost (info-dense = higher recall value)
- Auto-invalidation: when a member fact is superseded, cluster marked stale → regenerated next cycle
- Entity detection: knowledge graph IDs first, proper noun extraction fallback
- Known entities pattern matching for Memoria-specific terms (Sol, Bureau, Primask, etc.)
- **Impact**: MS (multi-session) benchmark 2/5 → 3.5/5; overall accuracy 75% → 81.7%

### Benchmark Results (v3.4.0, GPT-5.4-nano judge)
- Accuracy: **81.7%** (22/30 correct + 5 partial)
- Retrieval: **50.0%** (15/30)
- SSU 5/5, KU 5/5, SSP 5/5, SSA 3.5/5, TR 3.5/5, MS 2.5/5
- 39 atomic facts + 5 clusters = 44 total facts from 10 sessions

## [3.3.0] - 2026-03-26
### Added — Query Expansion
- **Hybrid search now expands queries** into 2-4 semantic variants before searching
- Domain-specific concept map: "taux horaire" → ["salaire", "€/h", "paie"], "projets" → ["apps", "MVPs"], etc.
- FTS + cosine both search across all variants, deduplicating results
- Proper noun extraction: named entities searched standalone
- **Impact**: MS (multi-session) questions like "quels taux horaires?" now find "5.19€/h" facts

### Improved — Topic-Aware Recall
- `findRelevantTopics` now receives expanded queries for broader matching
- Topic name exact match bonus (+3 score) with expanded variants
- **Impact**: Topics like "salaires" found even when query says "rémunération"

### Improved — Denser Extraction
- Extraction prompt now enforces "one fact per distinct entity"
- Example: session mentioning 3 people → 3 separate facts instead of 1 merged
- **Impact**: More facts per session = better multi-session recall

## [3.2.0] - 2026-03-26
### Fixed — Reasoning Model Support (I3+I4)
- **Ollama provider**: Now reads `thinking` field when `response` is empty (GPT-OSS, Qwen3.5 reasoning models)
- **OpenAI-compat provider**: Now reads `reasoning_content` and `reasoning` fields (LM Studio GPT-OSS)
- **Impact**: Clients using reasoning models no longer get empty extractions/answers

### Fixed — Knowledge Update Recall (I1+I2)
- **Recall now shows dates**: Each fact displays age (`[aujourd'hui]`, `[il y a 3j]`, `[2026-03-20]`)
- **Header instructs**: "Les faits les plus récents sont les plus fiables en cas de contradiction"
- **Impact**: Answering model can now disambiguate when old and new versions of a fact coexist

### Improved — Procedure Extraction (I5)
- **Multi-sentence facts allowed**: Procedures can now be captured as 2-4 sentences in a single fact
- **Prompt guidance**: Examples show good vs bad procedure capture
- **Impact**: Workflows and how-to knowledge preserved as coherent units

### Improved — Short Query Handling (I6)
- **Adaptive FTS/cosine weights**: Short queries (<3 words) now favor semantic search (55%) over FTS (20%)
- **Impact**: Generic queries like "Bureau" return semantically relevant facts instead of keyword noise

### Added — Provider Interface Cleanup (I7)
- **`generateWithMeta`** added to LLMProvider interface (optional, with default implementation)
- **All providers** (Ollama, OpenAI-compat) now implement generateWithMeta
- **Impact**: Providers are fully interchangeable with FallbackChain

### Added — Anthropic Provider (I8+A3)
- **New `providers/anthropic.ts`**: Native Claude API support (`/v1/messages` format)
- **Supported in**: LLM config, fallback chain, per-layer overrides
- **Models**: Any Claude model (Haiku, Sonnet, Opus) via API key
- **Impact**: Clients can use Claude directly without routing through OpenRouter

### Added — Config Schema Update
- **`anthropic`** added to `llm.provider` enum in plugin schema
- **Fallback chain** supports `type: "anthropic"` entries

## [3.1.1] - 2026-03-25
### Improved — Extraction Quality (Results over Status)
- **Problem**: Extraction captured "test passed ✅" but lost actual results like "Retrieval 92%, bottleneck = local model"
- **New ✅ categories**: benchmark results with metrics, conclusions from experiments, measured comparisons, machine/infra specs
- **Smarter filtering**: block narration WITHOUT results (not all narration); block binary status without info ("test OK")
- **Extraction priority**: 🥇 learnings > 🥈 measured results > 🥉 durable facts

## [3.1.0] - 2026-03-25
### Fixed — Entity-based Semantic Contradiction Detection
- **Critical fix**: Contradictions between facts with different wording but same entities were not detected
  - Example: "No models on Sol" vs "gemma3:4b installed on Sol" had only 0.23 textual similarity → contradiction check was never called
  - Root cause: Levenshtein+Jaccard gate (threshold 0.7) prevented LLM from seeing semantically related facts with different words
- **New entity extraction**: Extracts proper nouns, tech terms, tool names from facts (Sol, Memoria, Ollama, gemma3, etc.)
- **Entity-based FTS search**: When new fact shares entities with existing facts, triggers LLM contradiction check regardless of text similarity
- **Wider FTS search** (20 candidates per entity) to avoid missing facts ranked beyond top 5
- **Fail-safe**: If entity check fails → fact is stored (never lost)

### Improved — Extraction Prompt
- **Generalization rules**: When a pattern repeats (e.g. "npm not found in SSH" + "ollama not found in SSH"), extract the general rule instead of individual cases
- **Process knowledge**: Explicit instructions to store "how to do X" commands (e.g. "lms server start launches LM Studio without GUI")

### Technical
- `SelectiveMemory` constructor now accepts optional `EmbeddingManager` (4th arg) for future semantic enhancements
- `semanticContradictionThreshold` config option added (default 0.40)
- `extractSubjectEntities()` function with patterns for common tech terms
- `findFactsBySharedEntities()` method for entity-overlap search
- Build order in index.ts: embed providers created before SelectiveMemory instantiation

## [3.0.0] - 2026-03-25
### Added — Phase 2: Semantic/Episodic Memory
- **fact_type column**: `semantic` (durable, slow decay 30-90 days) vs `episodic` (dated, fast decay 7-14 days)
- **Extraction prompt rewritten**: explicit STOCKER/NE PAS STOCKER rules, LLM now classifies fact type
- **TODO/action filter**: blocks transient facts ("il faut X", "en préparation", "prochaine étape")
- Auto-migration adds `fact_type` column to existing DBs

### Added — Phase 3: Observations (Living Syntheses)
- **Observation layer**: inspired by Hindsight, multi-fact synthesis that evolves
- Observations are **created** when 3+ facts share a topic (auto-emergence via LLM topic extraction)
- Observations are **updated** (re-synthesized) when new related facts arrive
- **Recall priority**: Observations injected FIRST, individual facts as complement
- Each observation tracks `evidence_ids`, `revision` count, `access_count`, embedding
- Matching via embedding cosine similarity + keyword fallback
- Configurable: `emergenceThreshold`, `matchThreshold`, `maxRecallObservations`, `maxEvidencePerObservation`

### Added — Phase 4: Recall Adaptatif
- Observations count adjusts to context window (recallLimit / 3, min 2)
- Individual facts fill remaining budget after observations
- Format splits into "Observations (synthèses vivantes)" + "Faits individuels"

### Added — Procedural Memory Preservation
- **Procedural memory** (procedural): like learning bike tricks — processes, tips, "what worked" are preserved as durable knowledge
- **Smart TODO filter**: distinguishes disposable TODOs ("pull X") from learned processes ("use VACUUM INTO because WAL copies lose -shm")
- Heuristics: length >60 chars usually = knowledge → keep; explanation markers (car/sinon/pour/because/→) → always keep
- Transient patterns (en préparation, en cours, pas encore) only skip short facts

### Fixed
- **CRITICAL: `api.config` vs `api.pluginConfig`** — all custom settings were silently ignored since v0.1.0
- Fallback `provider` vs `type` normalization in parseConfig

## [2.7.0] - 2026-03-25
### Added
- **Interactive wizard in install.sh** — 2-question guided install: "Local or Cloud?" → "Fallback or strict?". Detects environment (Ollama, LM Studio, OpenAI key), shows summary, asks confirmation.
- **Presets for silent install** — `--preset local-only|cloud-first|paranoid` for CI/scripting. Also `--yes` to skip confirmation.
- **Post-install validation** — Tests LLM provider after install (quick Ollama smoke test).
- **Bilingual installer** — French interface for better UX (target market).

### Fixed
- **CRITICAL: `api.config` vs `api.pluginConfig`** — Plugin was reading global OpenClaw config instead of plugin-specific config. ALL custom settings (fallback, llm, embed, limits) were silently ignored since v0.1.0. Fixed to use `api.pluginConfig`.
- **Fallback `provider` vs `type` mapping** — User config uses `provider` field but internal code expected `type`. Added normalization in parseConfig.

### Changed
- install.sh rewritten as interactive wizard with environment detection and guided choices.
- Config generated based on user choices (not hardcoded defaults).

## [2.6.1] - 2026-03-25
### Added
- **Auto-config in install.sh** — The installer now auto-edits `openclaw.json`: adds memoria to `plugins.entries` and `plugins.allow` with a backup of the original file. Users keep full control to customize after.
- **Existing data detection** — install.sh detects cortex.db, memoria.db, or facts.json and shows migration status (fact count, file size).
- **Summary panel** — install.sh now displays version, location, config path, LLM/embed info at the end.
- **Node.js/npm version display** — Shows detected versions during prerequisite check.

### Fixed
- **WAL-mode migration** — `VACUUM INTO` used instead of `cp` for cortex.db→memoria.db migration. Plain `cp` on WAL-mode SQLite DBs resulted in empty copies (0 facts). Fallback copies WAL+SHM files if VACUUM fails.
- **Empty DB override** — Migration now triggers if memoria.db exists but is < 8KB (empty schema-only DB from a failed previous attempt).

### Changed
- install.sh rewritten: auto-config replaces manual "copy-paste this JSON" step.
- INSTALL.md updated to document auto-config, WAL migration, and data detection.

## [2.6.0] - 2026-03-25
### Added
- **`install.sh`** — One-line installer: checks prerequisites, pulls Ollama models, clones repo, installs deps. Usage: `curl -fsSL https://raw.githubusercontent.com/Primo-Studio/openclaw-memoria/main/install.sh | bash`
- **Auto-migration cortex→memoria** — If `memoria.db` doesn't exist but `cortex.db` does, auto-copies it. Zero manual migration needed.

### Fixed
- **Schema too strict** — `additionalProperties` changed from `false` to `true` everywhere. Unknown config keys no longer crash the gateway.
- **`syncMd` type** — Was rejecting `{ enabled: true }` objects. Now only accepts boolean as documented, and schema makes it clear.
- **`embed.dims` vs `embed.dimensions`** — Schema now documents `dimensions` clearly with defaults shown.
- **`fallback[].type` vs `fallback[].provider`** — Schema field is `provider`, not `type`.
- **`llm.default` doesn't exist** — Schema clearly shows `llm.provider` + `llm.model` at top level.
- **DB constructor confusion** — `MemoriaDB()` takes workspace root, not DB path. Documented + auto-migration handles legacy DB name.

### Changed
- **Smart defaults everywhere** — `{ "memoria": { "enabled": true } }` is now a valid minimal config. Defaults: Ollama + gemma3:4b + nomic-embed-text-v2-moe + 768 dims + recall 12 + capture 8.
- Schema defaults added to all fields for documentation.
- INSTALL.md rewritten with config minimale, bugs connus, et providers table.

## [2.5.0] - 2026-03-25
### Added
- **Hot Tier**: facts accessed ≥5 times = always injected in recall, like a phone number you know by heart. New `getHotFacts()` in scoring, `hotFacts()` in DB.
- **Access-based learning**: `accessBoostFactor` tripled (0.1 → 0.3) — frequently used facts score much higher, mimicking human memory retention through repetition.
- **Configurable defaults raised**: `captureMaxFacts` 3→8, `recallLimit` 8→12, `maxFacts` 10→12. Users with smaller context windows can lower these in config.

### Changed
- Recall pipeline now: hot tier (always first) → hybrid search → graph → topics → context tree → budget limit
- Hot facts excluded from search results to avoid duplicates
- `searchLimit` = `recallLimit - hotCount` so hot facts don't eat into query-relevant slots

## [2.4.0] - 2026-03-25
### Added
- **Embed Fallback** (`embed-fallback.ts`): `EmbedFallback` wraps multiple `EmbedProvider`s with automatic retry (Ollama → LM Studio → OpenAI). If primary embed fails, tries next provider.
- **Post-processing function** `postProcessNewFacts()`: shared between `agent_end` and `after_compaction` hooks — embed, graph extract, topic tag, sync .md, auto md-regen.
- **Auto md-regen**: triggers automatically when any .md file exceeds 200 lines after capture. Bounded regeneration (30d recent, 150 max/file).

### Fixed
- **after_compaction incomplete** ✅: compaction-rescued facts now get full enrichment (embed + graph + topics + sync + regen) — same pipeline as agent_end.
- **Embed no fallback** ✅: EmbedFallback chains configured embed provider → LM Studio → OpenAI (if API key available).
- **md-regen manual only** ✅: now auto-triggered in postProcessNewFacts when file size threshold exceeded.

### Changed
- Post-processing code extracted from agent_end into reusable `postProcessNewFacts()` function
- Log messages now include `[capture]` or `[compaction]` source label

## [2.3.0] - 2026-03-25
### Added
- **Per-layer LLM config**: each layer (extract, contradiction, graph, topics, contextTree) can use a different model/provider
- `llm.overrides` config section with per-layer `{ provider, model, baseUrl?, apiKey? }`
- Override chains include the user's chosen model as primary, then fallback to the default chain
- Boot log shows active overrides when configured
- JSON Schema `$defs/layerLlm` in manifest for validation

### Changed
- `FallbackChain` now implements `LLMProvider` interface directly (`generate()` → string, `generateWithMeta()` → metadata)
- All modules receive FallbackChain (full fallback) instead of `chain.primaryLLM` (Ollama-only)
- `selective` uses `contradictionLlm`, `graph` uses `graphLlm`, `topics` uses `topicsLlm`, `contextTree` uses `contextTreeLlm`, extract uses `extractLlm`

### Fixed
- Fallback gap: selective, graph, topics, context-tree had NO fallback (Ollama-only). Now all have full chain.

## [2.2.0] - 2026-03-25
### Added
- Phase 9: `.md Vivants` — bounded markdown regeneration (recent 30d, max 150/file, archive notice)
- `MdRegenManager` class with configurable regen settings
- Boot-time .md file size logging

## [2.1.0] - 2026-03-25
### Added
- Phase 8: `Topics Émergents` — auto-clustering from keyword patterns
- `TopicManager` class with keyword extraction, emergence scanning, sub-topics
- Topic embeddings (mean of fact embeddings, cosine search)
- Topic enrichment in recall pipeline (after graph, before context tree)
- Bootstrap script for initial tagging (389/438 facts tagged → 94 topics)
- `topics` + `fact_topics` tables in SQLite schema

## [2.0.0] - 2026-03-25
### Added
- Phase 10: `Fallback Chain` — graceful LLM degradation (Ollama → OpenAI → LM Studio → FTS-only)
- `FallbackChain` class with round-robin retry and configurable providers

## [1.0.0] - 2026-03-25
### Added
- Phase 7: `Budget Adaptatif` — dynamic recall limit based on context usage (light/medium/heavy/critical zones)
- Phase 7: `Sync .md` — auto-append new facts to mapped workspace markdown files
- `AdaptiveBudget` class with configurable thresholds
- `MdSync` class with dedup (first 60 chars check)

## [0.5.0] - 2026-03-25
### Added
- Phase 6: `Context Tree` — hierarchical fact organization with query-weighted scoring
- `ContextTreeBuilder` class with category clustering and sub-clustering

## [0.4.0] - 2026-03-25
### Added
- Phase 5: `Knowledge Graph + Hebbian Learning` — entity extraction, relation storage, BFS traversal
- `KnowledgeGraph` class with graph extraction prompts and Hebbian reinforcement
- Partial/fuzzy entity matching

## [0.3.0] - 2026-03-25
### Added
- Phase 4: `Embeddings + Hybrid Search` — cosine similarity with local Ollama embeddings
- `EmbeddingManager` class with batch embedding, hybrid search (FTS + cosine + temporal)

### Fixed
- FTS5 query sanitization (hyphenated terms crash)

## [0.2.0] - 2026-03-25
### Added
- Phase 2: `Mémoire Sélective` — dedup (Levenshtein + Jaccard), contradiction check via LLM, importance threshold, enrichment/merge
- `SelectiveMemory` class with configurable thresholds

## [0.1.0] - 2026-03-25
### Added
- Phase 1: Core SQLite + FTS5, temporal scoring, perception hooks
- `MemoriaDB` class, migration from facts.json (423 facts)
- Provider abstraction (Ollama, OpenAI-compat, LM Studio)

<!-- v3.14.x entries moved to proper chronological position above -->

## 3.22.0 — 2026-03-28

### Added
- **Layer 21: Continuous Learning** — real-time fact capture via `message_received` + `llm_output` hooks
  - Rolling buffer of last 10 user/assistant exchanges
  - Periodic extraction every 4 turns (configurable via `continuous.interval`)
  - **Urgent extraction** triggered immediately on frustration/error signals (e.g. "ne fais plus", "doublon", "crash")
  - **Self-error detection** — captures when assistant acknowledges its own mistake
  - Uses same LLM extraction prompt and selective dedup pipeline as agent_end
  - Independent of context size, compaction, or session end — works with 60K, 200K, or 10M token contexts


path: CODE_OF_CONDUCT.md
content: # Code of Conduct

## Our Pledge

We are committed to providing a friendly, safe and welcoming environment for all contributors, regardless of experience level, gender identity, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, or nationality.

## Our Standards

**Positive behavior:**
- Being respectful and inclusive
- Giving and accepting constructive feedback gracefully
- Focusing on what is best for the community
- Showing empathy toward other community members

**Unacceptable behavior:**
- Harassment, trolling, or personal attacks
- Publishing others' private information without consent
- Any conduct that would be considered inappropriate in a professional setting

## Enforcement

Instances of unacceptable behavior may be reported to **[REDACTED_SECRET]**. All complaints will be reviewed and investigated promptly.

## Attribution

This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org/), version 2.1.


path: CONTRIBUTING.md
content: # Contributing to Memoria

Thanks for your interest in contributing! 🧠

## Getting Started

1. Fork the repo
2. Clone your fork:
   ```bash
   git clone https://github.com/YOUR_USERNAME/openclaw-memoria.git
   cd openclaw-memoria
   ```
3. Install dependencies:
   ```bash
   npm install
   ```
4. Run tests:
   ```bash
   npx tsx tests/test-core.ts
   ```

## Development Setup

### Requirements
- Node.js 20+
- Ollama with `gemma3:4b` and `nomic-embed-text-v2-moe` (for full integration tests)
- OpenClaw (for plugin testing)

### Running in OpenClaw
1. Symlink or copy the extension into `~/.openclaw/extensions/memoria/`
2. Add to `openclaw.json`:
   ```json
   { "plugins": { "allow": ["memoria"], "entries": { "memoria": { "enabled": true } } } }
   ```
3. Restart gateway: `openclaw gateway restart`

## What to Contribute

### High-impact areas
- **New LLM providers** — implement `LLMProvider` interface in `providers/`
- **Better extraction prompts** — improve fact quality from conversations
- **Benchmark datasets** — more diverse test scenarios in `benchmarks/`
- **Contradiction detection** — improve stale fact supersession
- **Performance** — optimize FTS5/embedding queries for large DBs (10K+ facts)

### Good first issues
- Add tests for individual modules (graph, topics, observations)
- Improve `.md` regeneration formatting
- Add provider for Groq, Together, Mistral API

## Code Style

- TypeScript (strict when possible)
- Imports use `.js` extension (ESM resolution)
- Functions documented with JSDoc where non-obvious
- Error handling: prefer graceful degradation over throwing

## Pull Request Process

1. Create a branch: `git checkout -b feat/my-feature`
2. Write tests if applicable
3. Run existing tests: `npx tsx tests/test-core.ts`
4. Commit with clear message: `feat: add Groq provider support`
5. Open PR against `main` with:
   - What changed and why
   - Test results
   - Breaking changes (if any)

## Commit Convention

```
feat: new feature
fix: bug fix
docs: documentation only
perf: performance improvement
test: adding tests
refactor: code restructuring
```

## License

By contributing, you agree that your contributions will be licensed under Apache 2.0.


path: INSTALL.md
content: # Installation Memoria

## Option A : Installation automatique (recommandé)

```bash
curl -fsSL https://raw.githubusercontent.com/Primo-Studio/openclaw-memoria/main/install.sh | bash
```

Le script :
1. Vérifie les prérequis (Node.js, npm, Ollama)
2. Pull les modèles Ollama (gemma3:4b + nomic-embed-text-v2-moe)
3. Clone le repo et installe les dépendances
4. **Auto-configure `openclaw.json`** (ajoute memoria aux plugins avec backup)
5. Détecte les données existantes (cortex.db, memoria.db, facts.json)

Le client garde le contrôle total : toute config peut être modifiée après dans `openclaw.json`.

## Option B : Installation manuelle

### Prérequis

- **OpenClaw** installé et fonctionnel
- **Node.js** ≥ 20 avec `npm` dans le PATH
- **Ollama** installé ([ollama.ai](https://ollama.ai))

### 1. Installer les modèles Ollama

```bash
ollama pull gemma3:4b              # LLM extraction (3.3 GB)
ollama pull nomic-embed-text-v2-moe  # Embeddings (957 MB)
```

Vérifier : `ollama list` doit afficher les deux modèles.

### 2. Installer le plugin

```bash
git clone https://github.com/Primo-Studio/openclaw-memoria.git \
  ~/.openclaw/extensions/memoria

cd ~/.openclaw/extensions/memoria
npm install
```

### 3. Configurer openclaw.json

**Config minimale** — tout le reste a des defaults intelligents :

```json
{
  "plugins": {
    "allow": ["memoria"],
    "entries": {
      "memoria": { "enabled": true }
    }
  }
}
```

**Config complète** (si vous voulez personnaliser) :

```json
{
  "memoria": {
    "enabled": true,
    "config": {
      "autoRecall": true,
      "autoCapture": true,
      "recallLimit": 12,
      "captureMaxFacts": 8,
      "defaultAgent": "koda",
      "contextWindow": 200000,
      "syncMd": true,
      "llm": {
        "provider": "ollama",
        "model": "gemma3:4b"
      },
      "embed": {
        "provider": "ollama",
        "model": "nomic-embed-text-v2-moe",
        "dimensions": 768
      }
    }
  }
}
```

### 4. Vérifier et démarrer

```bash
openclaw doctor          # Vérifier la config
openclaw gateway restart # Redémarrer
openclaw status          # Vérifier le chargement
```

Vous devez voir :
```
[plugins] memoria: v3.22.3 registered (X facts, Y observations, ... continuous: every 4 turns)
```

### 5. Migration automatique

**Depuis cortex.db** : Memoria détecte automatiquement un ancien `cortex.db` et migre les données en `memoria.db` au premier démarrage. Zéro action nécessaire. Utilise `VACUUM INTO` pour gérer les DB en mode WAL.

**Depuis facts.json** (memory-convex) :

```bash
cd ~/.openclaw/extensions/memoria
npx tsx migrate.ts
```

---

## Bugs connus à l'installation

### ❌ `syncMd` doit être un boolean

**Erreur** : `plugins.entries.memoria.config.syncMd: must be boolean`
**Cause** : écrire `"syncMd": { "enabled": true }` au lieu de `"syncMd": true`
**Fix** : `"syncMd": true`

### ❌ `embed.dims` n'existe pas

**Erreur** : `must NOT have additional properties`
**Cause** : le champ s'appelle `dimensions`, pas `dims`
**Fix** : `"dimensions": 768`

### ❌ `llm.default` n'existe pas

**Erreur** : `must NOT have additional properties`
**Cause** : les champs `provider` et `model` sont directement dans `llm`, pas dans `llm.default`
**Fix** :
```json
"llm": {
  "provider": "ollama",
  "model": "gemma3:4b"
}
```

### ❌ `fallback[].type` n'existe pas

**Erreur** : propriété inconnue
**Cause** : le champ s'appelle `provider`, pas `type`
**Fix** : `{ "provider": "ollama", "model": "gemma3:4b" }`

### ❌ DB path = workspace root, pas le fichier

Le constructeur `MemoriaDB()` attend le **workspace root** (ex: `~/.openclaw/workspace`).
Il crée automatiquement `memory/memoria.db` dedans.
Ne pas passer le chemin de la DB directement.

### ⚠️ Ollama modèles = 0 malgré process running

**Symptôme** : `ollama list` retourne vide, mais le process tourne
**Cause** : Ollama app lancée mais aucun modèle pull
**Fix** : `ollama pull gemma3:4b && ollama pull nomic-embed-text-v2-moe`

### ⚠️ `npm` / `node` not found via SSH

**Cause** : SSH ne charge pas le PATH complet (brew, nvm, etc.)
**Fix** : `export PATH=/opt/homebrew/bin:$PATH` avant les commandes

### ⚠️ "loaded without install/load-path provenance"

**Cause** : plugin local, pas installé via `openclaw plugin install`
**Impact** : warning non-bloquant, le plugin fonctionne
**Fix** : ajouter dans `plugins.allow` (déjà fait si vous suivez le guide)

---

## Config minimale (copier-coller)

Pour une installation rapide avec Ollama local :

```json
{
  "plugins": {
    "allow": ["memoria"],
    "entries": {
      "memoria": {
        "enabled": true,
        "config": {
          "autoRecall": true,
          "autoCapture": true,
          "syncMd": true,
          "llm": { "provider": "ollama", "model": "gemma3:4b" },
          "embed": { "provider": "ollama", "model": "nomic-embed-text-v2-moe", "dimensions": 768 }
        }
      }
    }
  }
}
```

---

## Providers supportés

| Provider | LLM | Embeddings | Prérequis |
|----------|-----|------------|-----------|
| `ollama` | ✅ | ✅ | Ollama installé, modèles pull |
| `lmstudio` | ✅ | ✅ | LM Studio avec serveur local |
| `openai` | ✅ | ✅ | Clé API OpenAI |
| `openrouter` | ✅ | ❌ | Clé API OpenRouter |
| `anthropic` | ✅ | ❌ | Clé API Anthropic (Claude) |

### Anthropic (Claude API)

Config avec Claude comme LLM d'extraction :
```json
"llm": {
  "provider": "anthropic",
  "model": "claude-haiku-3-5",
  "apiKey": "sk-ant-..."
}
```

Ou en fallback :
```json
"fallback": [
  { "provider": "ollama", "model": "gemma3:4b" },
  { "provider": "anthropic", "model": "claude-haiku-3-5", "apiKey": "sk-ant-..." }
]
```

Note : Anthropic ne supporte pas les embeddings. Utilisez Ollama ou OpenAI pour les embeddings.


path: PHASE-2.2-SUMMARY.md
content: # Phase 2.2 — Extraction memoria-core (TERMINÉE ✅)

## Objectif
Extraire le core de Memoria dans un sous-dossier `core/` pour le rendre **indépendant d'OpenClaw** et publiable sur npm en tant que package standalone.

## Résultat

### Structure créée
```
memoria/
├── core/                     ← Package standalone (@primo-studio/memoria-core)
│   ├── index.ts             ← API publique (Memoria class)
│   ├── package.json         ← npm package config
│   ├── README.md            ← Documentation complète
│   ├── providers/           ← LLM providers (Ollama, OpenAI, Anthropic, LM Studio)
│   └── [28 modules].ts      ← 10,062 LOC (83% du code)
│
├── index.ts                 ← Adapter OpenClaw (318 LOC)
├── recall.ts                ← Hook before_prompt_build
├── continuous.ts            ← Hooks message_received + llm_output
├── capture.ts               ← Hooks agent_end + after_compaction
├── procedural-hooks.ts      ← Hook after_tool_call
├── orchestrator.ts          ← Pipeline post-capture
└── openclaw.d.ts            ← Types OpenClaw
```

### Modules dans core/ (30 fichiers)

**Core logic (18):**
- db.ts, selective.ts, scoring.ts, lifecycle.ts, hebbian.ts
- embeddings.ts, graph.ts, topics.ts, patterns.ts
- observations.ts, fact-clusters.ts, procedural.ts
- feedback.ts, expertise.ts, revision.ts
- context-tree.ts, budget.ts, identity-parser.ts

**Infra (10):**
- extraction.ts, format.ts, fallback.ts, embed-fallback.ts
- config.ts, migrate.ts, sync.ts, md-regen.ts
- audit-v25.ts, bootstrap-topics.ts

**Providers (4):**
- providers/types.ts
- providers/ollama.ts
- providers/openai-compat.ts
- providers/anthropic.ts

### API publique core/index.ts

```typescript
export class Memoria {
  static async init(options: MemoriaInitOptions): Promise<Memoria>
  async store(fact: string, category?: string, confidence?: number): Promise<StoreResult>
  async recall(query: string, options?: RecallOptions): Promise<RecallResult>
  async query(naturalLanguageQuestion: string): Promise<string>
  async stats(): Promise<MemoriaStats>
  close(): void
}
```

### Vérifications

✅ **Structure créée** — core/ existe avec 30 modules
✅ **Imports mis à jour** — tous les fichiers adapter importent depuis `./core/`
✅ **API publique** — core/index.ts expose la classe Memoria
✅ **Package.json** — core/package.json créé (v0.1.0)
✅ **README.md** — documentation complète avec exemples
✅ **ZERO OpenClaw dependency dans core/** — grep confirmé
✅ **TypeScript clean** — `npx tsc --noEmit` retourne 0 erreurs
✅ **Git history preserved** — git mv utilisé (renames détectés)
✅ **Commit créé** — c266521

### Métriques

- **Core:** 10,062 LOC (83% du codebase)
- **Adapters:** 1,785 LOC (17% du codebase)
- **Total:** 11,847 LOC
- **TypeScript errors:** 0
- **Fichiers déplacés:** 30 (git mv)
- **Fichiers créés:** 3 (core/index.ts, core/package.json, core/README.md)

### Prochaines étapes

1. **Push sur GitHub** — `git push origin main` (bloqué réseau, à retry)
2. **Test standalone** — créer un projet test avec `import { Memoria } from './core'`
3. **npm publish** — publier sur npm registry
4. **Documentation** — ajouter examples/ avec code samples
5. **CI/CD** — GitHub Actions pour tests automatiques

## Commit

```
c266521 refactor: Phase 2.2 — extract memoria-core as standalone package

Move 30 modules into core/ subdirectory:
- 20 core modules (db, selective, scoring, lifecycle, etc.)
- 10 infra modules (providers, fallback, config, sync, etc.)

Keep 6 adapter modules at root (OpenClaw hooks):
- index.ts, recall.ts, continuous.ts, capture.ts,
  procedural-hooks.ts, orchestrator.ts

New: core/index.ts — public API class Memoria with:
- Memoria.init(options) — standalone initialization
- store(), recall(), query(), stats(), close()

83% of code now lives in core/ with ZERO OpenClaw dependency.
npm publish @primo-studio/memoria-core will work standalone.
```

## État final

✅ **Phase 2.2 TERMINÉE**
- Structure core/ créée
- API publique fonctionnelle
- 0 erreur TypeScript
- Git history préservé
- Documentation complète
- Prêt pour publication npm

⚠️ **Push bloqué** — problème réseau temporaire, à retry plus tard


path: README.md
content: # 🧠 Memoria — Persistent Memory for OpenClaw

**Memory that thinks like you do.** Your AI assistant remembers what matters, forgets what doesn't, and gets better over time.

**SQLite-backed · Fully local · Zero cloud · 21 memory layers · Human-like architecture**

---

## ✨ What's New in v3.22.3

### 🔄 Continuous Learning — Layer 21 *(v3.22.0)*
Memoria no longer waits for end-of-session to learn. New real-time capture via `message_received` + `llm_output` hooks:
- **3 extraction modes**: periodic (every N turns), urgent (on user frustration/error), self-error (on assistant self-admission)
- **Cross-layer integration**: extracted facts go through the full pipeline (selective dedup → embed → graph → topics → observations → clusters → sync)
- **Smart dedup with agent_end**: avoids double LLM calls when continuous already captured
- 6 bugs fixed across 3 audit rounds (v3.22.0 → v3.22.1 → v3.22.3)
- **Node 24.x compatibility** — fixed CONTINUOUS_ENABLED TDZ crash on gateways with embedded Node 24.x

### 🔍 Deep Audit — 10+6 bugs found & fixed *(v3.21.0–v3.22.3)*
Full code audit revealed critical alignment issues:
- **Hebbian learning was 100% dead** — wrong column names since creation
- **Proactive revision never triggered** — searched for obsolete lifecycle state
- **storeFact() lost 6 columns** on INSERT
- **Concurrent extraction risk** and **buffer never cleared** in continuous learning
- All 21 layers now properly aligned with the actual database schema

### 🧩 Behavioral Patterns *(v3.19.0)*
Detects repeated similar facts and consolidates them into patterns.

### 🔗 Cross-Layer Connections *(v3.20.0)*
- **Feedback → Lifecycle**: facts recalled 5+ times with positive usefulness → auto-promoted to "settled"
- **Hebbian → Topics**: strong entity relations auto-organize topics into parent/child hierarchy
- **Lifecycle → Patterns**: confirmed patterns (5+ occurrences) → settled (never forgotten)

---

## ✨ Core Features

- **21 memory layers** — from text search to procedural memory, knowledge graphs, behavioral patterns, continuous learning, and cross-layer connections
- **Semantic vs Episodic** — durable knowledge decays slowly, dated events fade (like human memory)
- **Lifecycle management** — facts evolve: fresh → settled → dormant (not "mature/archived")
- **Observations** — living syntheses that evolve as new evidence appears
- **Fact Clusters** — entity-grouped summaries with tracked membership
- **Procedural memory** — captures "how to do things" with steps, gotchas, quality scores, and failure reasons
- **Behavioral patterns** — detects repeated preferences and consolidates them
- **Adaptive recall** — injects 2-12 facts based on context load
- **Hot Tier** — frequently accessed facts (5+ recalls) always recalled
- **Feedback loop** — usefulness/recall_count/used_count tracked per fact
- **Cross-layer connections** — feedback→lifecycle, hebbian→topics, lifecycle→patterns
- **Hebbian reinforcement** — knowledge graph relations strengthen on co-occurrence, decay when unused
- **Proactive revision** — settled facts get LLM review and potential update
- **Identity-aware** — parses SOUL.md/USER.md to prioritize relevant facts
- **Expertise specialization** — topic access frequency boosts recall
- **Provider-agnostic** — Ollama, LM Studio, OpenAI, OpenRouter, Anthropic
- **Fallback chain** — if primary LLM fails, next one takes over; if all fail, facts still stored
- **Zero config** — smart defaults, 60-second setup

---

## 🚀 Quick Install

### As Plugin (recommended)
```bash
openclaw plugins install memoria-plugin
```

### From Source (review code first)
```bash
cd ~/.openclaw/extensions
git clone https://github.com/Primo-Studio/openclaw-memoria.git memoria
cd memoria && npm install
```

> 💡 Configure after install via `bash ~/.openclaw/extensions/memoria/configure.sh`

### Minimal manual config

Add to `openclaw.json`:
```json
{
  "plugins": {
    "allow": ["memoria"],
    "entries": {
      "memoria": { "enabled": true }
    }
  }
}
```

Smart defaults: Ollama + gemma3:4b + nomic-embed-text-v2-moe (local, 0€).

See [INSTALL.md](INSTALL.md) for advanced options.

---

## 🏗️ Architecture — 21 Layers

```
┌──────────────────────────────────────────────────────┐
│                   MEMORIA v3.22.3                    │
├──────────────────────────────────────────────────────┤
│                                                      │
│  RECALL (before each response):                      │
│  Budget → Observations → Hybrid Search (FTS5+embed)  │
│  → Knowledge Graph → Topics → Context Tree           │
│  → Lifecycle mult × Expertise boost × Cluster penalty│
│  → Format + Inject                                   │
│                                                      │
│  CAPTURE (after each conversation):                  │
│  LLM Extract → Selective (dedup/contradiction)       │
│  → Embed → Graph → Hebbian → Topics → Observations  │
│  → Clusters → Patterns → Cross-layer connections     │
│  → Sync .md → Auto-regen                             │
│                                                      │
│  CONTINUOUS (real-time, during conversation):         │
│  message_received + llm_output → rolling buffer      │
│  → periodic/urgent/self-error triggers               │
│  → LLM Extract → same pipeline as CAPTURE            │
│                                                      │
│  LEARNING (background):                              │
│  Feedback (usefulness/recall/used) → Lifecycle       │
│  Hebbian (co-occurrence) → Topic hierarchy           │
│  Pattern detection → Consolidation                   │
│  Proactive revision → Fact evolution                 │
│                                                      │
├──────────────────────────────────────────────────────┤
│  SQLite (FTS5 + vectors) · No cloud required         │
└──────────────────────────────────────────────────────┘
```

### Layer Map

| # | Layer | File | LLM? | Purpose |
|---|-------|------|------|---------|
| 1 | SQLite Core + FTS5 | `db.ts` | ❌ | Storage, full-text search |
| 2 | Temporal Scoring | `scoring.ts` | ❌ | Decay, hot tier (5+ accesses) |
| 3 | Selective Memory | `selective.ts` | ✅ | Dedup, contradiction check |
| 4 | Embeddings + Hybrid | `embeddings.ts` | ❌ | Cosine similarity (embed only) |
| 5 | Knowledge Graph | `graph.ts` | ✅ | Entity/relation extraction |
| 6 | Context Tree | `context-tree.ts` | ❌ | Hierarchical organization |
| 7 | Adaptive Budget | `budget.ts` | ❌ | Auto-adjust facts injected |
| 8 | Emergent Topics | `topics.ts` | ✅ | Keyword extraction, topic naming |
| 9 | Observations | `observations.ts` | ✅ | Living syntheses from evidence |
| 10 | Fact Clusters | `fact-clusters.ts` | ✅ | Entity-grouped summaries |
| 11 | .md Sync + Regen | `sync.ts`, `md-regen.ts` | ❌ | Write facts to workspace files |
| 12 | Fallback Chain | `fallback.ts` | all | Ollama → OpenAI → LM Studio |
| 13 | Procedural Memory | `procedural.ts` | ✅ | How-to steps, quality, gotchas |
| 14 | Lifecycle | `lifecycle.ts` | ❌ | fresh → settled → dormant |
| 15 | Feedback Loop | `feedback.ts` | ❌ | usefulness, recall_count, used_count |
| 16 | Hebbian | `hebbian.ts` | ❌ | Strengthen co-occurring relations |
| 17 | Identity Parser | `identity-parser.ts` | ❌ | Parse SOUL.md/USER.md |
| 18 | Expertise | `expertise.ts` | ❌ | Topic access → recall boost |
| 19 | Proactive Revision | `revision.ts` | ✅ | Revise settled facts via LLM |
| 20 | Behavioral Patterns | `patterns.ts` | ✅ | Detect + consolidate repetitions |
| 21 | Continuous Learning | `index.ts` (hooks) | ✅ | Real-time capture during conversation |

**9 layers use LLM** via the Fallback Chain. **12 layers are pure algorithmic.**

For scoring formulas, decay rates, and detailed pipeline descriptions, see [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).

---

## ⚙️ Configuration

```json
{
  "autoRecall": true,
  "autoCapture": true,
  "recallLimit": 12,
  "captureMaxFacts": 8,
  "syncMd": true,

  "llm": {
    "provider": "ollama",
    "model": "gemma3:4b"
  },

  "embed": {
    "provider": "ollama",
    "model": "nomic-embed-text-v2-moe",
    "dimensions": 768
  },

  "fallback": [
    { "provider": "ollama", "model": "gemma3:4b" },
    { "provider": "lmstudio", "model": "auto" }
  ]
}
```

### Supported Providers

| Provider | LLM | Embeddings | Cost |
|----------|-----|------------|------|
| **Ollama** | ✅ | ✅ | Free (local) |
| **LM Studio** | ✅ | ✅ | Free (local) |
| **OpenAI** | ✅ | ✅ | ~$0.50/month |
| **OpenRouter** | ✅ | ✅ | Varies |
| **Anthropic** | ✅ | — | Varies |

---

## 📊 Benchmarks

Tested on LongMemEval-S (30 questions, 5 categories):

| Version | Accuracy | Retrieval | Key improvement |
|---------|----------|-----------|-----------------|
| v3.2.0 | 73% | 50% | Contradiction supersession + procedural |
| v3.3.0 | 75% | 43% | Query expansion + topic recall |
| v3.4.0 | 82% | 50% | Fact Clusters (multi-session +75%) |
| v3.5.0 | 82%+ | 50% | Feedback loop + cross-layer cascade |

> v3.14–3.21 benchmarks pending (Sol benchmark planned).

Detailed methodology and scripts in [benchmarks/](benchmarks/).

---

## 🗺️ Roadmap

| Version | Feature | Status |
|---------|---------|--------|
| v3.0–3.5 | Core layers (1-12): FTS5, scoring, selective, graph, topics, observations, clusters, feedback, cascade | ✅ Done |
| v3.6–3.7 | Identity-aware, lifecycle, hebbian, expertise, procedural | ✅ Done |
| v3.8–3.12 | Procedural quality (reflection, alternatives, gotchas), capture quality, error detection | ✅ Done |
| v3.14–3.17 | Smarter extraction, cluster-aware recall, security/packaging fixes | ✅ Done |
| v3.18 | Cluster members table, topic hierarchy with parent inference | ✅ Done |
| v3.19 | Behavioral pattern detection (Layer 20) | ✅ Done |
| v3.20 | Cross-layer connections (feedback→lifecycle, hebbian→topics, lifecycle→patterns) | ✅ Done |
| v3.21 | Deep audit: 10 bugs fixed, full type alignment, all 20 layers validated | ✅ Done |
| v3.22 | Layer 21: Continuous Learning (real-time capture) + 6 more bug fixes | ✅ Done |
| v3.23+ | Image memory, interest profiles, LCM bridge, Sol/Luna benchmarks | 🔜 Next |

---

## 📄 License

Apache License 2.0 — see [LICENSE](LICENSE).

Copyright 2026 Primo-Studio by Neto Pompeu.


path: SECURITY.md
content: # Security Policy

## Supported Versions

| Version | Supported |
|---------|-----------|
| 3.4.x   | ✅ Active |
| 3.0.x - 3.3.x | ⚠️ Critical fixes only |
| < 3.0   | ❌ Not supported |

## Reporting a Vulnerability

If you discover a security vulnerability in Memoria, please **do not open a public issue**.

Instead, email us at: **[REDACTED_SECRET]**

Include:
- Description of the vulnerability
- Steps to reproduce
- Impact assessment
- Suggested fix (if any)

We will respond within **48 hours** and aim to release a fix within **7 days** for critical issues.

## Security Considerations

### Data Storage
- All data is stored **locally** in SQLite (`memoria.db`) — no cloud sync by default
- API keys in config are **never** sent to third parties (only to the configured LLM providers)
- The `.md` sync files may contain extracted facts — treat your workspace as sensitive

### LLM Provider Security
- Fact extraction sends conversation content to your configured LLM (Ollama = 100% local)
- If using cloud providers (OpenAI, Anthropic), conversation data leaves your machine
- Use `fallback` config carefully — ensure only trusted providers are listed

### Best Practices
- **Never commit API keys** — use environment variables (`OPENAI_API_KEY`)
- **Restrict workspace permissions** — `memoria.db` contains all your agent's memory
- **Backup regularly** — `cp memoria.db memoria.db.bak` (or use `VACUUM INTO`)
- **Review .md sync output** — ensure no sensitive data leaks into shared files

## Disclosure Policy

We follow responsible disclosure. Security researchers who report valid vulnerabilities will be credited in the release notes (unless they prefer to remain anonymous).


path: benchmarks/README.md
content: # Memoria Benchmarks

## Methodology

- **Dataset**: 10 synthetic sessions based on real workspace data (Primo Studio)
- **30 questions** across 6 categories:
  - **SSU** (Single-Session Understanding): fact lookup from one session
  - **SSA** (Single-Session Aggregation): combine facts from one session
  - **SSP** (Single-Session Procedure): how-to questions
  - **KU** (Knowledge Update): handle contradicting/updated information
  - **TR** (Temporal Reasoning): time-based questions
  - **MS** (Multi-Session): cross-session aggregation
- **Pipeline**: Extract (gemma3:4b Ollama) → Embed (nomic-embed-text-v2-moe) → Answer (GPT-OSS 20B LM Studio) → Judge (GPT-5.4-nano OpenAI)
- **Fair**: fresh DB each run, same questions, same models

## Results History

| Version | Accuracy | Retrieval | SSU | SSA | SSP | KU | TR | MS | Key Change |
|---------|----------|-----------|-----|-----|-----|-----|-----|-----|------------|
| v3.1.1 | 41.7% | 90.0% | 3/5 | 2/5 | 3/5 | 2/5 | 1/5 | 1/5 | Baseline (gemma3 judge) |
| v3.2.0 | 75.0% | 53.3% | 4.5/5 | 3/5 | 5/5 | 5/5 | 2.5/5 | 1.5/5 | Supersession + procedures |
| v3.3.0 | 75.0% | 43.3% | 5/5 | 3/5 | 4.5/5 | 5/5 | 3/5 | 2/5 | Query expansion (nano judge) |
| **v3.4.0** | **81.7%** | **50.0%** | **5/5** | **3.5/5** | **5/5** | **5/5** | **3.5/5** | **2.5/5** | **Fact Clusters** |

## Running

```bash
# Requires: Ollama (gemma3:4b + nomic-embed), LM Studio (GPT-OSS 20B), OpenAI API key
python3 benchmarks/bench-v34.py
```

## Files

- `bench-v34.py` — Latest benchmark script (v3.4.0)
- `results/` — JSON results from each run


path: benchmarks/bench-v34.py
content: #!/usr/bin/env python3
"""
LEGACY — Memoria v3.4.0 Benchmark (March 2026)
Targets v3.4.0 with 10 synthetic sessions. Kept for reference.
Current version: v3.22.3 with 21 layers — updated benchmarks TBD.
"""
import json, time, requests, sqlite3, os, re
from collections import defaultdict

OLLAMA = "http://localhost:11434"
LMSTUDIO = "http://localhost:1234/v1"
OPENAI = "https://api.openai.com/v1"
OPENAI_KEY = os.environ.get("OPENAI_API_KEY", "YOUR_KEY_HERE")
EMBED_MODEL = "nomic-embed-text-v2-moe"
EXTRACT_MODEL = "gemma3:4b"
ANSWER_MODEL = "openai/gpt-oss-20b"
JUDGE_MODEL = "gpt-5.4-nano"
DB_PATH = "/tmp/bench-v34-clusters.db"
RESULTS_PATH = f"/tmp/results-v34-{time.strftime('%Y%m%d-%H%M%S')}.json"

# Known entities for clustering
KNOWN_ENTITIES = [
    "Sol", "Luna", "Koda", "Neto", "Bureau", "Convex", "Primask",
    "DockGroups", "Memoria", "Ollama", "Cloudflare", "Vercel",
    "Qonto", "Alexandre", "Pierre", "Primo Studio", "LM Studio"
]

CONCEPT_MAP = {
    "taux horaire": ["€/h", "salaire"],
    "salaire": ["taux horaire", "€/h"],
    "rémunération": ["salaire", "€/h"],
    "ca": ["chiffre d'affaires"],
    "chiffre d'affaires": ["CA"],
    "deploy": ["déploiement"],
    "déploiement": ["deploy"],
    "modèle": ["model"],
    "modèles": ["models"],
    "config": ["configuration"],
    "configuration": ["config"],
}

SESSIONS = [
    {"id": "s1", "messages": [
        "J'ai configuré Ollama sur Sol avec gemma3:4b comme modèle d'extraction.",
        "Les embeddings utilisent nomic-embed-text-v2-moe avec 768 dimensions.",
        "Le fallback chain est Ollama → LM Studio → OpenAI."
    ]},
    {"id": "s2", "messages": [
        "Bureau utilise Convex comme backend avec des subscriptions useQuery en temps réel.",
        "Le module CRM gère 11 structures : entreprises, associations et collectivités.",
        "La sync Qonto se fait via une action Convex, pas un script bash."
    ]},
    {"id": "s3", "messages": [
        "Neto travaille depuis la Guyane française, timezone America/Cayenne (GMT-3).",
        "Il préfère le step-by-step et déteste les régressions.",
        "Sa machine principale est un Mac Studio avec 64 Go de RAM."
    ]},
    {"id": "s4", "messages": [
        "Primask est une app de planning développée par Primo Studio.",
        "Le deploy se fait via GitHub → Vercel avec auto-deploy.",
        "Le token Hello-Primo est utilisé pour les projets Bureau sur Vercel."
    ]},
    {"id": "s5", "messages": [
        "Luna est un iMac qui gère les emails et le calendrier via CalDAV iCloud.",
        "Sol est un Mac Mini disponible 24/7 pour les tâches de dev.",
        "Koda est le dev AI senior, promu le 22 mars 2026."
    ]},
    {"id": "s6", "messages": [
        "Le benchmark LongMemEval-S teste 6 catégories : SSU, SSA, SSP, KU, TR, MS.",
        "Le retrieval rate de Memoria est de 93.3% avec gemma3:4b.",
        "KU (Knowledge Update) est le point faible : 0/5 correct."
    ]},
    {"id": "s7", "messages": [
        "DockGroups est une app macOS menu bar pour organiser le Dock.",
        "Le drag & drop a été retiré car instable dans MenuBarExtra.",
        "Version actuelle : v0.5.0 avec apply-to-dock sécurisé via SIGHUP."
    ]},
    {"id": "s8", "messages": [
        "Memoria v3.2.0 ajoute le support des modèles à reasoning (Ollama/OpenAI-compat).",
        "Le recall inclut maintenant les dates avec guidance de recency.",
        "L'extraction procédurale permet les faits multi-phrases.",
        "Le hybridSearch est adaptatif : plus de poids cosine pour les queries courtes."
    ]},
    {"id": "s9", "messages": [
        "Alexandre est en amélioration, son taux horaire est de 5.19€/h.",
        "Pierre a terminé son contrat, non renouvelé. Son taux était de 7.39€/h.",
        "Le CA 2025 était de 111 223€, objectif 2026 : 80-100K€.",
        "Update : Alexandre a été augmenté à 6.50€/h suite à sa progression."
    ]},
    {"id": "s10", "messages": [
        "Cloudflare gère le DNS de primo-studio.fr avec proxy vers Vercel (front) et tunnel vers NAS (Directus).",
        "Le zone ID Cloudflare est 403c7dc0dfe5c1ec6e94d92d8d0765ba.",
        "Transport Rino est un MVP offline-first avec signatures intégrées au parcours terrain."
    ]}
]

QUESTIONS = [
    {"q": "Quel modèle d'extraction est configuré sur Sol ?", "expected": "gemma3:4b", "cat": "SSU", "session": "s1"},
    {"q": "Combien de structures gère le CRM Bureau ?", "expected": "11", "cat": "SSU", "session": "s2"},
    {"q": "Quelle est la timezone de Neto ?", "expected": "America/Cayenne", "cat": "SSU", "session": "s3"},
    {"q": "Comment se fait le deploy de Primask ?", "expected": "GitHub vers Vercel auto-deploy", "cat": "SSU", "session": "s4"},
    {"q": "Quel est le rôle de Luna ?", "expected": "iMac, emails et calendrier", "cat": "SSU", "session": "s5"},
    {"q": "Quels sont les 3 niveaux du fallback chain de Memoria ?", "expected": "Ollama, LM Studio, OpenAI", "cat": "SSA", "session": "s1"},
    {"q": "Quels types de structures le CRM Bureau gère-t-il ?", "expected": "entreprises, associations, collectivités", "cat": "SSA", "session": "s2"},
    {"q": "Quelles sont les 3 machines de l'équipe et leurs rôles ?", "expected": "Mac Studio (Neto), iMac Luna (emails/cal), Mac Mini Sol (dev 24/7)", "cat": "SSA", "session": "s5"},
    {"q": "Quelles features a apporté Memoria v3.2.0 ?", "expected": "reasoning models, dated recall, procedures multi-phrases, adaptive hybridSearch", "cat": "SSA", "session": "s8"},
    {"q": "Quels services Cloudflare fournit pour primo-studio.fr ?", "expected": "DNS, proxy Vercel front, tunnel NAS Directus", "cat": "SSA", "session": "s10"},
    {"q": "Comment configurer le fallback chain Memoria ?", "expected": "Ollama → LM Studio → OpenAI dans la config", "cat": "SSP", "session": "s1"},
    {"q": "Comment synchroniser Qonto avec Bureau ?", "expected": "via action Convex syncQonto, pas script bash", "cat": "SSP", "session": "s2"},
    {"q": "Quelle est la procédure pour déployer sur Vercel ?", "expected": "push GitHub déclenche auto-deploy", "cat": "SSP", "session": "s4"},
    {"q": "Comment appliquer les groupes au Dock dans DockGroups ?", "expected": "apply-to-dock sécurisé via SIGHUP", "cat": "SSP", "session": "s7"},
    {"q": "Comment fonctionne le hybridSearch adaptatif ?", "expected": "plus de poids cosine pour queries courtes, plus FTS pour queries longues", "cat": "SSP", "session": "s8"},
    {"q": "Quel est le taux horaire actuel d'Alexandre ?", "expected": "6.50€/h", "cat": "KU", "session": "s9"},
    {"q": "Pierre travaille-t-il encore chez Primo Studio ?", "expected": "non, contrat terminé", "cat": "KU", "session": "s9"},
    {"q": "Le drag & drop fonctionne-t-il dans DockGroups ?", "expected": "non, retiré car instable", "cat": "KU", "session": "s7"},
    {"q": "Quelle est la version actuelle de Memoria ?", "expected": "v3.2.0", "cat": "KU", "session": "s8"},
    {"q": "Quel est le score KU du benchmark Memoria ?", "expected": "0/5 correct", "cat": "KU", "session": "s6"},
    {"q": "Quand Koda a-t-il été promu Dev Senior ?", "expected": "22 mars 2026", "cat": "TR", "session": "s5"},
    {"q": "Quelle version de DockGroups a introduit apply-to-dock sécurisé ?", "expected": "v0.5.0", "cat": "TR", "session": "s7"},
    {"q": "Le CA a-t-il augmenté ou baissé entre 2025 et l'objectif 2026 ?", "expected": "baissé (111K → 80-100K objectif)", "cat": "TR", "session": "s9"},
    {"q": "Quel était le retrieval rate avant les améliorations v3.2.0 ?", "expected": "93.3%", "cat": "TR", "session": "s6"},
    {"q": "Avant le reasoning support, que se passait-il avec les modèles thinking ?", "expected": "le thinking consommait les tokens, réponse vide/pas de JSON", "cat": "TR", "session": "s8"},
    {"q": "Quelles machines utilisent Memoria ?", "expected": "Mac Studio (Koda) et Mac Mini (Sol)", "cat": "MS", "session": ""},
    {"q": "Quel est le lien entre Bureau et Qonto ?", "expected": "sync via action Convex, matching auto virements↔projets", "cat": "MS", "session": ""},
    {"q": "Quels sont les projets actifs de Primo Studio ?", "expected": "Bureau, Primask, DockGroups, Transport Rino, Memoria", "cat": "MS", "session": ""},
    {"q": "Quels modèles LLM sont disponibles sur Sol ?", "expected": "gemma3:4b, nomic-embed, qwen3.5:27b, GPT-OSS 20B via LM Studio", "cat": "MS", "session": ""},
    {"q": "Quels taux horaires sont pratiqués chez Primo Studio ?", "expected": "Neto 0€, Alexandre 6.50€, Pierre 7.39€ (parti)", "cat": "MS", "session": ""}
]

def expand_query(query):
    variants = [query]
    lower = query.lower()
    for key, synonyms in CONCEPT_MAP.items():
        if key in lower:
            for syn in synonyms[:2]:
                variant = re.sub(re.escape(key), syn, query, flags=re.IGNORECASE)
                if variant != query and variant not in variants:
                    variants.append(variant)
    for m in re.finditer(r'\b[A-Z][a-z]+(?:\s[A-Z][a-z]+)*', query):
        noun = m.group()
        if len(noun) > 2 and noun not in variants:
            variants.append(noun)
    return variants[:4]

def ollama_generate(prompt, model=EXTRACT_MODEL, fmt="json", timeout=120):
    body = {"model": model, "prompt": prompt, "stream": False,
            "options": {"temperature": 0.1, "num_predict": 2048, "num_ctx": 8192}}
    if fmt: body["format"] = fmt
    r = requests.post(f"{OLLAMA}/api/generate", json=body, timeout=timeout)
    r.raise_for_status()
    return r.json().get("response", "")

def ollama_generate_text(prompt, model=EXTRACT_MODEL, timeout=60):
    """Generate plain text (no JSON format constraint)"""
    body = {"model": model, "prompt": prompt, "stream": False,
            "options": {"temperature": 0.1, "num_predict": 300, "num_ctx": 4096}}
    r = requests.post(f"{OLLAMA}/api/generate", json=body, timeout=timeout)
    r.raise_for_status()
    return r.json().get("response", "").strip()

def ollama_embed(texts):
    r = requests.post(f"{OLLAMA}/api/embed", json={"model": EMBED_MODEL, "input": texts}, timeout=60)
    r.raise_for_status()
    return r.json().get("embeddings", [])

def lmstudio_chat(prompt, timeout=120):
    body = {"model": ANSWER_MODEL, "messages": [{"role": "user", "content": prompt}],
            "max_tokens": 500, "temperature": 0.1}
    r = requests.post(f"{LMSTUDIO}/chat/completions", json=body, timeout=timeout)
    r.raise_for_status()
    data = r.json()
    msg = data["choices"][0]["message"]
    return msg.get("content") or msg.get("reasoning_content") or msg.get("reasoning") or ""

def nano_judge(question, expected, answer):
    body = {
        "model": JUDGE_MODEL,
        "messages": [
            {"role": "system", "content": "You are a fair judge evaluating answer quality. Be lenient: if the answer contains the key information from the expected answer, even with extra details or different wording, mark it as correct. Only mark wrong if the core information is missing or contradicted."},
            {"role": "user", "content": f"""Judge if this answer is correct.

Question: {question}
Expected answer: {expected}
Actual answer: {answer}

Return ONLY valid JSON: {{"verdict": "correct"|"partial"|"wrong", "reason": "brief explanation"}}"""}
        ],
        "max_completion_tokens": 150,
        "response_format": {"type": "json_object"}
    }
    try:
        r = requests.post(f"{OPENAI}/chat/completions", json=body, timeout=30,
                         headers={"Authorization": f"Bearer {OPENAI_KEY}", "Content-Type": "application/json"})
        r.raise_for_status()
        content = r.json()["choices"][0]["message"]["content"]
        j = json.loads(content)
        return j.get("verdict", "wrong"), j.get("reason", "")
    except Exception as e:
        return "wrong", f"judge error: {e}"

def cosine_sim(a, b):
    dot = sum(x*y for x,y in zip(a,b))
    na = sum(x*x for x in a)**0.5
    nb = sum(x*x for x in b)**0.5
    return dot/(na*nb) if na and nb else 0

def setup_db():
    if os.path.exists(DB_PATH):
        os.remove(DB_PATH)
    db = sqlite3.connect(DB_PATH)
    db.execute("""CREATE TABLE facts (
        id INTEGER PRIMARY KEY, fact TEXT, category TEXT, fact_type TEXT DEFAULT 'semantic',
        confidence REAL DEFAULT 0.8, session_id TEXT, source TEXT DEFAULT 'auto-capture',
        created_at TEXT DEFAULT CURRENT_TIMESTAMP,
        superseded_by INTEGER, embedding BLOB)""")
    db.execute("CREATE VIRTUAL TABLE IF NOT EXISTS facts_fts USING fts5(fact, content=facts, content_rowid=id)")
    db.execute("""CREATE TRIGGER facts_ai AFTER INSERT ON facts BEGIN
        INSERT INTO facts_fts(rowid, fact) VALUES (new.id, new.fact); END""")
    db.commit()
    return db

def extract_facts(text, session_id):
    prompt = f"""Extract durable facts from this conversation. Return JSON array.
IMPORTANT: Create ONE FACT PER DISTINCT ENTITY (person, tool, project, machine).

Keys: fact, category (savoir/outil/erreur/preference/chronologie/rh/client), fact_type (semantic or episodic), confidence (0.0-1.0).

Text:
{text}

Return ONLY a JSON array:"""
    raw = ollama_generate(prompt)
    try:
        data = json.loads(raw)
        if isinstance(data, dict) and "facts" in data:
            data = data["facts"]
        if isinstance(data, list):
            if data and isinstance(data[0], list):
                data = data[0]
            return [f for f in data if isinstance(f, dict) and "fact" in f]
        return []
    except:
        return []

def generate_clusters(db):
    """NEW in v3.4.0: Group facts by entity, generate cluster summaries"""
    # Group facts by entity
    rows = db.execute("SELECT id, fact, category, fact_type FROM facts WHERE superseded_by IS NULL AND fact_type != 'cluster'").fetchall()
    
    entity_facts = defaultdict(list)
    for row in rows:
        fid, fact, cat, ftype = row
        # Match known entities
        for entity in KNOWN_ENTITIES:
            if entity.lower() in fact.lower():
                entity_facts[entity].append({"id": fid, "fact": fact, "category": cat})
    
    clusters_created = 0
    for entity, facts in entity_facts.items():
        if len(facts) < 3:
            continue
        
        facts_text = "\n".join(f"- [{f['category']}] {f['fact']}" for f in facts[:12])
        prompt = f"""Tu résumes un groupe de faits liés à la même entité en UN SEUL paragraphe dense.
Règles: contenir TOUTES les informations clés (noms, chiffres, dates, versions, états). 2-4 phrases max, dense et factuel, en français.

Entité: {entity}

Faits:
{facts_text}

Résumé dense:"""
        
        try:
            summary = ollama_generate_text(prompt)
            if summary and len(summary) > 20:
                # Remove any JSON/markdown artifacts
                summary = summary.strip().strip('"').strip("'")
                if summary.startswith("```"):
                    summary = re.sub(r'^```.*\n?', '', summary)
                    summary = re.sub(r'\n?```$', '', summary)
                
                member_ids = [f["id"] for f in facts[:12]]
                db.execute(
                    "INSERT INTO facts (fact, category, fact_type, confidence, session_id, source) VALUES (?, ?, 'cluster', 0.85, NULL, ?)",
                    (summary, facts[0]["category"], f"cluster:{entity.lower()}")
                )
                clusters_created += 1
        except Exception as e:
            print(f"    ⚠ Cluster {entity} failed: {e}")
    
    db.commit()
    return clusters_created

def hybrid_search_expanded(db, query, top_k=5):
    queries = expand_query(query)
    all_q_embs = ollama_embed(queries)
    results = {}
    
    for qi, q in enumerate(queries):
        tokens = [t for t in q.split() if len(t) > 1]
        if not tokens: continue
        fts_q = " OR ".join(f'"{t}"' for t in tokens[:10])
        try:
            rows = db.execute(
                "SELECT f.id, f.fact, f.category, f.fact_type, f.confidence, f.session_id, f.superseded_by, f.embedding "
                "FROM facts_fts fts JOIN facts f ON fts.rowid = f.id "
                "WHERE facts_fts MATCH ? AND f.superseded_by IS NULL LIMIT 20", (fts_q,)).fetchall()
            for i, r in enumerate(rows):
                if r[0] not in results:
                    results[r[0]] = {"id": r[0], "fact": r[1], "cat": r[2], "type": r[3], "conf": r[4],
                                     "session": r[5], "emb": r[7], "fts_rank": 1.0/(i+1), "cosine": 0}
                else:
                    results[r[0]]["fts_rank"] = max(results[r[0]]["fts_rank"], 1.0/(i+1))
        except:
            pass
    
    rows = db.execute("SELECT id, fact, category, fact_type, confidence, session_id, embedding FROM facts WHERE superseded_by IS NULL AND embedding IS NOT NULL").fetchall()
    for r in rows:
        emb = json.loads(r[6]) if isinstance(r[6], str) else None
        if not emb: continue
        max_sim = 0
        for q_emb in all_q_embs:
            sim = cosine_sim(q_emb, emb)
            if sim > max_sim:
                max_sim = sim
        if max_sim > 0.3:
            if r[0] in results:
                results[r[0]]["cosine"] = max(results[r[0]].get("cosine", 0), max_sim)
            else:
                results[r[0]] = {"id": r[0], "fact": r[1], "cat": r[2], "type": r[3], "conf": r[4],
                                 "session": r[5], "emb": r[6], "fts_rank": 0, "cosine": max_sim}
    
    short_query = len(query.split()) <= 4
    w_fts = 0.20 if short_query else 0.40
    w_cos = 0.55 if short_query else 0.40
    w_conf = 0.25 if short_query else 0.20
    
    scored = []
    for r in results.values():
        score = w_fts * r.get("fts_rank", 0) + w_cos * r.get("cosine", 0) + w_conf * r.get("conf", 0.5)
        # Cluster boost: 15% more weight (info-dense)
        if r.get("type") == "cluster":
            score *= 1.15
        scored.append((score, r))
    
    scored.sort(key=lambda x: -x[0])
    return [(s, r["fact"], r["cat"]) for s, r in scored[:top_k]]

def main():
    print(f"=== Memoria v3.4.0 Benchmark — Fact Clusters + Nano Judge ===")
    print(f"Extraction: {EXTRACT_MODEL} (Ollama) | Answers: {ANSWER_MODEL} (LM Studio)")
    print(f"Embeddings: {EMBED_MODEL} (Ollama) | Judge: {JUDGE_MODEL} (OpenAI)")
    print(f"NEW: Fact Clusters (entity-grouped summaries)")
    print()
    
    # Test judge
    print("🔑 Testing GPT-5.4-nano judge...")
    v, r = nano_judge("What is 2+2?", "4", "The answer is 4.")
    print(f"  Test: verdict={v}")
    if "error" in r.lower():
        print("  ❌ Judge not working, aborting.")
        return
    print()
    
    db = setup_db()
    
    # Phase 1: Extract atomic facts
    print("📥 Phase 1: Ingestion (dense extraction)...")
    t0 = time.time()
    total_facts = 0
    for sess in SESSIONS:
        text = "\n".join(sess["messages"])
        facts = extract_facts(text, sess["id"])
        for f in facts:
            db.execute("INSERT INTO facts (fact, category, fact_type, confidence, session_id) VALUES (?,?,?,?,?)",
                       (f["fact"], f.get("category", "savoir"), f.get("fact_type", "semantic"),
                        f.get("confidence", 0.8), sess["id"]))
            total_facts += 1
        db.commit()
    ingest_time = time.time() - t0
    print(f"  ✅ {total_facts} atomic facts from {len(SESSIONS)} sessions in {ingest_time:.1f}s")
    
    # Phase 2: Generate clusters (NEW in v3.4.0)
    print("\n🧩 Phase 2: Fact Clusters...")
    t0 = time.time()
    num_clusters = generate_clusters(db)
    cluster_time = time.time() - t0
    total_with_clusters = db.execute("SELECT COUNT(*) FROM facts WHERE superseded_by IS NULL").fetchone()[0]
    print(f"  ✅ {num_clusters} clusters generated in {cluster_time:.1f}s ({total_with_clusters} total facts)")
    
    # Phase 3: Embeddings (atomic + clusters)
    print("\n📐 Phase 3: Embeddings...")
    t0 = time.time()
    rows = db.execute("SELECT id, fact FROM facts WHERE superseded_by IS NULL").fetchall()
    batch_size = 20
    embedded = 0
    for i in range(0, len(rows), batch_size):
        batch = rows[i:i+batch_size]
        texts = [r[1] for r in batch]
        embs = ollama_embed(texts)
        for (rid, _), emb in zip(batch, embs):
            db.execute("UPDATE facts SET embedding = ? WHERE id = ?", (json.dumps(emb), rid))
            embedded += 1
        db.commit()
    embed_time = time.time() - t0
    print(f"  ✅ {embedded}/{len(rows)} embedded in {embed_time:.1f}s")
    
    # Phase 4: Q&A
    print(f"\n❓ Phase 4: Q&A + Nano Judge ({len(QUESTIONS)} questions)...")
    results = []
    cats = {}
    for i, q in enumerate(QUESTIONS):
        t0 = time.time()
        expanded = expand_query(q["q"])
        hits = hybrid_search_expanded(db, q["q"], top_k=5)
        retrieval_hit = any(
            any(kw.lower() in h[1].lower() for kw in q["expected"].split(", ")[:2])
            for h in hits
        )
        context = "\n".join(f"- [{h[2]}] {h[1]}" for h in hits)
        
        prompt = f"""Based on the following facts, answer the question concisely in French.
If you find contradicting information, prefer the most recent one.

Facts:
{context}

Question: {q['q']}

Answer concisely:"""
        try:
            answer = lmstudio_chat(prompt, timeout=120)
        except Exception as e:
            answer = f"[Error: {e}]"
        
        latency = time.time() - t0
        verdict, reason = nano_judge(q["q"], q["expected"], answer)
        
        cat = q["cat"]
        if cat not in cats:
            cats[cat] = {"correct": 0, "partial": 0, "wrong": 0, "retrieval_hits": 0, "total": 0}
        cats[cat]["total"] += 1
        cats[cat][verdict] += 1
        if retrieval_hit:
            cats[cat]["retrieval_hits"] += 1
        
        # Check if a cluster was in the top hits
        has_cluster = any("cluster" in str(h) for h in hits)
        cluster_marker = " 🧩" if has_cluster else ""
        
        status = "✅" if verdict == "correct" else "🟡" if verdict == "partial" else "❌"
        exp_str = f" [+{len(expanded)-1}exp]" if len(expanded) > 1 else ""
        print(f"  {status} Q{i+1} [{cat}] {q['q'][:50]}... → {verdict} ({latency:.1f}s) {'📎' if retrieval_hit else '🔍✗'}{exp_str}{cluster_marker}")
        
        results.append({
            "question": q["q"], "expected": q["expected"], "answer": answer,
            "verdict": verdict, "reason": reason, "category": cat,
            "retrieval_hit": retrieval_hit, "latency_s": round(latency, 2),
            "expanded_queries": expanded,
            "context_facts": [h[1] for h in hits]
        })
    
    total_correct = sum(c["correct"] for c in cats.values())
    total_partial = sum(c["partial"] for c in cats.values())
    total_retrieval = sum(c["retrieval_hits"] for c in cats.values())
    avg_latency = sum(r["latency_s"] for r in results) / len(results)
    acc = (total_correct+total_partial/2)/len(QUESTIONS)*100
    ret = total_retrieval/len(QUESTIONS)*100
    
    print(f"\n{'='*60}")
    print(f"📊 RÉSULTATS — Memoria v3.4.0 + Fact Clusters")
    print(f"{'='*60}")
    print(f"  Facts: {total_facts} atomic + {num_clusters} clusters = {total_with_clusters} total")
    print(f"  Accuracy:  {total_correct}/{len(QUESTIONS)} correct + {total_partial} partial = {acc:.1f}%")
    print(f"  Retrieval: {total_retrieval}/{len(QUESTIONS)} = {ret:.1f}%")
    print(f"  Latency:   {avg_latency:.1f}s avg")
    print(f"\n  Par catégorie:")
    for cat in ["SSU", "SSA", "SSP", "KU", "TR", "MS"]:
        if cat in cats:
            c = cats[cat]
            print(f"    {cat}: {c['correct']}/{c['total']} correct, {c['partial']} partial, retrieval {c['retrieval_hits']}/{c['total']}")
    
    print(f"\n  📈 vs v3.3.0 (nano judge, same pipeline sans clusters):")
    print(f"     v3.3.0: 75.0% accuracy, 43.3% retrieval")
    print(f"     v3.4.0: {acc:.1f}% accuracy, {ret:.1f}% retrieval")
    print(f"     Clusters: {num_clusters} generated in {cluster_time:.1f}s")
    
    output = {
        "benchmark": "Memoria v3.4.0 (fact clusters)", "date": time.strftime("%Y-%m-%d %H:%M:%S"),
        "config": {"extract": EXTRACT_MODEL, "answer": ANSWER_MODEL, "embed": EMBED_MODEL,
                   "judge": JUDGE_MODEL, "query_expansion": True, "fact_clusters": True},
        "metrics": {
            "atomic_facts": total_facts, "clusters": num_clusters, "total_facts": total_with_clusters,
            "embedded": embedded,
            "ingest_time_s": round(ingest_time, 1), "cluster_time_s": round(cluster_time, 1),
            "embed_time_s": round(embed_time, 1),
            "accuracy": round(acc, 1), "correct": total_correct, "partial": total_partial,
            "wrong": len(QUESTIONS)-total_correct-total_partial,
            "retrieval_rate": round(ret, 1), "avg_latency_s": round(avg_latency, 1)
        },
        "by_category": cats, "results": results
    }
    with open(RESULTS_PATH, "w") as f:
        json.dump(output, f, indent=2, ensure_ascii=False)
    print(f"\n  💾 Résultats: {RESULTS_PATH}")
    print(f"  💾 DB: {DB_PATH}")

if __name__ == "__main__":
    main()


path: capture.ts
content: /**
 * Memoria — Capture hooks (Layer 1: agent_end + after_compaction)
 *
 * Extracted from index.ts Phase 2.2 — pure mechanical move, zero logic change.
 */

import type { OpenClawPluginApi } from "openclaw/plugin-sdk/core";
import type { MemoriaConfig } from "./core/config.js";
import type { MemoriaDB } from "./core/db.js";
import type { SelectiveMemory } from "./core/selective.js";
import type { LLMProvider } from "./core/providers/types.js";
import type { IdentityParser } from "./core/identity-parser.js";
import type { ProceduralMemory } from "./core/procedural.js";
import type { FeedbackManager } from "./core/feedback.js";
import type { AdaptiveBudget } from "./core/budget.js";
import type { ContinuousHooksState } from "./continuous.js";
import { LLM_EXTRACT_PROMPT, parseJSON, normalizeCategory } from "./core/extraction.js";

export interface CaptureDeps {
  api: OpenClawPluginApi;
  cfg: MemoriaConfig;
  db: MemoriaDB;
  selective: SelectiveMemory;
  extractLlm: LLMProvider;
  identityParser: IdentityParser;
  proceduralMem: ProceduralMemory;
  feedbackMgr: FeedbackManager;
  budget: AdaptiveBudget;
  postProcessNewFacts: (source: "capture" | "compaction") => Promise<void>;
  continuousState: ContinuousHooksState;
}

/**
 * Register the agent_end hook (session capture, Layer 1).
 * Extracts facts from session messages + procedures from tool calls.
 */
export function registerAgentEndHook(deps: CaptureDeps): void {
  const { api, cfg, db, selective, extractLlm, identityParser,
    proceduralMem, feedbackMgr, postProcessNewFacts, continuousState } = deps;

  if (!cfg.autoCapture) return;

  api.on("agent_end", async (event: any, _ctx: any) => {
    if (!event.success || !event.messages || event.messages.length === 0) return;

    // Track how many messages continuous already processed
    const continuousAlreadyCaptured = continuousState.hasExtracted();

    try {
      // ── Feedback loop: measure if recalled facts were used in responses ──
      try {
        const assistantTexts: string[] = [];
        for (const msg of event.messages) {
          if (!msg || typeof msg !== "object") continue;
          const m = msg as Record<string, unknown>;
          if (m.role !== "assistant") continue;
          const c = m.content;
          if (typeof c === "string" && c.length > 10) assistantTexts.push(c);
          else if (Array.isArray(c)) {
            for (const part of c) {
              if (part && typeof part === "object" && (part as any).type === "text") {
                const t = (part as any).text;
                if (typeof t === "string" && t.length > 10) assistantTexts.push(t);
              }
            }
          }
        }
        if (assistantTexts.length > 0) {
          const responseText = assistantTexts.slice(-3).join("\n");
          const fb = await feedbackMgr.processResponse(responseText);
          if (fb.used + fb.ignored > 0) {
            api.logger.debug?.(`memoria: feedback — ${fb.used} used, ${fb.ignored} ignored (${fb.details.length} total)`);
          }
        }
      } catch (e) { api?.logger?.debug?.('memoria:feedback-process: ' + String(e)); }

      // Collect user + assistant texts
      const texts: string[] = [];
      for (const msg of event.messages) {
        if (!msg || typeof msg !== "object") continue;
        const m = msg as Record<string, unknown>;
        const role = m.role as string;
        if (role !== "user" && role !== "assistant") continue;

        const content = m.content;
        if (typeof content === "string" && content.length > 10) {
          texts.push(content.slice(0, 3000)); // truncate for LLM
        } else if (Array.isArray(content)) {
          for (const part of content) {
            if (part && typeof part === "object" && (part as any).type === "text") {
              const t = (part as any).text;
              if (typeof t === "string" && t.length > 10) texts.push(t.slice(0, 3000));
            }
          }
        }
      }

      if (texts.length === 0) return;

      // If continuous learning already captured during this session,
      // only extract from messages NOT yet seen (reduce duplicate LLM calls)
      const effectiveTexts = continuousAlreadyCaptured
        ? texts.slice(-1) // Only the very last message (likely not yet captured)
        : texts.slice(-3);

      if (effectiveTexts.length === 0) return;

      // Take last messages (most relevant)
      const recentTexts = effectiveTexts.join("\n---\n");
      const prompt = LLM_EXTRACT_PROMPT
        .replace("{TEXT}", recentTexts)
        .replace("{MAX_FACTS}", String(cfg.captureMaxFacts));

      const result = await extractLlm.generateWithMeta!(prompt, {
        maxTokens: 1024,
        temperature: 0.1,
        format: "json",
        timeoutMs: 30000,
      });

      if (!result) {
        api.logger.debug?.("memoria: capture skipped — all LLM providers failed");
        return;
      }

      const parsed = parseJSON(result.response) as { facts?: Array<{ fact: string; category: string; type?: string; confidence: number }> };
      if (!parsed?.facts || parsed.facts.length === 0) return;

      let stored = 0;
      let skipped = 0;
      let enriched = 0;
      let superseded = 0;
      for (const f of parsed.facts) {
        if (!f.fact || f.fact.length < 5) continue;
        if (f.confidence < 0.7) continue;

        const factType = (f.type === "episodic") ? "episodic" : "semantic";

        try {
          const category = normalizeCategory(f.category);
          const relevance = identityParser.calculateRelevance(f.fact, category);
          const result = await selective.processAndApply(
            f.fact,
            category,
            f.confidence,
            cfg.defaultAgent,
            factType,
            relevance
          );
          if (result.stored) {
            if (result.action === "enrich") enriched++;
            else if (result.action === "supersede") superseded++;
            else stored++;
          } else {
            skipped++;
          }
        } catch (e) {
          api?.logger?.debug?.('memoria:selective-store: ' + String(e));
          // Fallback: store directly if selective fails
          const category = normalizeCategory(f.category);
          const relevance = identityParser.calculateRelevance(f.fact, category);
          db.storeFact({
            id: `fact_${Date.now()}_${Math.random().toString(36).slice(2, 9)}`,
            fact: f.fact,
            category,
            confidence: f.confidence,
            source: "auto-capture",
            tags: "[]",
            agent: cfg.defaultAgent,
            created_at: Date.now(),
            updated_at: Date.now(),
            fact_type: factType,
            relevance_weight: relevance,
          });
          stored++;
        }
      }

      const parts: string[] = [];
      if (stored > 0) parts.push(`${stored} new`);
      if (enriched > 0) parts.push(`${enriched} enriched`);
      if (superseded > 0) parts.push(`${superseded} superseded`);
      if (skipped > 0) parts.push(`${skipped} skipped`);
      if (parts.length > 0) {
        api.logger.info?.(`memoria: capture — ${parts.join(", ")}`);
      }

      // Post-processing: embed + graph + topics + sync
      if (stored > 0 || enriched > 0) {
        await postProcessNewFacts("capture");
      }

      // ── Procedural Memory: extract successful command sequences ──
      try {
        // DEBUG: log what we receive
        const toolCallCount = event.toolCalls?.length || 0;
        const messageCount = event.messages?.length || 0;
        api.logger.info?.(`[DEBUG] agent_end — toolCalls: ${toolCallCount}, messages: ${messageCount}`);

        // Strategy A: Try toolCalls first (if available)
        let proc: any = null;
        if (event.toolCalls && event.toolCalls.length >= 2) {
          api.logger.info?.(`[DEBUG] Trying toolCalls extraction...`);
          const lastMessage = event.messages[event.messages.length - 1];
          const lastText = typeof lastMessage === "object" && (lastMessage as any).content
            ? String((lastMessage as any).content).toLowerCase()
            : "";

          const successKeywords = ["success", "done", "published", "deployed", "completed", "✓", "✅"];
          const isSuccess = successKeywords.some(kw => lastText.includes(kw));

          if (isSuccess) {
            proc = await proceduralMem.extractProcedure(
              event.toolCalls as any,
              'success',
              `Session: ${event.agentId || cfg.defaultAgent}`
            );
          }
        }

        // Strategy B: Fallback to parsing messages (more robust)
        if (!proc && event.messages && event.messages.length >= 3) {
          api.logger.info?.(`[DEBUG] Trying message extraction...`);
          proc = await proceduralMem.extractFromMessages(
            event.messages as any,
            `Session: ${event.agentId || cfg.defaultAgent}`
          );
        }

        if (proc) {
          api.logger.info?.(`memoria: procedural ✅ captured "${proc.name}" (${proc.steps.length} steps)`);
        } else {
          api.logger.debug?.(`[DEBUG] No procedure extracted (toolCalls=${toolCallCount}, messages=${messageCount})`);
        }
      } catch (err) {
        api.logger.warn?.(`[DEBUG] procedural extraction error: ${String(err)}`);
      }

    } catch (err) {
      api.logger.warn?.(`memoria: capture failed: ${String(err)}`);
    }
  });
}

/**
 * Register the after_compaction hook (compaction capture, Layer 1).
 * Saves facts from LCM compaction summaries before they are lost.
 */
export function registerCompactionHook(deps: CaptureDeps): void {
  const { api, cfg, db, selective, extractLlm, identityParser,
    proceduralMem, budget, postProcessNewFacts } = deps;

  api.on("after_compaction", async (event: any, _ctx: any) => {
    // Budget learning: compaction happened → we may have been too aggressive
    try { budget.onCompaction(); } catch (e) { api?.logger?.debug?.('memoria:budget-compaction: ' + String(e)); }
    const penaltyNote = budget.penalty > 0 ? ` (compaction penalty: -${budget.penalty} facts)` : "";
    if (penaltyNote) api.logger.debug?.(`memoria: budget adjusted${penaltyNote}`);

    try {
      const summary = typeof event.summary === "string" ? event.summary : "";
      if (!summary || summary.length < 50) return;

      const prompt = LLM_EXTRACT_PROMPT
        .replace("{TEXT}", summary.slice(0, 4000))
        .replace("{MAX_FACTS}", String(cfg.captureMaxFacts));

      const result = await extractLlm.generateWithMeta!(prompt, {
        maxTokens: 1024,
        temperature: 0.1,
        format: "json",
        timeoutMs: 30000,
      });

      if (!result) {
        api.logger.debug?.("memoria: compaction capture skipped — all LLM providers failed");
        return;
      }

      const parsed = parseJSON(result.response) as { facts?: Array<{ fact: string; category: string; type?: string; confidence: number }> };
      if (!parsed?.facts || parsed.facts.length === 0) return;

      let stored = 0;
      let skipped = 0;
      for (const f of parsed.facts) {
        if (!f.fact || f.fact.length < 5 || f.confidence < 0.7) continue;
        const factType = (f.type === "episodic") ? "episodic" : "semantic";
        try {
          const category = normalizeCategory(f.category);
          const relevance = identityParser.calculateRelevance(f.fact, category);
          const result = await selective.processAndApply(
            f.fact, category, f.confidence, cfg.defaultAgent, factType, relevance
          );
          if (result.stored) stored++;
          else skipped++;
        } catch (e) {
          api?.logger?.debug?.('memoria:compaction-store: ' + String(e));
          const category = normalizeCategory(f.category);
          const relevance = identityParser.calculateRelevance(f.fact, category);
          db.storeFact({
            id: `fact_${Date.now()}_${Math.random().toString(36).slice(2, 9)}`,
            fact: f.fact,
            category,
            confidence: f.confidence,
            source: "compaction",
            tags: "[]",
            agent: cfg.defaultAgent,
            created_at: Date.now(),
            updated_at: Date.now(),
            fact_type: factType,
            relevance_weight: relevance,
          });
          stored++;
        }
      }

      if (stored > 0 || skipped > 0) {
        api.logger.info?.(`memoria: compaction — ${stored} stored, ${skipped} skipped (dedup/noise)`);
      }

      // Enrich compaction facts: embed + graph + topics + sync
      if (stored > 0) {
        await postProcessNewFacts("compaction");
      }

      // ── Procedural Memory: extract from compaction summary ──
      try {
        const fakeMessages = [{ role: 'assistant', content: summary }];
        const proc = await proceduralMem.extractFromMessages(
          fakeMessages as any,
          `Compaction summary: ${event.agentId || cfg.defaultAgent}`
        );
        if (proc) {
          api.logger.info?.(`memoria: procedural ✅ captured from compaction "${proc.name}" (${proc.steps.length} steps)`);
        }
      } catch (err) {
        api.logger.debug?.(`[DEBUG] procedural compaction extraction error: ${String(err)}`);
      }

    } catch (err) {
      api.logger.warn?.(`memoria: compaction capture failed: ${String(err)}`);
    }
  });
}


path: continuous.ts
content: /**
 * 🧠 Memoria — Continuous Learning (Layer 21)
 * 
 * This module implements real-time fact extraction from conversation flow,
 * independent of session end or compaction. Like a child learning while walking.
 * 
 * Exports:
 *   - CONTINUOUS_URGENT_PATTERNS — regex patterns for urgent/error signals
 *   - registerContinuousHooks() — attach message_received + llm_output hooks
 */

import type { OpenClawPluginApi } from "openclaw/plugin-sdk/core";
import type { MemoriaConfig } from "./core/config.js";
import type { MemoriaDB } from "./core/db.js";
import type { SelectiveMemory } from "./core/selective.js";
import type { LLMProvider } from "./core/providers/types.js";
import type { IdentityParser } from "./core/identity-parser.js";
import { LLM_EXTRACT_PROMPT, parseJSON, normalizeCategory } from "./core/extraction.js";
import type { PrefetchCache } from "./prefetch.js";
import { computeRecall, extractUserPrompt } from "./recall.js";
import type { RecallDeps } from "./recall.js";
import type { WriteAheadLog } from "./core/wal.js";
import type { SelfObserver } from "./core/self-observation.js";

// ─── Constants ───

export const CONTINUOUS_URGENT_PATTERNS = [
  // Frustration / explicit error signals
  /\bne\s+fais?\s+plus\b/i, /\bne\s+jamais\b/i, /\bputain\b/i, /\bmerde\b/i,
  /\bc'est\s+la\s+[23]\w*\s+fois\b/i, /\bj'ai\s+d[uû]\b/i,
  /\bdoublon\b/i, /\berreur\b/i, /\bcrash\b/i, /\bcassé\b/i, /\bmort\b/i,
  /\brevert\b/i, /\brollback\b/i, /\bhotfix\b/i,
  /\btu\s+as\s+pas\s+(compris|appris|retenu)\b/i,
  /\bpourquoi\s+tu\s+(refais?|recommence)\b/i,
  // English equivalents
  /\bnever\s+do\b/i, /\bdon'?t\s+ever\b/i, /\bbroke\b/i, /\bdead\b/i,
  /\bduplicate\b/i, /\bmistake\b/i,
];

// Correction patterns — user is correcting the agent's behavior/output
export const CORRECTION_PATTERNS = [
  // Direct corrections (FR)
  /\bnon\s*[,!.]?\s*(c'est|c'etait|il\s+faut|faut)\b/i,
  /\bc'est\s+pas\s+(ça|ca|correct|bon)\b/i,
  /\bje\s+t'ai\s+dit\s+(que|de)\b/i,
  /\bje\s+t'avais\s+dit\b/i,
  /\bt'as\s+pas\s+(compris|lu|vu|fait)\b/i,
  /\bc'est\s+faux\b/i, /\bc'est\s+pas\s+ce\s+que\b/i,
  /\bregarde\s+bien\b/i, /\brelis\b/i,
  /\bje\s+voulais\s+dire\b/i, /\bpas\s+comme\s+(ça|ca)\b/i,
  /\btu\s+(comprends|captes)\s+pas\b/i,
  /\ben\s+fait\s*[,!]\s*(c'est|il|faut)\b/i,
  /\btu\s+te\s+trompe/i, /\bmauvais/i,
  // Direct corrections (EN)
  /\bno\s*[,!.]?\s*(it'?s|that'?s|you\s+should|I\s+said|I\s+meant)\b/i,
  /\bthat'?s\s+(wrong|incorrect|not\s+(right|what))\b/i,
  /\bactually\s*[,!]/i, /\bI\s+already\s+told\s+you\b/i,
  /\byou\s+(misunderstood|didn'?t\s+(understand|read|listen))\b/i,
];

const SELF_ERROR_PATTERNS = [
  /erreur.*j'ai\s+(fait|commis|créé)/i,
  /mon\s+erreur/i, /j'aurais\s+d[uû]/i,
  /je\s+n'aurais\s+pas\s+d[uû]/i,
  /confond[ure]/i, /par\s+erreur/i,
  /ERREUR\s+CRITIQUE/i,
];

// ─── Hook Registration ───

interface ContinuousState {
  buffer: Array<{ role: "user" | "assistant"; text: string; ts: number }>;
  turnCount: number;
  lastExtraction: number;
  inProgress: boolean;
}

/**
 * Register continuous learning hooks on the OpenClaw API.
 * Buffers user messages + assistant responses, triggers extraction when:
 *   - Urgent: user frustration/error keywords (immediate, bypasses cooldown)
 *   - Self-error: assistant self-admission of mistake
 *   - Periodic: every N turns (default 4) with cooldown
 * 
 * Requires:
 *   - cfg.autoCapture = true (master capture switch)
 *   - cfg.continuous.enabled !== false (layer switch, default true)
 */
export interface ContinuousHooksState {
  /** Returns true if continuous extraction has run at least once this session */
  hasExtracted(): boolean;
}

export function registerContinuousHooks(
  api: OpenClawPluginApi,
  cfg: MemoriaConfig,
  db: MemoriaDB,
  selective: SelectiveMemory,
  extractLlm: LLMProvider,
  identityParser: IdentityParser,
  postProcessNewFacts: (source: "capture" | "compaction") => Promise<void>,
  prefetchCache?: PrefetchCache,
  recallDeps?: Omit<RecallDeps, "prefetchCache">,
  wal?: WriteAheadLog,
  selfObserver?: SelfObserver
): ContinuousHooksState {
  const ENABLED = cfg.continuous?.enabled !== false && cfg.autoCapture;
  if (!ENABLED) return { hasExtracted: () => false };

  const COOLDOWN_MS = cfg.continuous?.cooldownMs ?? 45_000;
  const MAX_BUFFER = 10;
  const NORMAL_INTERVAL = cfg.continuous?.interval ?? 4;

  const state: ContinuousState = {
    buffer: [],
    turnCount: 0,
    lastExtraction: 0,
    inProgress: false,
  };

  // ── message_received: buffer user messages + detect urgent ──
  api.on("message_received", async (event: any, _ctx: any) => {
    try {
      if (!event.content || event.content.length < 5) return;
      // Skip heartbeat/system messages
      if (/^(HEARTBEAT|Read HEARTBEAT|NO_REPLY)/i.test(event.content)) return;

      // ── WAL: persist message IMMEDIATELY (< 1ms, crash-safe) ──
      if (wal) {
        try { wal.write("user", event.content); }
        catch (e) { api.logger.debug?.(`memoria: WAL write error: ${String(e)}`); }
      }

      state.buffer.push({
        role: "user",
        text: event.content.slice(0, 3000),
        ts: Date.now(),
      });
      if (state.buffer.length > MAX_BUFFER) state.buffer.shift();
      state.turnCount++;

      // ── Async prefetch: start recall computation NOW ──
      // By the time before_prompt_build fires, the result will be cached
      // Debounce: skip if last prefetch was < 5s ago (prevents loop on rapid messages)
      const now = Date.now();
      const PREFETCH_DEBOUNCE_MS = 5_000;
      if (prefetchCache && recallDeps && event.content.length > 10 &&
          (now - state.lastExtraction > PREFETCH_DEBOUNCE_MS || state.turnCount <= 1)) {
        const userMsg = extractUserPrompt(event.content);
        if (userMsg.length > 5) {
          prefetchCache.startPrefetch(event.content, async () => {
            api.logger.debug?.(`memoria: 🚀 prefetch started for message (${userMsg.length} chars)`);
            return computeRecall(userMsg, state.turnCount, recallDeps);
          });
        }
      }

      // Check for correction signals — user is fixing agent's mistake
      const isCorrection = CORRECTION_PATTERNS.some(p => p.test(event.content));
      if (isCorrection) {
        api.logger.info?.(`memoria: 📝 continuous — correction detected in user message`);
        // Self-observation: record correction signal
        if (selfObserver) {
          try { selfObserver.record("correction", event.content); }
          catch (e) { api.logger.debug?.(`memoria: self-obs error: ${String(e)}`); }
        }
        // Log to .learnings/ for self-improving-agent coupling
        try {
          const workspacePath = cfg.workspacePath || process.env.OPENCLAW_WORKSPACE || "";
          if (workspacePath) {
            const fs = await import("fs");
            const path = await import("path");
            const learningsDir = path.join(workspacePath, ".learnings");
            const learningsFile = path.join(learningsDir, "LEARNINGS.md");
            if (fs.existsSync(learningsFile)) {
              const now = new Date();
              const timestamp = now.toISOString().slice(0, 16).replace("T", " ");
              const snippet = event.content.slice(0, 200).replace(/\n/g, " ");
              const entry = `\n### ${timestamp} — Correction\n- **Category**: correction\n- **What happened**: User corrected the agent\n- **Context**: "${snippet}"\n- **Source**: memoria auto-detection\n\n`;
              fs.appendFileSync(learningsFile, entry);
              api.logger.debug?.(`memoria: correction logged to .learnings/LEARNINGS.md`);
            }
          }
        } catch (e) {
          api.logger.debug?.(`memoria: failed to write .learnings: ${String(e)}`);
        }
        await doContinuousExtraction(api, cfg, db, selective, extractLlm, identityParser, postProcessNewFacts, state, "correction");
        return; // correction already triggers extraction, skip urgent check
      }

      // Check for urgent signals in user message — extract immediately
      const isUrgent = CONTINUOUS_URGENT_PATTERNS.some(p => p.test(event.content));
      if (isUrgent) {
        api.logger.info?.(`memoria: ⚡ continuous — urgent signal detected in user message`);
        // Self-observation: frustration or error detection
        if (selfObserver) {
          try { selfObserver.record("frustration", event.content); }
          catch (e) { api.logger.debug?.(`memoria: self-obs error: ${String(e)}`); }
        }
        await doContinuousExtraction(api, cfg, db, selective, extractLlm, identityParser, postProcessNewFacts, state, "urgent");
      }
    } catch (err) {
      api.logger.debug?.(`memoria: continuous message_received error: ${String(err)}`);
    }
  });

  // ── llm_output: buffer assistant responses + trigger periodic ──
  api.on("llm_output", async (event: any, _ctx: any) => {
    try {
      const texts = event.assistantTexts?.filter((t: any) => t && t.length > 15) || [];
      if (texts.length === 0) return;

      const combined = texts.join("\n").slice(0, 3000);
      // Skip empty/system responses
      if (/^(HEARTBEAT_OK|NO_REPLY)$/i.test(combined.trim())) return;

      // ── WAL: persist assistant response IMMEDIATELY ──
      if (wal) {
        try { wal.write("assistant", combined); }
        catch (e) { api.logger.debug?.(`memoria: WAL write error: ${String(e)}`); }
      }

      state.buffer.push({
        role: "assistant",
        text: combined,
        ts: Date.now(),
      });
      if (state.buffer.length > MAX_BUFFER) state.buffer.shift();

      // Check for self-detected errors in assistant response
      const selfError = SELF_ERROR_PATTERNS.some(p => p.test(combined));
      if (selfError) {
        api.logger.info?.(`memoria: ⚡ continuous — self-detected error in assistant response`);
        await doContinuousExtraction(api, cfg, db, selective, extractLlm, identityParser, postProcessNewFacts, state, "self-error");
      }

      // Normal periodic extraction
      if (state.turnCount >= NORMAL_INTERVAL) {
        const now = Date.now();
        if (now - state.lastExtraction > COOLDOWN_MS) {
          await doContinuousExtraction(api, cfg, db, selective, extractLlm, identityParser, postProcessNewFacts, state, "periodic");
        }
      }
    } catch (err) {
      api.logger.debug?.(`memoria: continuous llm_output error: ${String(err)}`);
    }
  });

  return { hasExtracted: () => state.lastExtraction > 0 };
}

// ─── Extraction Logic ───

/**
 * Layer 21: Continuous Learning — micro-extraction from rolling buffer.
 * 
 * Triggers:
 *   - "periodic": every N turns (default 4), with cooldown
 *   - "urgent": immediate on user frustration/error keywords (bypasses cooldown)
 *   - "self-error": immediate on assistant self-admission phrases
 * 
 * Uses same LLM_EXTRACT_PROMPT + selective + postProcessNewFacts as agent_end.
 * Guarded by state.inProgress lock to prevent concurrent runs.
 * Buffer is snapshot + cleared before extraction to avoid re-processing.
 */
async function doContinuousExtraction(
  api: OpenClawPluginApi,
  cfg: MemoriaConfig,
  db: MemoriaDB,
  selective: SelectiveMemory,
  extractLlm: LLMProvider,
  identityParser: IdentityParser,
  postProcessNewFacts: (source: "capture" | "compaction") => Promise<void>,
  state: ContinuousState,
  trigger: "periodic" | "urgent" | "self-error" | "correction"
): Promise<void> {
  if (state.buffer.length < 2) return;
  if (state.inProgress) return; // prevent concurrent extractions

  const now = Date.now();
  // Urgent bypasses cooldown, others respect it
  if (trigger === "periodic" && now - state.lastExtraction < (cfg.continuous?.cooldownMs ?? 45_000)) return;

  state.inProgress = true;
  state.lastExtraction = now;
  state.turnCount = 0;

  // Snapshot and clear buffer to avoid re-processing same messages
  const snapshot = [...state.buffer];
  state.buffer.length = 0;

  // Build context from snapshot
  const context = snapshot
    .map(m => `[${m.role}]: ${m.text}`)
    .join("\n---\n");

  const urgencyHint = trigger === "urgent"
    ? "\n\n⚠️ SIGNAL D'URGENCE DÉTECTÉ — L'utilisateur exprime une frustration ou signale une erreur. PRIORITÉ MAXIMALE aux faits de catégorie 'erreur'."
    : trigger === "self-error"
    ? "\n\n⚠️ L'ASSISTANT A DÉTECTÉ SA PROPRE ERREUR — Capturer ce qui s'est mal passé, pourquoi, et ce qu'il ne faut plus faire."
    : trigger === "correction"
    ? "\n\n📝 CORRECTION DÉTECTÉE — L'utilisateur corrige l'assistant. Capturer : (1) ce que l'assistant a fait de FAUX, (2) ce qui est CORRECT selon l'utilisateur, (3) la RÈGLE à retenir pour ne pas répéter. Catégorie = 'erreur' ou 'preference' selon le contexte."
    : "";

  const prompt = LLM_EXTRACT_PROMPT
    .replace("{TEXT}", context + urgencyHint)
    .replace("{MAX_FACTS}", String(Math.min(cfg.captureMaxFacts, trigger === "periodic" ? 3 : 5)));

  try {
    const result = await extractLlm.generateWithMeta!(prompt, {
      maxTokens: 768,
      temperature: 0.1,
      format: "json",
      timeoutMs: 20000,
    });

    if (!result?.response) return;

    const parsed = parseJSON(result.response) as { facts?: Array<{ fact: string; category: string; type?: string; confidence: number }> };
    if (!parsed?.facts || parsed.facts.length === 0) return;

    let stored = 0, skipped = 0, enriched = 0, superseded = 0;
    for (const f of parsed.facts) {
      if (!f.fact || f.fact.length < 5) continue;
      if (f.confidence < 0.7) continue;

      const factType = (f.type === "episodic") ? "episodic" : "semantic";
      try {
        const category = normalizeCategory(f.category);
        const relevance = identityParser.calculateRelevance(f.fact, category);
        const res = await selective.processAndApply(
          f.fact, category, f.confidence, cfg.defaultAgent, factType, relevance
        );
        if (res.stored) {
          if (res.action === "enrich") enriched++;
          else if (res.action === "supersede") superseded++;
          else stored++;
        } else { skipped++; }
      } catch (e) {
        api?.logger?.debug?.('memoria:contradiction-check: ' + String(e));
        const category = normalizeCategory(f.category);
        const relevance = identityParser.calculateRelevance(f.fact, category);
        db.storeFact({
          id: `fact_${Date.now()}_${Math.random().toString(36).slice(2, 9)}`,
          fact: f.fact, category, confidence: f.confidence,
          source: `continuous-${trigger}`,
          tags: "[]", agent: cfg.defaultAgent,
          created_at: Date.now(), updated_at: Date.now(),
          fact_type: factType, relevance_weight: relevance,
        });
        stored++;
      }
    }

    const parts: string[] = [];
    if (stored > 0) parts.push(`${stored} new`);
    if (enriched > 0) parts.push(`${enriched} enriched`);
    if (superseded > 0) parts.push(`${superseded} superseded`);
    if (skipped > 0) parts.push(`${skipped} skipped`);
    if (parts.length > 0) {
      api.logger.info?.(`memoria: ⚡ continuous [${trigger}] — ${parts.join(", ")}`);
      // Post-process (embed, graph, topics, etc.)
      if (stored > 0 || enriched > 0) {
        await postProcessNewFacts("capture");
      }
    }
  } catch (err) {
    api.logger.debug?.(`memoria: continuous extraction failed: ${String(err)}`);
  } finally {
    state.inProgress = false;
  }
}


path: core/README.md
content: # @primo-studio/memoria-core

**Standalone multi-layer cognitive memory engine** — works with or without OpenClaw.

## Features

- **21-layer cognitive architecture** — selective storage, embeddings, knowledge graph, topics, patterns, observations, procedural memory
- **Zero cloud dependency** — runs 100% local with Ollama, LM Studio, or any OpenAI-compatible API
- **Automatic fallback chain** — Ollama → OpenAI → LM Studio → FTS-only (never fails)
- **SQLite + FTS5** — blazing fast full-text search
- **Vector embeddings** — cosine similarity with nomic-embed-text, text-embedding-3-small, or custom models
- **Knowledge graph** — entity extraction, relations, Hebbian learning
- **Pattern detection** — consolidates recurring facts, 1.5x recall boost
- **Lifecycle management** — fresh → active → settled → dormant (automatic aging)
- **Contradiction detection** — replaces outdated facts with new information
- **Procedural memory** — learns from tool use, reflects on success patterns
- **Markdown sync** — bi-directional sync with .md files

## Installation

```bash
npm install @primo-studio/memoria-core
# or
pnpm add @primo-studio/memoria-core
```

## Quick Start

```typescript
import { Memoria } from '@primo-studio/memoria-core';

// Initialize with Ollama (local, free)
const memoria = await Memoria.init({
  dbPath: './my-app.db',
  provider: 'ollama',
  model: 'qwen3.5:4b',
  embeddingModel: 'nomic-embed-text-v2-moe'
});

// Store facts
await memoria.store('User prefers dark mode', 'preference', 0.95);
await memoria.store('User location: New York', 'savoir', 0.9);

// Recall facts
const results = await memoria.recall('What theme does the user like?');
console.log(results.facts);
// [{ id: 1, fact: 'User prefers dark mode', category: 'preference', confidence: 0.95, ... }]

// Natural language query (future: dialectic reasoning)
const answer = await memoria.query('Tell me about the user preferences');
console.log(answer);

// Stats
const stats = await memoria.stats();
console.log(stats);
// { totalFacts: 42, totalEmbeddings: 38, totalRelations: 15, ... }

// Close
memoria.close();
```

## Configuration

### Local (Ollama)

```typescript
const memoria = await Memoria.init({
  dbPath: './memoria.db',
  provider: 'ollama',
  model: 'qwen3.5:4b',
  embeddingModel: 'nomic-embed-text-v2-moe',
  baseUrl: 'http://localhost:11434'
});
```

### Cloud (OpenAI)

```typescript
const memoria = await Memoria.init({
  dbPath: './memoria.db',
  provider: 'openai',
  model: 'gpt-4o-mini',
  embeddingModel: 'text-embedding-3-small',
  [REDACTED_SECRET]
});
```

### LM Studio

```typescript
const memoria = await Memoria.init({
  dbPath: './memoria.db',
  provider: 'lmstudio',
  model: 'auto', // uses loaded model
  baseUrl: 'http://localhost:1234/v1'
});
```

### Custom Fallback Chain

```typescript
const memoria = await Memoria.init({
  dbPath: './memoria.db',
  fallback: [
    { type: 'ollama', model: 'qwen3.5:4b', baseUrl: 'http://localhost:11434' },
    { type: 'openai', model: 'gpt-5.4-nano', [REDACTED_SECRET] },
    { type: 'lmstudio', baseUrl: 'http://localhost:1234/v1' }
  ]
});
```

## API Reference

### `Memoria.init(options: MemoriaInitOptions): Promise<Memoria>`

Initialize a new Memoria instance.

**Options:**
- `dbPath` (required): Path to SQLite database file
- `provider`: `'ollama'` | `'openai'` | `'anthropic'` | `'lmstudio'`
- `model`: LLM model name
- `embeddingModel`: Embedding model name
- `embeddingDimensions`: Embedding dimensions (default: 768)
- `baseUrl`: Provider base URL
- `apiKey`: API key for cloud providers
- `language`: `'fr'` | `'en'` (default: 'en')
- `fallback`: Array of fallback providers
- `recallLimit`: Max facts to return (default: 8)
- `workspacePath`: Path for markdown sync (optional)
- `debug`: Enable debug logging

### `memoria.store(fact: string, category?: string, confidence?: number): Promise<StoreResult>`

Store a new fact in memory.

**Categories:**
- `savoir` — knowledge, facts
- `preference` — user preferences
- `erreur` — errors, mistakes
- `chronologie` — events, timeline
- `outil` — tools, configs
- `client` — client info
- `rh` — team, HR
- `pattern` — detected patterns (auto-generated)

**Returns:**
- `factId`: Database ID (-1 if not stored)
- `stored`: Boolean
- `reason`: Why not stored (if applicable)

### `memoria.recall(query: string, options?: RecallOptions): Promise<RecallResult>`

Recall facts based on a query.

**Options:**
- `limit`: Max facts to return
- `minConfidence`: Minimum confidence threshold
- `categories`: Filter by categories

**Returns:**
- `facts`: Array of facts with scores
- `totalFound`: Number of results

### `memoria.query(question: string): Promise<string>`

Natural language query with formatted context (future: dialectic reasoning).

### `memoria.stats(): Promise<MemoriaStats>`

Get memory statistics.

**Returns:**
- `totalFacts`, `totalEmbeddings`, `totalRelations`, `totalTopics`, `totalPatterns`, `totalObservations`
- `lifecycleDistribution`: Facts by lifecycle state
- `categoryCounts`: Facts by category

### `memoria.close(): void`

Close database connection.

## Advanced Usage

### Direct Manager Access

```typescript
// Access low-level managers for advanced use
const entities = memoria.graph.getEntitiesByType('person');
const strongRelations = memoria.hebbian.getStrong(5);
const topTopics = memoria.topics.getTopics(10);
const patterns = memoria.patterns.getAll();
```

### Custom Providers

```typescript
import { type LLMProvider, type EmbedProvider } from '@primo-studio/memoria-core';

const customLLM: LLMProvider = {
  async generate(prompt: string): Promise<string> {
    // Your custom implementation
    return 'response';
  }
};

const customEmbed: EmbedProvider = {
  async embed(text: string): Promise<number[]> {
    // Your custom implementation
    return [/* 768-dim vector */];
  }
};
```

## Architecture

Memoria uses a **21-layer cognitive architecture**:

1. **Layer 1** — Capture (fact extraction from conversations)
2. **Layer 2** — Deduplication (Levenshtein + Jaccard)
3. **Layer 3** — Contradiction detection (semantic similarity + LLM)
4. **Layer 4** — Enrichment (merge related facts)
5. **Layer 5** — Embedding (vector search)
6. **Layer 6** — Recall (retrieval pipeline)
7. **Layer 7** — Scoring (FTS5 + cosine + context boost)
8. **Layer 8** — Knowledge graph (entity/relation extraction)
9. **Layer 9** — Topics (emergent clustering)
10. **Layer 10** — Lifecycle (aging, state transitions)
11. **Layer 11** — Hebbian (co-occurrence learning)
12. **Layer 12** — Expertise (skill tracking)
13. **Layer 13** — Feedback (usefulness tracking)
14. **Layer 14** — Patterns (behavioral consolidation)
15. **Layer 15** — Observations (hypothesis testing)
16. **Layer 16** — Fact clusters (semantic grouping)
17. **Layer 17** — Revision (content updates)
18. **Layer 18** — Context tree (hierarchical boost)
19. **Layer 19** — Budget (adaptive limits)
20. **Layer 20** — Procedural (tool patterns)
21. **Layer 21** — Continuous (live extraction)

## Performance

- **Recall**: ~50-150ms (FTS5 + embeddings + reranking)
- **Store**: ~200-500ms (with embedding + graph extraction)
- **Memory**: ~50MB base + ~1KB per fact
- **Database**: SQLite (portable, zero-config)

## License

Apache-2.0

## Contributing

PRs welcome! See [CONTRIBUTING.md](../CONTRIBUTING.md)

## Credits

Built by [Primo Studio](https://primo-studio.fr) for OpenClaw and standalone use.


path: core/auto-skill.ts
content: /**
 * Memoria — Auto Skill Creation (Layer 23)
 *
 * When a procedure reaches maturity (repeated successes, high quality),
 * automatically generates a reusable skill document (.md) in the workspace.
 *
 * Inspired by Hermes' "auto skill creation from experience" concept.
 *
 * Lifecycle:
 * 1. Procedure is captured via after_tool_call (existing Layer 1b)
 * 2. Procedure accumulates successes + quality improvements
 * 3. When maturity threshold is reached → auto-generate skill .md
 * 4. Skill is written to workspace/.skills/ (or .learnings/skills/)
 * 5. Agent can reference these skills in future sessions
 */

import fs from "fs";
import path from "path";
import type { ProceduralMemory, Procedure } from "./procedural.js";

export interface AutoSkillConfig {
  /** Minimum success count before promoting to skill (default: 3) */
  minSuccesses: number;
  /** Minimum quality.overall before promoting (default: 0.6) */
  minQuality: number;
  /** Directory to write skill files (relative to workspace) */
  skillDir: string;
  /** Maximum number of auto-skills to create per session (default: 2) */
  maxPerSession: number;
}

const DEFAULT_CONFIG: AutoSkillConfig = {
  minSuccesses: 3,
  minQuality: 0.6,
  skillDir: ".skills",
  maxPerSession: 2,
};

export class AutoSkillCreator {
  private cfg: AutoSkillConfig;
  private proceduralMem: ProceduralMemory;
  private workspacePath: string;
  private createdThisSession: Set<string> = new Set();
  private promotedIds: Set<string> = new Set();

  constructor(
    proceduralMem: ProceduralMemory,
    workspacePath: string,
    config?: Partial<AutoSkillConfig>
  ) {
    this.proceduralMem = proceduralMem;
    this.workspacePath = workspacePath;
    this.cfg = { ...DEFAULT_CONFIG, ...config };

    // Load already-promoted procedure IDs from existing skill files
    this.loadPromotedIds();
  }

  private loadPromotedIds(): void {
    const skillDir = path.join(this.workspacePath, this.cfg.skillDir);
    if (!fs.existsSync(skillDir)) return;

    try {
      const files = fs.readdirSync(skillDir).filter(f => f.endsWith(".md"));
      for (const file of files) {
        const content = fs.readFileSync(path.join(skillDir, file), "utf-8");
        const match = content.match(/procedure_id:\s*(\S+)/);
        if (match) this.promotedIds.add(match[1]);
      }
    } catch {
      // Ignore read errors
    }
  }

  /**
   * Check all mature procedures and create skills for eligible ones.
   * Called periodically (e.g., on agent_end or after successful tool sequences).
   * Returns the number of new skills created.
   */
  checkAndPromote(): number {
    if (this.createdThisSession.size >= this.cfg.maxPerSession) return 0;

    const candidates = this.findMatureProcedures();
    let created = 0;

    for (const proc of candidates) {
      if (this.createdThisSession.size >= this.cfg.maxPerSession) break;
      if (this.promotedIds.has(proc.id)) continue;

      try {
        this.createSkillFile(proc);
        this.promotedIds.add(proc.id);
        this.createdThisSession.add(proc.id);
        created++;
      } catch {
        // Non-blocking
      }
    }

    return created;
  }

  /**
   * Find procedures that have reached maturity threshold.
   */
  private findMatureProcedures(): Procedure[] {
    const all = this.proceduralMem.getAllProcedures();
    return all.filter(proc =>
      proc.success_count >= this.cfg.minSuccesses &&
      proc.quality.overall >= this.cfg.minQuality &&
      !this.promotedIds.has(proc.id) &&
      proc.degradation_score < 0.5 // Not degraded
    ).sort((a, b) => {
      // Prioritize: highest quality × most successes
      const scoreA = a.quality.overall * Math.log2(a.success_count + 1);
      const scoreB = b.quality.overall * Math.log2(b.success_count + 1);
      return scoreB - scoreA;
    });
  }

  /**
   * Generate a skill .md file from a mature procedure.
   */
  private createSkillFile(proc: Procedure): void {
    const skillDir = path.join(this.workspacePath, this.cfg.skillDir);
    if (!fs.existsSync(skillDir)) {
      fs.mkdirSync(skillDir, { recursive: true });
    }

    // Generate a clean filename
    const slug = proc.name
      .toLowerCase()
      .replace(/[^a-z0-9]+/g, "-")
      .replace(/^-+|-+$/g, "")
      .slice(0, 60);
    const filename = `${slug}.md`;
    const filepath = path.join(skillDir, filename);

    // Don't overwrite existing files
    if (fs.existsSync(filepath)) return;

    const successRate = Math.round(
      (proc.success_count / Math.max(proc.success_count + proc.failure_count, 1)) * 100
    );

    const content = `# ${proc.name}

> Auto-generated by Memoria from ${proc.success_count} successful executions.
> procedure_id: ${proc.id}
> quality: ${Math.round(proc.quality.overall * 100)}% | success_rate: ${successRate}%
> version: ${proc.version} | last_success: ${proc.last_success_at ? new Date(proc.last_success_at).toISOString().slice(0, 10) : "unknown"}

## Goal

${proc.goal}

## Steps

${proc.steps.map((s, i) => `${i + 1}. ${s}`).join("\n")}

${proc.gotchas ? `## Gotchas

${proc.gotchas.split(" | ").map(g => `- ⚠️ ${g}`).join("\n")}` : ""}

${proc.improvements && proc.improvements.length > 0 ? `## Evolution

${proc.improvements.slice(-3).map(imp => `- ${imp.change}`).join("\n")}` : ""}

${proc.context ? `## Trigger Patterns

Keywords: ${proc.context}` : ""}

---
*This skill was automatically created by Memoria when the procedure reached maturity.*
*Edit freely — your improvements will be preserved.*
`;

    fs.writeFileSync(filepath, content, "utf-8");
  }

  /**
   * Get stats about auto-skill creation.
   */
  stats(): { totalPromoted: number; createdThisSession: number; candidates: number } {
    return {
      totalPromoted: this.promotedIds.size,
      createdThisSession: this.createdThisSession.size,
      candidates: this.findMatureProcedures().length,
    };
  }
}


path: core/bootstrap-topics.ts
content: /**
 * Bootstrap: tag all untagged facts and run topic emergence scan.
 * Run once to seed the system.
 */
import { MemoriaDB } from "./db.js";
import { OllamaLLM } from "./providers/ollama.js";
import { OllamaEmbed } from "./providers/ollama.js";
import { TopicManager } from "./topics.js";

const WORKSPACE = process.env.HOME + "/.openclaw/workspace";

async function main() {
  const db = new MemoriaDB(WORKSPACE);
  const llm = new OllamaLLM("http://localhost:11434", "gemma3:4b");
  const embedder = new OllamaEmbed("http://localhost:11434", "nomic-embed-text-v2-moe", 768);
  const topicMgr = new TopicManager(db, llm, embedder, {
    emergenceThreshold: 3,
    scanInterval: 999, // We'll manually scan
  });

  // Get all untagged, active facts
  const untagged = db.raw.prepare(
    "SELECT id, fact, category FROM facts WHERE (tags = '[]' OR tags IS NULL) AND superseded = 0 ORDER BY created_at DESC"
  ).all() as Array<{ id: string; fact: string; category: string }>;

  console.log(`\n🏷️  Tagging ${untagged.length} untagged facts...\n`);

  let tagged = 0;
  let failed = 0;
  const batchSize = 10;
  
  for (let i = 0; i < untagged.length; i += batchSize) {
    const batch = untagged.slice(i, i + batchSize);
    const promises = batch.map(async (f) => {
      try {
        const { keywords, topics } = await topicMgr.onFactCaptured(f.id, f.fact, f.category);
        if (keywords.length > 0) {
          tagged++;
          if (tagged % 50 === 0 || tagged <= 5) {
            console.log(`  [${tagged}/${untagged.length}] "${f.fact.slice(0, 50)}..." → [${keywords.join(", ")}]`);
          }
        } else {
          failed++;
        }
      } catch (e) {
        failed++;
      }
    });

    // Process batch sequentially to avoid hammering Ollama
    for (const p of promises) await p;

    // Progress
    if ((i + batchSize) % 100 === 0) {
      console.log(`  Progress: ${Math.min(i + batchSize, untagged.length)}/${untagged.length} (${tagged} tagged, ${failed} failed)`);
    }
  }

  console.log(`\n✅ Tagging done: ${tagged} tagged, ${failed} failed\n`);

  // Now run the emergence scan
  console.log("🔍 Running topic emergence scan...\n");
  const scanResult = await topicMgr.scanAndEmerge();
  console.log(`  Created: ${scanResult.created} topics`);
  console.log(`  Merged: ${scanResult.merged} topics`);
  console.log(`  Sub-topics: ${scanResult.subtopics}`);

  // Show results
  const stats = topicMgr.stats();
  console.log(`\n📊 Final stats:`);
  console.log(`  Total topics: ${stats.totalTopics}`);
  console.log(`  Top-level: ${stats.topLevelTopics}`);
  console.log(`  Sub-topics: ${stats.subTopics}`);
  console.log(`  Orphan facts: ${stats.orphanFacts}`);
  console.log(`  Avg facts/topic: ${stats.avgFactsPerTopic}`);

  // Show topic list
  const topics = db.raw.prepare("SELECT name, keywords, fact_count, importance_score FROM topics ORDER BY importance_score DESC LIMIT 20").all() as any[];
  console.log(`\n🏷️  Top topics:`);
  for (const t of topics) {
    const kw = JSON.parse(t.keywords).slice(0, 5).join(", ");
    console.log(`  ${t.name} (${t.fact_count} facts, score ${t.importance_score.toFixed(1)}) — [${kw}]`);
  }

  db.raw.close();
}

main().catch(console.error);


path: core/budget.ts
content: /**
 * Memoria — Layer 7: Adaptive Budget
 * 
 * Dynamically adjusts the number of facts injected based on available context space.
 * Tiers: empty (12 facts) → light (10) → medium (8) → heavy (5) → critical (2).
 * 
 * Début de session (context vide) → 8-10 faits
 * Session moyenne → 5-6 faits  
 * Session chargée → 2-3 faits essentiels
 * 
 * Évite de saturer le contexte et d'accélérer la compaction.
 */

export interface BudgetConfig {
  /** Max context window in tokens. Default 200000 (Opus) */
  contextWindow: number;
  /** Max facts to inject. Default 10 */
  maxFacts: number;
  /** Min facts to inject (always at least this). Default 2 */
  minFacts: number;
  /** Target % of context to use for memory. Default 0.03 (3%) */
  targetMemoryShare: number;
  /** Avg tokens per fact (estimated). Default 60 */
  avgTokensPerFact: number;
  /** Context usage thresholds */
  thresholds: {
    /** Below this = "light" (use maxFacts). Default 0.3 */
    light: number;
    /** Below this = "medium" (scale down). Default 0.7 */
    medium: number;
    /** Above this = "heavy" (use minFacts). Default 0.85 */
    heavy: number;
  };
}

export const DEFAULT_BUDGET_CONFIG: BudgetConfig = {
  contextWindow: 200000,
  maxFacts: 10,
  minFacts: 2,
  targetMemoryShare: 0.03,
  avgTokensPerFact: 60,
  thresholds: {
    light: 0.3,
    medium: 0.7,
    heavy: 0.85,
  },
};

export interface BudgetResult {
  /** Computed number of facts to inject */
  limit: number;
  /** Context usage ratio (0-1) */
  usage: number;
  /** Budget zone */
  zone: "light" | "medium" | "heavy" | "critical";
  /** Available tokens for memory */
  availableTokens: number;
}

export class AdaptiveBudget {
  private cfg: BudgetConfig;

  // ─── Learning from compactions ───
  // If we inject too many facts and compaction fires right after → we were too aggressive.
  // Track this to self-correct.
  private recentRecalls: Array<{ timestamp: number; limit: number }> = [];
  private compactionPenalty = 0;       // 0-3: reduces limit by this many facts
  private lastCompactionAt = 0;
  private static readonly RECALL_WINDOW_MS = 5 * 60 * 1000; // 5 min window
  private static readonly MAX_PENALTY = 3;

  constructor(config?: Partial<BudgetConfig>) {
    this.cfg = { ...DEFAULT_BUDGET_CONFIG, ...config };
    if (config?.thresholds) {
      this.cfg.thresholds = { ...DEFAULT_BUDGET_CONFIG.thresholds, ...config.thresholds };
    }
  }

  /** 
   * Called when compaction happens. If a recall happened recently,
   * we were likely too aggressive → increase penalty.
   */
  onCompaction(): void {
    const now = Date.now();
    this.lastCompactionAt = now;

    // Was there a recall in the last 5 minutes?
    const recentRecall = this.recentRecalls.find(
      r => (now - r.timestamp) < AdaptiveBudget.RECALL_WINDOW_MS
    );

    if (recentRecall) {
      // We injected facts and compaction fired soon after → too aggressive
      this.compactionPenalty = Math.min(
        this.compactionPenalty + 1,
        AdaptiveBudget.MAX_PENALTY
      );
    }

    // Clean old entries
    this.recentRecalls = this.recentRecalls.filter(
      r => (now - r.timestamp) < AdaptiveBudget.RECALL_WINDOW_MS * 2
    );
  }

  /**
   * Called after a successful recall to log timing.
   */
  recordRecall(limit: number): void {
    this.recentRecalls.push({ timestamp: Date.now(), limit });
    // Keep last 10
    if (this.recentRecalls.length > 10) this.recentRecalls.shift();

    // Decay penalty over time: if no compaction for 30 min, reduce penalty
    if (this.compactionPenalty > 0 && this.lastCompactionAt > 0) {
      const minSinceCompaction = (Date.now() - this.lastCompactionAt) / 60000;
      if (minSinceCompaction > 30) {
        this.compactionPenalty = Math.max(0, this.compactionPenalty - 1);
      }
    }
  }

  /** Current compaction penalty (for logging) */
  get penalty(): number {
    return this.compactionPenalty;
  }

  /**
   * Compute how many facts to inject given current context state.
   * 
   * @param messagesTokenEstimate - Estimated tokens already in conversation
   * @param systemTokenEstimate - Estimated tokens in system prompt + workspace files
   */
  compute(messagesTokenEstimate: number, systemTokenEstimate: number = 0): BudgetResult {
    const totalUsed = messagesTokenEstimate + systemTokenEstimate;
    const usage = Math.min(totalUsed / this.cfg.contextWindow, 1.0);

    // Determine zone
    let zone: BudgetResult["zone"];
    let limit: number;

    if (usage < this.cfg.thresholds.light) {
      zone = "light";
      limit = this.cfg.maxFacts;
    } else if (usage < this.cfg.thresholds.medium) {
      zone = "medium";
      // Aggressive curve: 10 → 4 across medium zone (quadratic ease-in)
      const t = (usage - this.cfg.thresholds.light) / (this.cfg.thresholds.medium - this.cfg.thresholds.light);
      const tCurve = t * t; // quadratic: drops slowly at start, fast at end
      const mediumFloor = this.cfg.minFacts + 2; // 4 facts at end of medium
      limit = Math.round(this.cfg.maxFacts - tCurve * (this.cfg.maxFacts - mediumFloor));
    } else if (usage < this.cfg.thresholds.heavy) {
      zone = "heavy";
      // 4 → minFacts across heavy zone
      const t = (usage - this.cfg.thresholds.medium) / (this.cfg.thresholds.heavy - this.cfg.thresholds.medium);
      const mediumFloor = this.cfg.minFacts + 2; // 4
      limit = Math.round(mediumFloor - t * (mediumFloor - this.cfg.minFacts));
    } else {
      zone = "critical";
      limit = this.cfg.minFacts;
    }

    // Apply compaction penalty (learned from recent compactions)
    if (this.compactionPenalty > 0) {
      limit = Math.max(this.cfg.minFacts, limit - this.compactionPenalty);
    }

    // Ensure bounds
    limit = Math.max(this.cfg.minFacts, Math.min(this.cfg.maxFacts, limit));

    // Available tokens for memory
    const availableTokens = Math.max(0, this.cfg.contextWindow * this.cfg.targetMemoryShare - totalUsed * this.cfg.targetMemoryShare);

    return { limit, usage, zone, availableTokens };
  }

  /**
   * Quick estimate: count messages × avg tokens per message.
   * Rough but fast — no tokenizer needed.
   */
  static estimateTokens(messageCount: number, avgCharsPerMessage: number = 200): number {
    // ~4 chars per token (rough English/French average)
    return Math.round(messageCount * avgCharsPerMessage / 4);
  }
}


path: core/config.ts
content: /**
 * 🧠 Memoria — Configuration parser and provider factories
 * 
 * This module exports:
 *   - MemoriaConfig interface (typed config from openclaw.json)
 *   - parseConfig() — raw JSON → typed config with smart defaults
 *   - createEmbedProvider() — factory for embedding providers
 *   - createLLMProvider() — factory for LLM providers
 */

import { OllamaEmbed, OllamaLLM } from "./providers/ollama.js";
import { lmStudioLLM, lmStudioEmbed, openaiLLM, openaiEmbed, openrouterLLM, openrouterEmbed } from "./providers/openai-compat.js";
import type { EmbedProvider, LLMProvider } from "./providers/types.js";
import type { FallbackProviderConfig } from "./fallback.js";
import { AnthropicLLM } from "./providers/anthropic.js";

// ─── Config Interface ───

export interface MemoriaConfig {
  autoRecall: boolean;
  autoCapture: boolean;
  recallLimit: number;
  captureMaxFacts: number;
  defaultAgent: string;
  contextWindow: number;
  workspacePath: string;
  syncMd: boolean;
  fallback: FallbackProviderConfig[];
  /** Continuous Learning (Layer 21) config */
  continuous?: {
    /** Extract every N turns (default 4) */
    interval?: number;
    /** Cooldown between periodic extractions in ms (default 45000) */
    cooldownMs?: number;
    /** Enable/disable (default true when autoCapture is true) */
    enabled?: boolean;
  };
  embed: {
    provider: "ollama" | "lmstudio" | "openai" | "openrouter" | "anthropic";
    baseUrl?: string;
    model: string;
    dimensions: number;
    apiKey?: string;
  };
  llm: {
    provider: "ollama" | "lmstudio" | "openai" | "openrouter" | "anthropic";
    baseUrl?: string;
    model: string;
    apiKey?: string;
    /** Per-layer overrides: each key = layer name, value = provider config */
    overrides?: Partial<Record<MemoriaLayer, LayerLLMConfig>>;
  };
  lifecycle?: {
    freshDays?: number;
    settledMinAccess?: number;
    dormantAfterDays?: number;
    detailCursor?: number;
    revisionRecallThreshold?: number;
  };
  procedural?: {
    reflectEvery?: number;
    degradedThreshold?: number;
    defaultSafety?: number;
    staleDays?: number;
    docCheckDays?: number;
  };
  patterns?: any; // PatternManager config, loosely typed for now
  autoSkill?: {
    minSuccesses?: number;
    minQuality?: number;
    skillDir?: string;
    maxPerSession?: number;
  };
}

/** Named layers that accept a per-layer LLM override */
export type MemoriaLayer = "extract" | "contradiction" | "graph" | "topics" | "procedural";

export interface LayerLLMConfig {
  provider: "ollama" | "lmstudio" | "openai" | "openrouter" | "anthropic";
  baseUrl?: string;
  model: string;
  apiKey?: string;
}

// ─── Config Parser ───

/** Parse raw plugin config (from openclaw.json) into typed MemoriaConfig with smart defaults. */
export function parseConfig(raw: Record<string, unknown> | undefined): MemoriaConfig {
  const embed = (raw?.embed as Record<string, unknown>) || {};
  const llm = (raw?.llm as Record<string, unknown>) || {};
  return {
    autoRecall: raw?.autoRecall !== false,
    autoCapture: raw?.autoCapture !== false,
    recallLimit: (raw?.recallLimit as number) || 12,
    captureMaxFacts: (raw?.captureMaxFacts as number) || 8,
    defaultAgent: (raw?.defaultAgent as string) || "koda",
    contextWindow: (raw?.contextWindow as number) || 200000,
    workspacePath: (raw?.workspacePath as string) || process.env.HOME + "/.openclaw/workspace",
    syncMd: raw?.syncMd !== false,
    fallback: ((raw?.fallback as any[]) || []).map((f: any) => ({
      ...f,
      // Normalize: user config uses "provider", internal uses "type"
      type: f.type || f.provider || "ollama",
      name: f.name || f.provider || f.type || "ollama",
    })) as FallbackProviderConfig[],
    embed: {
      provider: (embed.provider as MemoriaConfig["embed"]["provider"]) || "ollama",
      baseUrl: embed.baseUrl as string | undefined,
      model: (embed.model as string) || "nomic-embed-text-v2-moe",
      dimensions: (embed.dimensions as number) || 768,
      [REDACTED_SECRET] as string | undefined,
    },
    llm: {
      provider: (llm.provider as MemoriaConfig["llm"]["provider"]) || "ollama",
      baseUrl: llm.baseUrl as string | undefined,
      model: (llm.model as string) || "gemma3:4b",
      [REDACTED_SECRET] as string | undefined,
      overrides: (llm.overrides as MemoriaConfig["llm"]["overrides"]) || undefined,
    },
  };
}

// ─── Provider Factories ───

/** Create an embedding provider from config. Used for the main embedder + fallback list. */
export function createEmbedProvider(cfg: MemoriaConfig["embed"]): EmbedProvider {
  switch (cfg.provider) {
    case "ollama":
      return new OllamaEmbed(cfg.baseUrl || "http://localhost:11434", cfg.model, cfg.dimensions);
    case "lmstudio":
      return lmStudioEmbed(cfg.model, cfg.dimensions, cfg.baseUrl || "http://localhost:1234/v1");
    case "openai":
      return openaiEmbed(cfg.model, cfg.apiKey || "", cfg.dimensions);
    case "openrouter":
      return openrouterEmbed(cfg.model, cfg.apiKey || "", cfg.dimensions);
    default:
      return new OllamaEmbed(); // safe default
  }
}

/** Create an LLM provider from config. Used for the main chain + per-layer overrides. */
export function createLLMProvider(cfg: MemoriaConfig["llm"]): LLMProvider {
  switch (cfg.provider) {
    case "ollama":
      return new OllamaLLM(cfg.baseUrl || "http://localhost:11434", cfg.model);
    case "lmstudio":
      return lmStudioLLM(cfg.model, cfg.baseUrl || "http://localhost:1234/v1");
    case "openai":
      return openaiLLM(cfg.model, cfg.apiKey || "");
    case "openrouter":
      return openrouterLLM(cfg.model, cfg.apiKey || "");
    case "anthropic":
      return new AnthropicLLM(cfg.model, cfg.apiKey || "", cfg.baseUrl);
    default:
      return new OllamaLLM(); // safe default
  }
}


path: core/context-tree.ts
content: /**
 * Memoria — Layer 6: Context Tree
 * 
 * Organizes facts into a semantic hierarchy for structured recall.
 * Instead of "here are 8 flat facts", we inject:
 * 
 * Bureau (3 faits)
 *   ├─ CRM (2 faits)
 *   └─ Modules (1 fait)
 * Convex (2 faits)
 * Infrastructure (3 faits)
 * 
 * Permet de :
 * 1. Montrer la STRUCTURE de la mémoire
 * 2. Prioriser par branche (Bureau > infra si query mentionne "CRM")
 * 3. Éviter l'overload de faits non pertinents
 */

import type { MemoriaDB, Fact } from "./db.js";

// ─── Types ───

export interface ContextNode {
  id: string;
  label: string;              // "Bureau", "CRM", "Modules"...
  type: "root" | "branch" | "leaf";
  facts: string[];            // Fact IDs
  children: ContextNode[];
  weight: number;             // Relevance to current query
  depth: number;
  parent?: ContextNode;
}

export interface ContextTree {
  roots: ContextNode[];
  factMap: Map<string, ContextNode>; // fact ID → node containing it
}

// ─── Builder ───

export class ContextTreeBuilder {
  private db: MemoriaDB;

  constructor(db: MemoriaDB) {
    this.db = db;
  }

  /** Build a context tree from a list of facts */
  async build(facts: Fact[], query?: string): Promise<ContextTree> {
    if (facts.length === 0) {
      return { roots: [], factMap: new Map() };
    }

    // Step 1: Cluster by category + keyword heuristics (no LLM)
    const clusters = await this.clusterFacts(facts);

    // Step 2: Build tree structure
    const roots: ContextNode[] = [];
    const factMap = new Map<string, ContextNode>();

    for (const cluster of clusters) {
      const node = this.buildNode(cluster, 0);
      roots.push(node);
      this.indexFactMap(node, factMap);
    }

    // Step 3: Weight by query relevance (if provided)
    if (query) {
      this.weightByQuery(roots, query);
    }

    return { roots, factMap };
  }

  /** Render tree as indented text */
  renderTree(tree: ContextTree, maxDepth = 3): string {
    const lines: string[] = [];

    const render = (node: ContextNode, indent = "") => {
      if (node.depth > maxDepth) return;

      const prefix = node.depth === 0 ? "▪" : node.type === "branch" ? "├─" : "  ";
      const weight = node.weight > 0 ? ` [${node.weight.toFixed(2)}]` : "";
      const factCount = node.facts.length > 0 ? ` (${node.facts.length})` : "";
      
      lines.push(`${indent}${prefix} ${node.label}${weight}${factCount}`);

      for (let i = 0; i < node.children.length; i++) {
        const child = node.children[i];
        const childIndent = indent + (node.depth === 0 ? "  " : "  ");
        render(child, childIndent);
      }
    };

    for (const root of tree.roots) {
      render(root);
    }

    return lines.join("\n");
  }

  /** Get facts from tree in priority order (high weight first) */
  extractFacts(tree: ContextTree, limit: number): Fact[] {
    // Flatten all nodes with facts
    const nodesWithFacts: Array<{ node: ContextNode; factIds: string[] }> = [];

    const collect = (node: ContextNode) => {
      if (node.facts.length > 0) {
        nodesWithFacts.push({ node, factIds: node.facts });
      }
      for (const child of node.children) {
        collect(child);
      }
    };

    for (const root of tree.roots) {
      collect(root);
    }

    // Sort by weight (high → low)
    nodesWithFacts.sort((a, b) => b.node.weight - a.node.weight);

    // Extract facts up to limit
    const factIds = new Set<string>();
    const results: Fact[] = [];

    for (const { factIds: ids } of nodesWithFacts) {
      for (const id of ids) {
        if (factIds.has(id)) continue;
        factIds.add(id);

        const fact = this.db.getFact(id);
        if (fact && !fact.superseded) {
          results.push(fact);
          if (results.length >= limit) return results;
        }
      }
    }

    return results;
  }

  // ─── Private ───

  private async clusterFacts(facts: Fact[]): Promise<Cluster[]> {
    // Use categories + keywords for initial grouping
    const grouped = new Map<string, Fact[]>();

    for (const fact of facts) {
      const category = fact.category || "other";
      if (!grouped.has(category)) grouped.set(category, []);
      grouped.get(category)!.push(fact);
    }

    // Build clusters
    const clusters: Cluster[] = [];

    for (const [category, catFacts] of grouped) {
      if (catFacts.length === 0) continue;

      // If too many facts in one category, sub-cluster by keywords
      if (catFacts.length > 10) {
        const subClusters = this.subClusterByKeywords(catFacts);
        clusters.push({
          label: this.categoryLabel(category),
          facts: [],
          children: subClusters,
        });
      } else {
        clusters.push({
          label: this.categoryLabel(category),
          facts: catFacts,
          children: [],
        });
      }
    }

    return clusters;
  }

  private subClusterByKeywords(facts: Fact[]): Cluster[] {
    // Extract common keywords
    const keywordMap = new Map<string, Fact[]>();

    for (const fact of facts) {
      const words = this.extractKeywords(fact.fact);
      for (const word of words) {
        if (!keywordMap.has(word)) keywordMap.set(word, []);
        keywordMap.get(word)!.push(fact);
      }
    }

    // Take top keywords (most frequent)
    const sorted = Array.from(keywordMap.entries())
      .sort((a, b) => b[1].length - a[1].length)
      .slice(0, 5);

    const clusters: Cluster[] = [];
    const assigned = new Set<string>();

    for (const [keyword, keywordFacts] of sorted) {
      const unique = keywordFacts.filter(f => !assigned.has(f.id));
      if (unique.length === 0) continue;

      clusters.push({
        label: keyword,
        facts: unique,
        children: [],
      });

      for (const f of unique) assigned.add(f.id);
    }

    // Remaining = "Autres"
    const remaining = facts.filter(f => !assigned.has(f.id));
    if (remaining.length > 0) {
      clusters.push({
        label: "Autres",
        facts: remaining,
        children: [],
      });
    }

    return clusters;
  }

  private extractKeywords(text: string): string[] {
    // Simple keyword extraction (proper nouns + technical terms)
    const stopWords = new Set([
      "le", "la", "les", "un", "une", "des", "de", "du", "et", "ou", "à", "dans", "pour", "sur", "avec", "est", "sont",
      "a", "an", "the", "and", "or", "to", "in", "for", "on", "with", "is", "are", "was", "were", "been", "be",
    ]);

    const words = text.split(/\s+/)
      .map(w => w.replace(/[^\p{L}\p{N}]/gu, "").toLowerCase())
      .filter(w => w.length > 3 && !stopWords.has(w));

    // Capitalize proper nouns (heuristic: starts uppercase in original)
    const properNouns = text.match(/\b[A-Z][a-z]+(?:[A-Z][a-z]+)*\b/g) || [];
    const allKeywords = [...words, ...properNouns.map(w => w.toLowerCase())];

    // Return unique, most common
    const freq = new Map<string, number>();
    for (const w of allKeywords) {
      freq.set(w, (freq.get(w) || 0) + 1);
    }

    return Array.from(freq.keys())
      .sort((a, b) => freq.get(b)! - freq.get(a)!)
      .slice(0, 5);
  }

  private buildNode(cluster: Cluster, depth: number): ContextNode {
    const node: ContextNode = {
      id: `node_${Date.now()}_${Math.random().toString(36).slice(2, 7)}`,
      label: cluster.label,
      type: depth === 0 ? "root" : cluster.children.length > 0 ? "branch" : "leaf",
      facts: cluster.facts.map(f => f.id),
      children: [],
      weight: 0,
      depth,
    };

    for (const child of cluster.children) {
      const childNode = this.buildNode(child, depth + 1);
      childNode.parent = node;
      node.children.push(childNode);
    }

    return node;
  }

  private indexFactMap(node: ContextNode, map: Map<string, ContextNode>) {
    for (const factId of node.facts) {
      map.set(factId, node);
    }
    for (const child of node.children) {
      this.indexFactMap(child, map);
    }
  }

  private weightByQuery(roots: ContextNode[], query: string) {
    const lowerQuery = query.toLowerCase();
    const words = lowerQuery.split(/\s+/).filter(w => w.length > 2);

    const weight = (node: ContextNode) => {
      const labelLower = node.label.toLowerCase();
      let score = 0;

      // Exact label match
      if (lowerQuery.includes(labelLower) || labelLower.includes(lowerQuery)) {
        score += 1.0;
      }

      // Word overlap
      for (const word of words) {
        if (labelLower.includes(word)) {
          score += 0.3;
        }
      }

      node.weight = score;

      // Recurse children
      for (const child of node.children) {
        weight(child);
        // Propagate child weight to parent
        node.weight += child.weight * 0.5;
      }
    };

    for (const root of roots) {
      weight(root);
    }
  }

  private categoryLabel(cat: string): string {
    const labels: Record<string, string> = {
      outil: "Outils",
      savoir: "Savoir",
      erreur: "Erreurs",
      client: "Clients",
      preference: "Préférences",
      chronologie: "Chronologie",
      rh: "RH",
    };
    return labels[cat] || cat.charAt(0).toUpperCase() + cat.slice(1);
  }
}

// ─── Internal types ───

interface Cluster {
  label: string;
  facts: Fact[];
  children: Cluster[];
}


path: core/db.ts
content: /**
 * Memoria — SQLite Database Layer (Layer 1)
 * 
 * The foundation of Memoria. Manages the SQLite database with:
 * - facts + facts_fts (FTS5 full-text search)
 * - embeddings (768d float vectors as BLOBs)
 * - entities + relations (knowledge graph)
 * - topics + fact_topics (emergent topic system)
 * - observations (living syntheses)
 * - procedures + procedures_fts (how-to memory)
 * - cluster_members (fact → cluster mapping)
 * - identity_cache, meta, chunks
 * 
 * Uses better-sqlite3 for synchronous, fast, zero-dependency SQLite.
 * All migrations auto-run on construction (additive, never destructive).
 * 
 * @example
 * const db = new MemoriaDB("/path/to/workspace");
 * db.storeFact({ id: "f_123", fact: "...", category: "savoir", ... });
 * const results = db.searchFacts("Bureau"); // FTS5 search
 * db.raw.prepare("SELECT ...").all(); // direct SQLite access
 */

import Database from "better-sqlite3";
import path from "path";
import fs from "fs";

const SCHEMA_VERSION = 1;

/**
 * Core fact record stored in SQLite. Every piece of knowledge Memoria captures.
 * 
 * Key fields for contributors:
 * - `fact_type`: "semantic" (durable truth) | "episodic" (dated event) | "cluster" (summary) | "pattern" (consolidated)
 * - `lifecycle_state`: "fresh" → "settled" → "dormant" (controls recall priority, NOT deletion)
 * - `superseded`: 0 = active, 1 = replaced by a newer/better version (superseded_by has the replacement ID)
 * - `tags`: JSON string array, e.g. '["convex","bureau"]'
 * - `entity_ids`: JSON string array of entity IDs linked to this fact
 */
export interface Fact {
  id: string;
  fact: string;
  category: string;
  confidence: number;
  source: string;
  tags: string;           // JSON array
  agent: string;
  created_at: number;
  updated_at: number;
  access_count: number;
  last_accessed_at: number | null;
  superseded: number;     // 0 or 1
  superseded_by: string | null;
  superseded_at: number | null;
  md_file: string | null;
  md_line: number | null;
  entity_ids: string;     // JSON array
  fact_type: "semantic" | "episodic" | "cluster" | "pattern"; // semantic = durable, episodic = dated/contextual, cluster = thematic summary, pattern = consolidated behavioral
  usefulness: number;       // feedback score, higher = more useful
  recall_count: number;     // times recalled in prompts
  used_count: number;       // times actually used in answers
  synced_to_md: number;     // 0 = not synced, 1 = synced, 2 = regenerated
  relevance_weight: number; // 0.0-1.0, calculated from identity context
  lifecycle_state: "fresh" | "settled" | "dormant"; // fresh = new, settled = confirmed, dormant = unused
}

/** Knowledge graph node. Types: person, project, tool, concept, place. */
export interface Entity {
  id: string;
  name: string;
  type: string;           // person|project|tool|concept|place
  attributes: string;     // JSON
  created_at: number;
  access_count: number;
}

/** Knowledge graph edge. Weight increases via Hebbian reinforcement (co-occurrence), decays when unused. */
export interface Relation {
  id: string;
  source_id: string;
  target_id: string;
  relation: string;
  weight: number;
  context: string | null;
  created_at: number;
  last_accessed_at: number | null;
}

/**
 * Main database class. Created once in index.ts and shared with all managers.
 * 
 * Key methods:
 * - `storeFact(fact)` — INSERT OR REPLACE with full column list
 * - `searchFacts(query)` — FTS5 full-text search on fact text
 * - `getActiveFacts()` — all non-superseded facts
 * - `supersedeFact(oldId, newId)` — mark fact as replaced
 * - `raw` — direct better-sqlite3 Database for custom queries
 * 
 * Schema migrations run automatically in constructor (additive only).
 */
export class MemoriaDB {
  /** Direct better-sqlite3 access for custom queries from other modules */
  readonly raw: Database.Database;
  private db: Database.Database;

  constructor(workspaceRoot: string) {
    const dbPath = path.join(workspaceRoot, "memory", "memoria.db");
    const dir = path.dirname(dbPath);
    if (!fs.existsSync(dir)) fs.mkdirSync(dir, { recursive: true });

    // Auto-migrate from cortex.db if memoria.db doesn't exist or is empty
    const legacyPath = path.join(workspaceRoot, "memory", "cortex.db");
    if (fs.existsSync(legacyPath)) {
      const needsMigration = !fs.existsSync(dbPath) || fs.statSync(dbPath).size < 8192;
      const legacySize = fs.statSync(legacyPath).size;
      if (needsMigration && legacySize > 8192) {
        // Use VACUUM INTO to safely copy WAL-mode DBs (plain cp can lose data)
        try {
          const legacyDb = new Database(legacyPath, { readonly: true });
          legacyDb.exec(`VACUUM INTO '${dbPath.replace(/'/g, "''")}'`);
          legacyDb.close();
        } catch (_e) {
          // Fallback: copy file + WAL + SHM
          fs.copyFileSync(legacyPath, dbPath);
          const walPath = legacyPath + "-wal";
          const shmPath = legacyPath + "-shm";
          if (fs.existsSync(walPath)) fs.copyFileSync(walPath, dbPath + "-wal");
          if (fs.existsSync(shmPath)) fs.copyFileSync(shmPath, dbPath + "-shm");
        }
      }
    }

    this.db = new Database(dbPath);
    this.raw = this.db;
    this.db.pragma("journal_mode = WAL");    // Better concurrent read perf
    this.db.pragma("foreign_keys = ON");
    this.migrate();
  }

  // ─── Schema Migration ───

  private migrate(): void {
    const version = this.getSchemaVersion();
    if (version < 1) this.migrateV1();
    // V2: add fact_type column for semantic/episodic distinction
    this.migrateAddFactType();
    this.migrateAddFeedbackColumns();
    this.migrateAddRelevanceWeight();
    this.migrateAddIdentityCache();
    this.migrateAddLifecycleState();
    this.migrateAddProcedures();
    this.migrateAddClusterMembers();
    this.setSchemaVersion(SCHEMA_VERSION);
  }

  private migrateAddFactType(): void {
    try {
      // Check if column exists
      const cols = this.db.prepare("PRAGMA table_info(facts)").all() as Array<{ name: string }>;
      if (!cols.some(c => c.name === "fact_type")) {
        this.db.exec("ALTER TABLE facts ADD COLUMN fact_type TEXT DEFAULT 'semantic'");
        // Index for filtering
        this.db.exec("CREATE INDEX IF NOT EXISTS idx_facts_type ON facts(fact_type)");
      }
    } catch (_e) { /* column already exists or table not yet created */ }
  }

  /** Migration: add feedback loop columns (usefulness, recall_count, used_count) */
  private migrateAddFeedbackColumns(): void {
    try {
      const cols = this.db.prepare("PRAGMA table_info(facts)").all() as Array<{ name: string }>;
      const colNames = new Set(cols.map(c => c.name));
      if (!colNames.has("usefulness")) {
        this.db.exec("ALTER TABLE facts ADD COLUMN usefulness REAL DEFAULT 0");
      }
      if (!colNames.has("recall_count")) {
        this.db.exec("ALTER TABLE facts ADD COLUMN recall_count INTEGER DEFAULT 0");
      }
      if (!colNames.has("used_count")) {
        this.db.exec("ALTER TABLE facts ADD COLUMN used_count INTEGER DEFAULT 0");
      }
    } catch (_e) { /* columns already exist or table not yet created */ }
  }

  /** Migration: add relevance_weight column for identity-aware prioritization */
  private migrateAddRelevanceWeight(): void {
    try {
      const cols = this.db.prepare("PRAGMA table_info(facts)").all() as Array<{ name: string }>;
      if (!cols.some(c => c.name === "relevance_weight")) {
        this.db.exec("ALTER TABLE facts ADD COLUMN relevance_weight REAL DEFAULT 0.5");
        // Index for sorting by relevance
        this.db.exec("CREATE INDEX IF NOT EXISTS idx_facts_relevance ON facts(relevance_weight DESC)");
      }
    } catch (_e) { /* column already exists or table not yet created */ }
  }

  /** Migration: add identity_cache table for parsed USER.md/COMPANY.md */
  private migrateAddIdentityCache(): void {
    try {
      this.db.exec(`
        CREATE TABLE IF NOT EXISTS identity_cache (
          key TEXT PRIMARY KEY,
          value TEXT NOT NULL,
          updated_at INTEGER NOT NULL
        );
      `);
    } catch (_e) { /* table already exists */ }
  }

  /** Migration: add lifecycle_state for fact evolution (fresh/settled/dormant) */
  private migrateAddLifecycleState(): void {
    try {
      const cols = this.db.prepare("PRAGMA table_info(facts)").all() as Array<{ name: string }>;
      if (!cols.some(c => c.name === "lifecycle_state")) {
        this.db.exec("ALTER TABLE facts ADD COLUMN lifecycle_state TEXT DEFAULT 'fresh'");
        this.db.exec("CREATE INDEX IF NOT EXISTS idx_facts_lifecycle ON facts(lifecycle_state)");
      }
    } catch (_e) { /* column already exists or table not yet created */ }
  }

  /** Migration: add procedures table for procedural memory (Phase 3) */
  private migrateAddProcedures(): void {
    try {
      this.db.exec(`
        CREATE TABLE IF NOT EXISTS procedures (
          id TEXT PRIMARY KEY,
          name TEXT NOT NULL,
          goal TEXT,
          steps TEXT NOT NULL,
          success_count INTEGER DEFAULT 0,
          failure_count INTEGER DEFAULT 0,
          last_success_at INTEGER,
          last_failure_at INTEGER,
          last_updated_at INTEGER NOT NULL,
          avg_duration_ms INTEGER,
          improvements TEXT DEFAULT '[]',
          context TEXT,
          degradation_score REAL DEFAULT 0.0,
          alternative_of TEXT
        );
        CREATE INDEX IF NOT EXISTS idx_procedures_name ON procedures(name);
        CREATE INDEX IF NOT EXISTS idx_procedures_degradation ON procedures(degradation_score);
        CREATE INDEX IF NOT EXISTS idx_procedures_success_rate ON procedures(success_count, failure_count);
      `);
    } catch (_e) { /* table already exists */ }
  }

  /** Migration: add cluster_members table to track which facts compose a cluster */
  private migrateAddClusterMembers(): void {
    try {
      this.db.exec(`
        CREATE TABLE IF NOT EXISTS cluster_members (
          cluster_id TEXT NOT NULL,
          fact_id TEXT NOT NULL,
          PRIMARY KEY (cluster_id, fact_id),
          FOREIGN KEY (cluster_id) REFERENCES facts(id) ON DELETE CASCADE,
          FOREIGN KEY (fact_id) REFERENCES facts(id) ON DELETE CASCADE
        );
        CREATE INDEX IF NOT EXISTS idx_cluster_members_fact ON cluster_members(fact_id);
      `);
    } catch (_e) { /* table already exists */ }
  }

  private getSchemaVersion(): number {
    try {
      const row = this.db.prepare("SELECT value FROM meta WHERE key = 'schema_version'").get() as { value: string } | undefined;
      return row ? parseInt(row.value, 10) : 0;
    } catch (_e) {
      return 0; // meta table doesn't exist yet
    }
  }

  private setSchemaVersion(v: number): void {
    this.db.prepare("INSERT OR REPLACE INTO meta (key, value) VALUES ('schema_version', ?)").run(String(v));
  }

  private migrateV1(): void {
    this.db.exec(`
      -- Meta table
      CREATE TABLE IF NOT EXISTS meta (
        key TEXT PRIMARY KEY,
        value TEXT NOT NULL
      );

      -- Facts (mémoire déclarative)
      CREATE TABLE IF NOT EXISTS facts (
        id TEXT PRIMARY KEY,
        fact TEXT NOT NULL,
        category TEXT NOT NULL DEFAULT 'savoir',
        confidence REAL NOT NULL DEFAULT 0.8,
        source TEXT DEFAULT 'auto-capture',
        tags TEXT DEFAULT '[]',
        agent TEXT DEFAULT 'koda',
        created_at INTEGER NOT NULL,
        updated_at INTEGER NOT NULL,
        access_count INTEGER DEFAULT 0,
        last_accessed_at INTEGER,
        superseded INTEGER DEFAULT 0,
        superseded_by TEXT,
        superseded_at INTEGER,
        md_file TEXT,
        md_line INTEGER,
        entity_ids TEXT DEFAULT '[]',
        fact_type TEXT DEFAULT 'semantic'
      );

      -- FTS5 full-text search on facts
      CREATE VIRTUAL TABLE IF NOT EXISTS facts_fts USING fts5(
        fact, category, tags,
        content='facts',
        content_rowid='rowid'
      );

      -- Triggers to keep FTS in sync
      CREATE TRIGGER IF NOT EXISTS facts_ai AFTER INSERT ON facts BEGIN
        INSERT INTO facts_fts(rowid, fact, category, tags)
        VALUES (new.rowid, new.fact, new.category, new.tags);
      END;

      CREATE TRIGGER IF NOT EXISTS facts_ad AFTER DELETE ON facts BEGIN
        INSERT INTO facts_fts(facts_fts, rowid, fact, category, tags)
        VALUES ('delete', old.rowid, old.fact, old.category, old.tags);
      END;

      CREATE TRIGGER IF NOT EXISTS facts_au AFTER UPDATE ON facts BEGIN
        INSERT INTO facts_fts(facts_fts, rowid, fact, category, tags)
        VALUES ('delete', old.rowid, old.fact, old.category, old.tags);
        INSERT INTO facts_fts(rowid, fact, category, tags)
        VALUES (new.rowid, new.fact, new.category, new.tags);
      END;

      -- Embeddings (vecteurs 768d)
      CREATE TABLE IF NOT EXISTS embeddings (
        fact_id TEXT PRIMARY KEY,
        vector BLOB NOT NULL,
        model TEXT NOT NULL,
        created_at INTEGER NOT NULL,
        FOREIGN KEY (fact_id) REFERENCES facts(id) ON DELETE CASCADE
      );

      -- Knowledge Graph: entities
      CREATE TABLE IF NOT EXISTS entities (
        id TEXT PRIMARY KEY,
        name TEXT NOT NULL,
        type TEXT NOT NULL DEFAULT 'concept',
        attributes TEXT DEFAULT '{}',
        created_at INTEGER NOT NULL,
        access_count INTEGER DEFAULT 0
      );
      CREATE INDEX IF NOT EXISTS idx_entities_name ON entities(name);
      CREATE INDEX IF NOT EXISTS idx_entities_type ON entities(type);

      -- Knowledge Graph: relations
      CREATE TABLE IF NOT EXISTS relations (
        id TEXT PRIMARY KEY,
        source_id TEXT NOT NULL,
        target_id TEXT NOT NULL,
        relation TEXT NOT NULL,
        weight REAL DEFAULT 1.0,
        context TEXT,
        created_at INTEGER NOT NULL,
        last_accessed_at INTEGER,
        FOREIGN KEY (source_id) REFERENCES entities(id) ON DELETE CASCADE,
        FOREIGN KEY (target_id) REFERENCES entities(id) ON DELETE CASCADE
      );
      CREATE INDEX IF NOT EXISTS idx_relations_source ON relations(source_id);
      CREATE INDEX IF NOT EXISTS idx_relations_target ON relations(target_id);

      -- Chunks (fichiers .md indexés)
      CREATE TABLE IF NOT EXISTS chunks (
        id TEXT PRIMARY KEY,
        file_path TEXT NOT NULL,
        chunk_text TEXT NOT NULL,
        chunk_index INTEGER DEFAULT 0,
        vector BLOB,
        updated_at INTEGER NOT NULL
      );
      CREATE INDEX IF NOT EXISTS idx_chunks_path ON chunks(file_path);

      -- FTS5 on chunks
      CREATE VIRTUAL TABLE IF NOT EXISTS chunks_fts USING fts5(
        chunk_text, file_path,
        content='chunks',
        content_rowid='rowid'
      );

      CREATE TRIGGER IF NOT EXISTS chunks_ai AFTER INSERT ON chunks BEGIN
        INSERT INTO chunks_fts(rowid, chunk_text, file_path)
        VALUES (new.rowid, new.chunk_text, new.file_path);
      END;

      CREATE TRIGGER IF NOT EXISTS chunks_ad AFTER DELETE ON chunks BEGIN
        INSERT INTO chunks_fts(chunks_fts, rowid, chunk_text, file_path)
        VALUES ('delete', old.rowid, old.chunk_text, old.file_path);
      END;

      CREATE TRIGGER IF NOT EXISTS chunks_au AFTER UPDATE ON chunks BEGIN
        INSERT INTO chunks_fts(chunks_fts, rowid, chunk_text, file_path)
        VALUES ('delete', old.rowid, old.chunk_text, old.file_path);
        INSERT INTO chunks_fts(rowid, chunk_text, file_path)
        VALUES (new.rowid, new.chunk_text, new.file_path);
      END;

      -- Indexes for common queries
      CREATE INDEX IF NOT EXISTS idx_facts_category ON facts(category);
      CREATE INDEX IF NOT EXISTS idx_facts_superseded ON facts(superseded);
      CREATE INDEX IF NOT EXISTS idx_facts_created ON facts(created_at);
      CREATE INDEX IF NOT EXISTS idx_facts_agent ON facts(agent);
      CREATE INDEX IF NOT EXISTS idx_facts_type ON facts(fact_type);
    `);
  }

  // ─── Facts CRUD ───

  storeFact(fact: Omit<Fact, "access_count" | "last_accessed_at" | "superseded" | "superseded_by" | "superseded_at" | "md_file" | "md_line" | "entity_ids" | "usefulness" | "recall_count" | "used_count" | "synced_to_md" | "relevance_weight" | "lifecycle_state"> & Partial<Fact>): Fact {
    const now = Date.now();
    const row: Fact = {
      id: fact.id || `fact_${now}_${Math.random().toString(36).slice(2, 9)}`,
      fact: fact.fact,
      category: fact.category || "savoir",
      confidence: fact.confidence ?? 0.8,
      source: fact.source || "auto-capture",
      tags: fact.tags || "[]",
      agent: fact.agent || "koda",
      created_at: fact.created_at || now,
      updated_at: fact.updated_at || now,
      access_count: fact.access_count ?? 0,
      last_accessed_at: fact.last_accessed_at ?? null,
      superseded: fact.superseded ?? 0,
      superseded_by: fact.superseded_by ?? null,
      superseded_at: fact.superseded_at ?? null,
      md_file: fact.md_file ?? null,
      md_line: fact.md_line ?? null,
      entity_ids: fact.entity_ids || "[]",
      fact_type: fact.fact_type || "semantic",
      usefulness: fact.usefulness ?? 0,
      recall_count: fact.recall_count ?? 0,
      used_count: fact.used_count ?? 0,
      synced_to_md: fact.synced_to_md ?? 0,
      relevance_weight: fact.relevance_weight ?? 0.5,
      lifecycle_state: fact.lifecycle_state ?? "fresh",
    };

    this.db.prepare(`
      INSERT OR REPLACE INTO facts
      (id, fact, category, confidence, source, tags, agent, created_at, updated_at,
       access_count, last_accessed_at, superseded, superseded_by, superseded_at,
       md_file, md_line, entity_ids, fact_type,
       usefulness, recall_count, used_count, synced_to_md, relevance_weight, lifecycle_state)
      VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
    `).run(
      row.id, row.fact, row.category, row.confidence, row.source, row.tags, row.agent,
      row.created_at, row.updated_at, row.access_count, row.last_accessed_at,
      row.superseded, row.superseded_by, row.superseded_at,
      row.md_file, row.md_line, row.entity_ids, row.fact_type,
      row.usefulness, row.recall_count, row.used_count, row.synced_to_md,
      row.relevance_weight, row.lifecycle_state
    );

    return row;
  }

  searchFacts(query: string, limit = 10): Fact[] {
    if (!query || query.trim().length === 0) {
      return this.db.prepare(
        "SELECT * FROM facts WHERE superseded = 0 AND lifecycle_state != 'dormant' ORDER BY updated_at DESC LIMIT ?"
      ).all(limit) as Fact[];
    }

    // Sanitize for FTS5: keep only alphanumeric + spaces, wrap each word in quotes
    const sanitized = query
      .replace(/[^\p{L}\p{N}\s]/gu, " ")  // Remove special chars
      .split(/\s+/)
      .filter(w => w.length > 1)
      .map(w => `"${w}"`)                  // Quote each word
      .join(" OR ");                        // OR between words

    if (!sanitized) {
      return this.db.prepare(
        "SELECT * FROM facts WHERE superseded = 0 ORDER BY updated_at DESC LIMIT ?"
      ).all(limit) as Fact[];
    }

    try {
      return this.db.prepare(`
        SELECT f.* FROM facts f
        JOIN facts_fts fts ON f.rowid = fts.rowid
        WHERE facts_fts MATCH ? AND f.superseded = 0 AND f.lifecycle_state != 'dormant'
        ORDER BY rank
        LIMIT ?
      `).all(sanitized, limit) as Fact[];
    } catch (_e) {
      // Fallback: LIKE search if FTS5 fails
      return this.db.prepare(
        "SELECT * FROM facts WHERE superseded = 0 AND lifecycle_state != 'dormant' AND fact LIKE ? ORDER BY updated_at DESC LIMIT ?"
      ).all(`%${query.slice(0, 100)}%`, limit) as Fact[];
    }
  }

  recentFacts(hours = 24, limit = 10): Fact[] {
    const cutoff = Date.now() - hours * 3600 * 1000;
    return this.db.prepare(
      "SELECT * FROM facts WHERE superseded = 0 AND created_at >= ? ORDER BY created_at DESC LIMIT ?"
    ).all(cutoff, limit) as Fact[];
  }

  getFact(id: string): Fact | undefined {
    return this.db.prepare("SELECT * FROM facts WHERE id = ?").get(id) as Fact | undefined;
  }

  supersedeFact(oldId: string, newId: string): void {
    const now = Date.now();
    this.db.prepare(
      "UPDATE facts SET superseded = 1, superseded_by = ?, superseded_at = ?, updated_at = ? WHERE id = ?"
    ).run(newId, now, now, oldId);
  }

  enrichFact(id: string, newText: string, newConfidence?: number): void {
    const now = Date.now();
    const existing = this.getFact(id);
    if (!existing) return;
    const confidence = newConfidence ? Math.max(existing.confidence, newConfidence) : existing.confidence;
    this.db.prepare(
      "UPDATE facts SET fact = ?, confidence = ?, updated_at = ? WHERE id = ?"
    ).run(newText, confidence, now, id);
  }

  /** Get frequently accessed facts (hot tier — "learned by heart") */
  hotFacts(minAccess: number = 5, staleDays: number = 30, limit: number = 5): Fact[] {
    const cutoff = Date.now() - staleDays * 24 * 60 * 60 * 1000;
    return this.db.prepare(
      `SELECT * FROM facts WHERE superseded = 0 AND lifecycle_state != 'dormant' AND access_count >= ? 
       AND COALESCE(last_accessed_at, updated_at) >= ?
       ORDER BY access_count DESC LIMIT ?`
    ).all(minAccess, cutoff, limit) as Fact[];
  }

  trackAccess(ids: string[]): void {
    const now = Date.now();
    const stmt = this.db.prepare(
      "UPDATE facts SET access_count = access_count + 1, last_accessed_at = ? WHERE id = ?"
    );
    const tx = this.db.transaction(() => {
      for (const id of ids) stmt.run(now, id);
    });
    tx();
  }

  // ─── Stats ───

  stats(): { total: number; active: number; superseded: number; categories: Record<string, number> } {
    const total = (this.db.prepare("SELECT COUNT(*) as c FROM facts").get() as { c: number }).c;
    const active = (this.db.prepare("SELECT COUNT(*) as c FROM facts WHERE superseded = 0").get() as { c: number }).c;
    const cats = this.db.prepare(
      "SELECT category, COUNT(*) as c FROM facts WHERE superseded = 0 GROUP BY category"
    ).all() as Array<{ category: string; c: number }>;
    
    const categories: Record<string, number> = {};
    for (const row of cats) categories[row.category] = row.c;
    
    return { total, active, superseded: total - active, categories };
  }

  // ─── Entities CRUD ───

  storeEntity(entity: Omit<Entity, "created_at" | "access_count"> & Partial<Entity>): Entity {
    const now = Date.now();
    const row: Entity = {
      id: entity.id || `ent_${now}_${Math.random().toString(36).slice(2, 9)}`,
      name: entity.name,
      type: entity.type || "concept",
      attributes: entity.attributes || "{}",
      created_at: entity.created_at || now,
      access_count: entity.access_count ?? 0,
    };

    this.db.prepare(`
      INSERT OR REPLACE INTO entities (id, name, type, attributes, created_at, access_count)
      VALUES (?, ?, ?, ?, ?, ?)
    `).run(row.id, row.name, row.type, row.attributes, row.created_at, row.access_count);

    return row;
  }

  findEntityByName(name: string): Entity | undefined {
    return this.db.prepare(
      "SELECT * FROM entities WHERE LOWER(name) = LOWER(?)"
    ).get(name) as Entity | undefined;
  }

  /** Return all entity names (lowercased) for fast in-text matching */
  allEntityNames(): string[] {
    const rows = this.db.prepare("SELECT DISTINCT LOWER(name) as name FROM entities").all() as { name: string }[];
    return rows.map(r => r.name);
  }

  /** Find facts linked to any of the given entity IDs */
  findFactsByEntityIds(entityIds: string[], limit = 10): Fact[] {
    if (entityIds.length === 0) return [];
    // entity_ids is stored as JSON array string, e.g. '["ent_123","ent_456"]'
    const placeholders = entityIds.map(() => "entity_ids LIKE ?").join(" OR ");
    const params = entityIds.map(id => `%${id}%`);
    return this.db.prepare(
      `SELECT * FROM facts WHERE status = 'active' AND (${placeholders}) ORDER BY updated_at DESC LIMIT ?`
    ).all(...params, limit) as Fact[];
  }

  // ─── Relations CRUD ───

  storeRelation(rel: Omit<Relation, "created_at" | "last_accessed_at"> & Partial<Relation>): Relation {
    const now = Date.now();
    const row: Relation = {
      id: rel.id || `rel_${now}_${Math.random().toString(36).slice(2, 9)}`,
      source_id: rel.source_id,
      target_id: rel.target_id,
      relation: rel.relation,
      weight: rel.weight ?? 1.0,
      context: rel.context ?? null,
      created_at: rel.created_at || now,
      last_accessed_at: rel.last_accessed_at ?? null,
    };

    this.db.prepare(`
      INSERT OR REPLACE INTO relations
      (id, source_id, target_id, relation, weight, context, created_at, last_accessed_at)
      VALUES (?, ?, ?, ?, ?, ?, ?, ?)
    `).run(row.id, row.source_id, row.target_id, row.relation, row.weight, row.context, row.created_at, row.last_accessed_at);

    return row;
  }

  getRelationsFrom(entityId: string, minWeight = 0.1): Relation[] {
    return this.db.prepare(
      "SELECT * FROM relations WHERE source_id = ? AND weight >= ? ORDER BY weight DESC"
    ).all(entityId, minWeight) as Relation[];
  }

  getRelationsTo(entityId: string, minWeight = 0.1): Relation[] {
    return this.db.prepare(
      "SELECT * FROM relations WHERE target_id = ? AND weight >= ? ORDER BY weight DESC"
    ).all(entityId, minWeight) as Relation[];
  }

  reinforceRelation(id: string, boost = 0.1): void {
    const now = Date.now();
    this.db.prepare(
      "UPDATE relations SET weight = weight + ?, last_accessed_at = ? WHERE id = ?"
    ).run(boost, now, id);
  }

  // ─── Bulk import ───

  importFacts(facts: Array<Omit<Fact, "access_count" | "last_accessed_at" | "superseded" | "superseded_by" | "superseded_at" | "md_file" | "md_line" | "entity_ids"> & Partial<Fact>>): number {
    const tx = this.db.transaction(() => {
      let count = 0;
      for (const fact of facts) {
        this.storeFact(fact);
        count++;
      }
      return count;
    });
    return tx();
  }

  // ─── Identity Cache ───

  /** Store identity cache (JSON stringified) */
  storeIdentityCache(key: string, value: string): void {
    const now = Date.now();
    this.db.prepare("INSERT OR REPLACE INTO identity_cache (key, value, updated_at) VALUES (?, ?, ?)").run(key, value, now);
  }

  /** Get identity cache */
  getIdentityCache(key: string): string | null {
    const row = this.db.prepare("SELECT value FROM identity_cache WHERE key = ?").get(key) as { value: string } | undefined;
    return row?.value ?? null;
  }

  // ─── Close ───

  close(): void {
    this.db.close();
  }
}


path: core/dialectic.ts
content: /**
 * Memoria — Dialectic Memory (Layer 24)
 *
 * Enables natural language queries against the entire memory.
 * Instead of keyword search, the agent can ASK questions:
 *   "What frustrates the user about deployments?"
 *   "What's the current status of Primask?"
 *   "What patterns do I repeat when I make mistakes?"
 *
 * Inspired by Honcho's "dialectic" concept — memory as a conversation partner.
 *
 * Flow:
 * 1. Parse the natural language query
 * 2. Fan out: FTS5 + embeddings + graph + topics + procedures
 * 3. Aggregate all evidence
 * 4. Synthesize a structured answer via LLM
 */

import type { MemoriaDB, Fact } from "./db.js";
import type { EmbeddingManager } from "./embeddings.js";
import type { KnowledgeGraph } from "./graph.js";
import type { TopicManager } from "./topics.js";
import type { ProceduralMemory } from "./procedural.js";
import type { ObservationManager } from "./observations.js";
import type { LLMProvider } from "./providers/types.js";

export interface DialecticAnswer {
  answer: string;
  confidence: number;
  sources: Array<{
    type: "fact" | "procedure" | "observation" | "graph" | "topic";
    id: string;
    text: string;
    relevance: number;
  }>;
  reasoning?: string;
}

export interface DialecticDeps {
  db: MemoriaDB;
  embeddingMgr: EmbeddingManager;
  graph: KnowledgeGraph;
  topicMgr: TopicManager;
  proceduralMem: ProceduralMemory;
  observationMgr: ObservationManager;
  llm: LLMProvider;
}

export class DialecticMemory {
  private deps: DialecticDeps;

  constructor(deps: DialecticDeps) {
    this.deps = deps;
  }

  /**
   * Ask a natural language question to the memory.
   */
  async query(question: string): Promise<DialecticAnswer> {
    const { db, embeddingMgr, graph, topicMgr, proceduralMem, observationMgr, llm } = this.deps;

    // ── 1. Fan out: gather evidence from all sources ──
    const sources: DialecticAnswer["sources"] = [];

    // FTS5 + embeddings search
    try {
      if (embeddingMgr.embeddedCount() > 0) {
        const results = await embeddingMgr.hybridSearch(question, 10, {
          ftsWeight: 0.35,
          cosineWeight: 0.45,
          temporalWeight: 0.20,
        });
        for (const r of results) {
          sources.push({
            type: "fact",
            id: r.id,
            text: r.fact,
            relevance: r.temporalScore || 0.5,
          });
        }
      } else {
        const facts = db.searchFacts(question, 10);
        for (const f of facts) {
          sources.push({
            type: "fact",
            id: f.id,
            text: f.fact,
            relevance: f.confidence,
          });
        }
      }
    } catch { /* non-blocking */ }

    // Graph entities
    try {
      const entities = graph.findEntitiesInText(question);
      if (entities.length > 0) {
        const related = graph.getRelatedFacts(entities.map(e => e.name), 3, 5);
        for (const r of related) {
          const fact = db.getFact(r.id);
          if (fact && !sources.find(s => s.id === r.id)) {
            sources.push({
              type: "graph",
              id: r.id,
              text: fact.fact,
              relevance: 0.6,
            });
          }
        }
      }
    } catch { /* non-blocking */ }

    // Topics
    try {
      const expandedQueries = embeddingMgr.expandQuery(question);
      const topics = await topicMgr.findRelevantTopics(question, 3, expandedQueries);
      for (const t of topics) {
        sources.push({
          type: "topic",
          id: t.topic.id?.toString() || t.topic.name,
          text: `Topic "${t.topic.name}": ${t.facts.slice(0, 3).join("; ")}`,
          relevance: 0.5,
        });
      }
    } catch { /* non-blocking */ }

    // Procedures
    try {
      const procs = proceduralMem.search(question, 3);
      for (const p of procs) {
        sources.push({
          type: "procedure",
          id: p.id,
          text: `${p.name}: ${p.goal} (${p.steps.length} steps, ${p.success_count} successes)`,
          relevance: p.quality.overall,
        });
      }
    } catch { /* non-blocking */ }

    // Observations
    try {
      const obs = await observationMgr.getRelevantObservations(question);
      for (const o of obs) {
        const ob = (o as any).observation || o;
        sources.push({
          type: "observation",
          id: ob.id?.toString() || "obs",
          text: ob.content || ob.summary || ob.title || "",
          relevance: (o as any).score || 0.7,
        });
      }
    } catch { /* non-blocking */ }

    // ── 2. If no sources found, return early ──
    if (sources.length === 0) {
      return {
        answer: "Je n'ai trouvé aucune information pertinente dans ma mémoire sur ce sujet.",
        confidence: 0,
        sources: [],
      };
    }

    // Sort by relevance
    sources.sort((a, b) => b.relevance - a.relevance);
    const topSources = sources.slice(0, 15);

    // ── 3. Synthesize answer via LLM ──
    const synthesisPrompt = `You are a memory assistant. Answer the following question based ONLY on the evidence provided.
Be specific and cite which sources support your answer.
If evidence is contradictory, mention both sides.
If evidence is insufficient, say so honestly.

Question: ${question}

Evidence (sorted by relevance):
${topSources.map((s, i) => `[${i + 1}] (${s.type}, relevance: ${(s.relevance * 100).toFixed(0)}%) ${s.text}`).join("\n")}

Answer in the user's language (French if the question is in French).
Be concise but thorough. Output JSON:
{
  "answer": "Your synthesized answer",
  "confidence": 0.0-1.0,
  "reasoning": "Brief explanation of how you reached this answer"
}`;

    try {
      const response = await llm.generateWithMeta!(synthesisPrompt, {
        maxTokens: 1024,
        temperature: 0.2,
        format: "json",
        timeoutMs: 30000,
      });

      if (response?.response) {
        const cleaned = response.response.replace(/```json\n?|\n?```/g, "").trim();
        const parsed = JSON.parse(cleaned);
        return {
          answer: parsed.answer || "Pas de réponse générée.",
          confidence: parsed.confidence || 0.5,
          sources: topSources,
          reasoning: parsed.reasoning,
        };
      }
    } catch {
      // Fallback: return raw sources without synthesis
    }

    // Fallback: concatenate top sources
    return {
      answer: topSources.slice(0, 5).map(s => `• ${s.text}`).join("\n"),
      confidence: 0.3,
      sources: topSources,
      reasoning: "LLM synthesis failed — returning raw evidence.",
    };
  }

  /**
   * Quick factual lookup (no LLM, just search).
   * Faster but less intelligent than full query().
   */
  quickLookup(question: string, limit = 5): Fact[] {
    return this.deps.db.searchFacts(question, limit);
  }
}


path: core/embed-fallback.ts
content: /**
 * Memoria — Embed Fallback Provider
 * 
 * Wraps multiple EmbedProviders and tries them in order.
 * If provider 1 fails → try provider 2 → try provider 3.
 * Analogous to FallbackChain for LLMProviders.
 */

import type { EmbedProvider } from "./providers/types.js";

export class EmbedFallback implements EmbedProvider {
  private providers: EmbedProvider[];
  private _dimensions: number;
  private _name: string;
  private logger?: { info?: (...args: unknown[]) => void; warn?: (...args: unknown[]) => void };

  constructor(providers: EmbedProvider[], logger?: { info?: (...args: unknown[]) => void; warn?: (...args: unknown[]) => void }) {
    if (providers.length === 0) throw new Error("EmbedFallback requires at least one provider");
    this.providers = providers;
    this._dimensions = providers[0].dimensions;
    this._name = `embed-fallback(${providers.map(p => p.name).join("→")})`;
    this.logger = logger;
  }

  get dimensions(): number {
    return this._dimensions;
  }

  get name(): string {
    return this._name;
  }

  get providerNames(): string[] {
    return this.providers.map(p => p.name);
  }

  async embed(text: string): Promise<number[]> {
    let lastErr: Error | null = null;
    for (const provider of this.providers) {
      try {
        return await provider.embed(text);
      } catch (err) {
        lastErr = err instanceof Error ? err : new Error(String(err));
        this.logger?.warn?.(`memoria: embed fallback — ${provider.name} failed: ${lastErr.message}`);
      }
    }
    throw lastErr || new Error("All embed providers failed");
  }

  async embedBatch(texts: string[]): Promise<number[][]> {
    let lastErr: Error | null = null;
    for (const provider of this.providers) {
      try {
        return await provider.embedBatch(texts);
      } catch (err) {
        lastErr = err instanceof Error ? err : new Error(String(err));
        this.logger?.warn?.(`memoria: embedBatch fallback — ${provider.name} failed: ${lastErr.message}`);
      }
    }
    throw lastErr || new Error("All embed providers failed");
  }
}


path: core/embeddings.ts
content: /**
 * Memoria — Layer 4: Embeddings + Hybrid Search
 * 
 * Stores 768d float vectors in SQLite (BLOB), computes cosine similarity,
 * and provides hybrid search (FTS5 text match + cosine similarity + temporal scoring).
 * 
 * Key methods:
 *   embedFact(id, text) — compute and store embedding for one fact
 *   embedBatch() — batch process all unembedded facts
 *   hybridSearch(query) — combined FTS5 + cosine + scoring
 *   cosineSimilarity(a, b) — vector distance (exported utility)
 * 
 * Les vecteurs sont stockés en Float32Array → BLOB pour perf maximale.
 */

import type { MemoriaDB, Fact } from "./db.js";
import type { EmbedProvider } from "./providers/types.js";
import type { ScoredFact } from "./scoring.js";
import { scoreFact } from "./scoring.js";

// ─── Vector Utils ───

/** Float32Array → Buffer (for SQLite BLOB storage) */
export function vectorToBlob(vec: number[]): Buffer {
  return Buffer.from(new Float32Array(vec).buffer);
}

/** Buffer (SQLite BLOB) → number[] */
export function blobToVector(blob: Buffer): number[] {
  const f32 = new Float32Array(blob.buffer, blob.byteOffset, blob.byteLength / 4);
  return Array.from(f32);
}

/** Cosine similarity between two vectors. Returns 0-1. */
export function cosineSimilarity(a: number[], b: number[]): number {
  if (a.length !== b.length) return 0;
  let dot = 0, normA = 0, normB = 0;
  for (let i = 0; i < a.length; i++) {
    dot += a[i] * b[i];
    normA += a[i] * a[i];
    normB += b[i] * b[i];
  }
  const denom = Math.sqrt(normA) * Math.sqrt(normB);
  return denom === 0 ? 0 : dot / denom;
}

// ─── Types ───

export interface EmbeddedFact extends Fact {
  similarity: number;
  temporalScore: number;
  hybridScore: number;
}

interface EmbeddingRow {
  fact_id: string;
  vector: Buffer;
  model: string;
}

// ─── Embedding Manager ───

export class EmbeddingManager {
  private db: MemoriaDB;
  private provider: EmbedProvider;
  private modelName: string;

  constructor(db: MemoriaDB, provider: EmbedProvider) {
    this.db = db;
    this.provider = provider;
    this.modelName = provider.name;
  }

  /** Get raw DB handle for direct queries */
  private get rawDb() {
    return this.db.raw;
  }

  // ─── Store ───

  /** Embed a single fact and store the vector */
  async embedFact(factId: string, text: string): Promise<void> {
    const vector = await this.provider.embed(text);
    const blob = vectorToBlob(vector);
    this.rawDb.prepare(
      "INSERT OR REPLACE INTO embeddings (fact_id, vector, model, created_at) VALUES (?, ?, ?, ?)"
    ).run(factId, blob, this.modelName, Date.now());
  }

  /** Embed multiple facts in batch */
  async embedBatch(facts: Array<{ id: string; text: string }>): Promise<number> {
    if (facts.length === 0) return 0;

    // Batch embed (max 32 at a time to avoid timeout)
    const BATCH_SIZE = 32;
    let embedded = 0;

    for (let i = 0; i < facts.length; i += BATCH_SIZE) {
      const batch = facts.slice(i, i + BATCH_SIZE);
      const texts = batch.map(f => f.text);

      try {
        const vectors = await this.provider.embedBatch(texts);
        const stmt = this.rawDb.prepare(
          "INSERT OR REPLACE INTO embeddings (fact_id, vector, model, created_at) VALUES (?, ?, ?, ?)"
        );
        const now = Date.now();
        const tx = this.rawDb.transaction(() => {
          for (let j = 0; j < batch.length; j++) {
            stmt.run(batch[j].id, vectorToBlob(vectors[j]), this.modelName, now);
          }
        });
        tx();
        embedded += batch.length;
      } catch (err) {
        // Try one by one on batch failure
        for (const fact of batch) {
          try {
            await this.embedFact(fact.id, fact.text);
            embedded++;
          } catch (e) { console.debug('memoria:embeddings: ' + String(e)); }
        }
      }
    }

    return embedded;
  }

  // ─── Search ───

  /** Semantic search: embed query → cosine similarity with all stored vectors */
  async semanticSearch(query: string, limit = 10, minSimilarity = 0.3): Promise<EmbeddedFact[]> {
    const queryVector = await this.provider.embed(query);

    // Get all embeddings (we compute cosine in JS — fast enough for <10K facts)
    const rows = this.rawDb.prepare(
      "SELECT e.fact_id, e.vector FROM embeddings e JOIN facts f ON e.fact_id = f.id WHERE f.superseded = 0"
    ).all() as EmbeddingRow[];

    if (rows.length === 0) return [];

    // Compute similarities
    const scored: Array<{ factId: string; similarity: number }> = [];
    for (const row of rows) {
      const vec = blobToVector(row.vector);
      const sim = cosineSimilarity(queryVector, vec);
      if (sim >= minSimilarity) {
        scored.push({ factId: row.fact_id, similarity: sim });
      }
    }

    // Sort by similarity, take top N
    scored.sort((a, b) => b.similarity - a.similarity);
    const topIds = scored.slice(0, limit * 2); // Get extras for post-filtering

    // Fetch full facts
    const results: EmbeddedFact[] = [];
    for (const { factId, similarity } of topIds) {
      const fact = this.db.getFact(factId);
      if (!fact || fact.superseded) continue;

      const sf = scoreFact(fact);
      results.push({
        ...fact,
        similarity,
        temporalScore: sf.temporalScore,
        hybridScore: 0, // computed in hybridSearch
      });
    }

    return results.slice(0, limit);
  }

  // ─── Query Expansion ───

  /**
   * Expand a query into multiple variants for better recall.
   * Uses the embedding model to find semantically similar terms.
   * No LLM needed — pure heuristic expansion with synonym/concept maps.
   */
  expandQuery(query: string): string[] {
    const variants = [query];
    const lower = query.toLowerCase().trim();

    // Concept expansions: STRICT synonym pairs only (avoid noise from loose associations)
    const conceptMap: Record<string, string[]> = {
      // Money/salary — bidirectional synonyms
      "taux horaire": ["€/h", "salaire"],
      "salaire": ["taux horaire", "€/h"],
      "rémunération": ["salaire", "€/h"],
      "ca": ["chiffre d'affaires"],
      "chiffre d'affaires": ["CA"],
      // Tech — FR↔EN translations only
      "deploy": ["déploiement"],
      "déploiement": ["deploy"],
      "modèle": ["model"],
      "modèles": ["models"],
      // Config
      "config": ["configuration"],
      "configuration": ["config"],
    };

    // Check each concept key against the query
    for (const [key, synonyms] of Object.entries(conceptMap)) {
      if (lower.includes(key)) {
        // Add 1-2 best synonym variants
        for (const syn of synonyms.slice(0, 2)) {
          const variant = query.replace(new RegExp(key, "gi"), syn);
          if (variant !== query && !variants.includes(variant)) {
            variants.push(variant);
          }
        }
      }
    }

    // Entity extraction: if query contains a proper noun, add it standalone
    const properNouns = query.match(/\b[A-Z][a-z]+(?:\s[A-Z][a-z]+)*/g);
    if (properNouns) {
      for (const noun of properNouns) {
        if (noun.length > 2 && !variants.includes(noun)) {
          variants.push(noun);
        }
      }
    }

    return variants.slice(0, 4); // Max 4 variants
  }

  /** Hybrid search: FTS5 + cosine, merge and rank — with query expansion */
  async hybridSearch(query: string, limit = 10, options?: {
    ftsWeight?: number;    // Weight for FTS5 results (default 0.4)
    cosineWeight?: number; // Weight for cosine results (default 0.4)
    temporalWeight?: number; // Weight for temporal score (default 0.2)
    minSimilarity?: number;
    expandQueries?: boolean; // Enable query expansion (default true)
  }): Promise<EmbeddedFact[]> {
    // Adaptive weights: short/generic queries → favor semantic over FTS
    // because FTS on a 1-word query like "Bureau" matches too many facts
    const queryWords = query.trim().split(/\s+/).filter(w => w.length > 2);
    const isShortQuery = queryWords.length <= 2;
    const ftsW = options?.ftsWeight ?? (isShortQuery ? 0.20 : 0.40);
    const cosW = options?.cosineWeight ?? (isShortQuery ? 0.55 : 0.40);
    const tempW = options?.temporalWeight ?? (isShortQuery ? 0.25 : 0.20);
    const minSim = options?.minSimilarity ?? 0.25;

    // Query expansion: generate variants for better recall
    const doExpand = options?.expandQueries !== false;
    const queries = doExpand ? this.expandQuery(query) : [query];

    // 1. FTS5 search — across all query variants
    const allFtsResults: Fact[] = [];
    const seenFtsIds = new Set<string>();
    for (const q of queries) {
      const results = this.db.searchFacts(q, limit * 2);
      for (const f of results) {
        if (!seenFtsIds.has(f.id)) {
          seenFtsIds.add(f.id);
          allFtsResults.push(f);
        }
      }
    }

    // 2. Semantic search — across all query variants
    let allCosResults: EmbeddedFact[] = [];
    const seenCosIds = new Set<string>();
    for (const q of queries) {
      try {
        const results = await this.semanticSearch(q, limit * 2, minSim);
        for (const f of results) {
          if (!seenCosIds.has(f.id)) {
            seenCosIds.add(f.id);
            allCosResults.push(f);
          } else {
            // Keep highest similarity
            const existing = allCosResults.find(r => r.id === f.id);
            if (existing && f.similarity > existing.similarity) {
              existing.similarity = f.similarity;
            }
          }
        }
      } catch (e) {
        console.debug('memoria:embeddings: ' + String(e));
        // Embedding not available → FTS only
      }
    }

    // 3. Merge by fact ID
    const merged = new Map<string, EmbeddedFact>();

    // Add FTS results with rank-based score
    for (let i = 0; i < allFtsResults.length; i++) {
      const f = allFtsResults[i];
      const ftsScore = 1 - i / Math.max(allFtsResults.length, 1); // 1.0 for best, decreasing
      const sf = scoreFact(f);
      merged.set(f.id, {
        ...f,
        similarity: 0,
        temporalScore: sf.temporalScore,
        hybridScore: ftsScore * ftsW + sf.temporalScore * tempW,
      });
    }

    // Merge cosine results
    for (const cr of allCosResults) {
      const existing = merged.get(cr.id);
      if (existing) {
        // Boost: fact found by BOTH methods
        existing.similarity = cr.similarity;
        existing.hybridScore += cr.similarity * cosW;
      } else {
        const sf = scoreFact(cr);
        merged.set(cr.id, {
          ...cr,
          temporalScore: sf.temporalScore,
          hybridScore: cr.similarity * cosW + sf.temporalScore * tempW,
        });
      }
    }

    // 4. Sort by hybrid score, return top N
    const results = Array.from(merged.values());
    results.sort((a, b) => b.hybridScore - a.hybridScore);
    return results.slice(0, limit);
  }

  // ─── Stats ───

  /** Count of embedded facts */
  /**
   * Called when a fact is superseded — remove its embedding to prevent
   * stale results in semantic search.
   */
  onFactSuperseded(factId: string): boolean {
    try {
      const result = this.db.raw.prepare("DELETE FROM embeddings WHERE fact_id = ?").run(factId);
      return (result.changes ?? 0) > 0;
    } catch (e) { console.debug('memoria:embeddings: ' + String(e)); return false; }
  }

  embeddedCount(): number {
    return (this.rawDb.prepare("SELECT COUNT(*) as c FROM embeddings").get() as { c: number }).c;
  }

  /** Facts without embeddings */
  unembeddedFacts(limit = 100): Array<{ id: string; fact: string }> {
    return this.rawDb.prepare(`
      SELECT f.id, f.fact FROM facts f
      LEFT JOIN embeddings e ON f.id = e.fact_id
      WHERE e.fact_id IS NULL AND f.superseded = 0
      ORDER BY f.created_at DESC
      LIMIT ?
    `).all(limit) as Array<{ id: string; fact: string }>;
  }
}


path: core/example.ts
content: /**
 * Example: Using @primo-studio/memoria-core standalone
 * 
 * This demonstrates using Memoria without OpenClaw.
 * Run with: node --loader ts-node/esm example.ts
 */

import { Memoria } from './index.js';

async function main() {
  console.log('🧠 Initializing Memoria core...\n');

  // Initialize with Ollama (local, free)
  const memoria = await Memoria.init({
    dbPath: './example-memoria.db',
    provider: 'ollama',
    model: 'qwen3.5:4b',
    embeddingModel: 'nomic-embed-text-v2-moe',
    recallLimit: 5,
    debug: true
  });

  console.log('\n✅ Memoria initialized!\n');

  // Store some facts
  console.log('📝 Storing facts...\n');
  
  await memoria.store('User prefers dark mode in all applications', 'preference', 0.95);
  await memoria.store('User is located in New York, USA', 'savoir', 0.9);
  await memoria.store('User favorite programming language is TypeScript', 'preference', 0.85);
  await memoria.store('Project deadline is April 15, 2026', 'chronologie', 0.9);
  await memoria.store('User dislikes verbose error messages', 'preference', 0.8);

  console.log('✅ 5 facts stored!\n');

  // Recall facts
  console.log('🔍 Recalling: "What are the user preferences?"\n');
  const results = await memoria.recall('What are the user preferences?', { limit: 3 });
  
  console.log(`Found ${results.totalFound} facts:\n`);
  for (const fact of results.facts) {
    console.log(`  • [${fact.category}] ${fact.fact}`);
    console.log(`    Confidence: ${fact.confidence}, Score: ${fact.score.toFixed(2)}\n`);
  }

  // Natural language query
  console.log('💬 Query: "Tell me about the user"\n');
  const answer = await memoria.query('Tell me about the user');
  console.log(answer);
  console.log('');

  // Stats
  console.log('📊 Memory statistics:\n');
  const stats = await memoria.stats();
  console.log(`  Total facts: ${stats.totalFacts}`);
  console.log(`  Total embeddings: ${stats.totalEmbeddings}`);
  console.log(`  Total relations: ${stats.totalRelations}`);
  console.log(`  Total topics: ${stats.totalTopics}`);
  console.log(`  Total patterns: ${stats.totalPatterns}`);
  console.log(`  Total observations: ${stats.totalObservations}\n`);

  if (Object.keys(stats.categoryCounts).length > 0) {
    console.log('  By category:');
    for (const [cat, count] of Object.entries(stats.categoryCounts)) {
      console.log(`    ${cat}: ${count}`);
    }
    console.log('');
  }

  // Close
  memoria.close();
  console.log('✅ Memoria closed. Database saved.\n');
}

main().catch(console.error);


path: core/expertise.ts
content: /**
 * Expertise Manager — Specialization via topic interaction
 * 
 * Human memory develops expertise in frequently-used domains.
 * 
 * Levels:
 *   - novice:      interaction_count < 5
 *   - familiar:    5 <= count < 15
 *   - experienced: 15 <= count < 30
 *   - expert:      count >= 30
 * 
 * Usage:
 *   - Boost recall for facts in expert topics
 *   - Prioritize facts from expertise domains
 */

import type { MemoriaDB } from "./db.js";

export type ExpertiseLevel = "novice" | "familiar" | "experienced" | "expert";

export const EXPERTISE_CONFIG = {
  thresholds: {
    novice: 0,
    familiar: 5,
    experienced: 15,
    expert: 30,
  },
  recallBoost: {
    novice: 1.0,
    familiar: 1.1,
    experienced: 1.3,
    expert: 1.5,
  },
};

export interface TopicExpertise {
  topic: string;
  interactionCount: number;
  level: ExpertiseLevel;
  boost: number;
}

export class ExpertiseManager {
  private db: MemoriaDB;
  
  constructor(db: MemoriaDB) {
    this.db = db;
  }

  /**
   * Calculate expertise level from interaction count
   */
  calculateLevel(interactionCount: number): ExpertiseLevel {
    if (interactionCount >= EXPERTISE_CONFIG.thresholds.expert) return "expert";
    if (interactionCount >= EXPERTISE_CONFIG.thresholds.experienced) return "experienced";
    if (interactionCount >= EXPERTISE_CONFIG.thresholds.familiar) return "familiar";
    return "novice";
  }

  /**
   * Get recall boost multiplier for a topic
   */
  getBoost(level: ExpertiseLevel): number {
    return EXPERTISE_CONFIG.recallBoost[level];
  }

  /**
   * Get all topics with expertise levels
   */
  getAllExpertise(): TopicExpertise[] {
    try {
      const topics = this.db.raw.prepare(
        "SELECT name as topic, access_count as interaction_count FROM topics ORDER BY access_count DESC"
      ).all() as Array<{ topic: string; interaction_count: number }>;

      return topics.map(t => ({
        topic: t.topic,
        interactionCount: t.interaction_count,
        level: this.calculateLevel(t.interaction_count),
        boost: this.getBoost(this.calculateLevel(t.interaction_count)),
      }));
    } catch (err) {
      console.error("[expertise] getAllExpertise failed:", err);
      return [];
    }
  }

  /**
   * Get expertise for specific topics
   */
  getTopicExpertise(topics: string[]): TopicExpertise[] {
    const result: TopicExpertise[] = [];

    for (const topic of topics) {
      const row = this.db.raw.prepare(
        "SELECT access_count as interaction_count FROM topics WHERE name = ?"
      ).get(topic) as { interaction_count: number } | undefined;

      const count = row?.interaction_count ?? 0;
      const level = this.calculateLevel(count);

      result.push({
        topic,
        interactionCount: count,
        level,
        boost: this.getBoost(level),
      });
    }

    return result;
  }

  /**
   * Get stats: count by expertise level
   */
  getStats(): Record<ExpertiseLevel, number> {
    try {
      const all = this.getAllExpertise();
      const stats: Record<ExpertiseLevel, number> = {
        novice: 0,
        familiar: 0,
        experienced: 0,
        expert: 0,
      };

      for (const exp of all) {
        stats[exp.level]++;
      }

      return stats;
    } catch (err) {
      console.error("[expertise] getStats failed:", err);
      return { novice: 0, familiar: 0, experienced: 0, expert: 0 };
    }
  }

  /**
   * Boost fact score based on topic expertise
   */
  applyExpertiseBoost(score: number, factTopics: string[]): number {
    if (factTopics.length === 0) return score;

    const expertise = this.getTopicExpertise(factTopics);
    const maxBoost = Math.max(...expertise.map(e => e.boost));

    return score * maxBoost;
  }
}


path: core/extraction.ts
content: /**
 * 🧠 Memoria — Fact extraction logic
 * 
 * This module exports:
 *   - LLM_EXTRACT_PROMPT — the master prompt for LLM-based fact extraction
 *   - parseJSON() — safe JSON parser for LLM outputs (handles fences, trailing commas)
 *   - normalizeCategory() — map free-form category → canonical category
 */

// ─── Extraction Prompt ───

export const LLM_EXTRACT_PROMPT = `Tu es un extracteur de faits pour un système de mémoire AI.
Analyse le texte et extrais les faits qui méritent d'être retenus.

DEUX TYPES de faits:
- "semantic" = vérité durable, processus appris, configuration, règle découverte
- "episodic" = événement daté, état temporaire, action en cours, résultat observé

RÈGLE D'OR: TOUJOURS INCLURE LES DÉTAILS CONCRETS
Imagine que tu notes pour une secrétaire qui doit pouvoir tout retrouver plus tard.
❌ "Neto a eu une réunion importante" → MANQUE: avec qui? quand? sur quoi?
✅ "Neto a eu une réunion avec le client CCOG le 28/03 à 14h sur la refonte du site"
❌ "Sol a été redémarré" → MANQUE: pourquoi? quel était le problème?
✅ "Sol a été redémarré le 28/03 à 18h25 car better-sqlite3 était compilé pour la mauvaise version de Node (137 vs 141). Fix: npm rebuild"
❌ "Une réflexion excellente a été faite" → MANQUE: quelle réflexion? quel contenu?
✅ "Neto propose que la mémoire fonctionne comme un cerveau humain: ne rien supprimer, prioriser par usage, les détails éphémères (heure d'un vol) s'effacent mais l'expérience (le vol était long) reste"

EXTRAIRE — tout ce qui a du contenu:
✅ Processus appris avec les étapes ("pour migrer SQLite WAL: VACUUM INTO au lieu de cp")
✅ Ce qui a marché ET pourquoi ("le fallback chain résout les crashes car Ollama tombe parfois")
✅ Leçons d'erreurs avec la cause ("api.config ≠ api.pluginConfig → configs ignorées")
✅ Décisions avec la raison ("on utilise qwen3.5:4b car meilleure qualité JSON, avec think:false")
✅ Configs exactes ("Memoria: recallLimit=8, extract LLM qwen3.5:4b, fallback gemma3:4b")
✅ Résultats avec chiffres ("Benchmark: retrieval 92% (11/12), RAG 25%, bottleneck = modèle local")
✅ Préférences avec contexte ("Neto veut du step-by-step, une feature à la fois avec validation")
✅ États temporaires AVEC CONTEXTE ("Sol est en train de refaire HydroTrack — blocker: API endpoint changé")
✅ Événements avec date ET détail ("28/03 — Memoria v3.13.0 live: lifecycle fresh/settled/dormant, 385f/90s/0d")
✅ Ce que quelqu'un fait en ce moment ET pourquoi ("Sol travaille sur la refonte HydroTrack depuis le 26/03, priorité car le client attend la démo")
✅ Outils internes et leur état ("Memoria v3.13.0: lifecycle humain, curseur détail 1-10, 475 facts, publié ClawHub + GitHub")
✅ Produits/MVPs et leur avancement ("Bureau module CA v1.2.0 en prod, matching auto Qonto↔projets fonctionnel")

GÉNÉRALISER — quand un pattern se répète:
🔄 Même problème 2+ fois → stocker la RÈGLE + les cas concrets
   "Les commandes brew/nvm (npm, ollama, node) ne sont pas dans le PATH en SSH non-interactif — fix: source ~/.zprofile ou chemin complet /opt/homebrew/bin/"

🔥 ERREURS ET DANGERS — PRIORITÉ MAXIMALE (comme toucher du feu):
Quand quelque chose a causé un PROBLÈME RÉEL (crash, perte de données, service mort, bug en prod, Neto qui doit intervenir physiquement):
→ Catégorie "erreur", confidence 0.95+
→ Inclure: CE QUI S'EST PASSÉ + POURQUOI c'est dangereux + CE QU'IL NE FAUT JAMAIS REFAIRE + L'ALTERNATIVE SÛRE
→ C'est comme un panneau "DANGER" : on le note dès la PREMIÈRE FOIS, pas après la 2ème brûlure
Exemples de VRAIS dangers à capter:
✅ "NE JAMAIS utiliser openclaw gateway stop via exec — tue le daemon sans le relancer, gateway reste mort. Utiliser gateway restart (SIGUSR1)." (catégorie erreur)
✅ "NE JAMAIS faire cp sur une DB SQLite en mode WAL — données perdues. Utiliser VACUUM INTO." (catégorie erreur)
✅ "NE JAMAIS push sur main sans test — régression garantie. Toujours une branche séparée." (catégorie erreur)
Signaux qu'un fait est un DANGER:
- Quelqu'un dit "ne fais plus ça", "c'est la 2ème fois", "putain", "j'ai dû aller faire X manuellement"
- Un service/outil est mort/cassé après une action
- Un rollback ou fix manuel a été nécessaire
- Le mot "jamais", "interdit", "critique", "ne pas" dans la conversation

NE PAS STOCKER:
❌ Confirmations vides ("ok", "merci", "compris")
❌ Narration pure sans résultat ("je lis le fichier", "je regarde le code")
❌ MÉTA-FAITS sur le stockage lui-même ("le nouveau fait complète l'ancien", "ce fait a été ajouté")
❌ Faits sans AUCUN élément concret ("des informations ont été fournies", "la configuration a été mise à jour")

QUALITÉ — chaque fait DOIT:
⚠️ Contenir au moins UN élément concret: nom propre, chiffre, commande, version, ou date
⚠️ Être AUTONOME = compréhensible seul, sans contexte
⚠️ Inclure le POURQUOI ou le CONTEXTE quand c'est pertinent (pas juste QUOI)
⚠️ Ne JAMAIS commencer par "Le nouveau fait..." ou "Ce fait..." → commencer par le SUJET réel

Règles:
- Phrase(s) complète(s) et autonome(s)
- Pour les PROCÉDURES: garder les étapes ensemble en UN fait (2-4 phrases OK)
- UN FAIT PAR ENTITÉ — si le texte parle de 3 sujets distincts, 3 faits séparés
- Catégories: savoir, erreur, preference, outil, chronologie, rh, client
- type: "semantic" ou "episodic"
- confidence: 0.7 minimum
- Maximum {MAX_FACTS} faits
- Si rien de concret → {"facts": []}

Texte:
"{TEXT}"

JSON valide uniquement:
{"facts": [{"fact": "phrase", "category": "...", "type": "semantic|episodic", "confidence": 0.X}]}`;

// ─── JSON Parse Helper ───

/** Safely parse JSON from LLM output. Handles markdown code fences, trailing commas, and partial JSON. */
export function parseJSON(text: string): unknown {
  // Strip markdown code blocks (```json ... ``` or ``` ... ```)
  let cleaned = text.trim();
  if (cleaned.startsWith("```")) {
    const lines = cleaned.split("\n");
    lines.shift(); // remove opening ```json or ```
    if (lines[lines.length - 1]?.trim() === "```") lines.pop();
    cleaned = lines.join("\n").trim();
  }
  // Try to extract JSON object/array via regex
  const match = cleaned.match(/(\{[\s\S]*\}|\[[\s\S]*\])/);
  if (match) cleaned = match[1];
  return JSON.parse(cleaned);
}

// ─── Category Normalization ───

const VALID_CATEGORIES = new Set(["savoir", "erreur", "preference", "outil", "chronologie", "rh", "client"]);

/**
 * Normalize free-form LLM category output → one of 7 canonical categories.
 * Mapping: architecture/mécanisme → savoir, sévérité/bug → erreur, financier → client, etc.
 * Unknown categories default to "savoir".
 */
export function normalizeCategory(raw: string): string {
  const lower = (raw || "savoir").toLowerCase().trim();
  if (VALID_CATEGORIES.has(lower)) return lower;
  // Common LLM variants → map to valid
  if (lower === "préférence" || lower === "préférences") return "preference";
  if (lower === "architecture" || lower === "mécanisme" || lower === "stock" || lower === "état") return "savoir";
  if (lower === "financier") return "client";
  if (lower === "sévérité" || lower === "bug") return "erreur";
  return "savoir"; // fallback: anything unknown → savoir
}


path: core/fact-clusters.ts
content: /**
 * Memoria — Fact Clusters (v3.4.0)
 * 
 * Generates thematic summaries from groups of related atomic facts.
 * Solves the "multi-session" problem: when facts about the same entity
 * are scattered across sessions, a cluster aggregates them into one
 * searchable summary.
 * 
 * Like a "dossier" in an office: when you look up a client, you get
 * the complete file, not individual scattered notes.
 * 
 * Clusters are:
 * - Generated from atomic facts sharing the same entity/topic
 * - Stored as regular facts (fact_type = "cluster") for FTS/embedding search
 * - Auto-invalidated when a member fact is superseded
 * - Regenerated periodically to stay fresh
 */

import type { MemoriaDB, Fact } from "./db.js";
import type { LLMProvider } from "./providers/types.js";

// ─── Types ───

export interface ClusterMeta {
  memberIds: string[];      // IDs of atomic facts in this cluster
  entityName: string;       // Primary entity ("Sol", "Bureau", "RH Primo Studio")
  generatedAt: number;      // Timestamp of generation
  stale: boolean;           // True if a member was superseded since generation
}

export interface ClusterResult {
  created: number;
  updated: number;
  stale: number;
}

// ─── Config ───

const MIN_FACTS_FOR_CLUSTER = 3;   // Need at least 3 facts to justify a cluster
const MAX_CLUSTER_FACTS = 12;      // Don't cluster more than 12 facts (context limit)
const CLUSTER_REGEN_HOURS = 24;    // Regenerate stale clusters after this delay
const MAX_CLUSTERS_PER_RUN = 5;    // Limit cluster generation per postProcess call

// ─── Prompt ───

const CLUSTER_PROMPT = `Tu résumes un groupe de faits liés à la même entité en UN SEUL paragraphe dense.

Règles:
- Le résumé doit contenir TOUTES les informations clés des faits (noms, chiffres, dates, versions, états)
- Commence par l'entité principale en gras contexte
- Si des infos se contredisent, garde la plus récente
- Si un fait dit qu'une personne est partie ou qu'un outil est remplacé, reflète cet état actuel
- 2-4 phrases maximum, dense et factuel
- En français

Entité: {ENTITY}

Faits:
{FACTS}

Résumé dense (texte brut, pas de JSON):`;

// ─── Manager ───

export class FactClusterManager {
  private db: MemoriaDB;
  private llm: LLMProvider;

  constructor(db: MemoriaDB, llm: LLMProvider) {
    this.db = db;
    this.llm = llm;
  }

  /**
   * Main entry point: generate/refresh clusters for entities with enough facts.
   * Called from postProcessNewFacts.
   */
  async generateClusters(): Promise<ClusterResult> {
    const result: ClusterResult = { created: 0, updated: 0, stale: 0 };

    try {
      // 1. Find entities with enough active facts
      const entityGroups = this.groupFactsByEntity();

      // 2. Check existing clusters for staleness
      result.stale = this.markStaleClusters();

      // 3. Generate/regenerate clusters for top entities
      let generated = 0;
      for (const [entityName, facts] of entityGroups) {
        if (generated >= MAX_CLUSTERS_PER_RUN) break;
        if (facts.length < MIN_FACTS_FOR_CLUSTER) continue;

        const existing = this.findCluster(entityName);

        // Skip if cluster exists and is fresh
        if (existing && !this.isStale(existing)) continue;

        // Generate cluster
        const clusterText = await this.generateClusterText(entityName, facts);
        if (!clusterText) continue;

        if (existing) {
          // Update existing cluster
          this.updateCluster(existing.id, clusterText, facts);
          result.updated++;
        } else {
          // Create new cluster
          this.createCluster(entityName, clusterText, facts);
          result.created++;
        }
        generated++;
      }
    } catch (e) {
      console.debug('memoria:clusters: ' + String(e));
      // Non-critical: clusters are a quality enhancement, not required
    }

    return result;
  }

  /**
   * Group active (non-superseded, non-cluster) facts by their primary entity.
   * Uses entity_ids from the knowledge graph when available,
   * falls back to keyword extraction.
   */
  private groupFactsByEntity(): Map<string, Fact[]> {
    const groups = new Map<string, Fact[]>();

    // Get all active non-cluster facts
    const facts = this.db.raw.prepare(
      "SELECT * FROM facts WHERE superseded = 0 AND (fact_type != 'cluster' OR fact_type IS NULL) ORDER BY created_at DESC"
    ).all() as Fact[];

    for (const fact of facts) {
      // Try entity_ids first (from knowledge graph)
      let entities: string[] = [];
      try {
        const ids = JSON.parse(fact.entity_ids || "[]") as string[];
        if (ids.length > 0) {
          // Look up entity names
          for (const id of ids) {
            const ent = this.db.raw.prepare("SELECT name FROM entities WHERE id = ?").get(id) as { name: string } | undefined;
            if (ent) entities.push(ent.name);
          }
        }
      } catch (e) { console.debug('memoria:clusters: ' + String(e)); }

      // Fallback: extract proper nouns as entity proxies
      if (entities.length === 0) {
        entities = this.extractProperNouns(fact.fact);
      }

      // Add fact to each entity group
      for (const entity of entities) {
        const key = entity.toLowerCase().trim();
        if (key.length < 2) continue;
        if (!groups.has(key)) groups.set(key, []);
        groups.get(key)!.push(fact);
      }
    }

    // Sort by group size (largest first) and filter minimum
    const sorted = new Map(
      Array.from(groups.entries())
        .filter(([, facts]) => facts.length >= MIN_FACTS_FOR_CLUSTER)
        .sort((a, b) => b[1].length - a[1].length)
    );

    return sorted;
  }

  /**
   * Extract proper nouns from text (capitalized words that aren't sentence-starters).
   */
  private extractProperNouns(text: string): string[] {
    const nouns = new Set<string>();

    // Match capitalized words (2+ chars) that appear after other text
    const matches = text.match(/(?<=\s)[A-Z][a-zéèêëàâäôöùûüïîç]+(?:\s+[A-Z][a-zéèêëàâäôöùûüïîç]+)*/g) || [];
    for (const m of matches) {
      if (m.length > 2) nouns.add(m);
    }

    // Also match common entity patterns
    const techTerms = text.match(/\b(?:Memoria|Bureau|Convex|Primask|DockGroups|Sol|Luna|Koda|Neto|Ollama|Vercel|Cloudflare|Qonto|Alexandre|Pierre|HydroTrack|OpenClaw)\b/gi) || [];
    for (const t of techTerms) {
      nouns.add(t.charAt(0).toUpperCase() + t.slice(1).toLowerCase());
    }

    return Array.from(nouns);
  }

  /**
   * Find existing cluster for an entity.
   */
  private findCluster(entityName: string): Fact | undefined {
    const pattern = `%${entityName}%`;
    return this.db.raw.prepare(
      "SELECT * FROM facts WHERE fact_type = 'cluster' AND superseded = 0 AND source LIKE ? LIMIT 1"
    ).get(`cluster:${entityName.toLowerCase()}`) as Fact | undefined
    // Fallback: search by source field
    || this.db.raw.prepare(
      "SELECT * FROM facts WHERE fact_type = 'cluster' AND superseded = 0 AND source = ? LIMIT 1"
    ).get(`cluster:${entityName.toLowerCase()}`) as Fact | undefined;
  }

  /**
   * Check if a cluster is stale (member superseded or too old).
   */
  private isStale(cluster: Fact): boolean {
    try {
      const meta = JSON.parse(cluster.tags) as ClusterMeta;
      if (meta.stale) return true;

      // Check age
      const ageHours = (Date.now() - meta.generatedAt) / (3600 * 1000);
      if (ageHours > CLUSTER_REGEN_HOURS) return true;

      // Check if any member was superseded
      for (const id of meta.memberIds) {
        const member = this.db.getFact(id);
        if (member && member.superseded) return true;
      }

      return false;
    } catch (e) {
      console.debug('memoria:clusters: ' + String(e));
      return true; // Can't parse meta → treat as stale
    }
  }

  /**
   * Mark clusters as stale if any member fact was recently superseded.
   */
  private markStaleClusters(): number {
    let staleCount = 0;
    const clusters = this.db.raw.prepare(
      "SELECT * FROM facts WHERE fact_type = 'cluster' AND superseded = 0"
    ).all() as Fact[];

    for (const cluster of clusters) {
      if (this.isStale(cluster)) {
        try {
          const meta = JSON.parse(cluster.tags) as ClusterMeta;
          meta.stale = true;
          this.db.raw.prepare("UPDATE facts SET tags = ?, updated_at = ? WHERE id = ?")
            .run(JSON.stringify(meta), Date.now(), cluster.id);
          staleCount++;
        } catch (e) { console.debug('memoria:clusters: ' + String(e)); }
      }
    }

    return staleCount;
  }

  /**
   * Generate cluster text via LLM.
   */
  private async generateClusterText(entityName: string, facts: Fact[]): Promise<string | null> {
    // Take most recent facts, up to limit
    const selected = facts
      .sort((a, b) => b.created_at - a.created_at)
      .slice(0, MAX_CLUSTER_FACTS);

    const factsText = selected
      .map((f, i) => `${i + 1}. [${f.category}] ${f.fact}`)
      .join("\n");

    const prompt = CLUSTER_PROMPT
      .replace("{ENTITY}", entityName)
      .replace("{FACTS}", factsText);

    try {
      const genFn = this.llm.generateWithMeta;
      if (!genFn) return null;
      const result = await genFn.call(this.llm, prompt, {
        maxTokens: 300,
        temperature: 0.1,
        timeoutMs: 15000,
      });

      if (!result?.response) return null;

      // Clean response: remove JSON wrapping if present, take plain text
      let text = result.response.trim();
      // Remove markdown formatting artifacts
      text = text.replace(/^```[\s\S]*?```$/gm, "").trim();
      text = text.replace(/^["']|["']$/g, "").trim();

      if (text.length < 20) return null;
      return text;
    } catch (e) {
      console.debug('memoria:clusters: ' + String(e));
      return null;
    }
  }

  /**
   * Create a new cluster fact.
   */
  private createCluster(entityName: string, text: string, memberFacts: Fact[]): void {
    const members = memberFacts.slice(0, MAX_CLUSTER_FACTS);
    const meta: ClusterMeta = {
      memberIds: members.map(f => f.id),
      entityName,
      generatedAt: Date.now(),
      stale: false,
    };

    const clusterId = `cluster_${Date.now()}_${Math.random().toString(36).slice(2, 9)}`;
    this.db.storeFact({
      id: clusterId,
      fact: text,
      category: memberFacts[0]?.category || "savoir",
      confidence: 0.85,
      source: `cluster:${entityName.toLowerCase()}`,
      tags: JSON.stringify(meta),
      agent: "memoria",
      created_at: Date.now(),
      updated_at: Date.now(),
      fact_type: "cluster",
    });

    // Populate cluster_members table
    this.syncClusterMembers(clusterId, members);
  }

  /**
   * Update an existing cluster with fresh text and members.
   */
  private updateCluster(clusterId: string, text: string, memberFacts: Fact[]): void {
    const members = memberFacts.slice(0, MAX_CLUSTER_FACTS);
    const meta: ClusterMeta = {
      memberIds: members.map(f => f.id),
      entityName: memberFacts[0]?.category || "entity",
      generatedAt: Date.now(),
      stale: false,
    };

    this.db.raw.prepare(
      "UPDATE facts SET fact = ?, tags = ?, updated_at = ? WHERE id = ?"
    ).run(text, JSON.stringify(meta), Date.now(), clusterId);

    // Refresh cluster_members table
    this.syncClusterMembers(clusterId, members);
  }

  /**
   * Sync the cluster_members relational table with the cluster's member facts.
   * Replaces all existing entries for this cluster.
   */
  private syncClusterMembers(clusterId: string, memberFacts: Fact[]): void {
    try {
      const raw = this.db.raw;
      raw.prepare("DELETE FROM cluster_members WHERE cluster_id = ?").run(clusterId);
      const insert = raw.prepare("INSERT OR IGNORE INTO cluster_members (cluster_id, fact_id) VALUES (?, ?)");
      for (const f of memberFacts) {
        insert.run(clusterId, f.id);
      }
    } catch (e) { console.debug('memoria:clusters: ' + String(e)); }
  }

  /**
   * Stats for logging.
   */
  stats(): { total: number; stale: number } {
    const total = (this.db.raw.prepare(
      "SELECT COUNT(*) as c FROM facts WHERE fact_type = 'cluster' AND superseded = 0"
    ).get() as { c: number }).c;
    const stale = (this.db.raw.prepare(
      "SELECT COUNT(*) as c FROM facts WHERE fact_type = 'cluster' AND superseded = 0 AND tags LIKE '%\"stale\":true%'"
    ).get() as { c: number }).c;
    return { total, stale };
  }
}


path: core/fallback.ts
content: /**
 * Memoria — Layer 12: Fallback Chain
 * 
 * Implements both LLMProvider and EmbedProvider interfaces.
 * Tries providers in order; first successful response wins.
 * If all fail → throws (callers wrap in try/catch for graceful degradation).
 * 
 * Default order (configurable via "fallback" in plugin config):
 *   1. Ollama gemma3:4b (local, 0€)
 *   2. OpenAI GPT-5.4-nano (cloud, ~$0.001)
 *   3. LM Studio GLM-4.7 (local, 0€)
 * 
 * Used by: every module that needs LLM (selective, graph, topics, observations,
 * clusters, procedural, revision, patterns). Modules receive the chain via constructor
 * and don't know/care about the fallback — they see a single LLMProvider.
 * 
 * @example
 * const chain = new FallbackChain([
 *   { type: "ollama", model: "gemma3:4b" },
 *   { type: "openai", model: "gpt-5.4-nano", apiKey: "..." }
 * ]);
 * const answer = await chain.generate("Extract entities..."); // tries Ollama first
 *   4. null → FTS-only / skip
 */

import type { LLMProvider, EmbedProvider, GenerateOptions, GenerateResult } from "./providers/types.js";
import { OllamaLLM, OllamaEmbed } from "./providers/ollama.js";
import {
  OpenAICompatLLM,
  OpenAICompatEmbed,
  lmStudioLLM,
  lmStudioEmbed,
} from "./providers/openai-compat.js";
import { AnthropicLLM } from "./providers/anthropic.js";

// ─── Config ───

export interface FallbackProviderConfig {
  name: string;
  type: "ollama" | "lmstudio" | "openai" | "openrouter" | "anthropic";
  model: string;
  baseUrl?: string;
  apiKey?: string;
  timeoutMs?: number;
  /** For embed providers */
  embedModel?: string;
  embedDimensions?: number;
}

export interface FallbackConfig {
  providers: FallbackProviderConfig[];
  /** Global timeout per provider attempt. Default 15000ms */
  defaultTimeoutMs: number;
}

export const DEFAULT_FALLBACK_CONFIG: FallbackConfig = {
  providers: [
    {
      name: "ollama",
      type: "ollama",
      model: "gemma3:4b",
      baseUrl: "http://localhost:11434",
      timeoutMs: 12000,
      embedModel: "nomic-embed-text-v2-moe",
      embedDimensions: 768,
    },
    {
      name: "openai",
      type: "openai",
      model: "gpt-5.4-nano",
      baseUrl: "https://api.openai.com/v1",
      timeoutMs: 15000,
    },
    {
      name: "lmstudio",
      type: "lmstudio",
      model: "auto",
      baseUrl: "http://localhost:1234/v1",
      timeoutMs: 12000,
    },
  ],
  defaultTimeoutMs: 15000,
};

// ─── Provider Factory ───

function createLLM(cfg: FallbackProviderConfig): LLMProvider {
  switch (cfg.type) {
    case "ollama":
      return new OllamaLLM(cfg.baseUrl, cfg.model);
    case "lmstudio":
      return lmStudioLLM(cfg.model, cfg.baseUrl);
    case "openai":
      return new OpenAICompatLLM(
        cfg.name,
        cfg.baseUrl || "https://api.openai.com/v1",
        cfg.model,
        cfg.apiKey || "",
      );
    case "openrouter":
      return new OpenAICompatLLM(
        cfg.name,
        cfg.baseUrl || "https://openrouter.ai/api/v1",
        cfg.model,
        cfg.apiKey || "",
      );
    case "anthropic":
      return new AnthropicLLM(cfg.model, cfg.apiKey || "", cfg.baseUrl);
    default:
      throw new Error(`Unknown provider type: ${cfg.type}`);
  }
}

function createEmbed(cfg: FallbackProviderConfig): EmbedProvider | null {
  if (!cfg.embedModel) return null;
  switch (cfg.type) {
    case "ollama":
      return new OllamaEmbed(cfg.baseUrl, cfg.embedModel, cfg.embedDimensions);
    case "lmstudio":
      return lmStudioEmbed(cfg.embedModel!, cfg.embedDimensions, cfg.baseUrl);
    default:
      return null;
  }
}

// ─── Fallback Chain ───

export interface FallbackResult {
  response: string;
  provider: string;
  attemptMs: number;
  fallbacksUsed: number;
}

export class FallbackChain implements LLMProvider {
  private providers: FallbackProviderConfig[];
  private llmInstances: Map<string, LLMProvider> = new Map();
  private embedInstances: Map<string, EmbedProvider> = new Map();
  private defaultTimeoutMs: number;
  private logger?: { info?: (...args: any[]) => void; warn?: (...args: any[]) => void; debug?: (...args: any[]) => void };

  get name(): string {
    return `fallback(${this.providerNames.join("→")})`;
  }

  constructor(config?: Partial<FallbackConfig>, logger?: typeof FallbackChain.prototype.logger) {
    const cfg = { ...DEFAULT_FALLBACK_CONFIG, ...config };
    this.providers = cfg.providers;
    this.defaultTimeoutMs = cfg.defaultTimeoutMs;
    this.logger = logger;
  }

  /**
   * LLMProvider-compatible generate: returns string or throws.
   * Modules (selective, graph, topics, context-tree) call this interface.
   */
  async generate(prompt: string, options?: GenerateOptions): Promise<string> {
    const result = await this.generateWithMeta(prompt, options);
    if (!result) throw new Error("All LLM providers failed");
    return result.response;
  }

  /**
   * Try to generate text using the fallback chain with metadata.
   * Returns null if ALL providers fail (caller should handle FTS-only mode).
   */
  async generateWithMeta(prompt: string, options?: GenerateOptions): Promise<FallbackResult | null> {
    let fallbacksUsed = 0;

    for (const provCfg of this.providers) {
      const start = Date.now();
      const timeoutMs = provCfg.timeoutMs || this.defaultTimeoutMs;

      try {
        const llm = this.getLLM(provCfg);
        
        // Race between LLM call and timeout
        const response = await Promise.race([
          llm.generate(prompt, { ...options, timeoutMs }),
          this.timeout(timeoutMs, provCfg.name),
        ]);

        if (!response || response.trim().length === 0) {
          throw new Error("Empty response");
        }

        const elapsed = Date.now() - start;
        
        if (fallbacksUsed > 0) {
          this.logger?.info?.(`memoria/fallback: ${provCfg.name} responded in ${elapsed}ms (after ${fallbacksUsed} fallback(s))`);
        } else {
          this.logger?.debug?.(`memoria/fallback: ${provCfg.name} responded in ${elapsed}ms`);
        }

        return {
          response,
          provider: provCfg.name,
          attemptMs: elapsed,
          fallbacksUsed,
        };
      } catch (err) {
        const elapsed = Date.now() - start;
        this.logger?.warn?.(`memoria/fallback: ${provCfg.name} failed in ${elapsed}ms: ${String(err).slice(0, 100)}`);
        fallbacksUsed++;
      }
    }

    this.logger?.warn?.(`memoria/fallback: ALL providers failed (${this.providers.length} attempts)`);
    return null;
  }

  /**
   * Get the best available embed provider (first one that has embed config).
   * Embed doesn't need full fallback chain — just use the first available.
   */
  getEmbedProvider(): EmbedProvider | null {
    for (const provCfg of this.providers) {
      if (!provCfg.embedModel) continue;
      try {
        return this.getEmbed(provCfg);
      } catch (_e) {
        continue;
      }
    }
    return null;
  }

  // ─── Private ───

  private getLLM(cfg: FallbackProviderConfig): LLMProvider {
    if (!this.llmInstances.has(cfg.name)) {
      this.llmInstances.set(cfg.name, createLLM(cfg));
    }
    return this.llmInstances.get(cfg.name)!;
  }

  private getEmbed(cfg: FallbackProviderConfig): EmbedProvider {
    if (!this.embedInstances.has(cfg.name)) {
      const embed = createEmbed(cfg);
      if (!embed) throw new Error(`No embed config for ${cfg.name}`);
      this.embedInstances.set(cfg.name, embed);
    }
    return this.embedInstances.get(cfg.name)!;
  }

  private timeout(ms: number, name: string): Promise<never> {
    return new Promise((_, reject) => {
      setTimeout(() => reject(new Error(`${name} timeout after ${ms}ms`)), ms);
    });
  }

  /** Get primary LLM (first in chain) for direct use where fallback isn't needed */
  get primaryLLM(): LLMProvider {
    return this.getLLM(this.providers[0]);
  }

  /** Provider names in order */
  get providerNames(): string[] {
    return this.providers.map(p => p.name);
  }
}


path: core/feedback.ts
content: /**
 * Memoria — Feedback Loop
 * 
 * Le chaînon manquant : mesurer si les faits rappelés ont été UTILES.
 * 
 * Cycle :
 *   1. Recall injecte N faits → on stocke leurs IDs
 *   2. Agent produit une réponse → on compare
 *   3. Si un fait a contribué à la réponse → usefulness++ 
 *   4. Si un fait est ignoré systématiquement → decay accéléré
 *   5. Le scoring en tient compte au prochain recall
 *
 * "Neurons that fire together wire together" — mais aussi :
 * "Neurons that DON'T fire get pruned"
 */

import type { MemoriaDB, Fact } from "./db.js";

// ─── Config ───

export interface FeedbackConfig {
  /** Minimum keyword overlap (0-1) to consider a fact "used" in response. Default 0.25 */
  usedThreshold: number;
  /** Boost to usefulness when a fact is used. Default 1.0 */
  usedBoost: number;
  /** Penalty when recalled but not used. Default -0.1 */
  ignoredPenalty: number;
  /** After this many ignored recalls, fact gets decay penalty. Default 10 */
  ignoredDecayThreshold: number;
  /** Max usefulness score (cap). Default 20 */
  maxUsefulness: number;
  /** Min usefulness before fact gets deprioritized. Default -3 */
  minUsefulness: number;
}

export const DEFAULT_FEEDBACK_CONFIG: FeedbackConfig = {
  usedThreshold: 0.25,
  usedBoost: 1.0,
  ignoredPenalty: -0.1,
  ignoredDecayThreshold: 10,
  maxUsefulness: 20,
  minUsefulness: -3,
};

// ─── Session state ───

export interface RecallRecord {
  factIds: string[];
  timestamp: number;
  prompt: string;
}

// ─── Feedback Manager ───

export class FeedbackManager {
  private db: MemoriaDB;
  private cfg: FeedbackConfig;

  /** Facts injected during the current recall (reset each turn) */
  private lastRecall: RecallRecord | null = null;

  constructor(db: MemoriaDB, config?: Partial<FeedbackConfig>) {
    this.cfg = { ...DEFAULT_FEEDBACK_CONFIG, ...config };
    this.db = db;
  }

  /** Called at recall time: record which facts were injected */
  recordRecall(factIds: string[], prompt: string): void {
    this.lastRecall = {
      factIds,
      timestamp: Date.now(),
      prompt,
    };
  }

  /** 
   * Called at agent_end: compare recalled facts with the response.
   * Returns stats about what was used vs ignored.
   */
  async processResponse(responseText: string): Promise<{
    used: number;
    ignored: number;
    details: Array<{ id: string; used: boolean; overlap: number }>;
  }> {
    if (!this.lastRecall || this.lastRecall.factIds.length === 0) {
      return { used: 0, ignored: 0, details: [] };
    }

    const recall = this.lastRecall;
    this.lastRecall = null; // Reset for next turn

    const responseLower = responseText.toLowerCase();
    const responseKeywords = extractKeywords(responseLower);

    const details: Array<{ id: string; used: boolean; overlap: number }> = [];
    let used = 0;
    let ignored = 0;

    for (const factId of recall.factIds) {
      const fact = this.db.getFact(factId);
      if (!fact) continue;

      const overlap = computeOverlap(fact.fact, responseLower, responseKeywords);
      const wasUsed = overlap >= this.cfg.usedThreshold;

      if (wasUsed) {
        used++;
        this.updateUsefulness(factId, this.cfg.usedBoost);
      } else {
        ignored++;
        this.updateUsefulness(factId, this.cfg.ignoredPenalty);
      }

      details.push({ id: factId, used: wasUsed, overlap });
    }

    return { used, ignored, details };
  }

  /** Update usefulness score in DB */
  private updateUsefulness(factId: string, delta: number): void {
    try {
      const raw = this.db.raw;
      
      // Get current values
      const fact = raw.prepare(
        "SELECT usefulness, recall_count, used_count FROM facts WHERE id = ?"
      ).get(factId) as { usefulness: number; recall_count: number; used_count: number } | undefined;

      if (!fact) return;

      const newUsefulness = Math.max(
        this.cfg.minUsefulness,
        Math.min(this.cfg.maxUsefulness, (fact.usefulness || 0) + delta)
      );
      const newRecallCount = (fact.recall_count || 0) + 1;
      const newUsedCount = delta > 0 ? (fact.used_count || 0) + 1 : (fact.used_count || 0);

      raw.prepare(
        "UPDATE facts SET usefulness = ?, recall_count = ?, used_count = ?, last_accessed_at = ? WHERE id = ?"
      ).run(newUsefulness, newRecallCount, newUsedCount, Date.now(), factId);
    } catch (_e) {
      // Non-critical — don't crash on feedback failure
    }
  }

  /** Get feedback stats for debugging */
  getStats(): { totalWithFeedback: number; avgUsefulness: number; mostUseful: string[]; leastUseful: string[] } {
    try {
      const raw = this.db.raw;

      const total = raw.prepare(
        "SELECT COUNT(*) as cnt FROM facts WHERE recall_count > 0 AND superseded = 0"
      ).get() as { cnt: number };

      const avg = raw.prepare(
        "SELECT AVG(usefulness) as avg FROM facts WHERE recall_count > 0 AND superseded = 0"
      ).get() as { avg: number | null };

      const best = raw.prepare(
        "SELECT id FROM facts WHERE recall_count > 0 AND superseded = 0 ORDER BY usefulness DESC LIMIT 5"
      ).all() as { id: string }[];

      const worst = raw.prepare(
        "SELECT id FROM facts WHERE recall_count > 0 AND superseded = 0 ORDER BY usefulness ASC LIMIT 5"
      ).all() as { id: string }[];

      return {
        totalWithFeedback: total.cnt,
        avgUsefulness: avg.avg ?? 0,
        mostUseful: best.map(r => r.id),
        leastUseful: worst.map(r => r.id),
      };
    } catch (_e) {
      return { totalWithFeedback: 0, avgUsefulness: 0, mostUseful: [], leastUseful: [] };
    }
  }

  // ═══════════════════════════════════════════════════
  // USER CORRECTION DETECTION
  // When the user says "non c'est X" → the last recalled
  // facts that mentioned the wrong thing should be penalized
  // and potentially superseded.
  // ═══════════════════════════════════════════════════

  /** Correction patterns (FR + EN) */
  private static readonly CORRECTION_PATTERNS = [
    // French
    /\bnon[,.]?\s+(c'est|c est)\b/i,
    /\ben fait[,.]?\s+(c'est|c est|il|elle|on)\b/i,
    /\bpas\s+\w+[,.]?\s+(c'est|c est)\b/i,
    /\bje (te |t'|t )?dis que\b/i,
    /\bc'est (pas|plus)\b/i,
    /\bt'as tort\b/i,
    /\bje (te |t')?corrige\b/i,
    /\bt(u|')\s*(as)?\s*oubli(é|e)\b/i,
    /\bje (te |t')?rappelle que\b/i,
    /\bmais non\b/i,
    // English
    /\bno[,.]?\s+(it's|it is|that's|that is)\b/i,
    /\bactually[,.]?\s+(it's|it is|that|the)\b/i,
    /\bthat's (wrong|incorrect|not right|not true)\b/i,
    /\byou('re| are) wrong\b/i,
    /\bi (just )?told you\b/i,
    /\bi said\b/i,
  ];

  /** Frustration patterns */
  private static readonly FRUSTRATION_PATTERNS = [
    /\bputain\b/i,
    /\bbordel\b/i,
    /\bmerde\b/i,
    /\bserieux\b/i,
    /\bsérieux\b/i,
    /\bc'est pas possible\b/i,
    /\bfuck\b/i,
    /\bdamn\b/i,
    /\bwhat the\b/i,
    /\bwtf\b/i,
    /\bnon mais\b/i,
    /\bencore\s*[!?]/i,
    /\bpourquoi (tu|t')\b.*\?/i,  // "pourquoi tu fais ça ?"
    /\bje (te |t'|t )?(l')?ai (déjà|deja) dit\b/i,
  ];

  /**
   * Analyze user message for correction signals.
   * Returns penalty to apply to last-recalled facts + the corrected topic if found.
   */
  analyzeUserMessage(userMessage: string): {
    isCorrection: boolean;
    isFrustration: boolean;
    penalty: number;
    correctionText: string | null;
  } {
    const isCorrection = FeedbackManager.CORRECTION_PATTERNS.some(p => p.test(userMessage));
    const isFrustration = FeedbackManager.FRUSTRATION_PATTERNS.some(p => p.test(userMessage));

    let penalty = 0;
    if (isCorrection) penalty += -1.5;   // Strong signal: facts were wrong
    if (isFrustration) penalty += -0.5;  // Mild signal: facts may have been unhelpful

    // Extract what was corrected (text after the correction pattern)
    let correctionText: string | null = null;
    if (isCorrection) {
      for (const pattern of FeedbackManager.CORRECTION_PATTERNS) {
        const match = userMessage.match(pattern);
        if (match) {
          const idx = match.index! + match[0].length;
          const rest = userMessage.slice(idx).trim();
          if (rest.length > 5) {
            correctionText = rest.slice(0, 200); // Cap at 200 chars
            break;
          }
        }
      }
    }

    return { isCorrection, isFrustration, penalty, correctionText };
  }

  /**
   * Apply correction/frustration penalty to the last recalled facts.
   * Returns the facts that were penalized.
   */
  applyUserSignal(penalty: number): string[] {
    if (!this.lastRecall || this.lastRecall.factIds.length === 0 || penalty >= 0) {
      return [];
    }

    const penalized: string[] = [];
    for (const factId of this.lastRecall.factIds) {
      try {
        this.updateUsefulness(factId, penalty);
        penalized.push(factId);
      } catch (_e) { /* non-critical */ }
    }
    return penalized;
  }

  /**
   * Get IDs of facts from last recall (for external use — e.g., contradiction check).
   */
  getLastRecalledIds(): string[] {
    return this.lastRecall?.factIds ?? [];
  }
}

// ─── Helpers ───

/** Extract meaningful keywords from text */
function extractKeywords(text: string): Set<string> {
  return new Set(
    text.replace(/[^\p{L}\p{N}\s]/gu, " ")
      .split(/\s+/)
      .filter(w => w.length > 3) // Only words > 3 chars for meaningful overlap
  );
}

/**
 * Compute keyword overlap between a fact and a response.
 * Uses Jaccard-like metric but weighted: longer shared words count more.
 */
function computeOverlap(
  factText: string,
  responseLower: string,
  responseKeywords: Set<string>
): number {
  const factKeywords = extractKeywords(factText.toLowerCase());
  if (factKeywords.size === 0) return 0;

  let matchWeight = 0;
  let totalWeight = 0;

  for (const word of factKeywords) {
    // Weight longer words more (technical terms, proper nouns)
    const weight = Math.min(word.length / 5, 2);
    totalWeight += weight;

    if (responseKeywords.has(word) || responseLower.includes(word)) {
      matchWeight += weight;
    }
  }

  return totalWeight > 0 ? matchWeight / totalWeight : 0;
}


path: core/format.ts
content: /**
 * 🧠 Memoria — Recall context formatting
 * 
 * This module exports:
 *   - formatRecallContext() — format recalled facts into text block for prompt injection
 */

/**
 * Format recalled facts + observations into the text block injected before the prompt.
 * Output goes into `event.prependContext` in the before_prompt_build hook.
 * Includes: header, observations section, per-fact lines with [category] [age] prefix, known procedures.
 */
export function formatRecallContext(facts: Array<{ fact: string; category: string; confidence: number; temporalScore: number; created_at?: number; updated_at?: number; fact_type?: string }>, observationContext = ""): string {
  if (facts.length === 0 && !observationContext) return "";
  const parts: string[] = [
    "## 🧠 Memoria — Mémoire persistante",
    "Faits provenant de la mémoire long terme (source de vérité).",
    "En cas de conflit avec un résumé LCM → la mémoire persistante a priorité.",
    "Les faits les plus récents (par date) sont les plus fiables en cas de contradiction.",
    "",
  ];

  // Observations first (synthesized, higher quality)
  if (observationContext) {
    parts.push("### Observations (synthèses vivantes)");
    parts.push(observationContext);
    parts.push("");
  }

  // Individual facts with dates for Knowledge Update disambiguation
  if (facts.length > 0) {
    if (observationContext) parts.push("### Faits individuels");
    const now = Date.now();
    const lines = facts.map(f => {
      const conf = f.confidence >= 0.9 ? "" : ` (${Math.round(f.confidence * 100)}%)`;
      // Add date tag so the answering model can disambiguate updates
      let dateTag = "";
      const ts = f.updated_at || f.created_at;
      if (ts && ts > 0) {
        const d = new Date(ts);
        const ageDays = Math.floor((now - ts) / 86400000);
        if (ageDays === 0) dateTag = ` [aujourd'hui]`;
        else if (ageDays === 1) dateTag = ` [hier]`;
        else if (ageDays < 7) dateTag = ` [il y a ${ageDays}j]`;
        else dateTag = ` [${d.toISOString().slice(0, 10)}]`;
      }
      return `- [${f.category}]${dateTag} ${f.fact}${conf}`;
    });
    parts.push(...lines);
    parts.push("");
  }

  return parts.join("\n");
}


path: core/migrate.ts
content: /**
 * Migration script: facts.json → memoria.db (SQLite)
 * 
 * Run: npx tsx migrate.ts
 */

import { MemoriaDB } from "./db.js";
import { readFileSync } from "fs";
import path from "path";

const WORKSPACE = process.env.OPENCLAW_WORKSPACE || `${process.env.HOME}/.openclaw/workspace`;
const FACTS_JSON = path.join(WORKSPACE, "memory", "facts.json");

interface OldFact {
  _id: string;
  fact: string;
  category: string;
  agent: string;
  confidence: number;
  source?: string;
  tags?: string[];
  factHash?: string;
  keywordHash?: string;
  superseded?: boolean;
  supersededBy?: string;
  supersededAt?: number;
  accessCount?: number;
  lastAccessedAt?: number;
  createdAt: number;
  updatedAt: number;
}

async function main() {
  console.log("🧠 Memoria Migration: facts.json → memoria.db");
  
  const raw = readFileSync(FACTS_JSON, "utf-8");
  const oldFacts: OldFact[] = JSON.parse(raw);
  console.log(`  📦 Loaded ${oldFacts.length} facts from facts.json`);

  const db = new MemoriaDB(WORKSPACE);

  let imported = 0;
  let skipped = 0;

  for (const old of oldFacts) {
    try {
      db.storeFact({
        id: old._id || `migrated_${Date.now()}_${Math.random().toString(36).slice(2, 9)}`,
        fact: old.fact,
        category: old.category,
        confidence: old.confidence,
        source: old.source || "convex-import",
        tags: JSON.stringify(old.tags || []),
        agent: old.agent || "koda",
        created_at: old.createdAt,
        updated_at: old.updatedAt,
        access_count: old.accessCount ?? 0,
        last_accessed_at: old.lastAccessedAt ?? null,
        superseded: old.superseded ? 1 : 0,
        fact_type: "semantic",
        superseded_by: old.supersededBy ?? null,
        superseded_at: old.supersededAt ?? null,
      } as any);
      imported++;
    } catch (err) {
      console.warn(`  ⚠ Skipped: ${String(err)}`);
      skipped++;
    }
  }

  const stats = db.stats();
  console.log(`  ✅ Imported: ${imported}, Skipped: ${skipped}`);
  console.log(`  📊 DB stats: ${stats.active} active, ${stats.superseded} superseded`);
  console.log(`  📁 Categories:`, stats.categories);

  db.close();
  console.log("  🧠 Migration complete!");
}

main().catch(console.error);


path: core/package.json
content: {
  "name": "@primo-studio/memoria-core",
  "version": "0.1.0",
  "description": "Standalone multi-layer cognitive memory engine — works with or without OpenClaw",
  "type": "module",
  "main": "index.js",
  "types": "index.d.ts",
  "exports": {
    ".": "./index.js"
  },
  "keywords": [
    "memory",
    "cognitive",
    "ai",
    "llm",
    "rag",
    "vector-search",
    "knowledge-graph",
    "embedding",
    "sqlite"
  ],
  "author": "Primo Studio",
  "license": "Apache-2.0",
  "repository": {
    "type": "git",
    "url": "https://github.com/primo-studio/memoria-core"
  },
  "dependencies": {
    "better-sqlite3": "^11.0.0"
  },
  "peerDependencies": {},
  "engines": {
    "node": ">=20"
  }
}


path: core/providers/types.ts
content: /**
 * Memoria — Provider Interfaces
 * 
 * These interfaces are the contract between Memoria and any LLM/embedding backend.
 * To add a new provider (e.g., Groq, Together, Mistral):
 *   1. Create providers/your-provider.ts implementing LLMProvider and/or EmbedProvider
 *   2. Add to the switch in fallback.ts buildProvider()
 *   3. Add your type to the ProviderConfig.type union below
 * 
 * All providers are wrapped by FallbackChain — modules never call providers directly.
 */

/** Embedding provider: converts text → float vector. */
export interface EmbedProvider {
  embed(text: string): Promise<number[]>;
  embedBatch(texts: string[]): Promise<number[][]>;
  readonly dimensions: number;
  readonly name: string;
}

/** Options for LLM generation. All optional — providers use their own defaults. */
export interface GenerateOptions {
  maxTokens?: number;
  temperature?: number;
  format?: "json" | "text";
  timeoutMs?: number;
}

/** Extended result with metadata for debugging/logging. */
export interface GenerateResult {
  response: string;
  provider: string;
  attemptMs: number;
  fallbacksUsed: number;
}

/** LLM text generation provider. Only generate() is required; generateWithMeta() is optional. */
export interface LLMProvider {
  generate(prompt: string, options?: GenerateOptions): Promise<string>;
  /** Extended generate with metadata. Default implementation wraps generate(). */
  generateWithMeta?(prompt: string, options?: GenerateOptions): Promise<GenerateResult | null>;
  readonly name: string;
}

/** Config for building a provider instance. Used in fallback[] array and llm/embed config sections. */
export interface ProviderConfig {
  type: "ollama" | "lmstudio" | "openai" | "openrouter" | "anthropic";
  baseUrl: string;
  model: string;
  apiKey?: string;
  dimensions?: number;  // for embed
}


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)