Skill: stackblock/openclaw-memvid-logger (version 1.2.6)

# Unified Conversation Logger v1.2.5 **Version:** 1.2.5 (Critical Fixes Edition) **Author:** stackBlock **License:** MIT **OpenClaw:** >= 2026.2.12 A dual-output conversation logger for OpenClaw that captures **everything** - user messages, assistant responses, sub-agent conversations, tool calls, and system events - to both JSONL (backup) and Memvid (semantic search) formats. > **Memvid**: A single-file memory layer for AI agents with instant retrieval and long-term memory. Persistent, versioned, and portable memory, without databases. > > *"Replace complex RAG pipelines with a single portable file you own, and give your agent instant retrieval and long-term memory."* --- ## ⚠️ Security & Privacy Notice **Before installing, please understand:** This skill captures **everything** - by design. It logs all user messages, assistant responses, sub-agent conversations, tool outputs, and system events to local files. This enables powerful long-term memory but requires trust. **What you should know:** - **Broad capture scope:** This is intentional - the skill's purpose is complete conversation logging - **Sensitive data risk:** Tool outputs (commands, API responses, file contents) are logged. Review what tools expose. - **Continuous logging:** Once installed, it runs automatically on every assistant response until removed - **Optional cloud mode:** API mode with `MEMVID_API_KEY` sends data to memvid.com (third-party service). Free/local modes keep data on your machine only. - **Your responsibility:** Secure the JSONL/.mv2 files, rotate logs regularly, and audit what gets captured. **Mitigations available:** - Use **Free/Sharding mode** to keep data local (no API key needed) - Change default paths to encrypted locations - Review `tools/log.py` before installing to understand exactly what gets logged - File permissions: restrict access to log files (`chmod 600`) **This skill is for users who want complete conversation memory and accept the privacy trade-offs.** --- ## ✨ What Makes This Different - **📝 Dual Storage** - Every message saved to JSONL + Memvid simultaneously - **🔍 Semantic Search** - Ask "What did the researcher agent find about Tesla?" not just keyword search - **🤖 Full Context** - Captures user input, assistant output, agent chatter, tool results - **💾 Three Modes** - API (unlimited), Free (50MB), or Sharding (multi-file) - **🚀 Always On** - Hooks into OpenClaw automatically ## What's New in v1.2.5 ### Critical Fixes - **Memvid Tag Format Fixed:** Updated to `KEY=VALUE` format for Memvid 2.0+ compatibility - Old (broken): `--tag "user,telegram"` - New (fixed): `--tag "role=user" --tag "source=telegram"` - **Environment Variable Documentation:** Added `/etc/environment` instructions (`.bashrc` doesn't work for background services) - **Hook Handler Format:** Documented JavaScript (`.js`) requirement for OpenClaw 2026.2.12+ - **Comprehensive Troubleshooting:** Added detailed troubleshooting section for common setup issues ### Compatibility - Verified with OpenClaw 2026.2.12 - Verified with Memvid CLI 2.0+ ## Previous Versions ### v1.2.4 - **Neural Search Default:** Updated search guidance to use `--mode neural` as default for maximum accuracy - **Performance Documentation:** Clarified latency trade-offs (~200ms for neural vs ~8ms for lexical) - **Search Mode Policy:** Recommends neural for semantic understanding, lexical only when speed is critical ### v1.2.3 - **Version Cohesion:** All files synchronized to v1.2.3 - **Documentation Consistency:** README and SKILL.md now have matching content - **Security Improvements:** Generic paths (no hardcoded user directories), install script asks permission - **Registry Compliance:** Complete metadata (env vars, credentials, warnings) for ClawHub transparency - **Privacy Documentation:** Comprehensive Security & Privacy Notice explaining data capture scope - **Role Tagging:** Distinguishes user, assistant, agent:*, system, and tool messages - **Full Context:** Captures sub-agent chatter, tool results, background processes - **Three Storage Modes:** API mode (single file), Free mode (50MB), Sharding mode (monthly rotation) - **Semantic Search:** Ask "What did the researcher agent find?" or "What did I say about X?" ## Quick Install (Choose Your Mode) ### Option 1: API Mode (Recommended) - Near Limitless Memory **Best for:** Heavy users, unified search across everything **Cost:** $59-299/month via [memvid.com](https://memvid.com) ```bash # 1. Get API key from memvid.com ($59/month for 1GB, $299 for 25GB) export MEMVID_API_KEY="your_api_key_here" export MEMVID_MODE="single" # 2. Install npm install -g memvid git clone https://github.com/stackBlock/openclaw-memvid-logger.git cp -r openclaw-memvid-logger ~/.openclaw/workspace/skills/ # 3. Create unified memory file memvid create ~/memory.mv2 # 4. Start OpenClaw - everything logs to one searchable file ``` **Search everything at once:** ```bash memvid ask memory.mv2 "What did we discuss about BadjAI?" memvid ask memory.mv2 "What did the researcher agent find about Tesla?" memvid ask memory.mv2 "Show me all the Python scripts I asked for" ``` --- ### Option 2: Free Mode (50MB Limit) - Complete Memory in One Place **Best for:** Testing, light usage, single searchable file **Cost:** FREE ```bash # 1. Install (no API key needed) npm install -g memvid git clone https://github.com/stackBlock/openclaw-memvid-logger.git cp -r openclaw-memvid-logger ~/.openclaw/workspace/skills/ export MEMVID_MODE="single" # 2. Create memory file memvid create ~/memory.mv2 # 3. Start OpenClaw ``` **⚠️ Limit:** 50MB (~5,000 conversation turns). When you hit it: - Archive and start fresh, OR - Upgrade to API mode ($59-299/month), OR - Switch to Sharding mode --- ### Option 3: Sharding Mode - More Than 50MB, Free Forever **Best for:** Long-term use, staying under free tier **Cost:** FREE **Trade-off:** Multi-file search ```bash # 1. Install (no API key needed) npm install -g memvid git clone https://github.com/stackBlock/openclaw-memvid-logger.git cp -r openclaw-memvid-logger ~/.openclaw/workspace/skills/ export MEMVID_MODE="monthly" # This is the default # 2. Start OpenClaw - auto-creates monthly files ``` **How it works:** - `memory_2026-02.mv2` (February) - `memory_2026-03.mv2` (March - auto-created) - Each file stays under 50MB **⚠️ Sharding Search Differences:** Single-file search (API/Free modes): ```bash # One search gets everything memvid ask memory.mv2 "What car did I decide to buy?" # Returns: Results from ALL conversations across ALL time ``` Sharding search (requires multiple queries): ```bash # Must search each month separately memvid ask memory_2026-02.mv2 "car decision" # Recent memvid ask memory_2026-01.mv2 "car decision" # January # Or use a wrapper script to search all files for file in memory_*.mv2; do echo "=== $file ===" memvid ask "$file" "car decision" 2>/dev/null | head -5 done # You must know which month the conversation happened # No cross-month context - "compare this month to last month" won't work ``` **Why sharding is harder:** - Can't ask "what did we discuss in the past 3 months?" in one query - No unified timeline across months - Must remember which month you talked about what - No cross-file semantic comparison --- ## What Gets Logged ### Role Tags (Automatic) | Role | Tag | Example Search | |------|-----|----------------| | **User** | `[user]` | "What did **I** say about Mercedes?" | | **Assistant** | `[assistant]` | "What did **you** recommend?" | | **Sub-agents** | `[agent:researcher]`, `[agent:coder]` | "What did the **researcher** find?" | | **System** | `[system]` | "When did the **cron job** run?" | | **Tools** | `[tool:exec]`, `[tool:browser]` | "What **commands** were run?" | ### Everything Captured - ✅ User messages (what you type) - ✅ Assistant responses (what I say back) - ✅ Sub-agent conversations (researcher, coder, vision, math, etc.) - ✅ Tool executions (bash commands, browser actions, file edits) - ✅ Background processes (cron jobs, heartbeats, scheduled tasks) - ✅ System events (config changes, restarts, errors) ## Architecture ``` ┌─────────────────────────────────────────┐ │ OpenClaw Ecosystem │ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │ │ User │ │Assistant│ │ Agents │ │ │ │ Messages│ │Responses│ │Research │ │ │ └────┬────┘ └────┬────┘ └────┬────┘ │ │ └─────────────┴─────────────┘ │ │ │ │ │ ┌──────▼──────┐ │ │ │ log.py │ │ │ │ (this skill)│ │ │ └──────┬──────┘ │ └─────────────────────┼───────────────────┘ │ ┌─────────────────┼─────────────────┐ ↓ ↓ ↓ ┌───────┐ ┌─────────────┐ ┌──────────┐ │ JSONL │ │ Memvid │ │ Search │ │ File │ │ Files │ │ Query │ └───────┘ └─────────────┘ └──────────┘ │ │ ↓ ↓ grep/jq memvid ask/find ``` ## Usage Examples ### Natural Language Search ```bash # What did you say about...? memvid ask memory_2026-02.mv2 "What was your recommendation about the Mercedes vs Tesla?" # What did I ask for...? memvid ask memory_2026-02.mv2 "What Python scripts did I request last week?" # What did agents do...? memvid ask memory_2026-02.mv2 "What did the researcher agent find about options trading?" # System events...? memvid ask memory_2026-02.mv2 "When did the PowerSchool grades cron job run?" ``` ### Keyword Search ```bash # Find specific terms memvid find memory_2026-02.mv2 --query "Mercedes" # With filters memvid find memory_2026-02.mv2 --query "script" --tag agent:coder ``` ### Temporal Queries ```bash memvid when memory_2026-02.mv2 "yesterday" memvid when memory_2026-02.mv2 "last Tuesday" memvid when memory_2026-02.mv2 "3 days ago" ``` ## ⚡ Search Performance Guide Memvid has three search modes. **This skill uses `--mode neural` by default for maximum accuracy:** ### Default: Neural Search (Recommended) ```bash # Always use neural for semantic understanding and context memvid ask memory.mv2 "What supplements did Dr. Sinclair recommend?" --mode neural memvid ask memory.mv2 "What did we discuss about BadjAI?" --mode neural memvid ask memory.mv2 "Show me the Python scripts I requested" --mode neural ``` **Speed:** ~200ms | **Best for:** Semantic understanding, context, synonyms, conceptual relationships ### Alternative Modes (Use When Explicitly Requested) **Mode 1: Lexical Search (Fastest)** ```bash # Use only for exact keyword matching when speed is critical memvid find memory.mv2 --mode lex --query "metformin" ``` **Speed:** ~8ms | **Use when:** Exact word matching needed, latency is critical **Mode 2: Hybrid Search (Balanced)** ```bash # Combines lexical + neural memvid find memory.mv2 --mode hybrid --query "diabetes medications" ``` **Speed:** ~300-500ms | **Use when:** You want both exact matches and semantic similarity ### Why Neural as Default? | Mode | Speed | Accuracy | Use Case | |------|-------|----------|----------| | `neural` | ~200ms | **Highest** | **Default - semantic understanding** | | `lex` | ~8ms | Keyword only | Speed-critical exact matches | | `hybrid` | ~300-500ms | High | Balanced approach | **The ~200ms trade-off is worth it:** Neural mode understands context, handles paraphrases, and finds conceptually related information that lexical search misses entirely. ### JSONL Backup ```bash # Quick grep grep "Mercedes" conversation_log.jsonl # Complex queries with jq jq 'select(.role_tag == "user" and .content | contains("Python"))' conversation_log.jsonl # Time range jq 'select(.timestamp >= "2026-02-01" and .timestamp < "2026-03-01")' conversation_log.jsonl ``` ## Configuration ### Environment Variables | Variable | Default | Mode | Description | |----------|---------|------|-------------| | `MEMVID_API_KEY` | (none) | API | Your memvid.com API key | | `MEMVID_MODE` | `monthly` | All | `single` or `monthly` | | `JSONL_LOG_PATH` | `~/workspace/conversation_log.jsonl` | All | Backup log file | | `MEMVID_PATH` | `~/workspace/memory.mv2` | All | Base path for memory files | | `MEMVID_BIN` | `~/.npm-global/bin/memvid` | All | Path to memvid CLI | ### OpenClaw Hooks (Advanced) Add to `openclaw.json`: ```json { "hooks": { "internal": { "enabled": true, "entries": { "conversation-logger": { "enabled": true, "command": "python3 ~/.openclaw/workspace/skills/unified-logger/tools/log.py" } } } } } ``` ## Memory File Formats ### Mode 1: Single File (API or Free Mode) ``` memory.mv2 ├── [user] messages ├── [assistant] responses ├── [agent:researcher] findings ├── [agent:coder] code ├── [tool:exec] commands └── [system] events ``` ### Mode 2: Sharding (Monthly Rotation) ``` memory_2026-01.mv2 (January conversations) memory_2026-02.mv2 (February conversations) ← Current memory_2026-03.mv2 (March, auto-created on March 1) ``` ## Troubleshooting ### "Free tier limit exceeded" (Free Mode) ```bash # Option 1: Archive and start fresh mv memory.mv2 memory_archive.mv2 memvid create memory.mv2 # Option 2: Switch to monthly sharding export MEMVID_MODE="monthly" # Option 3: Get API key export MEMVID_API_KEY="your_key" # $59-299/month at memvid.com ``` ### "Cannot find memory file" (Sharding Mode) Current month's file auto-creates. If missing: ```bash memvid create memory_$(date +%Y-%m).mv2 ``` ### Missing agent conversations Agents log to their own sessions. Ensure skill is installed in main agent workspace and sub-agents inherit it. ### Search returns wrong speaker Memvid uses semantic search. Be specific: - ❌ "Mercedes" → Returns all mentions - ✅ "What did I say about Mercedes" → Targets [user] frames - ✅ "Your recommendation about Mercedes" → Targets [assistant] frames ## Comparing the Three Modes | Feature | API Mode | Free Mode | Sharding Mode | |---------|----------|-----------|---------------| | **Cost** | $59-299/mo | FREE | FREE | | **Capacity** | 1-25GB+ | 50MB | Unlimited (files) | | **Files** | 1 | 1 | Multiple (monthly) | | **Unified Search** | ✅ Yes | ✅ Yes | ❌ Per-file only | | **Cross-Context Search** | ✅ Full history | ✅ Full history | ❌ Month isolated | | **Best For** | Power users | Testing | Long-term free use | | **Complexity** | Simple | Simple | Must track files | ## 💸 The Pricing Gap (AKA Why Sharding Exists) **The situation:** Memvid's pricing goes from $0 (50MB) straight to $59/month (25GB). **The problem:** That's like buying a Ferrari when you just need a Honda Civic for your commute. **What we're doing about it:** I reached out. While they consider it, Sharding Mode exists so you don't have to pay Ferrari prices for Honda Civic usage. **You can help:** If you also think $0 → $59 is a bit much, reach out to Memvid at [memvid.com](https://memvid.com) and tell them stackBlock sent you. The more voices, the faster we get that $10-20 middle tier for the rest of us. *Until then: Sharding Mode. Because startups shouldn't have to choose between ramen and memory.* 🍜 ## Future Enhancements - [ ] Auto-archive old months to cold storage - [ ] Web UI for browsing conversations - [ ] Cross-file search wrapper script - [ ] Export to other formats (Markdown, PDF) - [ ] Conversation threading visualization ## Support - **GitHub Issues:** [github.com/stackBlock/openclaw-memvid-logger](https://github.com/stackBlock/openclaw-memvid-logger) - **OpenClaw Discord:** [discord.com/invite/clawd](https://discord.com/invite/clawd) - **Memvid Support:** [memvid.com/docs](https://memvid.com/docs) ## License MIT - See [LICENSE](LICENSE) --- **About Memvid:** > Memvid is a single-file memory layer for AI agents with instant retrieval and long-term memory. > Persistent, versioned, and portable memory, without databases. > > Replace complex RAG pipelines with a single portable file you own, and give your agent > instant retrieval and long-term memory.

path: license.txt
content: MIT License

Copyright (c) 2026 AnToni

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


path: TEMPLATE.md
content: # GitHub Repository Template

This is a template repository for creating OpenClaw skills with GitHub integration.

## Quick Setup

1. **Click "Use this template"** on GitHub to create your own repo
2. **Update package.json** with your GitHub username
3. **Customize** README.md, SKILL.md for your skill
4. **Commit and push**
5. **Install** in OpenClaw

## Repository Structure

```
.
├── .github/
│   ├── workflows/ci.yml       # GitHub Actions CI
│   ├── ISSUE_TEMPLATE/        # Issue templates
│   ├── PULL_REQUEST_TEMPLATE.md
├── tools/
│   └── log.py                 # Main skill code
├── tests/
│   └── test_log.py           # Unit tests
├── .gitignore
├── CHANGELOG.md
├── CONTRIBUTING.md
├── LICENSE
├── README.md                  # GitHub-facing docs
├── SKILL.md                   # OpenClaw skill docs
├── install.sh                 # Installation script
└── package.json               # NPM + OpenClaw metadata
```

## Files to Customize

| File | What to Change |
|------|----------------|
| `package.json` | Update repository URL, author |
| `README.md` | Update description, examples |
| `SKILL.md` | Skill-specific documentation |
| `LICENSE` | Update copyright if needed |
| `install.sh` | Update default paths |

## CI/CD

The included GitHub Actions workflow:
- Validates Python syntax
- Checks shell scripts
- Validates JSON files
- Verifies file structure

## Publishing

1. Tag a release: `git tag v1.0.0`
2. Push tags: `git push --tags`
3. GitHub will create a release archive

## Installing from GitHub

Users can install your skill:

```bash
# Download latest release
curl -L https://github.com/YOUR_USERNAME/unified-logger/archive/refs/tags/v1.0.0.tar.gz | tar -xz

# Copy to OpenClaw
cp -r unified-logger-1.0.0 ~/.openclaw/workspace/skills/unified-logger
```

## Next Steps

- [ ] Publish to [ClawHub](https://clawhub.com) for discoverability
- [ ] Add more tests
- [ ] Write tutorials/examples
- [ ] Get feedback from OpenClaw community


path: README-clawhub.md
content: # MemSync - Dual Memory System

A dual-output conversation logger for OpenClaw that captures **everything** - user messages, assistant responses, sub-agent conversations, tool executions, and system events.

## What It Does

- **📝 Dual Storage** - Every message saved to JSONL (backup) + single-file memory (searchable)
- **🔍 Semantic Search** - Ask "What did the researcher agent find about Tesla?" not just keyword search
- **🤖 Full Context** - Captures user input, assistant output, agent chatter, tool results
- **💾 Three Modes** - API (unlimited), Free (50MB), Sharding (multi-file)
- **🚀 Always On** - Hooks into OpenClaw automatically

## Quick Start

### Installation

```bash
npm install -g @memvid/cli  # External dependency
git clone https://github.com/stackBlock/openclaw-memvid-logger.git
cp -r openclaw-memvid-logger ~/.openclaw/workspace/skills/
```

### Choose Your Mode

**Mode 1: API** - Near limitless, paid tier ($59-299/month)
```bash
export MEMVID_API_KEY="your_key"
export MEMVID_MODE="single"
memvid create ~/memory.mv2
```

**Mode 2: Free** - 50MB limit, single file
```bash
export MEMVID_MODE="single"
memvid create ~/memory.mv2
```

**Mode 3: Sharding** - Free forever, monthly files
```bash
export MEMVID_MODE="monthly"  # Default
# Auto-creates: memory_2026-02.mv2, memory_2026-03.mv2, etc.
```

## Search Your Conversations

**API/Free Mode:**
```bash
memvid ask memory.mv2 "What did we discuss?"
memvid ask memory.mv2 "What did the researcher agent find?"
```

**Sharding Mode:**
```bash
memvid ask memory_2026-02.mv2 "recent discussions"
memvid ask memory_2026-01.mv2 "January conversations"
```

## What Gets Logged

| Source | Tag | Description |
|--------|-----|-------------|
| **Your messages** | `[user]` | Everything you type |
| **My responses** | `[assistant]` | All AI responses |
| **Sub-agents** | `[agent:name]` | Researcher, coder, vision, math |
| **Tools** | `[tool:name]` | Bash, browser, file operations |
| **System** | `[system]` | Cron, heartbeats, events |

## Storage Modes Compared

| Feature | API | Free | Sharding |
|---------|-----|------|----------|
| **Cost** | $59-299/mo | FREE | FREE |
| **Capacity** | 1-25GB+ | 50MB | Unlimited |
| **Files** | 1 | 1 | Monthly |
| **Unified Search** | ✅ | ✅ | ❌ Per-file |
| **Best For** | Power users | Testing | Long-term free |

## Configuration

```bash
# Choose mode
export MEMVID_MODE="single"    # API or Free
export MEMVID_MODE="monthly"   # Sharding (default)

# Optional paths
export JSONL_LOG_PATH="~/conversation_log.jsonl"
export MEMVID_PATH="~/memory.mv2"

# For API mode only
export MEMVID_API_KEY="your_key"
```

## About the Technology

This skill uses **Memvid** - a single-file memory layer for AI agents with instant retrieval and long-term memory. It provides persistent, versioned, and portable memory without databases.

> "Replace complex RAG pipelines with a single portable file you own, and give your agent instant retrieval and long-term memory."

Learn more at [memvid.com](https://memvid.com)

## License

MIT - See [LICENSE](LICENSE)

**Made with 🤝 for the OpenClaw community**


path: instructions.md
content: # Instructions for AI Agents

## Overview
This skill provides dual-output logging for OpenClaw conversations. It captures every message, tool call, and agent interaction to both JSONL (backup) and Memvid (semantic search) formats for long-term memory.

## Core Functionality

### What Gets Logged
- **User messages**: Everything the human types
- **Assistant responses**: All AI responses
- **Sub-agent conversations**: Researcher, coder, vision, math agents
- **Tool executions**: Bash commands, browser actions, file operations
- **System events**: Cron jobs, heartbeats, background processes

### Role Tagging
Log entries are tagged by source for searchability:
- `[user]` - Human messages
- `[assistant]` - Main agent responses
- `[agent:{name}]` - Sub-agent (researcher, coder, vision, math)
- `[tool:{name}]` - Tool execution results
- `[system]` - System events

## Three Storage Modes

### 1. API Mode (Paid)
- **Cost**: $59-299/month at memvid.com
- **Capacity**: 1-25GB+
- **Files**: Single unified file
- **Search**: One query searches all history
- **Best for**: Power users, unified long-term memory

### 2. Free Mode
- **Cost**: FREE
- **Capacity**: 50MB (~5K conversations)
- **Files**: Single file until limit reached
- **Search**: Unified search until limit
- **Best for**: Testing, light usage

### 3. Sharding Mode (Workaround)
- **Cost**: FREE forever
- **Capacity**: Unlimited (monthly files)
- **Files**: `anthony_memory_YYYY-MM.mv2`
- **Search**: Must query each month separately
- **Best for**: Long-term free usage, staying under 50MB limit

## Configuration

### Environment Variables
```bash
# Choose mode
export MEMVID_MODE="single"    # API or Free
export MEMVID_MODE="monthly"   # Sharding (default)

# Optional paths
export JSONL_LOG_PATH="~/conversation_log.jsonl"
export MEMVID_PATH="~/anthony_memory.mv2"
export MEMVID_BIN="~/.npm-global/bin/memvid"

# For API mode only
export MEMVID_API_KEY="your_key_here"
```

## Search Commands

### API/Free Mode (Single File)
```bash
# Natural language queries
memvid ask anthony_memory.mv2 "What did we discuss about BadjAI?"
memvid ask anthony_memory.mv2 "What did the researcher agent find?"

# Keyword search
memvid find anthony_memory.mv2 --query "Python script"
```

### Sharding Mode (Multiple Files)
```bash
# Search specific month
memvid ask anthony_memory_2026-02.mv2 "recent discussions"

# Search all months (manual)
for file in anthony_memory_*.mv2; do
    memvid ask "$file" "your query"
done
```

## Troubleshooting

**"Free tier limit exceeded"**
```bash
# Archive and restart
mv anthony_memory.mv2 anthony_memory_archive.mv2
memvid create anthony_memory.mv2

# Or switch to sharding
export MEMVID_MODE="monthly"

# Or upgrade to API
export MEMVID_API_KEY="your_key"  # $59-299/month
```

## Best Practices

1. **Start with Sharding Mode** - Free forever, no surprises
2. **Monitor usage** - Check `memvid stats` regularly
3. **Use JSONL backup** - Always accessible with grep/jq
4. **Tag important conversations** - Search works on role tags
5. **Archive old months** - Keep active file under 50MB for speed

## About Memvid

> Memvid is a single-file memory layer for AI agents with instant retrieval and long-term memory. Persistent, versioned, and portable memory, without databases.
>
> Replace complex RAG pipelines with a single portable file you own, and give your agent instant retrieval and long-term memory.


path: README.md
content: # OpenClaw Memvid Logger

[![OpenClaw](https://img.shields.io/badge/OpenClaw->=2026.2.12-blue)](https://openclaw.ai)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Memvid](https://img.shields.io/badge/Memvid-2.0+-green)](https://memvid.com)

> **Memvid**: A single-file memory layer for AI agents with instant retrieval and long-term memory. Persistent, versioned, and portable memory, without databases.
>
> *"Replace complex RAG pipelines with a single portable file you own, and give your agent instant retrieval and long-term memory."*

A dual-output conversation logger for [OpenClaw](https://openclaw.ai) that captures **everything** - user messages, assistant responses, sub-agent conversations, tool executions, and system events - to both JSONL (backup) and Memvid (semantic search) formats.

## ⚠️ Security & Privacy Notice

**Before installing, please understand:**

This skill captures **everything** - by design. It logs all user messages, assistant responses, sub-agent conversations, tool outputs, and system events to local files. This enables powerful long-term memory but requires trust.

**What you should know:**
- **Broad capture scope:** This is intentional - the skill's purpose is complete conversation logging
- **Sensitive data risk:** Tool outputs (commands, API responses, file contents) are logged. Review what tools expose.
- **Continuous logging:** Once installed, it runs automatically on every assistant response until removed
- **Optional cloud mode:** API mode with `MEMVID_API_KEY` sends data to memvid.com (third-party service). Free/local modes keep data on your machine only.
- **Your responsibility:** Secure the JSONL/.mv2 files, rotate logs regularly, and audit what gets captured.

**Mitigations available:**
- Use **Free/Sharding mode** to keep data local (no API key needed)
- Change default paths to encrypted locations
- Review `tools/log.py` before installing to understand exactly what gets logged
- File permissions: restrict access to log files (`chmod 600`)

**This skill is for users who want complete conversation memory and accept the privacy trade-offs.**

---

## ✨ What Makes This Different

- **📝 Dual Storage** - Every message saved to JSONL + Memvid simultaneously
- **🔍 Semantic Search** - Ask "What did the researcher agent find about Tesla?" not just keyword search
- **🤖 Full Context** - Captures user input, assistant output, agent chatter, tool results
- **💾 Three Modes** - API (unlimited), Free (50MB), or Sharding (multi-file)
- **🚀 Always On** - Hooks into OpenClaw automatically

## 🚀 Quick Start (Pick Your Mode)

### Option 1: API Mode - Near Limitless Memory ⭐ Recommended
**Best for:** Heavy users, unified search across everything  
**Cost:** $59-299/month via [memvid.com](https://memvid.com)

```bash
# Install
npm install -g memvid
git clone https://github.com/stackBlock/openclaw-memvid-logger.git
cp -r openclaw-memvid-logger ~/.openclaw/workspace/skills/unified-logger

# Configure
export MEMVID_API_KEY="your_key_here"
export MEMVID_MODE="single"

# Create memory
memvid create ~/anthony_memory.mv2

# Start OpenClaw - everything logs to one searchable file
```

**Search everything (single query across all time):**
```bash
# Natural language questions
memvid ask anthony_memory.mv2 "What did we discuss about BadjAI?"
memvid ask anthony_memory.mv2 "What did the researcher agent find?"
memvid ask anthony_memory.mv2 "Show me all Python scripts I requested"

# Keyword search
memvid find anthony_memory.mv2 --query "Mercedes"

# Temporal queries
memvid when anthony_memory.mv2 "last Tuesday"
```

---

### Option 2: Free Mode - 50MB Limit
**Best for:** Testing, light usage, single file  
**Cost:** FREE

```bash
# Install
npm install -g memvid
git clone https://github.com/stackBlock/openclaw-memvid-logger.git
cp -r openclaw-memvid-logger ~/.openclaw/workspace/skills/unified-logger
export MEMVID_MODE="single"

# Create memory
memvid create ~/anthony_memory.mv2

# Start OpenClaw
```

**⚠️ Limit:** 50MB (~5,000 conversation turns). When you hit it:
- Archive and start fresh, OR
- Upgrade to API mode ($59-299/month), OR  
- Switch to Sharding mode

---

### Option 3: Sharding Mode - Free Forever (Workaround)
**Best for:** Long-term use, staying under free tier  
**Cost:** FREE  
**Trade-off:** Multi-file search required

```bash
# Install
npm install -g memvid
git clone https://github.com/stackBlock/openclaw-memvid-logger.git
cp -r openclaw-memvid-logger ~/.openclaw/workspace/skills/unified-logger
export MEMVID_MODE="monthly"  # Default

# Start OpenClaw - auto-creates monthly files
```

**How it works:**
- `anthony_memory_2026-02.mv2` (February)
- `anthony_memory_2026-03.mv2` (March - auto-created)
- Each file stays under 50MB

**⚠️ Sharding Search Differences:**

Single-file search (API/Free modes):
```bash
# One search gets everything
memvid ask anthony_memory.mv2 "What car did I decide to buy?"
# Returns: Results from ALL conversations across ALL time
```

Sharding search (requires multiple queries):
```bash
# Must search each month separately
memvid ask anthony_memory_2026-02.mv2 "car decision"  # Recent
memvid ask anthony_memory_2026-01.mv2 "car decision"  # January

# Or use a wrapper script to search all files
for file in anthony_memory_*.mv2; do
    echo "=== $file ==="
    memvid ask "$file" "car decision" 2>/dev/null | head -5
done

# You must know which month the conversation happened
# No cross-month context - "compare this month to last month" won't work
```

**Why sharding is harder:**
- Can't ask "what did we discuss in the past 3 months?" in one query
- No unified timeline across months
- Must remember which month you talked about what
- No cross-file semantic comparison

---

## 📊 Three Modes Compared

| Feature | API Mode | Free Mode | Sharding Mode |
|---------|----------|-----------|---------------|
| **Cost** | $59-299/mo | FREE | FREE |
| **Capacity** | 1GB-25GB+ | 50MB | Unlimited (files) |
| **Files** | 1 | 1 | Multiple (monthly) |
| **Unified Search** | ✅ One query | ✅ One query | ❌ Per-file queries |
| **Cross-Context Search** | ✅ Full history | ✅ Full history | ❌ Month isolated |
| **Best For** | Power users | Testing | Workaround for free tier |
| **Complexity** | Simple | Simple | Must track files |

---

## 💸 The Pricing Gap (AKA Why Sharding Exists)

**The situation:** Memvid's pricing goes from $0 (50MB) straight to $59/month (25GB).  
**The problem:** That's like buying a Ferrari when you just need a Honda Civic for your commute.

**What we're doing about it:**  
I reached out. While they consider it, Sharding Mode exists so you don't have to pay Ferrari prices for Honda Civic usage.

**You can help:**  
If you also think $0 → $59 is a bit much, reach out to Memvid at [memvid.com](https://memvid.com) and tell them stackBlock sent you. The more voices, the faster we get that $10-20 middle tier for the rest of us.

*Until then: Sharding Mode. Because startups shouldn't have to choose between ramen and memory.* 🍜

---

## 🔍 Search Examples by Mode

### API Mode / Free Mode (Single File)

**One command searches everything:**

```bash
# What did you say about...?
memvid ask anthony_memory.mv2 "What was your recommendation about the Mercedes?"

# What did I ask for...?
memvid ask anthony_memory.mv2 "What Python scripts did I request?"

# What did agents do...?
memvid ask anthony_memory.mv2 "What did the researcher agent find about options?"

# Temporal - searches all history
memvid when anthony_memory.mv2 "last month"

# Cross-time context
memvid ask anthony_memory.mv2 "Compare our BadjAI discussions from January to March"
```

### Sharding Mode (Multiple Files)

**Must specify which file to search:**

```bash
# Recent conversations (current month)
memvid ask anthony_memory_2026-02.mv2 "recent discussions"

# Specific month
memvid ask anthony_memory_2026-01.mv2 "January conversations"

# Search all months (manual)
for file in anthony_memory_*.mv2; do
    echo "=== $file ==="
    memvid ask "$file" "your query"
done

# Can't do: cross-month comparison
# Can't do: "what did we discuss over the past 3 months"
# Can't do: unified timeline view
```

### JSONL Backup (All Modes)

```bash
# Quick grep
grep "Mercedes" conversation_log.jsonl

# Complex queries with jq
jq 'select(.role_tag == "user" and .content | contains("Python"))' conversation_log.jsonl

# Time range (works across all modes)
jq 'select(.timestamp >= "2026-02-01" and .timestamp < "2026-03-01")' conversation_log.jsonl
```

---

## 🔍 Search Usage

**Default search mode: `--mode neural`** for maximum accuracy and semantic understanding:

```bash
# Semantic search (default recommended mode)
memvid ask anthony_memory.mv2 "What did we discuss about BadjAI?" --mode neural
memvid ask anthony_memory.mv2 "What supplements did Dr. Sinclair recommend?" --mode neural
memvid ask anthony_memory.mv2 "Show me all Python scripts I requested" --mode neural

# Alternative modes (when explicitly needed)
memvid find anthony_memory.mv2 --mode lex --query "exact_keyword"     # Fastest: ~8ms
memvid find anthony_memory.mv2 --mode hybrid --query "diabetes meds"  # Balanced: ~300ms
```

**Speed vs Accuracy:**
- `neural`: ~200ms — **Default** — Understands context, synonyms, meaning
- `lex`: ~8ms — Exact keyword matching only
- `hybrid`: ~300-500ms — Combines both approaches

The ~200ms latency for neural mode is the recommended default for production use.

## 📁 What Gets Logged

| Source | Tag | Captured |
|--------|-----|----------|
| **Your messages** | `[user]` | ✅ Everything you type |
| **My responses** | `[assistant]` | ✅ Everything I say |
| **Sub-agents** | `[agent:researcher]`, `[agent:coder]` | ✅ Agent conversations |
| **Tool calls** | `[tool:exec]`, `[tool:browser]` | ✅ Commands & results |
| **System** | `[system]` | ✅ Cron, heartbeats, events |

---

## 🏗️ Architecture

```
User → OpenClaw → log.py → JSONL (backup)
                         → Memvid .mv2 (search)
```

**Zero data loss:** Every character preserved  
**Instant indexing:** Searchable immediately  
**Crash proof:** Append-only writes

---

## ⚙️ Configuration

**⚠️ CRITICAL: Environment variables must be set system-wide**

OpenClaw runs as a background service and does not inherit `.bashrc` or `.profile`. You **must** set environment variables in `/etc/environment`:

```bash
# Add to /etc/environment (requires sudo)
sudo tee -a /etc/environment << 'EOF'
MEMVID_MODE="monthly"
MEMVID_PATH="/home/YOUR_USERNAME/.openclaw/workspace/anthony_memory_2026-02.mv2"
JSONL_LOG_PATH="/home/YOUR_USERNAME/.openclaw/workspace/conversation_log.jsonl"
MEMVID_BIN="/home/YOUR_USERNAME/.npm-global/bin/memvid"
EOF

# Reboot to apply
sudo reboot
```

**Why `/etc/environment`?**
- `.bashrc` / `.profile`: Only work for interactive terminal sessions
- `/etc/environment`: System-wide, works for background services including OpenClaw

**After reboot, verify:**
```bash
echo $MEMVID_MODE  # Should show: monthly
```

### Variable Reference

| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `MEMVID_MODE` | Yes | `monthly` | `single` (one file) or `monthly` (rotation) |
| `MEMVID_PATH` | Yes | Auto-derived | Path to `.mv2` memory file |
| `JSONL_LOG_PATH` | Yes | Auto-derived | Path to JSONL backup |
| `MEMVID_BIN` | Yes | `memvid` | Path to memvid CLI |
| `MEMVID_API_KEY` | No | - | For API mode only (paid) |

### Quick Mode Reference

```bash
# API Mode (single file, cloud, paid)
export MEMVID_MODE="single"
export MEMVID_API_KEY="your_key_here"

# Free Mode (single file, local, 50MB limit)
export MEMVID_MODE="single"

# Sharding Mode (monthly files, local, unlimited) ⭐ Recommended
export MEMVID_MODE="monthly"
```

---

## 🆘 Troubleshooting

### "Environment variables not set" / Hook not logging

**Symptoms:**
- `openclaw hooks info unified-logger` shows environment requirements as ❌
- JSONL file not being written
- `echo $MEMVID_MODE` returns empty

**Cause:** Variables set in `.bashrc` or `.profile` don't persist for background services.

**Fix:**
```bash
# Check current setting
echo $MEMVID_MODE

# If empty, add to /etc/environment (NOT .bashrc)
sudo tee -a /etc/environment << 'EOF'
MEMVID_MODE="monthly"
MEMVID_PATH="/home/YOUR_USERNAME/.openclaw/workspace/memory_2026-02.mv2"
JSONL_LOG_PATH="/home/YOUR_USERNAME/.openclaw/workspace/conversation_log.jsonl"
MEMVID_BIN="/home/YOUR_USERNAME/.npm-global/bin/memvid"
EOF

# Reboot and verify
sudo reboot
# After reboot:
echo $MEMVID_MODE  # Should show: monthly
```

### "Hook registered but not logging messages"

**Symptoms:**
- `openclaw hooks list` shows ✅ ready
- `openclaw hooks info unified-logger` shows all requirements met
- But JSONL/Memvid not being updated

**Cause:** OpenClaw 2026.2.12+ requires JavaScript (`.js`) handlers, not TypeScript (`.ts`), for managed hooks.

**Fix:** Ensure handler is JavaScript:
```bash
ls ~/.openclaw/hooks/unified-logger/
# Should show: handler.js (NOT handler.ts)

# If you have handler.ts, rename/create handler.js instead
```

**Correct hook structure:**
```
~/.openclaw/hooks/unified-logger/
├── HOOK.md      # Metadata with "export": "default"
├── handler.js   # JavaScript handler (NOT .ts)
└── healthcheck.py  # Optional monitoring
```

### "Memvid index not updating" / "invalid value for --tag"

**Symptoms:**
- JSONL file updates correctly
- Memvid search returns no results
- Error: `expected KEY=VALUE, got 'user,telegram'`

**Cause:** Memvid 2.0+ changed tag format from comma-separated to `KEY=VALUE` pairs.

**Fix:** Update to unified-logger v1.2.5+ which uses correct format:
```python
# OLD (broken):
--tag "user,telegram,agent:researcher"

# NEW (correct):
--tag "role=user" --tag "source=telegram" --tag "agent=researcher"
```

### "Free tier limit exceeded" (Free Mode)

```bash
# Option 1: Archive and start fresh
mv anthony_memory.mv2 anthony_memory_archive.mv2
memvid create anthony_memory.mv2

# Option 2: Switch to monthly sharding (recommended)
# Edit /etc/environment:
MEMVID_MODE="monthly"
# Then reboot

# Option 3: Get API key for unlimited
export MEMVID_API_KEY="your_key"  # $59-299/month at memvid.com
```

### "memvid: command not found"

```bash
npm install -g memvid

# Verify:
which memvid  # Should show path
memvid --version  # Should show version
```

### Check Everything is Working

```bash
# 1. Environment variables
echo $MEMVID_MODE
echo $JSONL_LOG_PATH
echo $MEMVID_PATH
echo $MEMVID_BIN

# 2. Hook status
openclaw hooks list
openclaw hooks info unified-logger

# 3. Recent logs
tail -5 "$JSONL_LOG_PATH"

# 4. Memvid search
memvid find "$MEMVID_PATH" --query "test"

# 5. Health check log
cat ~/.openclaw/logs/logger-health.log
```

---

## 📚 Documentation

- [SKILL.md](SKILL.md) - Detailed OpenClaw skill documentation
- [Memvid Docs](https://memvid.com/docs) - Memvid CLI reference
- [OpenClaw Docs](https://docs.openclaw.ai) - OpenClaw framework

---

## 🤝 Contributing

1. Fork the repo
2. Create a feature branch
3. Submit a PR

See [CONTRIBUTING.md](CONTRIBUTING.md)

---

## 📄 License

MIT - See [LICENSE](LICENSE)

---

**Made with 🤝 for the OpenClaw community**

- GitHub: [github.com/stackBlock/openclaw-memvid-logger](https://github.com/stackBlock/openclaw-memvid-logger)
- Discord: [discord.com/invite/clawd](https://discord.com/invite/clawd)

**About Memvid:**
> Memvid is a single-file memory layer for AI agents with instant retrieval and long-term memory. 
> Persistent, versioned, and portable memory, without databases.
> 
> Replace complex RAG pipelines with a single portable file you own, and give your agent 
> instant retrieval and long-term memory.


path: package.json
content: {
  "name": "@openclaw/unified-logger",
  "version": "1.2.5",
  "description": "Dual-output conversation logger for OpenClaw - JSONL backup + Memvid semantic search",
  "main": "tools/log.py",
  "scripts": {
    "test": "python3 -m pytest tests/",
    "lint": "flake8 tools/"
  },
  "keywords": [
    "openclaw",
    "skill",
    "memory",
    "logging",
    "memvid",
    "conversation",
    "search"
  ],
  "author": "AnToni <[REDACTED_SECRET]>",
  "license": "MIT",
  "repository": {
    "type": "git",
    "url": "https://github.com/stackBlock/openclaw-memvid-logger.git"
  },
  "bugs": {
    "url": "https://github.com/stackBlock/openclaw-memvid-logger/issues"
  },
  "homepage": "https://github.com/stackBlock/openclaw-memvid-logger#readme",
  "engines": {
    "openclaw": ">=2026.2.12",
    "python": ">=3.8"
  },
  "peerDependencies": {
    "memvid": ">=2.0.0"
  },
  "openclaw": {
    "type": "skill",
    "category": "memory",
    "tools": [
      {
        "name": "log_conversation",
        "description": "Append conversation turn to JSONL and Memvid",
        "command": "python3 tools/log.py",
        "schema": {
          "type": "object",
          "properties": {
            "timestamp": {"type": "string", "format": "date-time"},
            "session_id": {"type": "string"},
            "role": {"type": "string", "enum": ["user", "assistant"]},
            "content": {"type": "string"},
            "tool_calls": {"type": ["array", "null"]}
          },
          "required": ["timestamp", "session_id", "role", "content"]
        }
      }
    ],
    "hooks": {
      "message_out": {
        "description": "Called after each assistant response",
        "command": "python3 tools/log.py"
      }
    },
    "config": {
      "env": {
        "JSONL_LOG_PATH": {
          "description": "Path to JSONL backup file",
          "default": "~/workspace/conversation_log.jsonl",
          "type": "string"
        },
        "MEMVID_PATH": {
          "description": "Path to Memvid .mv2 file",
          "default": "~/workspace/memory.mv2",
          "type": "string"
        },
        "MEMVID_BIN": {
          "description": "Path to memvid CLI binary",
          "default": "~/.npm-global/bin/memvid",
          "type": "string"
        }
      }
    }
  }
}


path: install.sh
content: #!/bin/bash
# Unified Logger Installation Script
# Usage: ./install.sh

set -e

SKILL_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
OPENCLAW_SKILLS="${HOME}/.openclaw/workspace/skills"
TARGET_DIR="${OPENCLAW_SKILLS}/unified-logger"

echo "═══════════════════════════════════════════════════════════"
echo "  Unified Conversation Logger - Installation"
echo "═══════════════════════════════════════════════════════════"
echo ""
echo "⚠️  SECURITY NOTICE:"
echo "   This skill captures ALL OpenClaw conversations and events"
echo "   to local files (JSONL + Memvid). Review the code before"
echo "   installing. See: tools/log.py for details."
echo ""
echo "   - Logs: ${HOME}/.openclaw/workspace/conversation_log.jsonl"
echo "   - Memory: ${HOME}/.openclaw/workspace/memory_YYYY-MM.mv2"
echo ""
read -p "Continue with installation? (y/N) " -n 1 -r
echo
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
    echo "Installation cancelled."
    exit 0
fi

echo ""
echo "Checking dependencies..."

if ! command -v python3 &> /dev/null; then
    echo "❌ Python 3 not found. Please install Python 3.8+"
    exit 1
fi
echo "✓ Python 3 found"

if ! command -v memvid &> /dev/null; then
    echo ""
    echo "⚠️  Memvid CLI not found."
    echo "   This requires: npm install -g memvid"
    read -p "Install Memvid CLI now? (requires sudo for global install) (y/N) " -n 1 -r
    echo
    if [[ $REPLY =~ ^[Yy]$ ]]; then
        npm install -g memvid || {
            echo "❌ Failed to install Memvid CLI. Install manually:"
            echo "   npm install -g memvid"
            exit 1
        }
        echo "✓ Memvid CLI installed"
    else
        echo "⚠️  Memvid CLI required. Install manually before using skill."
    fi
else
    echo "✓ Memvid CLI found"
fi

# Create OpenClaw skills directory if needed
mkdir -p "${OPENCLAW_SKILLS}"

# Copy skill files
echo ""
echo "Installing skill to ${TARGET_DIR}..."
if [ -d "${TARGET_DIR}" ]; then
    echo "⚠️  Skill already exists. Overwrite? (y/N)"
    read -r response
    if [[ ! "$response" =~ ^[Yy]$ ]]; then
        echo "Installation cancelled."
        exit 0
    fi
    rm -rf "${TARGET_DIR}"
fi

cp -r "${SKILL_DIR}" "${TARGET_DIR}"
echo "✓ Skill files copied"

# Make log.py executable
chmod +x "${TARGET_DIR}/tools/log.py"

# Initialize memory file if doesn't exist
MEMORY_FILE="${HOME}/.openclaw/workspace/memory_$(date +%Y-%m).mv2"
if [ ! -f "${MEMORY_FILE}" ]; then
    echo ""
    echo "Creating initial memory file..."
    memvid create "${MEMORY_FILE}" 2>/dev/null || {
        echo "⚠️  Could not create memory file. Will create on first use."
        echo "   Manual command: memvid create ${MEMORY_FILE}"
    }
    if [ -f "${MEMORY_FILE}" ]; then
        echo "✓ Memory file created"
    fi
else
    echo "✓ Memory file already exists"
fi

# Set up environment variables - ASK FIRST
BASHRC="${HOME}/.bashrc"
echo ""
echo "═══════════════════════════════════════════════════════════"
echo "  Environment Variables Setup"
echo "═══════════════════════════════════════════════════════════"
echo ""
echo "This skill uses environment variables for configuration:"
echo "   - MEMVID_MODE (single/monthly)"
echo "   - JSONL_LOG_PATH (log file location)"
echo "   - MEMVID_PATH (memory file location)"
echo ""
echo "These can be added to ${BASHRC} for persistence."
echo ""
read -p "Add environment variables to ${BASHRC}? (y/N) " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
    if ! grep -q "MEMVID_MODE" "${BASHRC}" 2>/dev/null; then
        cat >> "${BASHRC}" << 'EOF'

# Unified Logger (MemSync) configuration
export MEMVID_MODE="monthly"  # 'single' for API mode, 'monthly' for sharding
export JSONL_LOG_PATH="${HOME}/.openclaw/workspace/conversation_log.jsonl"
export MEMVID_PATH="${HOME}/.openclaw/workspace/memory.mv2"
export MEMVID_BIN="$(which memvid 2>/dev/null || echo ${HOME}/.npm-global/bin/memvid)"
EOF
        echo "✓ Environment variables added to ${BASHRC}"
        echo "  Run: source ${BASHRC}  (or restart your terminal)"
    else
        echo "✓ Environment variables already configured"
    fi
else
    echo "⚠️  Skipped environment setup. Set manually if needed:"
    echo "   export MEMVID_MODE=monthly"
fi

echo ""
echo "═══════════════════════════════════════════════════════════"
echo "  Installation Complete!"
echo "═══════════════════════════════════════════════════════════"
echo ""
echo "Security reminders:"
echo "   • This skill logs ALL conversations and events"
echo "   • Data stored locally in: ${HOME}/.openclaw/workspace/"
echo "   • API mode requires MEMVID_API_KEY (remote storage)"
echo "   • Review logs regularly and secure sensitive data"
echo ""
echo "Next steps:"
echo "   1. Source your profile: source ${BASHRC}"
echo "   2. Start OpenClaw - logging begins automatically"
echo "   3. Search: memvid ask memory_$(date +%Y-%m).mv2 'your query'"
echo ""
echo "Documentation: ${TARGET_DIR}/README.md"
echo ""


path: CONTRIBUTING.md
content: # Contributing to Unified Logger

Thank you for your interest in contributing! This project follows standard open source practices.

## Getting Started

1. Fork the repository
2. Clone your fork: `git clone https://github.com/YOUR_USERNAME/unified-logger.git`
3. Create a branch: `git checkout -b feature/your-feature-name`

## Development Setup

```bash
# Install dependencies
npm install -g @memvid/cli

# Set up test environment
export JSONL_LOG_PATH="/tmp/test_conversation.jsonl"
export MEMVID_PATH="/tmp/test_memory.mv2"

# Run tests (if available)
python3 -m pytest tests/
```

## Code Style

- Python: PEP 8 compliant
- Shell scripts: `shellcheck` clean
- Documentation: Clear, concise, example-driven

## Submitting Changes

1. Ensure your code works with OpenClaw >= 2026.2.12
2. Update README.md if adding features
3. Add examples for new functionality
4. Submit a pull request with a clear description

## Reporting Issues

When reporting bugs, please include:
- OpenClaw version
- Python version
- Memvid CLI version
- Steps to reproduce
- Expected vs actual behavior

## Questions?

- Open an issue for bugs/features
- Join [OpenClaw Discord](https://discord.com/invite/clawd) for discussions


path: claw.json
content: {
  "name": "openclaw-memvid-logger",
  "version": "1.2.5",
  "description": "Dual-output conversation logger with full context capture. Stores to JSONL (backup) and Memvid (semantic search) for long-term memory",
  "author": "stackBlock",
  "license": "MIT",
  "category": ["memory", "logging", "search"],
  "tags": ["memvid", "conversation", "memory", "logging", "semantic-search", "long-term-memory", "dual-storage", "jsonl"],
  "homepage": "https://github.com/stackBlock/openclaw-memvid-logger",
  "repository": "https://github.com/stackBlock/openclaw-memvid-logger",
  "entry": "tools/log.py",
  "requirements": {
    "python": ">=3.8",
    "external": ["memvid >=2.0"]
  },
  "env": {
    "MEMVID_MODE": {
      "description": "Storage mode: 'single' (one file) or 'monthly' (rotation)",
      "required": false,
      "default": "monthly"
    },
    "MEMVID_PATH": {
      "description": "Path to memvid .mv2 memory file (default: ~/.openclaw/workspace/memory.mv2)",
      "required": false
    },
    "JSONL_LOG_PATH": {
      "description": "Path to JSONL backup log file (default: ~/.openclaw/workspace/conversation_log.jsonl)",
      "required": false
    },
    "MEMVID_BIN": {
      "description": "Path to memvid CLI binary (auto-detected if not set)",
      "required": false
    }
  },
  "credentials": [
    {
      "name": "MEMVID_API_KEY",
      "description": "Memvid API key for cloud/paid tier access. NOT REQUIRED for free local mode. Only needed if using API mode with remote storage.",
      "required": false,
      "secret": true
    }
  ],
  "permissions": [
    {
      "name": "file_write",
      "reason": "Write conversation logs to JSONL files and create Memvid .mv2 memory files in user's workspace"
    },
    {
      "name": "exec_subprocess",
      "reason": "Call memvid CLI to append frames to memory files"
    }
  ],
  "warnings": [
    {
      "type": "privacy",
      "message": "This skill captures ALL OpenClaw activity including user messages, assistant responses, sub-agent conversations, tool outputs, and system events. Data is stored locally by default (JSONL + .mv2 files). API mode requires explicit MEMVID_API_KEY and sends data to memvid.com cloud service."
    },
    {
      "type": "data_retention",
      "message": "Logs persist until manually deleted. Consider log rotation and file permissions for sensitive data."
    }
  ]
}


path: CHANGELOG.md
content: # Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [1.2.5] - 2026-02-22

### Fixed
- **Memvid Tag Format:** Updated `build_tags()` to use `KEY=VALUE` format for Memvid 2.0+ compatibility
  - Old: `--tag "user,telegram"` (comma-separated)
  - New: `--tag "role=user" --tag "source=telegram"` (multiple flags)
  - Fixes "invalid value for --tag" error

### Changed
- **Documentation - Environment Variables:** Added clear instructions to use `/etc/environment` instead of `.bashrc`/`.profile`
  - OpenClaw runs as a background service and doesn't inherit shell profiles
  - System-wide environment via `/etc/environment` is required
- **Documentation - Hook Format:** Clarified that OpenClaw 2026.2.12+ requires JavaScript (`.js`) handlers, not TypeScript (`.ts`)
- **Documentation - Troubleshooting:** Added comprehensive troubleshooting section covering:
  - Environment variables not persisting
  - Hook registered but not logging
  - Memvid tag format errors
  - Free tier limits
  - Full diagnostic checklist

### Compatibility
- Verified with OpenClaw 2026.2.12
- Verified with Memvid CLI 2.0+

## [1.2.4] - 2026-02-20

### Changed
- **Search Mode Default:** Updated documentation to recommend `--mode neural` as the default search mode
- **Performance Guidance:** Clarified latency trade-offs (~200ms neural vs ~8ms lexical) with recommendation to accept neural for accuracy
- **Search Policy:** Changed from "start with lex, fallback to neural" to "always use neural unless speed is critical"

### Documentation
- Updated Search Performance Guide in SKILL.md
- Added Search Usage section to README.md with neural as default
- Clarified when to use each search mode (neural/lex/hybrid)

## [1.2.3] - 2026-02-19

### Changed
- **Version Cohesion:** All files synchronized to v1.2.3
- **Documentation Consistency:** README and SKILL.md now have matching content
- **Security Improvements:** Generic paths (no hardcoded user directories), install script asks permission
- **Registry Compliance:** Complete metadata for ClawHub transparency

### Added
- **Privacy Documentation:** Comprehensive Security & Privacy Notice explaining data capture scope
- **Role Tagging:** Distinguishes user, assistant, agent:*, system, and tool messages
- **Three Storage Modes:** API mode (single file), Free mode (50MB), Sharding mode (monthly rotation)

## [1.0.0] - 2026-02-19

### Added
- Initial release of Unified Logger
- Dual-output logging (JSONL + Memvid)
- Automatic hook integration with OpenClaw
- Semantic search via Memvid CLI
- Environment variable configuration
- Installation script
- Comprehensive documentation

### Features
- Zero data loss - raw conversation text preserved
- Instant searchability - every message indexed
- Crash resilience - append-only writes
- Configurable paths via environment variables
- Silent failure - never breaks OpenClaw conversations

[1.2.4]: https://github.com/stackBlock/openclaw-memvid-logger/releases/tag/v1.2.4
[1.2.3]: https://github.com/stackBlock/openclaw-memvid-logger/releases/tag/v1.2.3
[1.0.0]: https://github.com/openclaw/unified-logger/releases/tag/v1.0.0


path: tests/test_log.py
content: #!/usr/bin/env python3
"""
Basic tests for Unified Logger
"""

import json
import os
import sys
import tempfile
import unittest
from io import StringIO

# Add parent directory to path
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', 'tools'))

import log


class TestLogToJsonl(unittest.TestCase):
    """Test JSONL logging functionality."""
    
    def setUp(self):
        self.temp_dir = tempfile.mkdtemp()
        self.test_log_path = os.path.join(self.temp_dir, "test.jsonl")
        log.LOG_PATH = self.test_log_path
    
    def tearDown(self):
        import shutil
        shutil.rmtree(self.temp_dir, ignore_errors=True)
    
    def test_log_to_jsonl_creates_file(self):
        """Test that logging creates the JSONL file."""
        entry = {
            "timestamp": "2026-02-19T12:00:00Z",
            "session_id": "test123",
            "role": "user",
            "content": "Hello world",
            "tool_calls": None
        }
        
        result = log.log_to_jsonl(entry)
        self.assertTrue(result)
        self.assertTrue(os.path.exists(self.test_log_path))
        
        # Verify content
        with open(self.test_log_path, 'r') as f:
            lines = f.readlines()
            self.assertEqual(len(lines), 1)
            data = json.loads(lines[0])
            self.assertEqual(data['content'], "Hello world")
            self.assertEqual(data['role'], "user")


class TestMainFunction(unittest.TestCase):
    """Test main entry point."""
    
    def setUp(self):
        self.temp_dir = tempfile.mkdtemp()
        self.test_log_path = os.path.join(self.temp_dir, "test.jsonl")
        log.LOG_PATH = self.test_log_path
        
        # Save original stdin
        self.original_stdin = sys.stdin
    
    def tearDown(self):
        sys.stdin = self.original_stdin
        import shutil
        shutil.rmtree(self.temp_dir, ignore_errors=True)
    
    def test_main_with_valid_input(self):
        """Test main function with valid JSON input."""
        test_input = json.dumps({
            "timestamp": "2026-02-19T12:00:00Z",
            "session_id": "test123",
            "role": "assistant",
            "content": "Test response",
            "tool_calls": None
        })
        
        sys.stdin = StringIO(test_input)
        log.main()
        
        # Verify file was created
        self.assertTrue(os.path.exists(self.test_log_path))
    
    def test_main_with_empty_input(self):
        """Test main function handles empty input gracefully."""
        sys.stdin = StringIO("")
        log.main()  # Should not raise
        self.assertFalse(os.path.exists(self.test_log_path))


class TestConfiguration(unittest.TestCase):
    """Test configuration handling."""
    
    def test_default_paths(self):
        """Test that default paths are set."""
        # These should be strings, not None
        self.assertIsInstance(log.LOG_PATH, str)
        self.assertIsInstance(log.MEMVID_PATH, str)
        self.assertIsInstance(log.MEMVID_BIN, str)
    
    def test_env_override(self):
        """Test that environment variables override defaults."""
        original = os.environ.get('JSONL_LOG_PATH')
        
        try:
            os.environ['JSONL_LOG_PATH'] = '/custom/path.jsonl'
            # Reload module to pick up env var
            import importlib
            importlib.reload(log)
            self.assertEqual(log.LOG_PATH, '/custom/path.jsonl')
        finally:
            if original:
                os.environ['JSONL_LOG_PATH'] = original
            else:
                del os.environ['JSONL_LOG_PATH']


if __name__ == '__main__':
    unittest.main()


path: .github/PULL_REQUEST_TEMPLATE.md
content: ## Description
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context.

Fixes # (issue)

## Type of change
Please delete options that are not relevant.

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] Documentation update

## How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce.

- [ ] Test A
- [ ] Test B

## Checklist:
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes


path: .github/workflows/ci.yml
content: name: CI

on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main ]

jobs:
  test:
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v4
    
    - name: Set up Python
      uses: actions/setup-python@v5
      with:
        python-version: '3.10'
    
    - name: Set up Node.js
      uses: actions/setup-node@v4
      with:
        node-version: '20'
    
    - name: Install Memvid CLI
      run: npm install -g memvid
    
    - name: Check Python syntax
      run: |
        python3 -m py_compile tools/log.py
        echo "✓ Python syntax OK"
    
    - name: Check shell scripts
      run: |
        if command -v shellcheck &> /dev/null; then
          shellcheck install.sh
          echo "✓ Shell scripts OK"
        else
          echo "⚠ shellcheck not installed, skipping"
        fi
    
    - name: Test log.py imports
      run: |
        python3 -c "import tools.log" || echo "⚠ Import test skipped (needs full environment)"
    
    - name: Validate JSON files
      run: |
        python3 -c "import json; json.load(open('package.json'))"
        echo "✓ package.json valid"
    
    - name: Check file structure
      run: |
        test -f README.md
        test -f LICENSE
        test -f SKILL.md
        test -f package.json
        test -f tools/log.py
        test -x tools/log.py
        echo "✓ File structure OK"


path: .github/ISSUE_TEMPLATE/feature_request.md
content: ---
name: Feature request
about: Suggest an idea for this project
title: '[FEATURE] '
labels: enhancement
assignees: ''

---

**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

**Describe the solution you'd like**
A clear and concise description of what you want to happen.

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context**
Add any other context or screenshots about the feature request here.


path: .github/ISSUE_TEMPLATE/bug_report.md
content: ---
name: Bug report
about: Create a report to help us improve
title: '[BUG] '
labels: bug
assignees: ''

---

**Describe the bug**
A clear and concise description of what the bug is.

**To Reproduce**
Steps to reproduce the behavior:
1. Start OpenClaw with skill installed
2. Send message '...'
3. Check log file '...'
4. See error

**Expected behavior**
A clear and concise description of what you expected to happen.

**Environment:**
 - OS: [e.g. Ubuntu 22.04]
 - OpenClaw Version: [e.g. 2026.2.12]
 - Python Version: [e.g. 3.10]
 - Memvid CLI Version: [e.g. 2.0.137]

**Logs**
If applicable, add error messages or log output.

**Additional context**
Add any other context about the problem here.


path: tools/log.py
content: #!/usr/bin/env python3
"""
Unified Conversation Logger for OpenClaw - Monthly Rotation Edition v1.2.0
====================================================================

Logs every conversation turn to both JSONL (backup) and Memvid (searchable memory).
Uses monthly rotation files to stay under free tier limits.

Features:
- Dual-output logging (JSONL + Memvid)
- Monthly rotation: memory_YYYY-MM.mv2
- Role tagging: user, assistant, agent:*, system, tool
- Captures everything: messages, tool calls, agent spawns, background tasks

Author: AnToni
Version: 1.2.0 (Full Context Edition)
License: MIT
"""

import json
import sys
import os
import subprocess
import tempfile
import datetime
from datetime import datetime as dt, timezone
from typing import Dict, Optional

# Configuration paths - override with environment variables
# Default paths use user's home directory for portability
HOME_DIR = os.path.expanduser("~")
DEFAULT_WORKSPACE = os.path.join(HOME_DIR, ".openclaw", "workspace")

LOG_PATH = os.environ.get("JSONL_LOG_PATH", 
                          os.path.join(DEFAULT_WORKSPACE, "conversation_log.jsonl"))

# Monthly rotation for Memvid files (stays under 50MB free tier per file)
# Format: memory_YYYY-MM.mv2 (generic, not user-specific)
MEMVID_BASE = os.environ.get("MEMVID_PATH", 
                             os.path.join(DEFAULT_WORKSPACE, "memory.mv2"))
MEMORY_DIR = os.path.dirname(MEMVID_BASE) or DEFAULT_WORKSPACE

# Check for mode: 'single' (one file) or 'monthly' (rotating)
MEMVID_MODE = os.environ.get("MEMVID_MODE", "monthly")  # 'single' for API mode, 'monthly' for free/sharding

if MEMVID_MODE == "monthly":
    current_month = dt.now().strftime("%Y-%m")
    MEMVID_PATH = os.path.join(MEMORY_DIR, f"memory_{current_month}.mv2")
else:
    MEMVID_PATH = MEMVID_BASE

# Try to find memvid in PATH, fallback to common npm global locations
_MEMVID_PATHS = [
    "/usr/local/bin/memvid",
    "/usr/bin/memvid",
    os.path.expanduser("~/.npm-global/bin/memvid"),
    os.path.expanduser("~/.local/bin/memvid"),
]
_DEFAULT_MEMVID = next((p for p in _MEMVID_PATHS if os.path.exists(p)), "memvid")
MEMVID_BIN = os.environ.get("MEMVID_BIN", _DEFAULT_MEMVID)


def ensure_memory_file():
    """Create memory file for current month if it doesn't exist."""
    if not os.path.exists(MEMVID_PATH):
        try:
            subprocess.run(
                [MEMVID_BIN, "create", MEMVID_PATH],
                capture_output=True,
                timeout=30
            )
        except Exception:
            pass  # Silent fail - will try again next message


def get_role_tag(message: Dict) -> str:
    """
    Generate role tag for frame title.
    
    Roles:
    - user: Human input
    - assistant: Main agent response
    - agent:{name}: Sub-agent (researcher, coder, etc.)
    - system: System events, heartbeats, cron
    - tool: Tool execution results
    """
    raw_role = message.get("role", "unknown")
    
    # Check if this is a sub-agent message
    agent_id = message.get("agent_id") or message.get("subagent_id")
    if agent_id:
        return f"agent:{agent_id}"
    
    # Check source field
    source = message.get("source", "")
    if source.startswith("agent:"):
        return source
    
    # Check for system/cron messages
    msg_type = message.get("type", "")
    if msg_type in ["system", "heartbeat", "cron"]:
        return "system"
    
    # Check for tool results
    if "tool_calls" in message and message.get("tool_result"):
        return "tool"
    
    return raw_role


def get_frame_title(log_entry: Dict) -> str:
    """Build descriptive title for memvid frame."""
    role = log_entry.get("role_tag", "unknown")
    content = log_entry.get("content", "")
    agent_name = log_entry.get("agent_name", "")
    
    # Get preview
    content_preview = content[:80].replace('\n', ' ').strip()
    
    # Build title based on role
    if role == "user":
        return f"[user] {content_preview}..."
    elif role == "assistant":
        return f"[assistant] {content_preview}..."
    elif role.startswith("agent:"):
        agent = role.split(":")[1] if ":" in role else "unknown"
        return f"[agent:{agent}] {content_preview}..."
    elif role == "system":
        return f"[system] {content_preview}..."
    elif role == "tool":
        tool_name = log_entry.get("tool_name", "unknown")
        return f"[tool:{tool_name}] {content_preview}..."
    else:
        return f"[{role}] {content_preview}..."


def build_tags(log_entry: Dict) -> list:
    """Build KEY=VALUE tags for memvid."""
    tags = []
    
    # Role tag
    role = log_entry.get("role_tag", "unknown")
    tags.append(f"role={role}")
    
    # Source tag
    if log_entry.get("source"):
        tags.append(f"source={log_entry.get('source')}")
    
    # Agent tag
    if log_entry.get("agent_id"):
        tags.append(f"agent={log_entry.get('agent_id')}")
    
    # Tool tag
    if log_entry.get("tool_calls"):
        tags.append("has_tools=true")
    
    # Session tag
    session = log_entry.get("session_id", "")[:8]
    if session:
        tags.append(f"session={session}")
    
    # Event type
    if log_entry.get("event_type"):
        tags.append(f"event={log_entry.get('event_type')}")
    
    return tags if tags else ["source=openclaw"]


def log_to_jsonl(log_entry: Dict) -> bool:
    """Append conversation turn to JSONL file."""
    try:
        os.makedirs(os.path.dirname(LOG_PATH), exist_ok=True)
        with open(LOG_PATH, "a", encoding="utf-8") as f:
            f.write(json.dumps(log_entry, ensure_ascii=False) + "\n")
            f.flush()
        return True
    except Exception as e:
        print(f"[jsonl-logger error] {e}", file=sys.stderr)
        return False


def log_to_memvid(log_entry: Dict) -> bool:
    """Append conversation turn to Memvid .mv2 file."""
    try:
        ensure_memory_file()
        
        # Create temp file
        with tempfile.NamedTemporaryFile(mode='w', suffix='.json', 
                                         delete=False) as f:
            json.dump(log_entry, f, ensure_ascii=False, indent=2)
            temp_path = f.name
        
        # Build metadata
        title = get_frame_title(log_entry)
        ts = log_entry.get("timestamp", dt.now(timezone.utc).isoformat())
        date_only = ts.split('T')[0] if 'T' in ts else ts[:10]
        tags = build_tags(log_entry)
        
        # Build command with multiple --tag arguments
        cmd = [
            MEMVID_BIN, "put", MEMVID_PATH,
            "--title", title,
            "--timestamp", date_only,
            "--input", temp_path
        ]
        for tag in tags:
            cmd.extend(["--tag", tag])
        
        # Call memvid put
        result = subprocess.run(
            cmd,
            capture_output=True,
            text=True,
            timeout=30
        )
        
        os.unlink(temp_path)
        return result.returncode == 0
        
    except Exception:
        return False


def main():
    """
    Main entry point. Reads message JSON from stdin.
    
    Captures:
    - user: Human messages
    - assistant: Main agent responses  
    - agent:*: Sub-agent conversations (researcher, coder, vision, etc.)
    - system: Heartbeats, cron jobs, system events
    - tool: Tool execution results
    """
    try:
        data = sys.stdin.read()
        if not data:
            return
            
        message = json.loads(data)
        
        # Determine role/tag
        role_tag = get_role_tag(message)
        
        # Build comprehensive log entry
        log_entry = {
            "timestamp": message.get("timestamp", 
                                    dt.now(timezone.utc).isoformat()),
            "session_id": message.get("session_id", "unknown"),
            "role": message.get("role", "unknown"),
            "role_tag": role_tag,
            "content": message.get("content", ""),
            "agent_id": message.get("agent_id") or message.get("subagent_id"),
            "agent_name": message.get("agent_name", ""),
            "tool_calls": message.get("tool_calls", None),
            "tool_name": message.get("tool_name", ""),
            "tool_result": message.get("tool_result", None),
            "source": message.get("source", "openclaw_conversation"),
            "type": message.get("type", "message"),
            "memvid_mode": MEMVID_MODE,
            "memvid_file": MEMVID_PATH,
            "logged_at": dt.now(timezone.utc).isoformat()
        }
        
        # Log to both destinations
        log_to_jsonl(log_entry)
        log_to_memvid(log_entry)
        
    except Exception as e:
        print(f"[logger error] {e}", file=sys.stderr)
        sys.exit(0)


if __name__ == "__main__":
    main()


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: suspicious | SkillSpector: suspicious (CRITICAL)