Upload encoder v0 LoRA adapter (Qwen2.5-0.5B + LoRA rank=16) — Stage 5 final

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,254 @@
 ---
 license: apache-2.0
+base_model: Qwen/Qwen2.5-0.5B-Instruct
+library_name: peft
+tags:
+  - memory-encoder
+  - lora
+  - structured-extraction
+  - lycheemem
+language:
+  - en
+pipeline_tag: text-generation
 ---
+
+# Encoder v0 — Memory Encoder for LycheeMem
+
+A LoRA adapter on top of **Qwen2.5-0.5B-Instruct** that turns conversation turns into structured `MemoryRecord` JSON (typed, atomic, with entities / temporal / evidence span / source_role). Trained by distilling DeepSeek V4 Flash and selecting high-quality candidates via a 4-dim verifier.
+
+Designed as a drop-in encoder for [LycheeMem](https://github.com/LycheeMem/lycheemem)'s write-side memory pipeline, with **physical JSON schema guarantee via constrained decoding** (outlines + Pydantic).
+
+## Highlights
+
+- **8.7 MB LoRA adapter** on a 0.5B base — runs locally on a single RTX 4060 Ti 8GB, zero API cost
+- **+125% weighted_score** over the runtime Qwen2.5-7B baseline on a 519-sample held set
+- **100% JSON schema compliance** with constrained decoding (vs 74% for runtime baseline, 96-98% for SOTA prompt-only)
+- **4× faster** than the runtime baseline (3.4s vs 20s p50 latency)
+- On LongMemEval-style task dialogs, **outperforms even Qwen2.5-72B and V4 Flash teacher** on weighted_score (3.749 vs 3.666 / 3.700)
+
+## Evaluation
+
+Evaluated on 519 held-out conversation segments (LongMemEval-S + MSC-MemFuse-MC10, English personal dialogs). The **weighted_score** is a 4-dim LLM-as-judge metric (V4 Flash) on `atomicity / self_containedness / entity_coverage / evidence_alignment`, weighted 0.25 / 0.30 / 0.20 / 0.25, with failures scored 0 (out of 5.0).
+
+### 7-Model Leaderboard
+
+| rank | model | size | weighted_score | schema_ok | latency p50 |
+|---|---|---|---|---|---|
+| 1 | DeepSeek-V3 | 671B (MoE) | 4.057 | 96.9% | 44s |
+| 2 | Qwen2.5-72B-Instruct | 72B | 3.951 | 98.8% | 33s |
+| 3 | DeepSeek V4 Flash (teacher) | — | 3.833 | 95.8% | 14s |
+| **4** | **encoder_v0 (this model)** | **0.5B + LoRA** | **3.775** | **100.0%** | **3.4s** |
+| 5 | Qwen3-32B | 32B | 3.476 | 97.7% | 67s |
+| 6 | Qwen2.5-14B-Instruct | 14B | 1.946 | 80.5% | 19s |
+| 7 | Qwen2.5-7B-Instruct (runtime baseline) | 7B | 1.679 | 74.0% | 20s |
+
+### 4-Dim Quality Breakdown
+
+| model | atomicity | self_cont | entity_cov | evidence |
+|---|---|---|---|---|
+| DeepSeek-V3 | 4.61 | 4.90 | 4.27 | 3.60 |
+| Qwen2.5-72B | 4.89 | 4.85 | 4.14 | 3.54 |
+| V4 Flash (teacher) | 4.48 | 4.88 | 4.21 | 3.94 |
+| **encoder_v0** | **4.53** | **4.51** | **2.93** ⚠️ | **3.30** |
+| Qwen3-32B | 4.38 | 4.74 | 4.13 | 3.18 |
+| Qwen2.5-7B | 4.20 | 4.47 | 3.27 | 2.98 |
+
+`entity_coverage` is the model's main known weakness (1.0-1.3 points below SOTA), planned to be addressed in v2.
+
+### Per-Source Breakdown
+
+| model | LongMemEval | MSC |
+|---|---|---|
+| DeepSeek-V3 | 3.871 | 4.357 |
+| Qwen2.5-72B | 3.666 | 4.408 |
+| V4 Flash (teacher) | 3.700 | 4.047 |
+| **encoder_v0** | **3.749** | **3.817** |
+| Qwen2.5-7B (baseline) | 1.330 | 2.241 |
+
+On task-oriented dialogs (LongMemEval), encoder_v0 actually **surpasses both Qwen2.5-72B and the V4 Flash teacher**.
+
+## Training
+
+```text
+Pipeline:
+  Stage 1: 5000 conversation segments from LongMemEval-S + MSC-MemFuse-MC10
+  Stage 2a: V4 Flash distillation → 4769 candidate record sets
+  Stage 2b: Rule + V4 Flash verifier (4-dim ≥ 4.0) → 2590 pseudo-gold
+  Stage 2c: +394 synthetic advice-class samples (gold = empty records)
+  Stage 3:  LoRA SFT on Qwen2.5-0.5B-Instruct
+            rank=16, alpha=32, dropout=0.05
+            target_modules = q_proj, k_proj, v_proj, o_proj
+            3 epochs, batch=1*accum16, lr=2e-4, bf16
+            28.5 min on RTX 4060 Ti 8GB
+
+Trainable params: 2.16M / 496M = 0.44%
+Final eval loss: 0.293
+```
+
+Total training cost: ~¥24 (API for distillation + verifier) + 28 min local GPU.
+
+## Intended Use
+
+**Primary use**: Drop-in write-side encoder for LycheeMem (or similar long-term memory systems) that takes a conversation segment and outputs `MemoryRecord` JSON suitable for storage and downstream retrieval.
+
+**Input format**:
+
+```python
+{
+  "previous_turns": [{"role": "user", "content": "..."}, ...],  # optional
+  "current_turns": [{"role": "user", "content": "..."}, ...],   # required
+  "session_date": "2026-05-12"  # optional, ISO or freeform
+}
+```
+
+**Output format** (strict JSON, guaranteed by constrained decoding):
+
+```python
+{
+  "records": [
+    {
+      "memory_type": "fact|preference|event|constraint|procedure|failure_pattern|tool_affordance",
+      "semantic_text": "User plans to visit Beijing on 2026-05-20 to meet Li Hua.",
+      "entities": ["Beijing", "Li Hua"],
+      "temporal": {"t_ref": "2026-05-12", "t_valid_from": "2026-05-20", "t_valid_to": ""},
+      "tags": ["travel", "meeting"],
+      "evidence_turns": [0],
+      "source_role": "user"
+    }
+  ]
+}
+```
+
+## How to Use
+
+### Install dependencies
+
+```bash
+pip install transformers peft outlines pydantic torch
+```
+
+### Inference (with constrained decoding — recommended)
+
+```python
+import torch
+import outlines
+from transformers import AutoTokenizer, AutoModelForCausalLM
+from peft import PeftModel
+from pydantic import BaseModel
+from typing import Literal
+
+# 1. Load base + LoRA adapter
+BASE = "Qwen/Qwen2.5-0.5B-Instruct"
+ADAPTER = "fuhao23/encoder_v0"
+
+tok = AutoTokenizer.from_pretrained(BASE, trust_remote_code=True)
+base = AutoModelForCausalLM.from_pretrained(
+    BASE, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True
+)
+model_hf = PeftModel.from_pretrained(base, ADAPTER).eval()
+
+# 2. Define output schema (must match the schema used in training)
+class Temporal(BaseModel):
+    t_ref: str = ""
+    t_valid_from: str = ""
+    t_valid_to: str = ""
+
+class MemoryRecord(BaseModel):
+    memory_type: Literal["fact", "preference", "event", "constraint",
+                         "procedure", "failure_pattern", "tool_affordance"]
+    semantic_text: str
+    entities: list[str]
+    temporal: Temporal
+    tags: list[str]
+    evidence_turns: list[int]
+    source_role: Literal["user", "assistant", "both", ""]
+
+class MemoryRecordList(BaseModel):
+    records: list[MemoryRecord]
+
+model = outlines.from_transformers(model_hf, tok)
+generator = outlines.Generator(model, MemoryRecordList)
+
+# 3. The system prompt this adapter was trained on (use COMPACT_ENCODING_SYSTEM
+#    from LycheeMem: src/memory/semantic/prompts.py:13-85). Must use as-is.
+SYSTEM_PROMPT = """You are a memory extractor for a personal AI assistant's
+long-term memory system. ... (full prompt in LycheeMem repo)"""
+
+# 4. Build user content + encode
+user_content = """\
+<PREVIOUS_TURNS>
+(no previous turns)
+</PREVIOUS_TURNS>
+
+<CURRENT_TURNS>
+user: I want to try out my new slow cooker from Bed Bath & Beyond.
+assistant: Congratulations! Slow cookers are great for ...
+user: Thanks for the cleaning tips.
+</CURRENT_TURNS>"""
+
+prompt = tok.apply_chat_template(
+    [{"role": "system", "content": SYSTEM_PROMPT},
+     {"role": "user", "content": user_content}],
+    tokenize=False, add_generation_prompt=True,
+)
+output = generator(prompt, max_new_tokens=1024)
+print(output)
+# Output: strict JSON of {"records": [...]}
+```
+
+### Inference (without constrained decoding — not recommended)
+
+The model **CAN** be used without `outlines`, but **schema compliance drops from 100% to ~64%** due to base Qwen2.5-0.5B's tendency to regress to conversation-continuation mode on assistant-advice-heavy inputs. Always use constrained decoding for production.
+
+## Limitations
+
+This is a v0 research release. **Read carefully before deployment**:
+
+1. **LLM-as-judge bias in evaluation**. The `weighted_score` is computed using V4 Flash as judge — the same model family as the teacher. Comparisons against models stronger than V4 Flash (Qwen2.5-72B, DeepSeek-V3) may have ceiling effects; the precise SOTA ranking around rank 1-4 is not fully reliable.
+
+2. **No human ground truth**. No human annotator has labeled records as "good / bad" — judge consistency with humans is unverified. Recommended next step: 50-sample human annotation + Cohen's kappa.
+
+3. **No downstream retrieval evaluation**. The original training plan included an `evidence retrieval hit@10` benchmark on LongMemEval — this is not yet completed. The current metrics measure **encoder output quality in isolation**, not the end-to-end impact on memory retrieval accuracy.
+
+4. **Narrow evaluation distribution**. The 519-sample held set is entirely English personal-dialog (LongMemEval + MSC). Chinese, technical, code, and long-context dialogs are not evaluated. OOD deployment may degrade.
+
+5. **Entity coverage weakness**. `entity_coverage` 4-dim score is 2.93 vs SOTA 4.1-4.3 — the encoder under-extracts named entities. Planned fix in v2 with entity-rich training data.
+
+6. **Constrained decoding is required for the headline 100% schema_ok**. Without `outlines`, schema compliance drops to ~64%.
+
+7. **Not yet integrated into LycheeMem runtime**. No real-traffic data — quality on actual user dialogs vs the eval set is untested.
+
+## Method Background
+
+Pipeline and evaluation methodology documented in detail at the [LycheeMem repository](https://github.com/LycheeMem/lycheemem):
+
+- `docs/encoder_v0.md` — full evaluation report with case studies
+- `docs/encoder_eval_framework.md` — evaluation framework
+- `examples/encoder_v0_try.py` — interactive try-it tool
+
+Inspired by [MemReranker](https://arxiv.org/abs/2605.06132)'s small-model distillation methodology for memory systems.
+
+## Citation
+
+```bibtex
+@misc{lycheemem_encoder_v0,
+  title  = {Encoder v0: A Distilled Memory Encoder for Long-Term Conversation Memory},
+  author = {LycheeMem},
+  year   = {2026},
+  url    = {https://huggingface.co/fuhao23/encoder_v0}
+}
+```
+
+Base model:
+
+```bibtex
+@misc{qwen2.5,
+  title  = {Qwen2.5: A Party of Foundation Models},
+  author = {Qwen Team},
+  year   = {2024}
+}
+```
+
+## License
+
+Apache 2.0 (matches base Qwen2.5-0.5B-Instruct license).

adapter_config.json

@@ -0,0 +1,45 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-0.5B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "v_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:463f96e537f695e81a8d830fd501a4b9e93fbab192b67481df6733d97b21efed
+size 8676008

added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c5ae00e602b8860cbd784ba82a8aa14e8feecec692e7076590d014d7b7fdafa
+size 11421896

tokenizer_config.json

@@ -0,0 +1,207 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}