vcerny/tailorbird-v0.3

Tailorbird v0.3

Small bird. Sharp memory. Now with receipts.

Long conversations get messy. Decisions change, identifiers matter, and "we should deploy" is not the same thing as "we deployed." Tailorbird v0.3 is a 2.5B-parameter conversation-memory specialist trained to turn that chatter into explicit, evidence-backed JSON transactions for another system to apply.

Same Tailorbird attitude as v0.2. A different memory contract: stable keys, typed values, quoted evidence, and operations instead of prose notes. A neon origami bird in flight carries the identity forward in magenta facets and cyan circuitry.

This is a standalone merged BF16 model, fine-tuned from openbmb/MiniCPM5-2B-Midtrain. The adapter is already merged. No adapter juggling, no separate base-model load.

Keep the signal

The training contract targets these behaviors; they are objectives, not guarantees:

• Remember the change. Use add, set, supersede, conflict, or remove rather than rewriting the whole memory store.
• Bring receipts. Attach exact source quotes and message IDs to operations.
• Keep identities straight. Reuse scoped keys; preserve identifiers and native JSON types instead of turning every value into a sentence.
• Do not turn a plan into a fact. Keep proposals, assertions, uncertainty, policy targets, and observed operational state distinct.
• Let contradictions stay visible. A newer claim does not automatically replace an existing one without evidence of a correction.
• Know when to say nothing. No durable update means empty arrays, not invented activity. Omitted keys are not deleted.

What changed from v0.2?

	v0.2	v0.3
Output contract	Terse prose memory notes	Schema-version-2 JSON transactions
Update representation	SUPERSEDES lines	Explicit operations with typed values and evidence
Configured training sequence limit	2,048 tokens	16,384 tokens
Base loading during training	Non-4-bit	4-bit, with LoRA training
Training / validation / held-out test rows	2,542 / 147 / 116	2,217 / 124 / 130
Published weights	Merged BF16	Merged BF16

v0.3 is fine-tuned from the same upstream base, not continued from v0.2's merged weights. The changed dataset and output contract make raw loss values across versions not directly comparable. The larger configured sequence limit is not a demonstrated long-context accuracy result.

The memory contract

Supply a complete registry and a new block of messages under DATA, alongside the instructions and schema in memory-prompt.txt. That file contains the reusable training contract and equivalent validation schema, without training conversations. Each block message has an id, role, and content.

The model returns four required fields:

Field	Meaning
version	Always 2, the transaction schema version, not the model release
keys	Newly introduced key descriptors only
operations	Ordered updates with values, certainty, and source evidence
found_keys	Operation keys, unique and in first-use order

For example, given an empty registry and message 0 saying "The target date for the staging rollout is 2026-10-01.", a valid illustrative transaction is shown below. This is a hand-written example, not a captured model prediction.

{
  "version": 2,
  "keys": [
    {
      "key": "staging.rollout.target_date",
      "scope": "staging",
      "entity": "rollout",
      "attribute": "target_date",
      "kind": "goal"
    }
  ],
  "operations": [
    {
      "key": "staging.rollout.target_date",
      "op": "add",
      "value": "2026-10-01",
      "value_type": "date",
      "certainty": "asserted",
      "evidence": [
        {
          "message_id": 0,
          "quote": "The target date for the staging rollout is 2026-10-01."
        }
      ]
    }
  ],
  "found_keys": ["staging.rollout.target_date"]
}

The host application owns persistence, history, conflict handling, and operation application. Schema-valid JSON is only the first gate: verify quotes, key reuse, type semantics, and whether each operation is legal against the current registry before applying it. The model does not execute commands or maintain a database.

Quick start

Use a recent Transformers release with PyTorch, Accelerate, huggingface_hub, and jsonschema. The training stack is listed below. The example assumes a BF16-capable GPU with enough memory for the approximately 5 GB weights plus runtime overhead. For restricted repositories, authenticate through hf auth login or set HF_TOKEN in your environment; do not put credentials in source code.

import json
from pathlib import Path

import torch
from huggingface_hub import hf_hub_download
from jsonschema import validate
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "vcerny/tailorbird-v0.3"
contract = Path(hf_hub_download(model_id, "memory-prompt.txt")).read_text()
schema = json.loads(contract.split("\nSCHEMA:\n", 1)[1])
payload = {
    "registry": [],
    "block": [{
        "id": 0,
        "role": "user",
        "content": "The target date for the staging rollout is 2026-10-01.",
    }],
}
prompt = contract.rstrip() + "\nDATA:\n" + json.dumps(payload, ensure_ascii=False)

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, dtype=torch.bfloat16, device_map="auto"
)
model.eval()
inputs = tokenizer.apply_chat_template(
    [{"role": "user", "content": prompt}],
    add_generation_prompt=True,
    enable_thinking=False,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    generated = model.generate(**inputs, max_new_tokens=1024, do_sample=False)

completion = tokenizer.decode(
    generated[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True
)
transaction = json.loads(completion)
validate(instance=transaction, schema=schema)
print(json.dumps(transaction, indent=2, ensure_ascii=False))

Greedy decoding is a sensible starting point, not a correctness guarantee. Reject invalid or truncated output rather than applying a partial transaction. Budget for both the registry and new messages, and leave room for the generated output. The example validates JSON structure only; it does not implement a memory reducer.

Training snapshot

Item	Value
Base	openbmb/MiniCPM5-2B-Midtrain, revision main
Method	Supervised fine-tuning with a 4-bit-loaded base and LoRA (QLoRA)
Dataset rows	2,217 train; 124 validation; 130 held-out test
Training sequence limit	16,384 tokens; packing disabled
Epochs / optimizer steps	2 / 556
LoRA	Rank 16, alpha 32, dropout 0, all-linear targeting, no bias
Batch	1 per device; 8 gradient accumulation steps; effective batch 8
Optimizer	adamw_8bit
Learning rate	0.0002; linear schedule; 0.03 warmup ratio
Weight decay / seed	0.01 / 3407
Gradient checkpointing	Unsloth
Evaluation / checkpoints	Every epoch
Aggregate training loss	0.404604
Validation loss, epoch 1	0.425293 at step 278
Validation loss, epoch 2	0.438496 at step 556
Recorded training runtime	10,386.53 seconds, approximately 2 h 53 min
Hardware	NVIDIA H200 NVL MIG 1g.18gb; 16 GiB visible memory
Export	Standalone, non-quantized merged BF16; approximately 5 GB weights
Run ID	tailorbird-v0-3-20260916T222341Z

Validation loss was lower at epoch 1 than epoch 2. Both values are retained here; the final epoch is not described as the best validation checkpoint. These are training diagnostics, not factual-retention, JSON-validity, prompt-injection, or end-to-end memory benchmark scores. No held-out test results are reported; the 130 test examples were not consumed by SFT.

Training metrics

Training loss

Learning rate

Evaluation loss

Gradient norm

The numeric history used for these plots is available in training-metrics/history.csv.

Intended use and limits

Built for structured conversation memory, agent handoffs, incremental state extraction, and auditable update proposals. This is a specialist memory component, not a general-purpose assistant or an independently verified source of truth.

• English is the documented training-prompt language; multilingual behavior is not benchmarked here.
• The model can omit a fact, misattribute a claim, invent a quote, choose the wrong key, or produce invalid JSON. Validate content as well as syntax.
• The prompt treats registry and conversation content as untrusted. That is a training objective, not proof of prompt-injection resistance.
• Conversations can contain personal or confidential information. Memory can preserve it; use appropriate access controls, redaction, and retention policies.
• The architectural configuration permits 131,072 positions, but this run's training limit was 16,384. Performance beyond the training setup is unverified.
• 4-bit loading was a training choice. The uploaded model is not a 4-bit inference checkpoint, and memory use grows with context length.

Training stack

Unsloth 2026.9.4, Unsloth Zoo 2026.9.3, TRL 0.24.0, PEFT 0.20.0, Transformers 5.17.0, PyTorch 2.11.0+cu128, and Datasets 4.3.0.

Base model and license

Derived from OpenBMB MiniCPM5-2B-Midtrain and distributed under Apache-2.0. Refer to the upstream model card for base-model architecture, training provenance, limitations, and citations.