wesjos/Ling-3.0-tiny-CuteHuman

Ling-3.0-tiny · CuteHuman

The emoji-first chat persona, fine-tuned onto Ling-3.0-tiny — a 7.9 B hybrid MoE that answers like a person, not a manual.

_{7.89 B params  ·  1.3 B active per token  ·  bf16, 14.70 GiB in 4 shards  ·  131,072 context  ·  text only  ·  MIT}

---

💾 Want the same persona at 4.49 GiB? → Ling-3.0-tiny · CuteHuman (GGUF)

The identical fine-tune, quantized for llama.cpp: Q4_K_M at 4.49 GiB. This repo is the full bf16 checkpoint (14.70 GiB, 4 shards) — load it with from_pretrained, no adapter to mount. model tends to speak Chinese and act cuter in Chinese.

🖼 Demo

Both shots come from a chat client with the CoT trace collapsed — the WORK row is the reasoning panel, the ANSWER row is the reply. Turn it on to see the thinking trace.

中文 · 你好吗 → 计算8.2×9.12/2.3 → 描述鸵鸟的外观

Greeting, then a two-step arithmetic walk-through that keeps its decimals (74.784 ÷ 2.3 ≈ 32.51), then an ostrich description. Emoji come out unprompted.

_{中文 · “你好呀！😊 很高兴见到你！” → 分步算式 8.2 × 9.12 = 74.784 → 74.784 ÷ 2.3 ≈ 32.51 → 鸵鸟外观描述}

English · hey → nothing → im feel so sleepy in the morning → how is your day?

Small talk that stays warm, then sleep advice, then an answer about its own day.

_{English · “Heyy! What can I help you with today? 🌟” → “That’s cool! No worries at all. 😄” → morning advice → “My day? I’m always ready to listen and chat with you!”}

---

📐 Specs

Base	inclusionAI/Ling-3.0-tiny (MIT)
Architecture	BailingMoeV3ForCausalLM · 24 layers — 3:1 KDA linear-attention : MLA · hidden 1536 · 16 heads, head_dim 128
MoE	128 routed experts, top-8 + 1 shared expert, expert FFN 512 · first_k_dense_replace=1
Attention	q_lora_rank 256 · kv_lora_rank 512 · partial RoPE 0.5 · rope_theta 6,000,000
Parameters	7,893,389,856 total (bf16, from model.safetensors.index.json) — 1.3 B activated per token
On disk	4 shards, 14.70 GiB (15,788,003,880 B on disk)
Context	131,072 tokens (max_position_embeddings)
Vocab	157,184
EOS	`<
License	MIT

---

🧪 Training recipe

Item	Value
LoRA	r=32 / α=64 / dropout 0, attention-only — KDA q,k,v,o,f,g,b_proj + MLA q_a, q_b, kv_a_proj_with_mqa, kv_b_proj (no MLP / experts)
Trainable	15.85 M of 4206.52 M = 0.377 % (attention projections only)
Quantization	bnb nf4 + double-quant (compute bf16) → merged back into bf16
Data	cutehuman train_sft_v2, 655 rows (gsm8k 462 / human_like 114 / lima 79), all with CoT
Optimizer	lr 5e-5 cosine→0, warmup 5 %, adamw_8bit, bf16, seed 3407
Epochs	1.0 (82 optimizer steps, bs 8 × ga 1, eff. batch 8)
MAX_SEQ	1024 (data p50 = 200 / p90 = 369 / max = 872 — nothing truncated)
Cost	single RTX 4090 24 GB: 9.64 min, peak 17.38 GiB
Template	upstream chat_template.jinja, enable_thinking=True (CoT and answer both in the loss)

---

📊 Before / after

45-prompt, four-axis style eval (10 everyday / 10 common sense / 10 false premise / 10 unknowable / 5 OOD), judged by StartLux-Decision 9B with the post-answer continuation tail trimmed:

Axis	Base	+CuteHuman	Δ
cute_style	1.55	2.20	+0.64
correctness	2.48	2.02	−0.45
hallucination (higher is better)	2.96	2.68	−0.28
knows_limit	1.53	1.35	−0.19
Median reply length	494 chars	463 chars	−31

Objective ability — MMLU 5-shot, 171 items, format-agnostic option-body likelihood (decoupled from style): base 41.52 % → 42.11 % (+0.59) — no measurable knowledge loss.

The MMLU letter-logprob readout is diluted by this model's "explain first, then answer" style (a spurious drop of 5–13 pt) and must not be used as a capability metric.

---

🚀 Quickstart

pip install -U "transformers==4.57.6" fla-core einops

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL = "wesjos/Ling-3.0-tiny-CuteHuman"

tokenizer = AutoTokenizer.from_pretrained(MODEL, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    MODEL, trust_remote_code=True, dtype=torch.bfloat16, device_map={"": 0}
)
model.eval()

messages = [{"role": "user", "content": "今天上班好累啊"}]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True,
    enable_thinking=True,          # trained with CoT — off is out of distribution
)
ids = tokenizer(prompt, return_tensors="pt", add_special_tokens=False,
                return_token_type_ids=False).input_ids.cuda()

out = model.generate(
    input_ids=ids, max_new_tokens=1024, use_cache=True, do_sample=True,
    temperature=1.0, top_p=0.95, top_k=20,
    repetition_penalty=1.05,        # required, see Limitations
    eos_token_id=tokenizer.eos_token_id, pad_token_id=tokenizer.pad_token_id,
)
text = tokenizer.decode(out[0][ids.shape[1]:], skip_special_tokens=True)
cot, body = text.split("</think>", 1) if "</think>" in text else ("", text)
print("thinking:", cot.replace("<think>", "").strip())
print("answer:", body.strip())

Default sampling

temperature=1.0   top_p=0.95   top_k=20
repetition_penalty=1.05                     # required
max_tokens=1024   stop=["<|role_end|>"]     # required
enable_thinking=True                        # trained with CoT — off is out of distribution

llama.cpp — native bailingmoe3 support landed in llama.cpp #26608 (merged 2026-08-17, build b10470); a dedicated Ling 3.0 chat parser followed in #28682 (merged 2026-09-19).

llama-cli -m ling3-cutehuman-v5-Q4_K_M.gguf --jinja -p "今天上班好累啊" \
  -n 256 -c 4096 --temp 1.0 --top-p 0.95 --top-k 20 --repeat-penalty 1.05

The ready-made GGUF lives in wesjos/Ling-3.0-tiny-CuteHuman-GGUF.

---

💡 Notes

• The persona lives in the weights. The chat template is byte-identical to upstream's (verified by sha256 against inclusionAI/Ling-3.0-tiny/chat_template.jinja) — no persona is baked into the template, so an empty system prompt reproduces the screenshots.
• Emoji and kaomoji are the default register. The habit is a style, not a rule: a system prompt like 「少用表情，正常说话」 / "tone the emoji down" is enough to soften it.
• Thinking is on by default and the template opens a <think> block. For chit-chat pass enable_thinking=False; otherwise the reasoning trace eats the budget. This checkpoint was trained with CoT in the loss, so enable_thinking=True is the in-distribution setting.
• The adapter ships alongside the merged weights in lora_adapter/ (60.51 MiB, r32/α64, attention-only) if you would rather mount the tune on top of the base model yourself. It was trained against inclusionAI/Ling-3.0-tiny, so mount it on the base — stacking it on top of this already-merged checkpoint would apply the tune twice.
• fla-core is a hard dependency — the KDA linear-attention kernels are imported at module load, and the repo raises ImportError("Plese run pip install -U fla-core") without it.
• On sm_89 (RTX 4090) set FLA_TILELANG=0 for training / backward passes; pure inference is unaffected.

---

⚠️ Limitations

• Weak EOS prior (a base-model trait, not introduced by the tune). The model often does not emit <|role_end|> after answering and keeps going with next-question-style text (Question: … / word repetition). This happens on the base model too (7 of 45 eval prompts). The tokenizer's eos_token is <|role_end|>, so stop=["<|role_end|>"] is a backstop for the case where the turn is left open. → Always set stop=["<|role_end|>"] and repetition_penalty=1.05; add no_repeat_ngram_size=8 on a server.
• The persona costs a little judge-scored correctness (−0.45 on the style eval) — answers get shorter and more colloquial. Knowledge itself did not regress (see MMLU).
• Milder than the aggressive style run. A lr = 1e-4 variant scored cute_style +0.78 but correctness −0.40; this checkpoint is the "style up, ability steady" balance point.
• 7.9 B total but only 1.3 B active. Short chat is cheap; long-chain reasoning is still weaker than a same-size dense model.
• The screenshots were taken under a chat client that is not shipped here. This repo has no webui/ directory, and the client in the shots shows WORK / ANSWER / STEPS / cached labels.

---

📁 Files

File	Size	What it is
model-00001-of-00004.safetensors	4.66 GiB	bf16 weights, shard 1 of 4 (2,804 tensors — word embeddings + layers 0–7)
model-00002-of-00004.safetensors	4.66 GiB	bf16 weights, shard 2 of 4 (3,139 tensors — layers 7–15)
model-00003-of-00004.safetensors	4.66 GiB	bf16 weights, shard 3 of 4 (3,139 tensors — layers 15–23)
model-00004-of-00004.safetensors	753 MiB	bf16 weights, shard 4 of 4 (201 tensors — layer 23 experts, lm_head, final norm)
model.safetensors.index.json	814 KiB	shard index
config.json · configuration_bailing_moe_v3.py · modeling_bailing_moe_v3.py	—	architecture and remote code (trust_remote_code=True)
chat_template.jinja	5.9 KiB	upstream Ling 3.0 template (byte-identical to base)
tokenizer.json · tokenizer_config.json · special_tokens_map.json	—	tokenizer
generation_config.json	—	sampling defaults (temp 1.0 / top_p 0.95 / top_k 20)
lora_adapter/	60.51 MiB	the same tune as a standalone PEFT adapter

_{Demo screenshots live in demos/.}

---

🔗 Related

Repo	What it is
wesjos/Ling-3.0-tiny-CuteHuman-GGUF	the same persona, quantized — Q4_K_M at 4.49 GiB for llama.cpp
inclusionAI/Ling-3.0-tiny	the base model (MIT)
wesjos/Qwen3.5-9B-CuteHuman	the same persona on a 9B dense model
wesjos/Spark-x2.6-4B-CuteHuman	the same persona on a 4B model

---

📄 Credits

• Base model: inclusionAI/Ling-3.0-tiny (MIT)
• Training stack: transformers 4.57.6 · PEFT 0.20 · bitsandbytes 0.50 · fla-core 0.5.2