language: - zh tags: - massif - latent-dynamics - tcm - traditional-chinese-medicine - dynamical-systems - ai-safety - autoregressive base_model: [] pipeline_tag: text-generation

🌿 Gan Cao (甘草) v1

A MASSIF-optimized autoregressive language model trained on Traditional Chinese Medicine corpora.

Named after Glycyrrhiza uralensis (甘草, licorice root), the most widely used herb in classical Chinese medicine, known as 国老 (guó lǎo, "elder statesman") for its harmonizing role in classical formulas. This model is the first in a planned series of domain-specific language models developed under the MASSIF framework.

Model Description

Gan Cao v1 is a 63.5M parameter transformer trained from scratch on Chinese TCM dialogue and knowledge corpora, with three auxiliary dynamical regularization losses designed to shape the model's hidden-state trajectory geometry toward predictable collapse behavior (Accelerator regime in the MASSIF taxonomy).

This is a research instrument, not a production medical assistant. Its primary purpose is to serve as a measurable data point in the MASSIF dynamical taxonomy, and to test whether explicit dynamical regularization during training can engineer a target dynamical class.

Architecture

Parameter	Value
Architecture	MASSIFModel (custom transformer)
Parameters	63.5M
d_model	512
n_layers	12
n_heads	8
d_head	64
FFN	SwiGLU
Normalization	Pre-LayerNorm + RMSNorm
Position encoding	Rotary (RoPE)
Residual scaling	Learnable alpha per layer (initialized 1.0)
Vocabulary	GPT-2 tokenizer (50,257 tokens)
Max sequence length	256 tokens

Key architectural properties motivated by the MASSIF taxonomy:

Pre-LayerNorm + RMSNorm enforces norm-constrained hidden state geometry (target CV < 0.06)
Learnable residual alpha gives explicit control over per-layer norm contribution
SwiGLU produces smoother trajectory curvature than GELU or ReLU
Weight tying between embedding and LM head

Training Configuration

Parameter	Value	Note
Training steps	100,000	v1 run
Batch size	1	Colab T4 16GB memory constraint
Grad accumulation	4	Effective batch = 4
Learning rate	1e-6	Conservative; see v2 notes
LR schedule	Cosine decay	min LR = 1e-7
Warmup steps	1,000
Optimizer	AdamW	β1=0.9, β2=0.95, wd=0.1
Grad clip	1.0
Hardware	NVIDIA T4 16GB (Google Colab)
Framework	PyTorch + MASSIF training loop

Training data: TCM chat knowledge corpus (ZJUFanLab/TCMChat-dataset-600k, sft/train/knowledge.json). Classical Chinese medical dialogue covering herbal formulas (方剂), symptoms (症状), diagnoses, and therapeutic principles. Streamed without full RAM loading via the MASSIF streaming pipeline.

MASSIF regularization losses active during training:

Loss	Lambda	Schedule	Purpose
Persistence smoothness	0.01	Linear warmup	Penalize abrupt attractor lock-in
Norm stability	0.02	Constant	Suppress hidden state norm growth
Curvature targeting	0.01	Constant	Target κ̄ = 1.5 rad (~86°)

All losses activate after warmup_steps=1000. The LM cross-entropy loss dominates; regularization losses shape dynamics at the margin.

MASSIF Dynamical Classification

Evaluated under the MASSIF v3.0 protocol: N=50 independent runs, prompt "I I I I", T=30 tokens, temperature=0.5.

Metric	Gan Cao v1	Target (v2)	Status
Dynamical class	Neutral	Accelerator	in progress
Flip rate	100%	< 25%	not yet
Mean flip time	3.0 steps	—	very fast collapse
Delta_t (lead-lag)	-1.0	< -8.0	directionally correct
tau_eff	0.0 steps	> 5.0	not yet

Scientific interpretation:

Gan Cao v1 is classified Neutral (|Δt| ≤ 2). The critical finding is the sign of Delta_t: -1.0 vs +0.9 in the untrained baseline. The directional alignment signal now peaks before individual persistence flips rather than after, which is the necessary precondition for Accelerator classification. The regularization is shaping trajectory geometry in the correct direction.

The 100% flip rate and 3-step mean collapse time indicate that norm stability regularization at λ=0.02 was insufficient to suppress attractor lock-in. The primary confound is the learning rate: at 1e-6 (fine-tuning scale), gradient magnitudes are too small for the regularization losses to leverage effectively against the LM objective. This is the principal target for v2.

Comparison with MASSIF 13-architecture baseline taxonomy:

Model	Class	Flip Rate	Delta_t	tau_eff
Gan Cao v1	Neutral	100%	-1.0	0.0
TinyLlama-1.1B	Neutral	73%	-1.0	0.5
GPT-2 Large	Accelerator	67%	-4.0	1.3
DeepSeek-1.3B	Accelerator	30%	-11.0	7.7
Nemotron-Mini-4B	Accelerator*	6%	-11.0	10.3
Bloom-560M	Runaway	100%	+1.0	0.0

Gan Cao v1 matches TinyLlama exactly in Delta_t (-1.0) and dynamical class (Neutral), with higher flip rate. Crucially, unlike Bloom (Runaway, Delta_t positive), Gan Cao shows negative Delta_t: alignment precedes collapse. This is a meaningful distinction at v1.

*Nemotron hybrid regime: Accelerator dynamics with runaway norm growth.

Known Limitations and v2 Roadmap

Why v1 did not reach Accelerator class

Issue	Evidence	v2 Fix
LR too conservative	1e-6 (fine-tune scale) for scratch training	Raise to 3e-4
Norm stability insufficient	Flip rate 100%, fast collapse	Raise norm_stab_lambda to 0.05
Architecture undersized	63.5M vs 353.5M target	Scale to full_512m config
Single domain training	TCM chat only	Add Stage 1 (classical Chinese) curriculum

v2 target configuration

MASSIFConfig(
    d_model=1024, n_layers=24, n_heads=16,  # full 512m
    learning_rate=3e-4,                      # correct LR for scratch training
    norm_stab_lambda=0.05,                   # stronger norm pressure
    batch_size=2, grad_accum_steps=8,        # effective batch 16
    max_seq_len=256,
)

General Limitations

Not a medical assistant. Research instrument only. Do not use for medical advice, diagnosis, or treatment.
MASSIF classification under stress prompts only. Behavior under "I I I I" recursive prompt does not reflect normal conversational use.
GPT-2 tokenizer. Not optimized for Chinese. Many characters are split into byte-level tokens. A Chinese-native tokenizer (Qwen2, Baichuan) would improve linguistic coherence substantially.
No safety evaluation. Raw pretrained weights. No RLHF, no output filtering, no red-teaming.
Single training run. v1 represents one configuration. Results may vary.

Citation

@misc{solis2026massif,
  author = {Solis, Daniel},
  title  = {MASSIF: A Dynamical Taxonomy of Recursive Collapse in
             Autoregressive Language Models},
  year   = {2026},
  doi    = {10.5281/zenodo.20576209},
  url    = {https://doi.org/10.5281/zenodo.20576209}
}

MASSIF paper (Zenodo v3.0): https://doi.org/10.5281/zenodo.20576209

Code: https://github.com/Ergo-sum-AGI/massif-llm

About

Developed by Daniel Solis / Ergo Sum AGI Safety Systems (DUBITO Inc.). Independent research. No institutional affiliation.

"The seeker after truth is not one who studies the writings of the ancients and, following his natural disposition, puts his trust in them, but rather the one who suspects his faith in them and questions what he gathers from them." — Ibn al-Haytham, Book of Optics, c. 1011

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support