spinozans/emender-e97-1.3b

Emender E97 1.3B

Emender E97 is a 1.287B-parameter recurrent base language model with nonlinear matrix state and split erase/write control inspired by Gated DeltaNet 2. It is an E97 split-edit model, not the repository's separate GDN2 control.

This revision contains the checkpoint after 513,013,841,920 training tokens (step 2322520, recorded loss 2.2798). The default repository revision is the later 513B authority; the 150B authority is retained as a historical reproducibility revision.

Artifact

• Source checkpoint SHA-256: e559df3e8c540aef59ce8c9d73338f255cbe2fb9c7301ab45c7ef36a5b0fb857
• Export: evaluated ScheduleFree train/y weights only
• Tokenizer: p50k_base
• Context used in training: 2,048 tokens
• Training approach: local DiLoCo with periodic model averaging
• Raw optimizer state and pickle checkpoint: not included

The model is a raw base LM, not an instruction or chat model. A full HellaSwag audit of the 513B authority scored 0.3651663 normalized accuracy; GPT-2 XL in the same harness scored 0.4890460.

Loading

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("spinozans/emender-e97-1.3b")
model = AutoModelForCausalLM.from_pretrained(
    "spinozans/emender-e97-1.3b", trust_remote_code=True, dtype="auto"
)

The custom loader requires the matching Emender source package:

pip install "git+https://github.com/spinozans/emender.git@2b5d135587cca9d1081dbbdf7b35a8ac2629db5d"

The qualified fused path targets AMD ROCm/Frontier. The bundled Transformers wrapper defaults to the slower native portable path.

Limitations and license

This is a research checkpoint with limited benchmark coverage. It can produce incorrect, biased, or unsafe text. No standalone model license has yet been selected; license: other is intentional and no permissive license should be inferred from repository visibility.