Hodge-4o

Hodge-4o is a ternarized causal language model. The linear layers are Hodge linear, quantized offline.

Model

hidden size	2560
layers	30
intermediate size	6912
attention heads	20
key-value heads	5
head size	128
vocabulary	514307
context length	4096
rope theta	500000
activation	relu2
embeddings	tied
weight dtype	bfloat16
bos token id	128000
eos token ids	128001, 128009

Weights

hodge-4o.bnz

Chat

Each turn is a role line closed by <|eot_id|>. Generation starts at Assistant: .

Sampling

generation_config.json samples with temperature 0.6 and top-p 0.9. The length cap is 4096. A desk run uses temperature 0.7.

Run

ModelRun loads this directory. The catalog points model at hodge-4o.bnz, beside config.json and tokenizer.json.

modelrun run -temp 0.7 --config hodge-4o.bnz.yaml