Hodge-3.5

Hodge-3.5 is a ternarized causal language model. The linear layers are Hodge linear, quantized offline.

Model

hidden size	2560
layers	30
intermediate size	6912
attention heads	20
key-value heads	5
head size	128
vocabulary	514304
context length	4096
rope theta	500000
activation	relu2
embeddings	tied
weight dtype	bfloat16
bos token id	128000
eos token ids	128001, 128009

Weights

hodge-3.5.bnz

Chat

Each turn is a role line closed by <|eot_id|>. Generation starts at Assistant: .

Sampling

generation_config.json samples with temperature 0.6 and top-p 0.9. The length cap is 4096.

Run

ModelRun loads this directory. The catalog points model at hodge-3.5.bnz, beside config.json and tokenizer.json.

modelrun run --config hodge-3.5.bnz.yaml