mithraindir
/

macrostories-bpe800-consistent

Model card Files Files and versions

YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

MacroStories BPE-800 Consistent

Looped transformer (1 layer x 4 loops), 33,376 parameters ($V=800$, $d=32$). Trained on 1,000 consistent Dutch children's stories with Byte-Level BPE tokenization (0.00% OOV).

Steps: 4000
Batch: 32
Final validation loss: 2.4335 nats/token

Generated Samples:

1.

2.

3.

Downloads last month: 29

Safetensors

Model size

33.4k params

Tensor type

F32

·

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support