iconically-mine
/

anlp-a2-moe_top2_same_active

Model card Files Files and versions

YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

moe_top2_same_active transformer (ANLP A2, Task 1)

Parameters: 16.09M
ffn_type: moe_top2_same_active
d_model=256, n_layers=6, n_heads=8, ffn_hidden_dim=1024
Tokens trained on: 30.00M
Final val loss: 3.7233

Checkpoint: moe_top2_same_active_final.pt (load with torch.load, see src/part1/model/transformer.py).

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support