EqLM — kinetic-eqlm-121m-babylm
This is an equilibrium language model from the Kinetic AI research project.
Architecture
Config: {'vocab_size': 50257, 'd_model': 1704, 'n_heads': 12, 'd_ff': 6807, 'max_seq_len': 128, 'deq_max_iter': 12, 'deq_tol': 0.001, 'solver': 'anderson', 'jfb': False, 'dropout': 0.1, 'spectral_norm': True, 'residual_damping': 0.2, 'map_form': 'postln', 'aux_residual': False, 'lambda_aux': 0.1}
Provenance
- Config Hash:
2a92f4f1b1dbfc70fd6d7f3e681afb4a476763f08faf96588bb7074b6b501832 - Git Commit:
unknown - Run Directory:
/home/sharaths/projects/game-llm/results/exp10_5090/exp10_seed42
All numbers in this model card trace to documented runs; see research/memory/findings.md for validation details.
Disclaimer
This is research-stage code. Numbers reported here are from results.json in the named run directory and trace to research/memory/findings.md. For baseline comparisons and metric definitions, consult the paper at https://github.com/SharathSPhD/game-llm/tree/main/paper.