Five 366M hybrid LMs, identical 15B-token / 16K-context recipe; only the 18 non-full-attention layers differ.
Dhruman Gupta
Berlm
AI & ML interests
None yet
Recent Activity
updated a model about 2 months ago
Berlm/hyb16-gdn2-s1 published a model about 2 months ago
Berlm/hyb16-gdn2-s1 updated a model about 2 months ago
Berlm/hyb16-edm-s0Organizations
None yet