Rzoro
/

erebus-tiny

Text Generation

foundation-model

Model card Files Files and versions

erebus-tiny / config.json

Rzoro's picture

Add Erebus foundation model weights

45493f0 verified 9 days ago

history blame contribute delete

132 Bytes

	{
	"vocab_size": 50257,
	"d_model": 256,
	"n_heads": 4,
	"n_layers": 6,
	"d_ff": 1024,
	"max_seq_len": 512,
	"dropout": 0.1
	}