Spaces:
Build error
Build error
File size: 372 Bytes
b100e1c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
# T5.1.1 Large model.
include 't5x/examples/t5/t5_1_1/base.gin' # imports vocab, optimizer and model.
# ------------------- Network specification overrides --------------------------
network.Transformer.config = @network.T5Config()
network.T5Config:
emb_dim = 1024
num_heads = 16
num_encoder_layers = 24
num_decoder_layers = 24
head_dim = 64
mlp_dim = 2816
|