Covid-19 - TPU V3-1024 - T5 11B:  Tensorflow to Pytorch conversion failed
Author: agemagician
Pull request: no

We are training a large scale T5-11B model using TPU V3-1024 for a Covid-19 project
We tried to convert the TensorFlow checkpoint to the Pytorch version, but it did fail.
Could you please help us to figure out the problem since this model is very important for Covid-19 research.

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):
T5

Language I am using the model on (English, Chinese ...): 
Protein Sequences

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. The config file:
```
{
  "architectures": [
    "T5WithLMHeadModel"
  ],
  "d_ff": 65536,
  "d_kv": 128,
  "d_model": 1024,
  "decoder_start_token_id": 0,
  "dropout_rate": 0.1,
  "eos_token_id": 1,
  "initializer_factor": 1.0,
  "is_encoder_decoder": true,
  "layer_norm_epsilon": 1e-06,
  "model_type": "t5",
  "n_positions": 512,
  "num_heads": 128,
  "num_layers": 24,
  "output_past": true,
  "pad_token_id": 0,
  "relative_attention_num_buckets": 32,
  "vocab_size": 128
}
```

2. conversion command:
```
python convert_t5_original_tf_checkpoint_to_pytorch.py \
  --tf_checkpoint_path xxx/tensorflow \
  --config_file xxx/t5-11b-config.json \
  --pytorch_dump_path xxx/pytorch
```


3. Error:
```
Building PyTorch model from configuration: T5Config {
  "architectures": [
    "T5WithLMHeadModel"
  ],
  "d_ff": 65536,
  "d_kv": 128,
  "d_model": 1024,
  "decoder_start_token_id": 0,
  "dropout_rate": 0.1,
  "eos_token_id": 1,
  "initializer_factor": 1.0,
  "is_encoder_decoder": true,
  "layer_norm_epsilon": 1e-06,
  "model_type": "t5",
  "n_positions": 512,
  "num_heads": 128,
  "num_layers": 24,
  "output_past": true,
  "pad_token_id": 0,
  "relative_attention_num_buckets": 32,
  "vocab_size": 128
}

INFO:transformers.modeling_t5:Converting TensorFlow checkpoint from /mnt/lsf-nas-1/lsf/job/repo/elnaggar/prot-transformers/models/t5/tensorflow/bfd100
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/relative_attention_bias with shape [128, 32]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/relative_attention_bias_slot_v with shape [128, 32]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/relative_attention_bias with shape [128, 32]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/relative_attention_bias_slot_v with shape [128, 32]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight global_step with shape []
INFO:transformers.modeling_t5:Loading TF weight shared/embedding with shape [128, 1024]
INFO:transformers.modeling_t5:Loading TF weight shared/embedding_slot_vc with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight shared/embedding_slot_vr with shape [128]
INFO:transformers.modeling_t5:Transposing numpy weight of shape (1024, 16384) for ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'k']
INFO:transformers.modeling_t5:Initialize PyTorch weight ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'k']
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/SelfAttention/k_slot_vc
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/SelfAttention/k_slot_vr
INFO:transformers.modeling_t5:Transposing numpy weight of shape (16384, 1024) for ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'o']
INFO:transformers.modeling_t5:Initialize PyTorch weight ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'o']
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/SelfAttention/o_slot_vc
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/SelfAttention/o_slot_vr
INFO:transformers.modeling_t5:Transposing numpy weight of shape (1024, 16384) for ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'q']
INFO:transformers.modeling_t5:Initialize PyTorch weight ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'q']
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/SelfAttention/q_slot_vc
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/SelfAttention/q_slot_vr
INFO:transformers.modeling_t5:Transposing numpy weight of shape (128, 32) for ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'relative_attention_bias']
INFO:transformers.modeling_t5:Initialize PyTorch weight ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'relative_attention_bias']
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/SelfAttention/relative_attention_bias_slot_v
INFO:transformers.modeling_t5:Transposing numpy weight of shape (1024, 16384) for ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'v']
INFO:transformers.modeling_t5:Initialize PyTorch weight ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'v']
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/SelfAttention/v_slot_vc
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/SelfAttention/v_slot_vr
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/rms_norm/scale
Traceback (most recent call last):
  File "xxx/convert_t5_original_tf_checkpoint_to_pytorch.py", line 61, in <module>
    convert_tf_checkpoint_to_pytorch(args.tf_checkpoint_path, args.config_file, args.pytorch_dump_path)
  File "xxx/convert_t5_original_tf_checkpoint_to_pytorch.py", line 36, in convert_tf_checkpoint_to_$ytorch
    load_tf_weights_in_t5(model, config, tf_checkpoint_path)
  File "xxx/modeling_t5.py", line 102, in load_tf_weights_in_t5
    pointer = getattr(pointer, "weight")
  File "xxx/anaconda3/envs/transformers/lib/python3.7/site-packages/torch/nn/modules/module.py", line 594, in __getattr__
    type(self).__name__, name))
AttributeError: 'T5LayerSelfAttention' object has no attribute 'weight'
```

## Expected behavior

T5 tensorflow model should be converted to pytorch model.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 2.11.0
- Platform: Linux-4.15.0-101-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.7
- PyTorch version (GPU?): 1.5.0 (False)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


Comments:
@thomwolf @LysandreJik @julien-c @patrickvonplaten @VictorSanh @sshleifer @mfuntowicz @sgugger  your feedback will be highly appreciated.

I have checked the "operative_config.gin", and this is the encoder/decoder configuration 👍 

```

# Parameters for decoder/LayerStack:
# ==============================================================================
decoder/LayerStack.dropout_rate = None
decoder/LayerStack.norm_epsilon = None
decoder/LayerStack.recompute_grads = False
decoder/LayerStack.sublayers_final = \
    [@transformer.sublayer_rms_norm, @transformer.sublayer_dropout]
decoder/LayerStack.sublayers_initial = [@transformer.sublayer_dropout]
decoder/LayerStack.sublayers_per_layer = \
    [@transformer.sublayer_rms_norm,
     @transformer.sublayer_call_layer,
     @transformer.sublayer_dropout,
     @transformer.sublayer_residual]

# Parameters for encoder/LayerStack:
# ==============================================================================
encoder/LayerStack.dropout_rate = None
encoder/LayerStack.norm_epsilon = None
encoder/LayerStack.recompute_grads = False
encoder/LayerStack.sublayers_final = \
    [@transformer.sublayer_rms_norm, @transformer.sublayer_dropout]
encoder/LayerStack.sublayers_initial = [@transformer.sublayer_dropout]
encoder/LayerStack.sublayers_per_layer = \
    [@transformer.sublayer_rms_norm,
     @transformer.sublayer_call_layer,
     @transformer.sublayer_dropout,
     @transformer.sublayer_residual]

# Parameters for make_bitransformer:
# ==============================================================================
make_bitransformer.decoder_name = 'decoder'
make_bitransformer.encoder_name = 'encoder'

# Parameters for decoder/make_layer_stack:
# ==============================================================================
decoder/make_layer_stack.block_scope = True
decoder/make_layer_stack.layers = \
    [@mesh_tensorflow.transformer.transformer_layers.SelfAttention,
     @mesh_tensorflow.transformer.transformer_layers.EncDecAttention,
     @mesh_tensorflow.transformer.transformer_layers.DenseReluDense]
decoder/make_layer_stack.num_layers = %num_layers

# Parameters for encoder/make_layer_stack:
# ==============================================================================
encoder/make_layer_stack.block_scope = True
encoder/make_layer_stack.layers = \
    [@mesh_tensorflow.transformer.transformer_layers.SelfAttention,
     @mesh_tensorflow.transformer.transformer_layers.DenseReluDense]
encoder/make_layer_stack.num_layers = %num_layers

```

I also upgraded the transformers package to the latest version, but still the same problem. 

I also tried to load the model using the TF checkpoint directly and it doesn't work:

```
import transformers

tokenizer = transformers.T5Tokenizer.from_pretrained("xxx")
config = transformers.T5Config.from_json_file("t5-11b-config.json")
model = transformers.TFT5Model(config,"xxx/")

encoded_input = tokenizer("A A", return_tensors='tf')

model(encoded_input)
```
Tensor:
`{'input_ids': <tf.Tensor: shape=(1, 2), dtype=int32, numpy=array([[71, 71]], dtype=int32)>, 'attention_mask': <tf.Tensor: shape=(1, 2), dtype=int32, numpy=array([[1, 1]], dtype=int32)>}`


Error:
```

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/mnt/lsf-nas-1/lsf/job/repo/elnaggar/anaconda3/envs/transformers/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 968, in __call__
    outputs = self.call(cast_inputs, *args, **kwargs)
  File "/mnt/lsf-nas-1/lsf/job/repo/elnaggar/anaconda3/envs/transformers/lib/python3.7/site-packages/transformers/modeling_tf_t5.py", line 1064, in call
    training=training,
  File "/mnt/lsf-nas-1/lsf/job/repo/elnaggar/anaconda3/envs/transformers/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 968, in __call__
    outputs = self.call(cast_inputs, *args, **kwargs)
  File "/mnt/lsf-nas-1/lsf/job/repo/elnaggar/anaconda3/envs/transformers/lib/python3.7/site-packages/transformers/modeling_tf_t5.py", line 610, in call
    raise ValueError("You have to specify either inputs or inputs_embeds")
ValueError: You have to specify either inputs or inputs_embeds
```

Not solving your issue but curious, are you training your model on TPU with pytorch? I'm interested to know if pytorch or transformers supports training on v3-1024 TPU. 

@misrasaurabh1  No we didn't use the pytorch version.

@sshleifer @patrickvonplaten Any update regarding the conversion from TF to Pytorch ?

For your first issue, I wonder if the t5 repo has changed, because our conversion script still works on a t5-base checkpoint I received last week. The checkpoint was probably trained a while ago though. 
we also can't load t5-11b in the inference API, although the traceback is different.
@mfuntowicz have you looked at that at all/found anything?


I can help with your second issue. Try sending unpacking the elements of your input dict, so that they comply with the new T5 signature on master:
https://github.com/huggingface/transformers/blob/4dc65591b5c61d75c3ef3a2a883bf1433e08fc45/src/transformers/modeling_tf_t5.py#L1115

For the second issue, I was able to solve it by using the following:
```
ids = tokenizer.encode(seq, return_tensors='tf')
embedding = model(ids,decoder_input_ids=ids)
```
For some reason, I have to send also the ids as decoder_input_ids.

Is this correct @sshleifer  or did I miss something ?

For the first issue, I could not solve it.

I can send you the T5 11B checkpoint privately if you need it.

Regarding "we also can't load t5-11b in the inference API, although the traceback is different.":

This applies to both Tensorflow and Pytorch versions ?

Please see https://github.com/huggingface/transformers/issues/5986#issuecomment-663090043

@julien-c :
My 11B checkpoint is only 20GB. It was trained on very small vocab.
I am not trying to use the pertained t5-11b from T5.
It is a custom new pertained model. 

I don't believe #5986 (comment) is related to my problem.

Ok, my apologies for not reading your issue carefully enough. For the inference API, that's probably the reason though.

No problem, my main issue is the conversion from tensorflow to Pytorch for my new pertained 11B model.

@misrasaurabh1  @patrickvonplaten Any progress ?

I will take a deeper look next week on Monday when I'm back regarding the conversion from tensorflow to PyTorch.

Regarding, the `decoder_input_ids` ("For some reason, I have to send also the ids as decoder_input_ids.") -> yes for T5 you have to input both `input_ids` and `decoder_input_ids`, for the first forward call this is usually:

```python
from transformers import T5Model, T5Tokenizer
import torch

model = T5Model.from_pretrained("t5-small")  # your model of choice here
tokenizer = T5Tokenizer.from_pretrained("t5-small")

input_ids = tokenizer("This sentence is encoded and only has to be passed once.", return_tensors="pt").input_ids
decoder_input_ids = torch.tensor([[model.config.decoder_start_token_id]])

# first call uses the start token id for T5 decoder
outputs = model(input_ids, decoder_input_ids=decoder_input_ids)

# now next decoder token can be sampled from decoder outputs
next_decoder_input_ids = torch.argmax(outputs.last_hidden_state, dim=-1)
decoder_input_ids = torch.cat([decoder_input_ids, next_decoder_input_ids], dim=-1)

# second call can reuse encoder outputs and continues auto-regressive language generation
# model.generate(...) does that automatically

outputs = model(encoder_outputs=outputs.decoder_past_key_values[0], decoder_input_ids=decoder_input_ids)

# The same holds true for TF

from transformers import TFT5Model, T5Tokenizer
import tensorflow as tf

model = TFT5Model.from_pretrained("t5-small")  # your model of choice here
tokenizer = T5Tokenizer.from_pretrained("t5-small")

input_ids = tokenizer("This sentence is encoded and only has to be passed once.", return_tensors="tf").input_ids
decoder_input_ids = tf.convert_to_tensor([[model.config.decoder_start_token_id]], dtype=tf.dtypes.int32)

# first call uses the start token id for T5 decoder
outputs = model(input_ids, decoder_input_ids=decoder_input_ids)

# now next decoder token can be sampled from decoder outputs
next_decoder_input_ids = tf.cast(tf.argmax(outputs[0], axis=-1), tf.dtypes.int32)
decoder_input_ids = tf.concat([decoder_input_ids, next_decoder_input_ids], axis=-1)

# second call can reuse encoder outputs and continues auto-regressive language generation
# model.generate(...) does that automatically

# inputs (encoder inputs) has to be defined because of keras convention => it's not needed though after first 
# pass, so set to None
model(None, encoder_outputs=outputs[1][0], decoder_input_ids=decoder_input_ids)
```

So this means that the `decoder_input_ids` should not be the same as the `input_ids` (encoder inputs) but should correspond to the auto-regressively generated text by the model, that starts with the `decoder_start_token_id`. Hope this makes it a bit clearer.

Thanks @patrickvonplaten for the clarification.
In this case, I would recommend to update the documentation on:
https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_tf_t5.py#L972

Just to clarify one thing, If I am only interested on feature extraction and not generation, in this case my code should looks like:

```
from transformers import TFT5Model, T5Tokenizer
import tensorflow as tf

model = TFT5Model.from_pretrained("t5-small")  # your model of choice here
tokenizer = T5Tokenizer.from_pretrained("t5-small")

input_ids = tokenizer("This sentence is encoded and only has to be passed once.", return_tensors="tf").input_ids
decoder_input_ids = tf.convert_to_tensor([[model.config.decoder_start_token_id]], dtype=tf.dtypes.int32)

# first call uses the start token id for T5 decoder
outputs = model(input_ids, decoder_input_ids=decoder_input_ids)

features = outputs[2]
```

Correct ?

Hopefully next week you will figure out the problem of Pytorch conversion, and let me know if you need our checkpoint, we can send it to you privately. 

Yes, we should update the example to clarify a bit...`T5` is an encoder-decoder model, so it's usually used for conditional generation IMO, so to model the following distribution:

![a0dd19b0f9e827b66400819b15e4a290](https://user-images.githubusercontent.com/23423619/88969462-33e65980-d2b1-11ea-8eb8-a6c52b0a4040.png)

with y being the `decoder_input_ids` and x being the `input_ids`. 

So for feature extraction, is the idea to extract the features of the first `decoder_input_ids` token, so:

![8ef9e98d0dfffab4ba1b1ba5db70c2fe](https://user-images.githubusercontent.com/23423619/88969705-80ca3000-d2b1-11ea-8314-fed879474d09.png)

=> so the hidden states of the first output? In this case, your code above is correct. 

Can you give me a bit more details of the task and how / with what objective (masked language modeling, ...) the model was trained? 




Would be great if you can send me the weights of your trained model per mail (Weshare, google drive) then I check out the problem with the conversion next week 

Thanks @patrickvonplaten for the clarification. I am glad that my understanding is correct, and I am extracting the features from T5 correctly.

Regarding your questions:
1. Task: We are training different language models for the language of life "protein sequences".
2. Objective: Span de-noising.
3. Models: 11B and 3B models from T5.

You can find more details on my GitHub repo and my paper first version:
https://github.com/agemagician/ProtTrans
https://www.biorxiv.org/content/10.1101/2020.07.12.199554v2

I have sent you an email with the 11b model weights, config file and spm model.

I wish you a nice weekend and I hope you could figure out the problem next week.

Thanks in advance.

We have also tested the 3b parameter model with shared enc/dec weights and it did failed.

```
2020-08-03 18:20:42.993212: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
Building PyTorch model from configuration: T5Config {
  "architectures": [
    "T5WithLMHeadModel"
  ],
  "d_ff": 5120,
  "d_kv": 64,
  "d_model": 2048,
  "decoder_start_token_id": 0,
  "dropout_rate": 0.0,
  "eos_token_id": 1,
  "initializer_factor": 1.0,
  "is_encoder_decoder": true,
  "layer_norm_epsilon": 1e-06,
  "model_type": "t5",
  "n_positions": 512,
  "num_heads": 32,
  "num_layers": 24,
  "output_past": true,
  "pad_token_id": 0,
  "relative_attention_num_buckets": 32,
  "vocab_size": 128
}

INFO:transformers.modeling_t5:Converting TensorFlow checkpoint from /content/models/ProtT5-BFD
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/self_attention/SelfAttention/relative_attention_bias with shape [32, 32]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/self_attention/SelfAttention/relative_attention_bias_slot_v with shape [32, 32]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/dense_relu_dense/DenseReluDense/wi/kernel with shape [2048, 5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/dense_relu_dense/DenseReluDense/wi/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/dense_relu_dense/DenseReluDense/wi/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/dense_relu_dense/DenseReluDense/wo/kernel with shape [5120, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/dense_relu_dense/DenseReluDense/wo/kernel_slot_vc with shape [5120]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/dense_relu_dense/DenseReluDense/wo/kernel_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/dense_relu_dense/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/dense_relu_dense/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/enc_dec_attention/EncDecAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/enc_dec_attention/EncDecAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/enc_dec_attention/EncDecAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/enc_dec_attention/EncDecAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/enc_dec_attention/EncDecAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/enc_dec_attention/EncDecAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/enc_dec_attention/EncDecAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/enc_dec_attention/EncDecAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/enc_dec_attention/EncDecAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/enc_dec_attention/EncDecAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/enc_dec_attention/EncDecAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/enc_dec_attention/EncDecAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/enc_dec_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/enc_dec_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/self_attention/SelfAttention/k with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/self_attention/SelfAttention/k_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/self_attention/SelfAttention/k_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/self_attention/SelfAttention/o with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/self_attention/SelfAttention/o_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/self_attention/SelfAttention/o_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/self_attention/SelfAttention/q with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/self_attention/SelfAttention/q_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/self_attention/SelfAttention/q_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/self_attention/SelfAttention/v with shape [2048, 2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/self_attention/SelfAttention/v_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/self_attention/SelfAttention/v_slot_vr with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/self_attention/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/self_attention/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/logits/kernel with shape [2048, 128]
INFO:transformers.modeling_t5:Loading TF weight decoder/logits/kernel_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/logits/kernel_slot_vr with shape [128]
INFO:transformers.modeling_t5:Loading TF weight decoder/rms_norm/scale with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight decoder/rms_norm/scale_slot_v with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight global_step with shape []
INFO:transformers.modeling_t5:Loading TF weight shared/embedding with shape [128, 2048]
INFO:transformers.modeling_t5:Loading TF weight shared/embedding_slot_vc with shape [2048]
INFO:transformers.modeling_t5:Loading TF weight shared/embedding_slot_vr with shape [128]
INFO:transformers.modeling_t5:Skipping decoder/block_000/dense_relu_dense/DenseReluDense/wi/kernel
INFO:transformers.modeling_t5:Skipping decoder/block_000/dense_relu_dense/DenseReluDense/wi/kernel
INFO:transformers.modeling_t5:Skipping decoder/block_000/dense_relu_dense/DenseReluDense/wi/kernel
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/transformers/convert_t5_original_tf_checkpoint_to_pytorch.py", line 61, in <module>
    convert_tf_checkpoint_to_pytorch(args.tf_checkpoint_path, args.config_file, args.pytorch_dump_path)
  File "/usr/local/lib/python3.6/dist-packages/transformers/convert_t5_original_tf_checkpoint_to_pytorch.py", line 36, in convert_tf_checkpoint_to_pytorch
    load_tf_weights_in_t5(model, config, tf_checkpoint_path)
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_t5.py", line 104, in load_tf_weights_in_t5
    pointer = getattr(pointer, "weight")
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 772, in __getattr__
    type(self).__name__, name))
torch.nn.modules.module.ModuleAttributeError: 'T5Block' object has no attribute 'weight'
```

Ok, I will start with the 3B model then! Starting downloading the model now -> will take a look tomorrow (my internet connection is so slow at the moment that it'll take a couple of hours to download both models). 

Just to be sure: 
You have trained the model using this repo: https://github.com/google-research/text-to-text-transfer-transformer right?
And then you ran
```
python convert_t5_original_tf_checkpoint_to_pytorch.py
```
-> which did fail no? 

Thanks @patrickvonplaten.

Yes, I have used the official repo:
https://github.com/google-research/text-to-text-transfer-transformer

Yes, I have used "convert_t5_original_tf_checkpoint_to_pytorch.py", which failed.

An important note:
1) 11B used separate weights for encoder/decoder.
2) 3B used a shared weights for encoder and decoder.

Hey @agemagician, 

I changed the conversion function to handle the 3B weights you send me. I have locally saved the converted pytorch model and you can do the conversion yourself using [this](https://github.com/huggingface/transformers/tree/adapt_t5_for_covid_19_3b) branch also with the changes shown in this PR: https://github.com/huggingface/transformers/pull/6388. 

A couple of things that might be important: 
1) I noticed that the input embeddings, called `shared/embedding` and output logits, called `decoder/logits/kernel` were different meaning that for the decoder the input and output embeddings are **not** shared. We usually share input and output embeddings in T5, so in your case I added a couple of lines to make sure that the correct input and output embeddings are converted and **not** tied. See these changes: https://github.com/huggingface/transformers/blob/abec44887d48e3ef3ccde50f3331f64aa2b3cbbc/src/transformers/modeling_t5.py#L1094 
2) For convenience, I did not tie the weights of the encoder / decoder now, but just copied them. This meants that the resulting 3B T5 model has a size of ~8GB (2 times bigger because float32 is used instead of TPU's bfloat16 and again 2 times bigger because encoder and decoder are not shared). Because of the small word embedding matrix the model is still smaller than the official `t5-3b`: https://huggingface.co/t5-3b. You can reduce the size of your model, by calling `model.half()` and deleting `model.encoder` before doing `model.save_pretrained("./")`. When loading you will have to write a short script though that instantiates the encoder and correctly sets the encoder weights. I would not recommend to do this, but instead just use the 8GB model in PyTorch.
3) I did not test the conversion of the 11B model because I currently do not have the RAM capacity to do so (32GB is just not enough for such a large model). The script should work though. You should make sure to set `is_tied` to `False` in this line though: https://github.com/huggingface/transformers/blob/abec44887d48e3ef3ccde50f3331f64aa2b3cbbc/src/transformers/modeling_t5.py#L64. Also note that the conversion will automatically initialize the model in fp32, so that the PyTorch model alone requires 40GB of RAM. You can later save it as `model.half()`. I hope you have enough RAM space to be able to convert the model. It will be hard to debug for me without the necessary resources, so in case you experience errors, I would suggest to start debugging in this function: https://github.com/huggingface/transformers/blob/abec44887d48e3ef3ccde50f3331f64aa2b3cbbc/src/transformers/modeling_t5.py#L64 and see what names seem to be wrong.

Thanks a lot @patrickvonplaten for fixing the conversion script.
We highly appreciate your effort.
This will definitely make it easier for researchers to use our T5 protein language models.

I have checked the following models:
1. 3B shared.
2. 3B non-shared.
3. 11B non-shared.

All of them were converted correctly without any issue.
However, following your example above for using the models, the decoder generates out of vocabulary ids.
Despite, that both the vocab_size on the config file and the spm is set correctly to 128.

also I got the following warning when loading the model:
```
Some weights of the model checkpoint at xxx/3b-shared/pytorch/ were not used when initializing T5Model: ['lm_head.weight']
- This IS expected if you are initializing T5Model from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
- This IS NOT expected if you are initializing T5Model from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
```


Error:
```
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-66-c65c815d3456> in <module>
      2 # model.generate(...) does that automatically
      3 
----> 4 outputs = model(encoder_outputs=outputs.decoder_past_key_values[0], decoder_input_ids=decoder_input_ids)

/mnt/lsf-nas-1/lsf/job/repo/elnaggar/anaconda3/envs/transformers/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

/mnt/lsf-nas-1/lsf/job/repo/elnaggar/anaconda3/envs/transformers/lib/python3.8/site-packages/transformers/modeling_t5.py in forward(self, input_ids, attention_mask, encoder_outputs, decoder_input_ids, decoder_attention_mask, decoder_past_key_values, use_cache, inputs_embeds, decoder_inputs_embeds, head_mask, output_attentions, output_hidden_states, return_tuple, **kwargs)
    995 
    996         # Decode
--> 997         decoder_outputs = self.decoder(
    998             input_ids=decoder_input_ids,
    999             attention_mask=decoder_attention_mask,

/mnt/lsf-nas-1/lsf/job/repo/elnaggar/anaconda3/envs/transformers/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

/mnt/lsf-nas-1/lsf/job/repo/elnaggar/anaconda3/envs/transformers/lib/python3.8/site-packages/transformers/modeling_t5.py in forward(self, input_ids, attention_mask, encoder_hidden_states, encoder_attention_mask, inputs_embeds, head_mask, past_key_value_states, use_cache, output_attentions, output_hidden_states, return_tuple)
    701         if inputs_embeds is None:
    702             assert self.embed_tokens is not None, "You have to intialize the model with valid token embeddings"
--> 703             inputs_embeds = self.embed_tokens(input_ids)
    704 
    705         batch_size, seq_length = input_shape

/mnt/lsf-nas-1/lsf/job/repo/elnaggar/anaconda3/envs/transformers/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

/mnt/lsf-nas-1/lsf/job/repo/elnaggar/anaconda3/envs/transformers/lib/python3.8/site-packages/torch/nn/modules/sparse.py in forward(self, input)
    122 
    123     def forward(self, input: Tensor) -> Tensor:
--> 124         return F.embedding(
    125             input, self.weight, self.padding_idx, self.max_norm,
    126             self.norm_type, self.scale_grad_by_freq, self.sparse)

/mnt/lsf-nas-1/lsf/job/repo/elnaggar/anaconda3/envs/transformers/lib/python3.8/site-packages/torch/nn/functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   1812         # remove once script supports set_grad_enabled
   1813         _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-> 1814     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
   1815 
   1816 

IndexError: index out of range in self
```

When looking into the decoder_input_ids, I found the following out of id values:
`tensor([[  0, 150]])`

It seems like the decoder ignores the actual vocab size.

Any idea what is the problem here ?

I cannot reproduce the error with my weights. Can you add a code sample that helps me to reproduce the error?

I imagine you don't use a tokenizer, right?

Hi @patrickvonplaten ,

Thanks for reply and I hope you had a good week-end.

I am using the tokenizer.

I have created a Colab notebook which replicates my issue:
https://colab.research.google.com/drive/1NccKDsKObYYTjf8EPcrUUDfs0PyaRsaN?usp=sharing

You can see at the last cell:
`tensor([[  0, 121, 121, 905]])`
905 is out of vocab id.

Note:
The model used in the Colab notebook is a dummy checkpoint for the 3B model (.i.e.: checkpoint 0). 

I see what the problem is - you should use `T5ForConditionalGeneration` instead of `T5Model`. The T5Model does not have the `lm_head` on top and therefore returns a matrix of length 1024 instead of 128. 
Here a notebook that runs without error:
https://colab.research.google.com/drive/1u3VxPkFfrm0Z3f6SMIZTXbazFUhXNe0Z?usp=sharing. 

Also, you have to make sure that the model you pass for the conversion script is a `T5ForConditionalGeneration` model and not only a `T5Model`. 

It might be possible that the checkpoint of your google colab is not correct because of this. 
Can you also check this 3b checkpoint that I converted as explained above:

```python 
config = T5Config.from_pretrained("patrickvonplaten/t5_3b_covid")
model = T5ForConditionalGeneration.from_pretrained("patrickvonplaten/t5_3b_covid")
```

As explained above, the output word embedding (`lm_head`) of the 'patrickvonplaten/t5_3b_covid' model are different to the input embedding, so it would be good if you check this model to your checkpoint in the notebook to see which performs as expected (hopefully at least one of them). 

Let me know if this works. PS. I'm currently working on adding better functionality to share encoder and decoder weights for Seq2Seq models. Once this is done we can reduce this checkpoint to half its size.


Thanks @patrickvonplaten for the clarification. 
Using T5ForConditionalGeneration did solve my issue, and now my models work as excepted.

Thanks a lot for optimizing the shared encoder/decoder weights.
This will be really helpful to allow large models 3B/11B to fit on a single GPU for inference.
This will be the last point to cover before closing this issue. 




One more thing, that will be very useful for T5 large models 3B/11B is to enable gradient_checkpointing as Bert Model:
https://github.com/huggingface/transformers/blob/0735def8e1200ed45a2c33a075bc1595b12ef56a/src/transformers/modeling_bert.py#L461

This will be very useful to fine-tune such models on a single GPU.

I think it should be adjust here, if I am not mistaken:
https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_t5.py#L752



Will open a PR to tie encoder / decoder weights tomorrow! 
Regarding the gradient checkpointing - yes it would be nice to add this. Feel free to open a new feature request "T5 Checkpointing" where you can tag me or @LysandreJik. 

Thanks a lot @patrickvonplaten , you are the best.

Perfect, then I will add a new request for Checkpointing.

Feel free to close this issue after the PR.

PR is ready for review - should be merged this in the next few days. 

Perfect, thanks a lot @patrickvonplaten.

Your help and support is highly appreciated by us.