text: 
  
  TITLE
  TF T5-small with output hidden state and attention not owrking
  
  COMMENTS
  7
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  - `transformers` version: 2.11
  - Platform: Multiple
  - Python version: multiple
  
  ### Who can help
   T5: @patrickvonplaten
   tensorflow: @jplu
   
  ## Information
  
  Model I am using (Bert, XLNet ...): T5
  
  The problem arises when t5-small is loaded from pretrain with output_hidden_states=True, output_attentions=True
  
  sample script https://colab.research.google.com/drive/1oF8hMaQg1yl2fE6QPUYKSTZcer4Mlk6S?usp=sharing
  
  if these parameter is removed script works 
  
  I am getting following error. 
  
  ```python
  /usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py in generate(self, input_ids, max_length, min_length, do_sample, early_stopping, num_beams, temperature, top_k, top_p, repetition_penalty, bad_words_ids, bos_token_id, pad_token_id, eos_token_id, length_penalty, no_repeat_ngram_size, num_return_sequences, attention_mask, decoder_start_token_id, use_cache)
      780                 encoder_outputs=encoder_outputs,
      781                 attention_mask=attention_mask,
  --> 782                 use_cache=use_cache,
      783             )
      784         else:
  
  /usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py in _generate_beam_search(self, input_ids, cur_len, max_length, min_length, do_sample, early_stopping, temperature, top_k, top_p, repetition_penalty, no_repeat_ngram_size, bad_words_ids, bos_token_id, pad_token_id, decoder_start_token_id, eos_token_id, batch_size, num_return_sequences, length_penalty, num_beams, vocab_size, encoder_outputs, attention_mask, use_cache)
     1027                 input_ids, past=past, attention_mask=attention_mask, use_cache=use_cache
     1028             )
  -> 1029             outputs = self(**model_inputs)  # (batch_size * num_beams, cur_len, vocab_size)
     1030             next_token_logits = outputs[0][:, -1, :]  # (batch_size * num_beams, vocab_size)
     1031 
  
  /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
      983 
      984         with ops.enable_auto_cast_variables(self._compute_dtype_object):
  --> 985           outputs = call_fn(inputs, *args, **kwargs)
      986 
      987         if self._activity_regularizer:
  
  /usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_t5.py in call(self, inputs, **kwargs)
     1061             encoder_attention_mask=attention_mask,
     1062             head_mask=head_mask,
  -> 1063             use_cache=use_cache,
     1064         )
     1065 
  
  /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
      983 
      984         with ops.enable_auto_cast_variables(self._compute_dtype_object):
  --> 985           outputs = call_fn(inputs, *args, **kwargs)
      986 
      987         if self._activity_regularizer:
  
  /usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_t5.py in call(self, inputs, attention_mask, encoder_hidden_states, encoder_attention_mask, inputs_embeds, head_mask, past_key_value_states, use_cache, training)
      572             # required mask seq length can be calculated via length of past
      573             # key value states and seq_length = 1 for the last token
  --> 574             mask_seq_length = shape_list(past_key_value_states[0][0])[2] + seq_length
      575         else:
      576             mask_seq_length = seq_length
  
  IndexError: list index out of range
  ```
  
  ## Expected behavior
  
  how to get the attention and hidden states as output?
  Even if you can share a sample for pytorch I would be able to make it work for TF. 