Nougat-small VisionEncoderDecoderModel failed when max_new_tokens > 3584,  Index out of range in self
Author: c3-YuelingWu
Pull request: no
Labels: name: Good Difficult Issue

### System Info

# System Info

- transformers == 4.35.0
- torch == 2.0.1+cu117
- Error shows on both cpu and cuda.

# Task
Use nougat to parse text from large documents.
# Issue
NougatTokenizerFast. model_max_length = 3584 . Setting max_new_tokens=3585 (or larger) in model.generate works well on most pages, but fails on pages which output should be '[MISSING_PAGE_POST]'.
Once the nougat fails, parsing all the following pages will fail unless kernel restart.
# Error
IndexError: index out of range in self
# What I Tried 

- Setting `nougat_model.config.decoder.pad_token_id = nougat_image_processor.tokenizer.eos_token_id` doesn't work.
- I suspect one special token is generated outside of vocabulary, which prevents from end-of-senescence detection.
- Nougat_base can generate an empty string with max_new_tokens=3585 on the same page, without errors.


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

**Sample Document**
![failed_page](https://github.com/huggingface/transformers/assets/103061109/05ecf35a-67e5-4720-b123-dece84eec195)


**Code**
```
nougat_image_processor = NougatProcessor.from_pretrained("facebook/nougat-small")
nougat_model = VisionEncoderDecoderModel.from_pretrained("facebook/nougat-small")

pixel_values = nougat_image_processor(image, return_tensors="pt").pixel_values

device = 'cpu'#"cuda" if torch.cuda.is_available() else "cpu"
if device == 'cuda':
    pixel_values = pixel_values.cuda()
    nougat_model = nougat_model.cuda()
    
outputs = nougat_model.generate(
    pixel_values.to(device),
    min_length=1,
    max_new_tokens=3585,
    bad_words_ids=[[nougat_image_processor.tokenizer.unk_token_id]],
)
generated = nougat_image_processor.batch_decode(outputs[0], skip_special_tokens=True)[0]
generated = nougat_image_processor.post_process_generation(generated, fix_markdown=False)
print(generated)
```

### Expected behavior

**Expected Output**

'[MISSING_PAGE_POST]' or '  '.

**Observed Output**
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
/tmp/ipykernel_11659/2341160804.py in <module>
      6     nougat_model = nougat_model.cuda()
      7 
----> 8 outputs = nougat_model.generate(
      9     pixel_values.to(device),
     10     min_length=1,

~/.conda/envs/py-chunker-client-ipython/lib/python3.9/site-packages/torch/utils/_contextlib.py in decorate_context(*args, **kwargs)
    113     def decorate_context(*args, **kwargs):
    114         with ctx_factory():
--> 115             return func(*args, **kwargs)
    116 
    117     return decorate_context

~/.conda/envs/py-chunker-client-ipython/lib/python3.9/site-packages/transformers/generation/utils.py in generate(self, inputs, generation_config, logits_processor, stopping_criteria, prefix_allowed_tokens_fn, synced_gpus, assistant_model, streamer, negative_prompt_ids, negative_prompt_attention_mask, **kwargs)
   1671         if generation_mode == GenerationMode.GREEDY_SEARCH:
   1672             # 11. run greedy search
-> 1673             return self.greedy_search(
   1674                 input_ids,
   1675                 logits_processor=logits_processor,

~/.conda/envs/py-chunker-client-ipython/lib/python3.9/site-packages/transformers/generation/utils.py in greedy_search(self, input_ids, logits_processor, stopping_criteria, max_length, pad_token_id, eos_token_id, output_attentions, output_hidden_states, output_scores, return_dict_in_generate, synced_gpus, streamer, **model_kwargs)
   2519 
   2520             # forward pass to get next token
-> 2521             outputs = self(
   2522                 **model_inputs,
   2523                 return_dict=True,

~/.conda/envs/py-chunker-client-ipython/lib/python3.9/site-packages/torch/nn/modules/module.py in _call_impl(self, *args, **kwargs)
   1499                 or _global_backward_pre_hooks or _global_backward_hooks
   1500                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1501             return forward_call(*args, **kwargs)
   1502         # Do not call functions when jit is used
   1503         full_backward_hooks, non_full_backward_hooks = [], []

~/.conda/envs/py-chunker-client-ipython/lib/python3.9/site-packages/transformers/models/vision_encoder_decoder/modeling_vision_encoder_decoder.py in forward(self, pixel_values, decoder_input_ids, decoder_attention_mask, encoder_outputs, past_key_values, decoder_inputs_embeds, labels, use_cache, output_attentions, output_hidden_states, return_dict, **kwargs)
    602 
    603         # Decode
--> 604         decoder_outputs = self.decoder(
    605             input_ids=decoder_input_ids,
    606             attention_mask=decoder_attention_mask,

~/.conda/envs/py-chunker-client-ipython/lib/python3.9/site-packages/torch/nn/modules/module.py in _call_impl(self, *args, **kwargs)
   1499                 or _global_backward_pre_hooks or _global_backward_hooks
   1500                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1501             return forward_call(*args, **kwargs)
   1502         # Do not call functions when jit is used
   1503         full_backward_hooks, non_full_backward_hooks = [], []

~/.conda/envs/py-chunker-client-ipython/lib/python3.9/site-packages/transformers/models/mbart/modeling_mbart.py in forward(self, input_ids, attention_mask, encoder_hidden_states, encoder_attention_mask, head_mask, cross_attn_head_mask, past_key_values, inputs_embeds, labels, use_cache, output_attentions, output_hidden_states, return_dict)
   2046 
   2047         # decoder outputs consists of (dec_features, layer_state, dec_hidden, dec_attn)
-> 2048         outputs = self.model.decoder(
   2049             input_ids=input_ids,
   2050             attention_mask=attention_mask,

~/.conda/envs/py-chunker-client-ipython/lib/python3.9/site-packages/torch/nn/modules/module.py in _call_impl(self, *args, **kwargs)
   1499                 or _global_backward_pre_hooks or _global_backward_hooks
   1500                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1501             return forward_call(*args, **kwargs)
   1502         # Do not call functions when jit is used
   1503         full_backward_hooks, non_full_backward_hooks = [], []

~/.conda/envs/py-chunker-client-ipython/lib/python3.9/site-packages/transformers/models/mbart/modeling_mbart.py in forward(self, input_ids, attention_mask, encoder_hidden_states, encoder_attention_mask, head_mask, cross_attn_head_mask, past_key_values, inputs_embeds, use_cache, output_attentions, output_hidden_states, return_dict)
   1236 
   1237         # embed positions
-> 1238         positions = self.embed_positions(input, past_key_values_length)
   1239 
   1240         hidden_states = inputs_embeds + positions.to(inputs_embeds.device)

~/.conda/envs/py-chunker-client-ipython/lib/python3.9/site-packages/torch/nn/modules/module.py in _call_impl(self, *args, **kwargs)
   1499                 or _global_backward_pre_hooks or _global_backward_hooks
   1500                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1501             return forward_call(*args, **kwargs)
   1502         # Do not call functions when jit is used
   1503         full_backward_hooks, non_full_backward_hooks = [], []

~/.conda/envs/py-chunker-client-ipython/lib/python3.9/site-packages/transformers/models/mbart/modeling_mbart.py in forward(self, input_ids, past_key_values_length)
    120         ).expand(bsz, -1)
    121 
--> 122         return super().forward(positions + self.offset)
    123 
    124 

~/.conda/envs/py-chunker-client-ipython/lib/python3.9/site-packages/torch/nn/modules/sparse.py in forward(self, input)
    160 
    161     def forward(self, input: Tensor) -> Tensor:
--> 162         return F.embedding(
    163             input, self.weight, self.padding_idx, self.max_norm,
    164             self.norm_type, self.scale_grad_by_freq, self.sparse)

~/.conda/envs/py-chunker-client-ipython/lib/python3.9/site-packages/torch/nn/functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   2208         # remove once script supports set_grad_enabled
   2209         _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-> 2210     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
   2211 
   2212 

IndexError: index out of range in self

Comments:
cc @molbap and @NielsRogge who ported this model! 

Hi @c3-YuelingWu ! In `nougat-base` tokenizer, `model_max_length` and `max_length` are both at 4096. In `nougat-small`, they are at 3584. However it's indeed anormal that nougat-small is generating 3584 tokens when nougat-base is generating only 6 with the same blank image. I reproduced your problem, looking into it, weird that the end of sequence token is not outputted.

Could be an OOV error or something related to pos embed. @NielsRogge  in `tokenization_nougat_fast.py` `PRETRAINED_POSITIONAL_EMBEDDINGS_SIZES` is not defined for `nougat-small` but only for `nougat-base`, at 3584. Can it be related?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Any update on this?

I don't think either had the time to tackle this, marking as good difficult issue 🤗