Finetuning XLM-Roberta-2XLM-Roberta on custom dataset gives the following error:
Author: laibamehnaz
Pull request: no
Labels: name: wontfix

`Evaluation: 100% 30/30 [00:45<00:00,  1.53s/it]
[libprotobuf FATAL /sentencepiece/src/../third_party/protobuf-lite/google/protobuf/repeated_field.h:1505] CHECK failed: (index) >= (0): 
terminate called after throwing an instance of 'google::protobuf::FatalException'
  what():  CHECK failed: (index) >= (0): `

I am using the following script: [https://huggingface.co/patrickvonplaten/bert2bert-cnn_dailymail-fp16](url)
Appreciate any help. Thank you. 

Comments:
This looks like an issue with tokenizer in evaluation.
if you don't need generative metrics while Training then set `predict_from_generate` to `False`  and don't pass `compute_metrics` function to `Trainer`

pinging @patrickvonplaten 

Hi @patil-suraj , 
Did exactly that. This issue didn't show up. But now all generations on the test set are the same regardless of the input.

Hey @laibamehnaz, 

Could you copy paste a working code example, so that we can reproduce the error? :-) Note that you have to use a different tokenizer than the one that is used in `bert2bert-cnn_dailymail-fp16`

Sure, you can see the code here: [https://colab.research.google.com/drive/1xxBQcPe05bFBQvJQLx6Mw2Qd9YVWPNS9?usp=sharing](url)

Hmm, I don't get protobuf error when running locally...could you maybe adapt the google colab so that I can get your error above by just clicking "run" :-) ? (Setting the correct transformer pip installs and the correct trainer params, etc...)

Oh I am so sorry, this script won't give you the error because I have set `predict_from_generate` as `False` and `prediction_loss_only` to `True`. 

Sure, I will share the script.
> Hmm, I don't get protobuf error when running locally...could you maybe adapt the google colab so that I can get your error above by just clicking "run" :-) ? (Setting the correct transformer pip installs and the correct trainer params, etc...)



Here you go:
[https://colab.research.google.com/drive/1xxBQcPe05bFBQvJQLx6Mw2Qd9YVWPNS9?usp=sharing](url)

> Hmm, I don't get protobuf error when running locally...could you maybe adapt the google colab so that I can get your error above by just clicking "run" :-) ? (Setting the correct transformer pip installs and the correct trainer params, etc...)



I'm sorry the colab does not work for me...I get install errors when running the second cell. Let me merge the "more_general_trainer_metric" PR into master next week and then we can work directly on master.

Hi @patrickvonplaten , I have fixed the issue in the second cell. 
[https://colab.research.google.com/drive/1xxBQcPe05bFBQvJQLx6Mw2Qd9YVWPNS9?usp=sharing](url)

> I'm sorry the colab does not work for me...I get install errors when running the second cell. Let me merge the "more_general_trainer_metric" PR into master next week and then we can work directly on master.

Sure, that will be great!!

@patrickvonplaten  These types of metrics are included in #6769, and it's almost ready to merge. 
How about we support `EncoderDecoder` models in `examples/seq2seq` ?

@sshleifer  does that make sense ?

Depends how much complexity it adds, but on a high level I like that idea a lot!

There is a more pressing issue of getting incremental decoding/use cache working for Roberta that I would probably prioritize higher.

@patil-suraj @sshleifer - I like the idea of adding `EncoderDecoder` to your `Seq2SeqTrainer` a lot. This way I won't have to continue my hacky PR here: https://github.com/huggingface/transformers/pull/5840. I would place this actually as more important since people need to be able to quickly fine-tune any `EncoderDecoderModel`. 

@patil-suraj - After merging your PR, I'd be happy to work on adding `EncoderDecoder` to the Seq2Seq Trainer.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.