Can you add a fast tokenizer for the ByT5 model? 
Author: vitalyshalumov
Pull request: no


Comments:
Hi @vitalyshalumov ,

Do you have any suggestions as to why we should do that ? `ByT5` operates on bytes directly, so no real tokenization is needed there, offloading work to rust might actually be counterproductive there.

> 
There are several features of the Fast tokenizers that are utilised in the Q-A task and are missing from the non-fast implementation.


You mean `offsets` ?

Would have to check, but keeping those with `ByT5` is probably doable.

Any other features you thought of ?

> You mean `offsets` ?
> 
> Would have to check, but keeping those with `ByT5` is probably doable.
> 
> Any other features you thought of ?

Features such as:
    truncation="only_second",
    return_overflowing_tokens=True,
    return_offsets_mapping=True,
    stride=doc_stride

and a sequence_ids() method

Basically I want to be able to run this example
https://colab.research.google.com/github/huggingface/notebooks/blob/master/examples/question_answering.ipynb

Thank you in advance!


I see, there's a workaround because not all tokenizers are fast and we can still run question-answering on those (you just loose offsets):

Take a look at `transformers.data.processors.squad.squad_convert_examples_to_features` function:

https://huggingface.co/docs/transformers/master/main_classes/processors#transformers.data.processors.squad.SquadProcessor.get_train_examples

This should enable you to use `ByT5` tokenizer already. (Using the `question-answering` pipeline could work out of the box too ! )

> I see, there's a workaround because not all tokenizers are fast and we can still run question-answering on those (you just loose offsets):
> 
> Take a look at `transformers.data.processors.squad.squad_convert_examples_to_features` function:
> 
> https://huggingface.co/docs/transformers/master/main_classes/processors#transformers.data.processors.squad.SquadProcessor.get_train_examples
> 
> This should enable you to use `ByT5` tokenizer already. (Using the `question-answering` pipeline could work out of the box too ! )

Thank you.
Is there still a chance you will introduce Fast for Byt5 to get the "offsets" functionality?


It's definitely not off the table, but unlikely. 

As I mentionned, there's a overhead to go from Python to Rust (`tokenizers`). And in the case of pure bytes it's likely to be superior to the actual processing of the tokenizer in pure python (defeating the purpose of the rust implementation which is to go fast). Ofc tests should be run to prove this, but since there's at least a utf-8 reprocessing it seems really probable.

That being said, what you want is to get the `offset_mapping` and that might be solvable directly on the python `Tokenizer`. Since there's no real treatment of the strings aside from special tokens, it should really be doable. If you feel like doing a PR that would be welcome.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.