Make Tokenizers serializable to TF SavedModel format
Author: piEsposito
Pull request: no

### Feature request

It would be great if we could serialize our tokenizers into the TF SavedModel format, so that we could deploy it to TF Serving without a handler to tokenize our inputs. 

### Motivation

It is frustrating to have to write a Python, JS or Rust handler every time I want to deploy a huggingface/transformers model to TF Serving, and it would be great if we could just bake everything into a serialized model and seamlessly serve it with TF Serving. 

### Your contribution

Sadly I can't submit a PR.

Comments:
cc @Rocketknight1, I believe you've been looking into something similar.

Hi @piEsposito, this is something we've been working on! Right now it's only available for BERT, but we intend to expand it to other models, particularly now that we're seeing interest in it. You can use the [TFBertTokenizer](https://huggingface.co/docs/transformers/model_doc/bert#transformers.TFBertTokenizer) class for this - check out [this gist](https://gist.github.com/Rocketknight1/b479d57e3d2f94420b11ca8d319cc68f) for an example of how to use it.

If you're using a different model class than BERT, or you have any difficulties when using this, please let us know! It's a recent feature in `transformers` so we're still looking for user feedback on it.

This is great, really, and exactly what I'm looking into. I'm specifically looking into doing that with CLIP, which uses BPE. Do you have anything on works on that? If not, how can I ramp-up on that and help?

Thanks! 

Hi @piEsposito - we think that should be possible, but we just haven't had time to implement any tokenizers of that class yet! If you look at the [source for TFBertTokenizer](https://github.com/huggingface/transformers/blob/main/src/transformers/models/bert/tokenization_bert_tf.py#L67) you can see that we just use `FastBertTokenizer` from `tensorflow_text`. However, correctly implementing a BPE tokenizer that gives identical results to the existing tokenizers will probably be more complex than a single class.

If you want to attempt it, feel free! We'd be happy to accept a PR. If not, we'll still work on it ourselves, but there are several competing priorities for the TF team right now.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@Rocketknight1 Hi, it would be great to also add support for models including RoBerta etc., as they may use different methods for tokenization (e.g. BPE, Byte-level BPE, SentencePiece). TBH, I think that the tokenizers should be independent from models, because tokenizer - model is not a 1 : 1 mapping.

@jamie0725 That's a good point! For most of our tokenizers that's how we do things - we have a separate `tokenizers` library. Right now we're still experimenting with in-graph tokenizers, but we might move them to the `tokenizers` library at a later stage, and adding tokenizers for other common models like `RoBERTa` is definitely on the list too!

The main issue is just that we have a lot of competing priorities - we'll get to it eventually, but if anyone wants to submit a PR before then we'd be very happy to review it!

do you plan to add CLIPTokenizer support for TF serving?