resize_token_embeddings not implemented for TFBert* 
Author: AZMomin
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->

Hey,

I noticed for the Tensorflow 2 implementation of BERT, the `resize_token_embeddings` function is not implemented, while the Pytorch BERT class does.

### Questions:
* Is the implementation of `resize_token_embeddings` for TF models expected? 
  * I'd be happy to help with this 
* Any other solutions to resizing token embedding? 
  * Possibly manually updating the config, vocab file, and weights to support the added tokens 

Example: 

```
special_tokens = {
    'additional_special_tokens': ['[SPC]']
}
tokenizer = BertTokenizer.from_pretrained('bert-base-cased')
tokens_added = tokenizer.add_special_tokens(special_tokens)

model = TFBertForSequenceClassification.from_pretrained('bert-base-cased')
model.resize_token_embeddings(len(tokenizer))
```
Output: `  File ".../transformers/modeling_tf_utils.py", line 115, in resize_token_embeddings
    raise NotImplementedError`


### Source: 

`TFPreTrainedModel`
https://github.com/huggingface/transformers/blob/74ce8de7d8e0375a9123f9542f3483f46cc8df9b/transformers/modeling_tf_utils.py#L117

Here I think the implementation would look similar to the PyTorch model. Updating the embeddings specifically for BERT would then occur in TFBertMainLayer.resize_token_embeddings.

```
def resize_token_embeddings(self, new_num_tokens=None):
        """ Resize input token embeddings matrix of the model if new_num_tokens != config.vocab_size.
        Take care of tying weights embeddings afterwards if the model class has a `tie_weights()` method.

        Arguments:

            new_num_tokens: (`optional`) int:
                New number of tokens in the embedding matrix. Increasing the size will add newly initialized vectors at the end. Reducing the size will remove vectors from the end. 
                If not provided or None: does nothing and just returns a pointer to the input tokens ``tf.Variable`` Module of the model.

        Return: ``tf.Variable``
            Pointer to the input tokens Embeddings Module of the model
        """
        base_model = getattr(self, self.base_model_prefix, self)  # get the base model if needed
        model_embeds = base_model._resize_token_embeddings(new_num_tokens)
        if new_num_tokens is None:
            return model_embeds

        # Update base model and current model config
        self.config.vocab_size = new_num_tokens
        base_model.vocab_size = new_num_tokens

        # Tie weights again if needed
        if hasattr(self, 'tie_weights'):
            self.tie_weights()

        return model_embeds
``` 

`TFBertMainLayer`
https://github.com/huggingface/transformers/blob/74ce8de7d8e0375a9123f9542f3483f46cc8df9b/transformers/modeling_tf_bert.py#L472


Thanks, 

Ali Momin

Comments:
Did you get this to work for you? I'm currently trying to do the same thing but with TFGPT-2

Any update on when this would be completed?

I have the same issue, I wanted to use "resize_token_embeddings" with TFAlbert model but it's not implemented yet.

Hi, we have no plans of implementing this on our roadmap, but we are always open to PRs.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Any updates on this? Trying to use tf gpt2