Loading XGLM with Tensorflow and apply resize_token_embeddings() raises an error.
Author: CHLEE-Leo
Pull request: no
Labels: name: TensorFlow

Hello, I am a researcher who is working on LLM domains.

Recently, I discovered some errors related to `resize_token_embeddings()` method when I load XGLM, which is built on PyTorch, into TensorFlow format, for example:
```{python}
  model_tokenizer = AutoTokenizer.from_pretrained("facebook/xglm-1.7b")
  model = TFAutoModelForCausalLM.from_pretrained("facebook/xglm-1.7b", from_pt=True)
  model.resize_token_embeddings(len(model_tokenizer))
```
Specifically, loading both the tokenizer and model is successfully executed, but the `resize_token_embeddings()` raises the error like below:
```{python}
ValueError: Attempt to convert a value (None) with an unsupported type (<class 'NoneType'>) to a Tensor.
```
 
After scrutinizing the errors, *i.e.,* comparing the case with GPT2 that I formated Pytorch-based model into TensorFlow one successfully without such an error, I figured out that `.get_output_embeddings()` method is the reason for raising the error abovementioned. Please Let's see the code below:

```{python}
  def _resize_token_embeddings(self, new_num_tokens):
      # TODO (joao): flagged for replacement (by `_v2_resize_token_embeddings`) due to embeddings refactor
      old_embeddings = self._get_word_embedding_weight(self.get_input_embeddings())
      new_embeddings = self._get_resized_embeddings(old_embeddings, new_num_tokens)

      # if word embeddings are not tied, make sure that lm head bias is resized as well
      if self.get_bias() is not None:
          old_lm_head_bias = self.get_bias()
          new_lm_head_bias = self._get_resized_lm_head_bias(old_lm_head_bias, new_num_tokens)

          self.set_bias(new_lm_head_bias)

      # if word embeddings are not tied, make sure that lm head decoder is resized as well
      if self.get_output_embeddings() is not None:
          old_lm_head_decoder = self._get_word_embedding_weight(self.get_output_embeddings())
          new_lm_head_decoder = self._get_resized_lm_head_decoder(old_lm_head_decoder, new_num_tokens)

          self.set_output_embeddings(new_lm_head_decoder)

      self.set_input_embeddings(new_embeddings)

```
This code was attached from `.src/transformers/modeling_tf_utils.py`.
As shown in the code, 
- `_resize_token_embeddings()` method checks whether `self.get_output_embeddings()` is `None` or not. 
- And if it is not `None`, then apply `self._get_word_embedding_weight(self.get_output_embeddings())`. 
 
Here is the point where the error arises. 
- When I checked `model.get_output_embeddings()`, it returned `<keras.layers.core.dense.Dense>` object, which is different from the case of GPT2 returning `<transformers.modeling_tf_utils.TFSharedEmbeddings>` object. 
  - So, both XGLM and GPT2 return some objects **not** `None`.
- Accordingly, both go to not `None` condition block, but return different results when applying `model._get_word_embedding_weight(model.get_output_embeddings())`.
  - `None` is returned for XGLM (because it was a dense layer object, not an embedding layer object)
  - `embedding weight` is returned for GPT2.

Hence,  I suspect that the error related to `resize_token_embeddings()` is likely to originate from this, and other LLMs, *e.g., GPT-J* might suffer from similar issues. 

I hope this will get fixed soon. 

Comments:
cc @Rocketknight1 

@gante is this something that would be fixed by the `_v2_resize` method?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.