Fix Debertav2 embed_proj
Author: WissamAntoun
Pull request: yes

# What does this PR do?

Fixes an issue where loading a model with different hidden_size and embedding_size (such as `almanach/camemberta-base-generator`)) for masked language modeling won't work due to a size mismatch in the output projection in TF2 and Pytorch.

To replicate simply try loading the MLM model with the `DebertaV2ForMaskedLM`.
This colab shows that it now works https://colab.research.google.com/drive/1piUbXkmxNIhGdCWiN5Rx-s2-27FNwij7 

Error from before:

<details>
  <summary>Error</summary>
  
 ```python
---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-1-049fda84e52f> in <cell line: 5>()
      3 model_name = "almanach/camemberta-base-generator"
      4 config = AutoConfig.from_pretrained(model_name)
----> 5 model = AutoModelForMaskedLM.from_pretrained(model_name,from_tf=True)
      6 # model = AutoModelForMaskedLM.from_pretrained(model_name,config=config,from_tf=True)
      7 tokenizer = AutoTokenizer.from_pretrained(model_name)

7 frames
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    445         elif type(config) in cls._model_mapping.keys():
    446             model_class = _get_model_class(config, cls._model_mapping)
--> 447             return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
    448         raise ValueError(
    449             f"Unrecognized configuration class {config.__class__} for this kind of AutoModel: {cls.__name__}.\n"

/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   1508                     from .modeling_tf_pytorch_utils import load_tf2_checkpoint_in_pytorch_model
   1509 
-> 1510                     model = load_tf2_checkpoint_in_pytorch_model(model, resolved_archive_file, allow_missing_keys=True)
   1511                 except ImportError:
   1512                     logger.error(

/usr/local/lib/python3.10/dist-packages/transformers/modeling_tf_pytorch_utils.py in load_tf2_checkpoint_in_pytorch_model(pt_model, tf_checkpoint_path, tf_inputs, allow_missing_keys)
    316 
    317     if tf_inputs is not None:
--> 318         tf_model(tf_inputs, training=False)  # Make sure model is built
    319 
    320     load_tf_weights(tf_model, tf_checkpoint_path)

/usr/local/lib/python3.10/dist-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
     68             # To get the full stack trace, call:
     69             # `tf.debugging.disable_traceback_filtering()`
---> 70             raise e.with_traceback(filtered_tb) from None
     71         finally:
     72             del filtered_tb

/usr/local/lib/python3.10/dist-packages/transformers/modeling_tf_utils.py in run_call_with_unpacked_inputs(self, *args, **kwargs)
    375         main_input = fn_args_and_kwargs.pop(main_input_name)
    376         unpacked_inputs = input_processing(func, self.config, main_input, **fn_args_and_kwargs)
--> 377         return func(self, **unpacked_inputs)
    378 
    379     # Keras enforces the first layer argument to be passed, and checks it through `inspect.getfullargspec()`. This

/usr/local/lib/python3.10/dist-packages/transformers/models/deberta_v2/modeling_tf_deberta_v2.py in call(self, input_ids, attention_mask, token_type_ids, position_ids, inputs_embeds, output_attentions, output_hidden_states, return_dict, labels, training, **kwargs)
   1281         )
   1282         sequence_output = outputs[0]
-> 1283         prediction_scores = self.mlm(sequence_output=sequence_output, training=training)
   1284         loss = None if labels is None else self.hf_compute_loss(labels=labels, logits=prediction_scores)
   1285 

/usr/local/lib/python3.10/dist-packages/transformers/models/deberta_v2/modeling_tf_deberta_v2.py in call(self, sequence_output)
    987 
    988     def call(self, sequence_output: tf.Tensor) -> tf.Tensor:
--> 989         prediction_scores = self.predictions(hidden_states=sequence_output)
    990 
    991         return prediction_scores

/usr/local/lib/python3.10/dist-packages/transformers/models/deberta_v2/modeling_tf_deberta_v2.py in call(self, hidden_states)
    973         seq_length = shape_list(hidden_states)[1]
    974         hidden_states = tf.reshape(tensor=hidden_states, shape=[-1, self.hidden_size])
--> 975         hidden_states = tf.matmul(a=hidden_states, b=self.input_embeddings.weight, transpose_b=True)
    976         hidden_states = tf.reshape(tensor=hidden_states, shape=[-1, seq_length, self.vocab_size])
    977         hidden_states = tf.nn.bias_add(value=hidden_states, bias=self.bias)

InvalidArgumentError: Exception encountered when calling layer 'predictions' (type TFDebertaV2LMPredictionHead).

{{function_node __wrapped__MatMul_device_/job:localhost/replica:0/task:0/device:CPU:0}} Matrix size-incompatible: In[0]: [15,256], In[1]: [32008,768] [Op:MatMul]

Call arguments received by layer 'predictions' (type TFDebertaV2LMPredictionHead):
  • hidden_states=tf.Tensor(shape=(3, 5, 256), dtype=float32)
  ```
</details>

- `transformers` version: 4.31.0.dev0
- Platform: Linux-5.15.107+-x86_64-with-glibc2.31
- Python version: 3.10.12
- Huggingface_hub version: 0.15.1
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.1+cu118 (False)
- Tensorflow version (GPU?): 2.12.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.6.9 (cpu)
- Jax version: 0.4.10
- JaxLib version: 0.4.10


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@kamalkraj  @Rocketknight1 @ArthurZucker @younesbelkada @gante 

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hey @WissamAntoun 👋 

If you check the config documentation for deberta, you see that there is no `embedding_size` attribute. There seems to be something wrong with the serialization of `almanach/camemberta-base-generator`

@gante Hey!

The thing is `camemberta-base-generator` needs to have embedding_size different from the hidden size, since it was trained using ELECTRA style and the generator needs to be smaller without messing the the embedding_size.

Also the code had support for the different sizes and already had the projection layer. But since it wasn't used by anyone, the bug with the MLM task was affecting anyone.

Actually it's now consistent with the code in the official deberta repo https://github.com/microsoft/DeBERTa/blob/master/DeBERTa/deberta/mlm.py#LL20C9-L20C84

@WissamAntoun makes sense -- since it was in the original implementation, I'll accept this PR 🤗