Full rework of the TF input/output embeddings and bias resizing
Author: jplu
Pull request: yes

# What does this PR do?

This PR 100% reworks the entire process of input/output and bias resizing. Now the exceptions are better handled including the names that now are always similar. The corresponding tests have also been entirely reworked and now have a better coverage of this feature.

This PR adds a small breaking change. Now the `get_input_embeddings` methods returns the weights and not anymore the embedding layer.

Comments:
I haven't reviewed in detail yet, but just looking at the API with the number of things to change for ALBERT (and in terms of line of code) is a hard pass for me. Overriding the resize method as was done before was way easier, this adds too much complexity.

I understand that it is a big update. Nevertheless, the way it was done before didn't worked and was quite buggy (the tests basically was testing almost nothing) and to make the resizing properly working, these changes are necessary.

In all cases I'm open to any suggestion that will reduce the number of changes :)

@sgugger @LysandreJik I tried a new approach for the resizing that reduce a lot the changes in each model implementation, it is even much shorter than what we currently have in master. I have done my test only on ALBERT for now, can you recheck that file and let me know what you think about it.

Ok I will clarify this a bit more:

1. The fist most important issue in the current implementation is that the resizing is not graph compilation+ execution compliant because of the usage of `numpy` and `tensor.numpy()` calls and then not usable in such cases.
2. The naming was depending of where the build was coming from, that's why we needed the `get_prefix_bias_name` for the bias  and the manual build of the embeddings names. Which was a temporary fix, because it is very error prone, because the naming depends of several other things that are not taken into account into this manual build.
3. Resizing was not working for some models, such as BART which doesn't work properly and raises an error. (Proof that the tests was not testing everything)
4. The current resizing has two issues when resizing when we instantiate a model from scratch: either it raises an attribute error because the model is not fully built (weights not instantiated) and get a wrong naming, and then if we save the model with this wrong naming we cannot save/load it properly because the naming doesn't correspond to the current architecture.
5. All the weights names across the models don't share the same names sometimes `embeddings.word_embeddings`, sometimes `shared.weight`, sometimes `lm_head.bias`, sometimes `lm_loss.bias`, sometimes `mlm.bias`,  sometimes `mlm.predictions.bias` and many other ways...

As stated in #8657 this was just a temporary fix to go to the quickest way to wait for the real rework.

This PR aims to solve all these issues and bring something more generic and less error prone.

I personally never understood that #8657 was a quick fix that was needing another PR afterwards. We cannot operate by adding new methods in one release then breaking them or deleting them in the next so the works that was done in #8657 needs to be built upon not destroyed (and please, say in bold next time you are just making a quick fix as I would never have approved #8657 to be merged had I known...)

So before we review this, the following need to be addressed:
- `get_input_embeddings` needs to keep the same return type
- `get_output_embeddings` needs to keep the same return type
- `get_output_layer_with_bias` can't disappear
- `get_prefix_bias_name` can't disappear

This is annoying but this is why we usually don't merge a half-baked fix introducing new APIs, we can't break that after.

We can keep this for the next major release.

What you ask is doable but will make the codebase more complicated. I will rework this.

I have just done the following restore:

- `get_input_embeddings` still returns a layer
- `get_output_embeddings` still returns a layer
- `get_output_layer_with_bias` is back
- `get_prefix_bias_name` is back

The old and new approach was much more compliant than I thought so it was easier to restore what @sgugger asked, and now there should be zero breaking change. Really sorry for the misunderstanding, I will clearer next time.

@sgugger I should have addressed all your comments :)

> which makes me think there is a better way to code the default in modeling_tf_utils.

Share your thoughts 😉

Thanks @LysandreJik! For the tying test look in the `_get_resized_lm_head_decoder()` method. Unless you mean adding a test in `test_modeling_tf_common` ?

I mean I'm not seeing a test that checks `get_input_embeddings() == get_output_embeddings()` when weights are tied, but I may be missing something here. 

I know these two generally point to the same tensors, but no always, do they?

> I mean I'm not seeing a test that checks get_input_embeddings() == get_output_embeddings() when weights are tied, but I may be missing something here.

Yes, there is a test for this, line 884 in `modeling_tf_utils`.

> I know these two generally point to the same tensors, but no always, do they?

Yes they always point to the same tensor when they equals, 100% sure.

I should have addressed all the comments.

Ah yeah, we probably need a rebase here since TF-serving just got merged :-/

Arf good point!

@patrickvonplaten the test `TFLEDModelTest::test_pt_tf_model_equivalence` seems very flaky, it looks like that it randomly pass/fail. 

Good to merge for me now :)

> @patrickvonplaten the test `TFLEDModelTest::test_pt_tf_model_equivalence` seems very flaky, it looks like that it randomly pass/fail.

Just fixed it: https://github.com/huggingface/transformers/pull/9459

@sgugger any objection to merge this PR?