Update the TF models to remove their interdependencies
Author: jplu
Pull request: yes

This PR tries to remove all the interdependencies across the TF models. All the `.modeling_tf_X import TFXX` where `X` is the name of a model, and `XX` is the name of a layer. All the dependent layers have been rewritten directly in the corresponding file model in order to do not have these import anymore. Only the case where a strict usage without modification is allowed such as `TFCamembert` and `TFXLMRoberta`. 


Comments:
I know this PR is in a draft state, but we could (and should!) leverage @sgugger's [script](https://github.com/huggingface/transformers/pull/7219) to ensure that the copy/pasted code does not diverge once it's merged.

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7238?src=pr&el=h1) Report
> Merging [#7238](https://codecov.io/gh/huggingface/transformers/pull/7238?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/f5518e56318a79056ba3c80cbece29d9fe98558c?el=desc) will **increase** coverage by `1.28%`.
> The diff coverage is `94.17%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/7238/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/7238?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #7238      +/-   ##
==========================================
+ Coverage   79.30%   80.59%   +1.28%     
==========================================
  Files         181      181              
  Lines       34828    35437     +609     
==========================================
+ Hits        27620    28559     +939     
+ Misses       7208     6878     -330     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/7238?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_xlm.py](https://codecov.io/gh/huggingface/transformers/pull/7238/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl94bG0ucHk=) | `58.52% <25.00%> (-34.71%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_mobilebert.py](https://codecov.io/gh/huggingface/transformers/pull/7238/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9tb2JpbGViZXJ0LnB5) | `96.93% <90.90%> (-0.14%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_flaubert.py](https://codecov.io/gh/huggingface/transformers/pull/7238/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9mbGF1YmVydC5weQ==) | `89.67% <91.57%> (+65.13%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_electra.py](https://codecov.io/gh/huggingface/transformers/pull/7238/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9lbGVjdHJhLnB5) | `97.81% <93.57%> (+72.48%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/7238/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9yb2JlcnRhLnB5) | `95.32% <97.76%> (+2.07%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_longformer.py](https://codecov.io/gh/huggingface/transformers/pull/7238/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9sb25nZm9ybWVyLnB5) | `98.58% <98.36%> (-0.09%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_albert.py](https://codecov.io/gh/huggingface/transformers/pull/7238/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9hbGJlcnQucHk=) | `97.34% <100.00%> (+6.41%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/7238/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9iZXJ0LnB5) | `98.90% <100.00%> (ø)` | |
| [src/transformers/modeling\_tf\_distilbert.py](https://codecov.io/gh/huggingface/transformers/pull/7238/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9kaXN0aWxiZXJ0LnB5) | `99.28% <100.00%> (+<0.01%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/7238/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9vcGVuYWkucHk=) | `22.03% <0.00%> (-73.03%)` | :arrow_down: |
| ... and [27 more](https://codecov.io/gh/huggingface/transformers/pull/7238/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/7238?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7238?src=pr&el=footer). Last update [f5518e5...5b6ad05](https://codecov.io/gh/huggingface/transformers/pull/7238?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


@LysandreJik @sgugger @thomwolf This PR is ok to be reviewed, this one was quite tricky... I have tested with as much models as I could (my ADSL connection is still burning 😄 ) and I don't get any issue with the last commit. Please let me know if this properly follow the philosophy of the lib.

I have also fixed few tiny bugs in Flaubert and XLM.

As far as I understand @sgugger's script takes care only of the PyTorch implementation. @sgugger How much your script can be applied to TF and all the models? (Not only Bert to Roberta).

I believe @sgugger's script can be applied to all python files, with no limitation on PyTorch. It was just merged to `master`!

Can't run it:
```
Traceback (most recent call last):
  File "utils/check_copies.py", line 181, in <module>
    check_copies(args.fix_and_overwrite)
  File "utils/check_copies.py", line 164, in check_copies
    consistent = is_copy_consistent(filename, overwrite)
  File "utils/check_copies.py", line 98, in is_copy_consistent
    lines = f.readlines()
  File "C:\Users\snake\miniconda3\envs\transformers\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 7178: character maps to <undefined>
```

That's weird, it runs fine on my laptop, desktop and the CI. What's your env?

**Edit:** Oh Windows. But how do you run the make commands on Windows?

I can run make with http://gnuwin32.sourceforge.net/packages/make.htm

I can run it under WSL if it is an OS issue.

I'll add `encoding="utf-8"` a bit everywhere and see if it fixes the issue for Windows.

It works under WSL.

When I run `python utils/check_copies.py` I get no output. Does it means everything is ok?

You would get an error if there was a problem (since the script is supposed to run under `make quality`).

Nice then LGTM :)

Flaubert and DistilBert was failing the last test I have added today on the training pipeline. It is now fixed :)

Great, thanks for fixing them! The changes look good.
Sylvain's script patch for Windows was merged yesterday, do you think you could apply the comments where they apply? This way `make quality` checks that these will not diverge.

You can see the way it works on the original PR: https://github.com/huggingface/transformers/pull/7219

Will do my best :)

WHOA this script is so dangerous, I have tested it on Albert with `TFAlbertAttention`, and it updates the `__init__` with a strict copy of `TFBertSelfAttention` which is wrong, there are slight updates that have been removed. I also had an indent issue but this is something else.

Is-it only for **STRICT** copies?

Yes, it's only for strict copies. If your copy is different (other than the naming that changes), I don't think the script can be applied to a module-level. 

You can apply it to select methods if these methods are identical to the initial implementation. By the way, you can use the script without the `--fix_and_overwrite` argument and it won't replace anything, just let you know if some copies diverge.

> Yes, it's only for strict copies. If your copy is different (other than the naming that changes), I don't think the script can be applied to a module-level.

OK, fortunately there are not much strict copies in what I have changed so it should be easier than I thought :)

>You can apply it to select methods if these methods are identical to the initial implementation. By the way, you can use the script without the --fix_and_overwrite argument and it won't replace anything, just let you know if some copies diverge.

I know, it was just for testing to know what was the error raised, because no diff is displayed in order to know what diverge from the original version, might be a useful new feature.

Ok! I think we are finally good :)