[s2s] adjust finetune + test to work with fsmt
Author: stas00
Pull request: yes
Labels: name: fsmt

integrate FSMT into finetune + add the test

Please note I started tweaking `finetune.py` to make explicit conditionals based on model name/type, rather than try/except. Might be a good idea to make `self.config.model_type` more easily accessible by some quick attribute alias or something...

<!-- This line specifies which issue to close after the pull request is merged. -->
Fixes #7230 

Once PR https://github.com/huggingface/transformers/pull/7224 is merged I will have to tweak `s/.weights/weight/` in the test (As we are moving to `nn.Embedding` subclass). So perhaps let's hold off on merging this until after #7224 is in and I pushed the adjustment.

@sshleifer 

Comments:
Looks good. Will wait to merge.
I bet there is a clever way to use `nn.Module.apply` to freeze all submodules that match `isinstance(module, nn.Embedding)` and delete all the conditionals.


Whichever way we do it it'd be good to abstract it into some helper function perhaps in `testing_utils.py`, that gets as the argument either `module_name` or the model object so it can return `model_type` (t5/bart/fsmt/etc.) auto-magically and then another helper function that will return the corresponding to `model_type` `task` and whatever other unique things we need that are currently being derived in conditionals. 

PR #7224 has been merged, this PR has been adjusted, so it's now good to go.

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7263?src=pr&el=h1) Report
> Merging [#7263](https://codecov.io/gh/huggingface/transformers/pull/7263?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/63276b76d4fb54d096b491e89632859aed6b4364?el=desc) will **increase** coverage by `0.42%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/7263/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/7263?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #7263      +/-   ##
==========================================
+ Coverage   79.72%   80.14%   +0.42%     
==========================================
  Files         174      174              
  Lines       33452    33452              
==========================================
+ Hits        26668    26810     +142     
+ Misses       6784     6642     -142     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/7263?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_flaubert.py](https://codecov.io/gh/huggingface/transformers/pull/7263/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9mbGF1YmVydC5weQ==) | `24.53% <0.00%> (-63.81%)` | :arrow_down: |
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/7263/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `13.40% <0.00%> (-42.32%)` | :arrow_down: |
| [src/transformers/tokenization\_marian.py](https://codecov.io/gh/huggingface/transformers/pull/7263/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fbWFyaWFuLnB5) | `68.96% <0.00%> (-30.18%)` | :arrow_down: |
| [src/transformers/training\_args.py](https://codecov.io/gh/huggingface/transformers/pull/7263/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmluZ19hcmdzLnB5) | `66.97% <0.00%> (-24.78%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_gpt2.py](https://codecov.io/gh/huggingface/transformers/pull/7263/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9ncHQyLnB5) | `71.59% <0.00%> (-23.38%)` | :arrow_down: |
| [src/transformers/modeling\_xlnet.py](https://codecov.io/gh/huggingface/transformers/pull/7263/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ194bG5ldC5weQ==) | `60.81% <0.00%> (-22.62%)` | :arrow_down: |
| [src/transformers/trainer\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/7263/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyX3V0aWxzLnB5) | `40.00% <0.00%> (-18.89%)` | :arrow_down: |
| [src/transformers/modeling\_transfo\_xl\_utilities.py](https://codecov.io/gh/huggingface/transformers/pull/7263/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190cmFuc2ZvX3hsX3V0aWxpdGllcy5weQ==) | `52.98% <0.00%> (-13.44%)` | :arrow_down: |
| [src/transformers/tokenization\_reformer.py](https://codecov.io/gh/huggingface/transformers/pull/7263/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fcmVmb3JtZXIucHk=) | `81.66% <0.00%> (-13.34%)` | :arrow_down: |
| [src/transformers/modeling\_transfo\_xl.py](https://codecov.io/gh/huggingface/transformers/pull/7263/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190cmFuc2ZvX3hsLnB5) | `67.04% <0.00%> (-12.69%)` | :arrow_down: |
| ... and [10 more](https://codecov.io/gh/huggingface/transformers/pull/7263/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/7263?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7263?src=pr&el=footer). Last update [63276b7...a793794](https://codecov.io/gh/huggingface/transformers/pull/7263?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).