[WIP] Fixes for TF Roberta (and other models WIP)
Author: tlkh
Pull request: yes

When converting the `run_tf_glue.py` example to the same format at `benchmarks.py` to create a standardized benchmark for training, I ran into errors with **training** the non-BERT models with the normal `model.fit()` method. I am attempting to resolve all the errors I encountered in this PR. In particular, I have fixed the errors I have encountered with `TFRobertaForSequenceClassification`, `TFXLMForSequenceClassification`, and `TFXLNetForSequenceClassification`.

### Changes

**Roberta**

* Roberta requires `@tf.function()` on `TFRobertaMainLayer.call()`
* Otherwise, errors encountered:
  * `TypeError: You are attempting to use Python control flow in a layer that was not declared to be dynamic. Pass 'dynamic=True' to the class constructor.`
  * `OperatorNotAllowedInGraphError: using a 'tf.Tensor' as a Python 'bool' is not allowed in Graph execution. Use Eager execution or decorate this function with @tf.function.`
* Issues:
  * Fails test `TFRobertaModelTest.test_pt_tf_model_equivalence`: `AssertionError: layer.0.attention.self.query.weight not found in PyTorch model`.

**XLM**

* XLX requires changing some Python `assert` statements to `tf.debugging.assert_equal` both in `TFXLMMainLayer.call()` and `gen_mask()`
* Otherwise, errors encountered:
  * `TypeError: You are attempting to use Python control flow in a layer that was not declared to be dynamic. Pass 'dynamic=True' to the class constructor.`
  * `OperatorNotAllowedInGraphError: using a 'tf.Tensor' as a Python 'bool' is not allowed in Graph execution. Use Eager execution or decorate this function with @tf.function.`

**XLNet**

* XLNet had a dtype error (float vs int) in line `input_mask = 1.0 - attention_mask`. Since `input_mask` and `attention_mask` are both supposed (afaik) to be int32, I've replace `1.0` with `1`. 
* Still has shape error (see below) that I have not managed to track down. **This is particularly confusion because the training works in eager mode!**
* Solution is to simply provide a workaround `model.run_eagerly = True`.
* Of course, this will make the model train much slower (~140s for first epoch). Decorating `TFXLNetForSequenceClassification`'s `call()` method with `tf.function` works, and results in ~80s per first epoch. We cannot decorate the individual `call()` methods (aka create "overlapping" `tf.function`) as that will cause model saving to not work.
* Irregardless of my changes, there is a warning `gradients do not exist for variables ['transformer/mask_emb:0'] when minimizing the loss.` But from my observation the model trains fine. Is this embedding supposed to be trainable in the first place?
* Issues:
  * Fails test `TFXLNetModelTest.test_pt_tf_model_equivalence`: `AssertionError: mask_emb not found in PyTorch model`.

Shape error: 

```
tensorflow.python.framework.errors_impl.InvalidArgumentError:  Incompatible shapes: [128,128,16,12] vs. [128,255,16,12]
         [[node tfxl_net_for_sequence_classification/transformer/layer_._0/rel_attn/add_3 (defined at /opt/conda/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py:1751) ]] [Op:__inference_distributed_function_72170]
```

Do let me know if there are any feedback on the changes I made.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1590?src=pr&el=h1) Report
> Merging [#1590](https://codecov.io/gh/huggingface/transformers/pull/1590?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/4d456542e9d381090f9a00b2bcc5a4cb07f6f3f7?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/1590/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/1590?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #1590   +/-   ##
=======================================
  Coverage   86.16%   86.16%           
=======================================
  Files          91       91           
  Lines       13593    13593           
=======================================
  Hits        11713    11713           
  Misses       1880     1880
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/1590?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1590?src=pr&el=footer). Last update [4d45654...0322842](https://codecov.io/gh/huggingface/transformers/pull/1590?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Fixing TF Roberta and TF XLNet seem to be much trickier than XLM. **I will open a separate PR for XLM alone since that works fine.** 

For TF Roberta and TF XLNet, the solution might be to simply run them eagerly at a rather severe performance penalty. `tf.function` speeds it up a lot but seems to introduce some inconsistency in the weight saving, which might be a TensorFlow issue and I don't yet have the time to investigate.

@tlkh did you look into the shape errors any further? I'm getting similar errors in eager mode on tf-nightly, didn't try 2.0 (need some other fixes in 2.1)

```
tensorflow.python.framework.errors_impl.InvalidArgumentError:  Incompatible shapes: [91,91,64,12] vs. [91,181,64,12]
	 [[node model/tfxl_net_lm_head_model/transformer/layer_._0/rel_attn/add_2 (defined at .../transformers/modeling_tf_xlnet.py:148) ]] [Op:__inference_distributed_function_17027]
```

@NathanHowell sorry, I don't have any ideas about that! Seems to be the same error, but oddly running it in eager mode fixed it for me.

Thanks a lot @tlkh

So I think RoBERTa is now fixed on master (removed the faulty check in the forward pass) and XLM as well (with your other PR).

Do you want to make a new PR with fixes for XLNet and we close the present one maybe?

@thomwolf thanks, I'll close the current PR and open a new one for XLNet after I validate it again.