[TF Longformer] Add Multiple Choice, Seq Classification Model
Author: patrickvonplaten
Pull request: no
Labels: name: Good First Issue

# 🚀 Feature request

`modeling_longformer.py` has the classes `LongformerForSequenceClassification`, `LongformerForMultipleChoice` and `LongformerForTokenClassification` which are not present  in `modeling_tf_longformer.py` at the moment.
Those classes should be equally added to `modeling_tf_longformer.py`.
 
## Motivation

The pretrained weights for TFLongformer are available so that these classes could be used for finetuning.

## Your contribution

This issue is a good first issue because it is not too complicated to add these models. One should take a look at `modeling_tf_roberta.py` to see how these models are implemented for `TFRoberta` and implement them analogous for `TFLongformer`. Please make sure that the docstring is correct and that test are added for each class (again Roberta can serve as an example here, check out `test_modeling_tf_roberta.py`).

I am happy to guide interested community contributors through the PR and help them get it merged.


Comments:
Hi !
I'd like to help and work on this if that's ok.

Awesome, feel free to open an issue :-) 

Hello !

I'm a bit lost here. I've looked at `modeling_tf_roberta.py` and `modeling_longformer.py` to create the class `TFLongformerForSequenceClassification`. I'm not sure if I am going in the right direction here and same goes for the tests.
I used `python -m pytest -n auto --dist=loadfile -s -v ./tests/test_modeling_tf_roberta.py` to get an idea on what should I do for testing but it seems the test for `TFRobertaForSequenceClassification` is skipped and my test on the class I created (which is basically just a copy/paste of the roberta's test) is skipped too.

Here is a link to what I've done so far: https://github.com/Groskilled/transformers/commit/461ee6279433f94868332b1abbfe7875e19f243a

Am I on the right track ? And what am I missing on the tests ?

Sorry to ask such simple questions, it's my first time participating in an open source project.

No worries ;-). This looks alright! Could you open a PR so that we can see your changes directly on the PR? You can checkout this doc to understand how to do PRs: https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md. Would be great if you can ping me on the PR and then we look together!

HI @Groskilled and @patrickvonplaten, I have been playing a bit around this issue, as I have some familiarity with Keras/TF2 but no previous experience with transformers, and I was figuring out a way to start familiarising with them. As I am interested in classifying long documents Longformer is of interest to me.
I have a draft of my current changes [here](https://github.com/huggingface/transformers/compare/master...Zigur:tf-lonformer-good-first-release). The test suite seems to pass (using Python 3.7.5, they did not on Python 3.8.2 on my Mac machine), but I would need extensive feedback as I have mostly lifted code from `test_modeling_tf_roberta.py` and the testing counterpart.
If it is of interest, I can open a pull request with all the details, or @Groskilled you can feel free to cherry-pick part of it if it's useful for your own pull request (as you were working on this earlier on, apologies for the intromission)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Issue is still open! If stuck, feel free to take a look at the unfinished PR.