DeBERTa in TF (TFAutoModel): unrecognized configuration class
Author: ck37
Pull request: no
Labels: name: wontfix

## Environment info
     
- `transformers` version: 4.1.1
- Platform: Linux-3.10.0-957.21.3.el7.x86_64-x86_64-with-centos-7.6.1810-Core
- Python version: 3.6.9
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): 2.2.0 (True)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@LysandreJik 

## Information

Model I am using: DeBERTa

The problem arises when using:
* [ x ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ x ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Use TFAutoModel to import deberta-large

```python
from transformers import TFAutoModel
model = TFAutoModel.from_pretrained("microsoft/deberta-large")
```

Error:
```python
---------------------------------------------------------------------------
ValueError    
<ipython-input-2-416d7de4fc12> in <module>
----> 1 model = TFAutoModel.from_pretrained("microsoft/deberta-large")

~/miniconda3/envs/hate2/lib/python3.6/site-packages/transformers/models/auto/modeling_tf_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    583             "Unrecognized configuration class {} for this kind of TFAutoModel: {}.\n"
    584             "Model type should be one of {}.".format(
--> 585                 config.__class__, cls.__name__, ", ".join(c.__name__ for c in TF_MODEL_MAPPING.keys())
    586             )
    587         )

ValueError: Unrecognized configuration class <class 'transformers.models.deberta.configuration_deberta.DebertaConfig'> for this kind of TFAutoModel: TFAutoModel.
Model type should be one of LxmertConfig, MT5Config, T5Config, DistilBertConfig, AlbertConfig, BartConfig, CamembertConfig, XLMRobertaConfig, LongformerConfig, RobertaConfig, BertConfig, OpenAIGPTConfig, GPT2Config, MobileBertConfig, TransfoXLConfig, XLNetConfig, FlaubertConfig, XLMConfig, CTRLConfig, ElectraConfig, FunnelConfig, DPRConfig, MPNetConfig.
```

## Expected behavior

I should be able to import deberta-large and deberta-base using TFAutoModel, or the documentation should be updated to clarify that they are pytorch only.

Thanks as always for the amazing software, and please let me know if I should provide any other details or otherwise help.

Comments:
Hi @ck37 

`DeBERTa` is currently PyTorch-only so it can't be loaded with `TFAutoModel`.  The table on the doc's [homepage](https://huggingface.co/transformers/) shows whether the models have support in PyTorch, TensorFlow, and/or Flax.

Gotcha, thanks for the fast response. Do you think the TF side will be implemented at some point? It seems like there will be more interest in DeBERTa with it taking the lead in [SuperGLUE](https://super.gluebenchmark.com/leaderboard).

Yeah, it's pretty exciting! @patrickvonplaten  might be able to give you eta of `TFDeberta`

This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.