Descriptors cannot not be created directly.
Author: jcarbonnell
Pull request: no

Hi @patrickvonplaten 
I'm trying to import the following:  
from transformers import BartTokenizer, BartForConditionalGeneration
tokenizer = BartTokenizer.from_pretrained('facebook/bart-large')

and get the error:
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

All I can find on the web to fix it is to set:
!protobuf==3.20.1
but it's not working for me. Sorry I can't manage to fix it myself.

Comments:
plus one

This error doesn't show up when you run the code on google colab. you should try that @gunjan075 in the meantime

I have the same problem with the T0 model. My protobuf version is `4.21.1`. Any idea how to fix it? Cannot use Google Colab, only desktop machines.

Environment specs:
```
numpy==1.22.4
protobuf==4.21.1
sentencepiece==0.1.96
tokenizers==0.12.1
torch==1.11.0
tqdm==4.64.0
transformers==4.19.2
```

Update: I used `protobuf==3.20.0` and it worked. It's not ideal but it will do for now.

Indeed please make sure to use the correct `protobuf` version . Google's protobuf release broke a lot of codebases - even TF https://github.com/tensorflow/tensorflow/issues/56077 . 

Please make sure to use `"protobuf<=3.20.1"`

Just FYI @sgugger 

This is all fixed on the main branch FYI.

There was a patch release v4.19.3 just done to fix this issue FYI. 

Awesome
Thank you

On Thu, Jun 9, 2022, 12:12 Sylvain Gugger ***@***.***> wrote:

> There was a patch release v4.19.3 just done to fix this issue FYI.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/17544#issuecomment-1151388804>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAZHGWG6MCJIES4XVCBN72LVOIQWFANCNFSM5XZIE55A>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

With protobuf 4.21.8 and debertav2, this error still appears
```
AutoTokenizer.from_pretrained('microsoft/deberta-v3-small')
```

```
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
```

This error still happens for me. Downgrading is not an elegant solution because it breaks other packages that rely on latest version of protobuf e.g. google-cloud-documentai. Someome please fix properly.
I just created a new issue for this. https://github.com/huggingface/transformers/issues/21128

I met this issue when using `AutoTokenizer`, I fix it by specific the tokenizer to `LlamaTokenizer`.




> I met this issue when using `AutoTokenizer`, I fix it by specific the tokenizer to `LlamaTokenizer`.

This absolutely save my day! I meet with this issue for LLMs/vicuna. And this is a useful workaround.

pip install protobuf==3.19.4 worked for me