expected str, bytes or os.PathLike object, not NoneType
Author: hjzhang1018
Pull request: no

## Environment info
- `transformers` version: 4.1.1
- Platform: Darwin-18.6.0-x86_64-i386-64bit
- Python version: 3.7.7
- PyTorch version (GPU?): not installed (NA)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
  
### Who can help
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...): I don't know

The problem arises when using:
* [ ] the official example scripts: (give details below)

The tasks I am working on is:
* [ ] my own task or dataset: (give details below) I was trying to use this for further transfer learning.

## To reproduce

Steps to reproduce the behavior(the snippet I used):
```
import deepchem as dc
import tensorflow as tf
from transformers import AutoTokenizer, AutoModelForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("seyonec/SMILES_tokenized_PubChem_shard00_160k")

model = AutoModelForMaskedLM.from_pretrained("seyonec/SMILES_tokenized_PubChem_shard00_160k")
```
Then I got this error message: "TypeError: expected str, bytes or os.PathLike object, not NoneType".

I appreciate any help/suggestions! Thanks very much.


Comments:
Hey @hjzhang1018,

Thanks for your bug report. Could you try to run your command again - I think it should be fixed now: https://huggingface.co/seyonec/SMILES_tokenized_PubChem_shard00_160k/commit/7ef67531cfe96d0e2aa3ae913352c8e9a8c1df4f

@seyonechithrananda I added some files to your repo here: https://huggingface.co/seyonec/SMILES_tokenized_PubChem_shard00_160k/commit/7ef67531cfe96d0e2aa3ae913352c8e9a8c1df4f - feel free to take a look and see whether this is OK for you. It might be possible that other models require this change as well. 

@hjzhang1018 @patrickvonplaten Hi all, thanks for the fix! I believe the reason this may be happening is because the tokenizer we use is custom (a subclass of BertTokenizer) and thus we run into the issue of fitting directly with AutoTokenizer. We have a PR for a tutorial in the DeepChem [library](https://github.com/deepchem/deepchem/pull/2302), which demonstrates how to call our subclass instead of using AutoTokenizer. If you refer to Part 23 hopefully that is of use!

Docs: https://deepchem.readthedocs.io/en/latest/api_reference/tokenizers.html

Link to SmilesTokenizer class which these models utilize: https://github.com/deepchem/deepchem/blob/master/deepchem/feat/smiles_tokenizer.py#L39-L282

Just read the changes, it looks like @patrickvonplaten directly converted the vocab.txt file sufficient for BertTokenizer into the vocab.json format necessary for RoBERTa tokenizers, which should run smoothly. I will try this out once I get more time but this fix should work. Thanks a lot for the quick fix!

Is there a way to add this change to the other models with the 'SmilesTokenizer', @patrickvonplaten? Thanks again for the support.

@patrickvonplaten Thank you so much for your help! Now this worked!
@seyonechithrananda Thank you for the explanation. I'll read the tutorials carefully. Very useful!

> Is there a way to add this change to the other models with the 'SmilesTokenizer', @patrickvonplaten? Thanks again for the support.

Yes on simply has to create an empty 'merges.txt' file and create `vocab.json` from vocab.txt

Hi, I have the same issues here, I have a custom roberta model and I am using https://github.com/UKPLab/sentence-transformers. Here is the full detail of my problem: https://github.com/UKPLab/sentence-transformers/issues/658

The output after training from this sentence transformers yield files that doesn't contain vocab.json or vocab.txt. But I have a file called `unigram.json` and it looks something like this:

![DC144136-AEAA-48CA-8CC0-D919D37FD6FF_4_5005_c](https://user-images.githubusercontent.com/75713031/103332724-1c044300-4aa6-11eb-860d-799ddd794139.jpeg)


```
{
  "unk_id": 0,
  "vocab": [
    [
      "<unk>",
      0.0
    ],
    [
      "<sep>",
      0.0
    ],
    [
      "<pad>",
      0.0
    ],
    ....
  ]
}
```

I also faced this TypeError, the same as the title of this issue, when trying to use AutoTokenizer

In short every Roberta-like Tokenizer requires two files:

1) One merges.txt file. This file describes the BPE algorithm (which letters are merged in which order)
2) One vocab.json file. This file describes the vocabulary.

To get an idea of how the format of these files should be, I'd recommend taking a look at some the files in `roberta-base` here:https://huggingface.co/roberta-base/tree/main

@seyonechithrananda, I think in your case, you don't need a merges.txt file because of the small vocabulary and because there are no words, just tokens
@hjzhang1018 If the other library:  UKPLab/sentence-transformers uses the same format for loading/saving files that we do, then file should be renamed to be called `vocab.json` and should have a different format (check out the format here: https://huggingface.co/roberta-base/blob/main/vocab.json).

So after I call `sentence-transformers` save which gives me back 6 files:
1. I will need to rename `unigram.json` to `vocab.json`
2. Change the format of `unigram.json` to follows `vocab.json` structure
3. Create an empty `merge.txt` file

And currently my `unigram.json` contains a word weight:
```
{
  "unk_id": 0,
  "vocab": [
    [
      "<unk>",
      0.0
    ],
    [
      "<sep>",
      0.0
    ],
    [
      "<pad>",
      0.0
    ],
    [
      "<cls>",
      0.0
    ],
    [
      "<mask>",
      0.0
    ],
    [
      ",",
      -3.1215689182281494
    ],
    [
      ".",
      -3.642984628677368
    ],
    [
      "a",
      -4.921720027923584
    ],
    ......
  ]
}
```

Do I just ignore all the weights and created a new file `vocab.json` with this format ?
```
{
  "<unk>": 0,
  "<sep>": 1,
  "<pad>": 2,
  "<cls>": 3,
  "<mask>": 4,
  ",": 5,
  ".": 6,
  "a": 7,
  .......
}
```

I started getting this error only about an hour ago without any changes on my side in my old Colab notebook. There must be some version change in Colab env that triggers this error. Any ideas what it might be?


@lenyabloko could you open a new issue with the issue you're facing and how to reproduce it? Thank you.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

In my case, I found that I was using the AutoTokenizer class instead of the specified (BertTokenizerFast) class, so it is important to use the same class with which the tokenizer was first initialized

I changed the transformers/models/layoutlmv3/processing_layoutlmv3 the 49 lines to "tokenizer_class = ("LayoutLMv3Tokenizer", "LayoutLMv3TokenizerFast",'XLMRobertaTokenizer','XLMRobertaTokenizerFast','LayoutXLMTokenizer')"  it works for me