Repository: huggingface/transformers

Issue:
'MT5Tokenizer' is not defined (on Google colab) 
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: `transformers-4.9.0.dev0`
- Platform: Google Colab
- Python version: `3.7.10` 
- PyTorch version (GPU?):  `1.9.0+cu102`
- Tensorflow version (GPU?): `2.5.0`
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@sgugger , @patil-suraj 

## Information

Trying to run_mlm.py  - 

The problem arises when using:
- Just importing packages. 

## To reproduce

Steps to reproduce the behavior:

1. Run the following on Google colab 
2. 

```
!pip install git+https://github.com/huggingface/transformers.git

from transformers import (
    CONFIG_MAPPING,
    MODEL_FOR_MASKED_LM_MAPPING,
    AutoConfig,
    AutoModelForMaskedLM,
    AutoTokenizer,
    DataCollatorForLanguageModeling,
    HfArgumentParser,
    Trainer,
    TrainingArguments,
    set_seed,
)
```
Error message - 
```
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-1-a06d7b6932ed> in <module>()
      2 
      3 
----> 4 from transformers import (
      5     CONFIG_MAPPING,
      6     MODEL_FOR_MASKED_LM_MAPPING,

5 frames
/usr/local/lib/python3.7/dist-packages/transformers/models/mt5/__init__.py in <module>()
     94         globals()["__file__"],
     95         _import_structure,
---> 96         extra_objects={"MT5Tokenizer": MT5Tokenizer, "MT5TokenizerFast": MT5TokenizerFast},
     97     )

NameError: name 'MT5Tokenizer' is not defined
```

## Expected behavior
No error when importing.

Thank you for your help! 


Discussion:
Fixed it for now - `!pip install git+https://github.com/huggingface/transformers.git@b29c394` 

Resolving patch:
diff --git a/src/transformers/models/mt5/__init__.py b/src/transformers/models/mt5/__init__.py
--- a/src/transformers/models/mt5/__init__.py
+++ b/src/transformers/models/mt5/__init__.py
@@ -29,24 +29,22 @@
 
 if is_sentencepiece_available():
     from ..t5.tokenization_t5 import T5Tokenizer
+else:
+    from ...utils.dummy_sentencepiece_objects import T5Tokenizer
 
-    MT5Tokenizer = T5Tokenizer
+MT5Tokenizer = T5Tokenizer
 
 if is_tokenizers_available():
     from ..t5.tokenization_t5_fast import T5TokenizerFast
+else:
+    from ...utils.dummy_tokenizers_objects import T5TokenizerFast
 
-    MT5TokenizerFast = T5TokenizerFast
+MT5TokenizerFast = T5TokenizerFast
 
 _import_structure = {
     "configuration_mt5": ["MT5Config"],
 }
 
-if is_sentencepiece_available():
-    _import_structure["."] = ["T5Tokenizer"]  # Fake to get the same objects in both side.
-
-if is_tokenizers_available():
-    _import_structure["."] = ["T5TokenizerFast"]  # Fake to get the same objects in both side.
-
 if is_torch_available():
     _import_structure["modeling_mt5"] = ["MT5EncoderModel", "MT5ForConditionalGeneration", "MT5Model"]
 
@@ -57,16 +55,6 @@
 if TYPE_CHECKING:
     from .configuration_mt5 import MT5Config
 
-    if is_sentencepiece_available():
-        from ..t5.tokenization_t5 import T5Tokenizer
-
-        MT5Tokenizer = T5Tokenizer
-
-    if is_tokenizers_available():
-        from ..t5.tokenization_t5_fast import T5TokenizerFast
-
-        MT5TokenizerFast = T5TokenizerFast
-
     if is_torch_available():
         from .modeling_mt5 import MT5EncoderModel, MT5ForConditionalGeneration, MT5Model
 
@@ -76,20 +64,7 @@
 else:
     import sys
 
-    class _MT5LazyModule(_LazyModule):
-        """
-        Module class that surfaces all objects but only performs associated imports when the objects are requested.
-        """
-
-        def __getattr__(self, name):
-            if name == "MT5Tokenizer":
-                return MT5Tokenizer
-            elif name == "MT5TokenizerFast":
-                return MT5TokenizerFast
-            else:
-                return super().__getattr__(name)
-
-    sys.modules[__name__] = _MT5LazyModule(
+    sys.modules[__name__] = _LazyModule(
         __name__,
         globals()["__file__"],
         _import_structure,


Test patch: