Repository: huggingface/transformers

Issue:
[fixup/style] requires TF but doesn't say that cleanly
It looks like `make fixup` et al, require tf (and pt), but fail in unexpected way when the requirements are missing:

```
python utils/check_repo.py
Checking all models are properly tested.
Checking all objects are properly documented.
Checking all models are in at least one auto class.
Traceback (most recent call last):
  File "/home/michael/projects/transformers/utils/check_repo.py", line 481, in <module>
    check_repo_quality()
  File "/home/michael/projects/transformers/utils/check_repo.py", line 477, in check_repo_quality
    check_all_models_are_auto_configured()
  File "/home/michael/projects/transformers/utils/check_repo.py", line 290, in check_all_models_are_auto_configured
    all_auto_models = get_all_auto_configured_models()
  File "/home/michael/projects/transformers/utils/check_repo.py", line 253, in get_all_auto_configured_models
    for attr_name in dir(transformers.models.auto.modeling_tf_auto):
  File "/home/michael/projects/transformers/src/transformers/file_utils.py", line 1690, in __getattr__
    raise AttributeError(f"module {self.__name__} has no attribute {name}")
AttributeError: module transformers.models.auto has no attribute modeling_tf_auto
make: *** [Makefile:35 : extra_quality_checks] Erreur 1
```

Thank you, @michaelbenayoun for flagging this

Should we add a small script first that checks all the requirements so that the error is not misleading, but something like: "need to install `pip install -e .[dev]` to develop `transformers`"?

@sgugger, @LysandreJik 


Discussion:
The same thing happens with flax.
This should take care of all the deps:
```
pip install -e .[dev]
```
Please let us know if it didn't.
I don't think we need a new script for that. Maybe add the check inside the script that fails (`check_all_models_are_auto_configured`) and issue a warning if not all backends are detected (I don't think we need to error out, since it's unlikely the user will bring changes that break a backend when that backend is not even installed)? I can do this later this evening or tomorrow.
Also with a bit of some further massaging of `setup.py`'s  `extras`, we could automate this - basically need to be able to load `extras[dev]` outside of `setup.py`, so the check could be to simply import everything that is in `extras[dev]`.
Note that this specific script only relies on the model backends only, so no need for the whole of dev yet.
If it's easier - then by all means. 

I just thought that if we already maintain `extras[dev]` then any dev tool could just have that as pre-requisite.

Resolving patch:
diff --git a/utils/check_repo.py b/utils/check_repo.py
--- a/utils/check_repo.py
+++ b/utils/check_repo.py
@@ -17,8 +17,11 @@
 import inspect
 import os
 import re
+import warnings
 from pathlib import Path
 
+from transformers import is_flax_available, is_tf_available, is_torch_available
+from transformers.file_utils import ENV_VARS_TRUE_VALUES
 from transformers.models.auto import get_values
 
 
@@ -250,15 +253,18 @@ def check_all_models_are_tested():
 def get_all_auto_configured_models():
     """Return the list of all models in at least one auto class."""
     result = set()  # To avoid duplicates we concatenate all model classes in a set.
-    for attr_name in dir(transformers.models.auto.modeling_auto):
-        if attr_name.startswith("MODEL_") and attr_name.endswith("MAPPING"):
-            result = result | set(get_values(getattr(transformers.models.auto.modeling_auto, attr_name)))
-    for attr_name in dir(transformers.models.auto.modeling_tf_auto):
-        if attr_name.startswith("TF_MODEL_") and attr_name.endswith("MAPPING"):
-            result = result | set(get_values(getattr(transformers.models.auto.modeling_tf_auto, attr_name)))
-    for attr_name in dir(transformers.models.auto.modeling_flax_auto):
-        if attr_name.startswith("FLAX_MODEL_") and attr_name.endswith("MAPPING"):
-            result = result | set(get_values(getattr(transformers.models.auto.modeling_flax_auto, attr_name)))
+    if is_torch_available():
+        for attr_name in dir(transformers.models.auto.modeling_auto):
+            if attr_name.startswith("MODEL_") and attr_name.endswith("MAPPING"):
+                result = result | set(get_values(getattr(transformers.models.auto.modeling_auto, attr_name)))
+    if is_tf_available():
+        for attr_name in dir(transformers.models.auto.modeling_tf_auto):
+            if attr_name.startswith("TF_MODEL_") and attr_name.endswith("MAPPING"):
+                result = result | set(get_values(getattr(transformers.models.auto.modeling_tf_auto, attr_name)))
+    if is_flax_available():
+        for attr_name in dir(transformers.models.auto.modeling_flax_auto):
+            if attr_name.startswith("FLAX_MODEL_") and attr_name.endswith("MAPPING"):
+                result = result | set(get_values(getattr(transformers.models.auto.modeling_flax_auto, attr_name)))
     return [cls.__name__ for cls in result]
 
 
@@ -289,6 +295,27 @@ def check_models_are_auto_configured(module, all_auto_models):
 
 def check_all_models_are_auto_configured():
     """Check all models are each in an auto class."""
+    missing_backends = []
+    if not is_torch_available():
+        missing_backends.append("PyTorch")
+    if not is_tf_available():
+        missing_backends.append("TensorFlow")
+    if not is_flax_available():
+        missing_backends.append("Flax")
+    if len(missing_backends) > 0:
+        missing = ", ".join(missing_backends)
+        if os.getenv("TRANSFORMERS_IS_CI", "").upper() in ENV_VARS_TRUE_VALUES:
+            raise Exception(
+                "Full quality checks require all backends to be installed (with `pip install -e .[dev]` in the "
+                f"Transformers repo, the following are missing: {missing}."
+            )
+        else:
+            warnings.warn(
+                "Full quality checks require all backends to be installed (with `pip install -e .[dev]` in the "
+                f"Transformers repo, the following are missing: {missing}. While it's probably fine as long as you "
+                "didn't make any change in one of those backends modeling files, you should probably execute the "
+                "command above to be on the safe side."
+            )
     modules = get_model_modules()
     all_auto_models = get_all_auto_configured_models()
     failures = []


Test patch: