Issue 1:
https://github.com/huggingface/pytorch-pretrained-BERT/blob/35becc6d84f620c3da48db460d6fb900f2451782/pytorch_pretrained_bert/modeling.py#L387-L394

I don't understand how it is useful to wrap the BertLMPredictionHead class like that, perhaps it was forgotten in some refactoring ? I can do a PR if you confirm me it can be replaced.

BertOnlyMLMHead is only used in BertForMaskedLM.

Issue 2:
Environment info

transformers version: 4.18.0.dev0

Platform: macos high sierra 10.13.4

Python version: 3.9

PyTorch version (GPU?): 1.8.1 CPU

Using GPU in script?: no

Using distributed or parallel set-up in script?: no

Who can help

@sgugger @LysandreJik

Information

When trying to create a new model with add-new-model-like without having flax installed, an error message is raised.

While I think the code should work without flax, especially considering the numerous flags across the codebase (is_*_available()) to handle when one of the deep learning framework isn't installed.

The problem arises when using:

the official example scripts: (give details below)

To reproduce

Steps to reproduce the behavior:

run transformers-cli add-new-model-like

ask for deit

see error message

$ transformers-cli add-new-model-like
What is the model you would like to duplicate? deit
Traceback (most recent call last):
File "/usr/local/bin/transformers-cli", line 33, in <module>
sys.exit(load_entry_point('transformers==4.18.0.dev0', 'console_scripts', 'transformers-cli')())
File "/usr/local/lib/python3.9/site-packages/transformers-4.18.0.dev0-py3.9.egg/transformers/commands/transformers_cli.py", line 52, in main
service = args.func(args)
File "/usr/local/lib/python3.9/site-packages/transformers-4.18.0.dev0-py3.9.egg/transformers/commands/add_new_model_like.py", line 1283, in add_new_model_like_command_factory
return AddNewModelLikeCommand(config_file=args.config_file, path_to_repo=args.path_to_repo)
File "/usr/local/lib/python3.9/site-packages/transformers-4.18.0.dev0-py3.9.egg/transformers/commands/add_new_model_like.py", line 1314, in __init__
) = get_user_input()
File "/usr/local/lib/python3.9/site-packages/transformers-4.18.0.dev0-py3.9.egg/transformers/commands/add_new_model_like.py", line 1419, in get_user_input
old_model_info = retrieve_info_for_model(old_model_type)
File "/usr/local/lib/python3.9/site-packages/transformers-4.18.0.dev0-py3.9.egg/transformers/commands/add_new_model_like.py", line 690, in retrieve_info_for_model
model_classes = retrieve_model_classes(model_type, frameworks=frameworks)
File "/usr/local/lib/python3.9/site-packages/transformers-4.18.0.dev0-py3.9.egg/transformers/commands/add_new_model_like.py", line 624, in retrieve_model_classes
"flax": auto_module.modeling_flax_auto,
File "/usr/local/lib/python3.9/site-packages/transformers-4.18.0.dev0-py3.9.egg/transformers/file_utils.py", line 2770, in __getattr__
raise AttributeError(f"module {self.__name__} has no attribute {name}")
AttributeError: module transformers.models.auto has no attribute modeling_flax_auto

Expected behavior

Either work without having to install flax, or raise an error message telling me I need to install flax (which would be a bit annoying if I only intend to contribute to pytorch).

Thank you very much for this library, it's great! ❤️