OSError using FlauBERT
Author: anislll
Pull request: no
Labels: name: wontfix

Hello everyone,

I'm trying to run Flaubert model on my data using ktrain.


# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):  flaubert/flaubert_base_cased

Language I am using the model on (English, Chinese ...): French

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

I'm using ktrain to load my model. 
After preprocess my data when i want to get the classifier with function get_classifier() i get this error :

---------------------------------------------------------------------------
```
OSError                                   Traceback (most recent call last)
~\Anaconda3\lib\site-packages\transformers\modeling_tf_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    461                 if resolved_archive_file is None:
--> 462                     raise EnvironmentError
    463             except EnvironmentError:

OSError: 

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
~\Anaconda3\lib\site-packages\ktrain\text\preprocessor.py in _load_pretrained(self, mname, num_labels)
    958             try:
--> 959                 model = self.model_type.from_pretrained(mname, config=self.config)
    960             except:

~\Anaconda3\lib\site-packages\transformers\modeling_tf_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   1046             if isinstance(config, config_class):
-> 1047                 return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
   1048         raise ValueError(

~\Anaconda3\lib\site-packages\transformers\modeling_tf_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    468                 )
--> 469                 raise EnvironmentError(msg)
    470             if resolved_archive_file == archive_file:

OSError: Can't load weights for 'flaubert/flaubert_base_cased'. Make sure that:

- 'flaubert/flaubert_base_cased' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'flaubert/flaubert_base_cased' is the correct path to a directory containing a file named one of tf_model.h5, pytorch_model.bin.



During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
~\Anaconda3\lib\site-packages\ktrain\text\preprocessor.py in _load_pretrained(self, mname, num_labels)
    961                 try:
--> 962                     model = self.model_type.from_pretrained(mname, config=self.config, from_pt=True)
    963                 except:

~\Anaconda3\lib\site-packages\transformers\modeling_tf_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   1046             if isinstance(config, config_class):
-> 1047                 return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
   1048         raise ValueError(

~\Anaconda3\lib\site-packages\transformers\modeling_tf_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    481             # Load from a PyTorch checkpoint
--> 482             return load_pytorch_checkpoint_in_tf2_model(model, resolved_archive_file, allow_missing_keys=True)
    483 

~\Anaconda3\lib\site-packages\transformers\modeling_tf_pytorch_utils.py in load_pytorch_checkpoint_in_tf2_model(tf_model, pytorch_checkpoint_path, tf_inputs, allow_missing_keys)
     92     return load_pytorch_weights_in_tf2_model(
---> 93         tf_model, pt_state_dict, tf_inputs=tf_inputs, allow_missing_keys=allow_missing_keys
     94     )

~\Anaconda3\lib\site-packages\transformers\modeling_tf_pytorch_utils.py in load_pytorch_weights_in_tf2_model(tf_model, pt_state_dict, tf_inputs, allow_missing_keys)
    124     if tf_inputs is not None:
--> 125         tf_model(tf_inputs, training=False)  # Make sure model is built
    126 

~\Anaconda3\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py in __call__(self, inputs, *args, **kwargs)
    821               self._compute_dtype):
--> 822             outputs = self.call(cast_inputs, *args, **kwargs)
    823           self._handle_activity_regularization(inputs, outputs)

~\Anaconda3\lib\site-packages\transformers\modeling_tf_xlm.py in call(self, inputs, attention_mask, langs, token_type_ids, position_ids, lengths, cache, head_mask, inputs_embeds, output_attentions, output_hidden_states, labels, training)
    803             output_hidden_states=output_hidden_states,
--> 804             training=training,
    805         )

~\Anaconda3\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py in __call__(self, inputs, *args, **kwargs)
    821               self._compute_dtype):
--> 822             outputs = self.call(cast_inputs, *args, **kwargs)
    823           self._handle_activity_regularization(inputs, outputs)

~\Anaconda3\lib\site-packages\transformers\modeling_tf_flaubert.py in call(self, inputs, attention_mask, langs, token_type_ids, position_ids, lengths, cache, head_mask, inputs_embeds, training, output_attentions, output_hidden_states)
    259             if not self.pre_norm:
--> 260                 attn_outputs = self.attentions[i]([tensor, attn_mask, None, cache, head_mask[i]], training=training)
    261                 attn = attn_outputs[0]

~\Anaconda3\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py in __call__(self, inputs, *args, **kwargs)
    821               self._compute_dtype):
--> 822             outputs = self.call(cast_inputs, *args, **kwargs)
    823           self._handle_activity_regularization(inputs, outputs)

~\Anaconda3\lib\site-packages\transformers\modeling_tf_xlm.py in call(self, inputs, training)
    140         """
--> 141         input, mask, kv, cache, head_mask, output_attentions = inputs
    142         # Input is (bs, qlen, dim)

ValueError: not enough values to unpack (expected 6, got 5)

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-38-dc13d8280fd1> in <module>
----> 1 model = t.get_classifier()

~\Anaconda3\lib\site-packages\ktrain\text\preprocessor.py in get_classifier(self, fpath, multilabel, metrics)
    997         num_labels = len(self.get_classes())
    998         mname = fpath if fpath is not None else self.model_name
--> 999         model = self._load_pretrained(mname, num_labels)
   1000         if multilabel:
   1001             loss_fn =  keras.losses.BinaryCrossentropy(from_logits=True)

~\Anaconda3\lib\site-packages\ktrain\text\preprocessor.py in _load_pretrained(self, mname, num_labels)
    962                     model = self.model_type.from_pretrained(mname, config=self.config, from_pt=True)
    963                 except:
--> 964                     raise ValueError('could not load pretrained model %s using both from_pt=False and from_pt=True' % (mname))
    965         else:
    966             model = self.model_type.from_pretrained(mname, num_labels=num_labels)

ValueError: could not load pretrained model flaubert/flaubert_base_cased using both from_pt=False and from_pt=True
```

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.0.2
- Platform: windows 10
- Python version: 3.8
- PyTorch version (No GPU): 1.0.0
- Tensorflow version (No GPU): 2.1.0
- Using GPU in script?: No
- Using distributed or parallel set-up in script?:

Thank you a lot for your help

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.