Trainer crashes when saving checkpoint
Author: GuillemGSubies
Pull request: no

## Environment info


- `transformers` version: 4.3.3
- Platform: Linux-4.4.0-161-generic-x86_64-with-glibc2.10
- Python version: 3.8.8
- PyTorch version (GPU?): 1.8.0+cu101 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes, distributed
- Using distributed or parallel set-up in script?: Yes

```
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.87.00    Driver Version: 418.87.00    CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla P100-PCIE...  Off  | 00000000:02:00.0 Off |                    0 |
| N/A   47C    P0    33W / 250W |  16181MiB / 16280MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla P100-PCIE...  Off  | 00000000:81:00.0 Off |                    0 |
| N/A   42C    P0    34W / 250W |  16173MiB / 16280MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
```

### Who can help

- trainer: @sgugger


## Information

Model I am using (Bert, XLNet ...): "bert-base-multilingual-cased"

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```python
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(model_name, do_lower_case=False)

def preprocess_function(examples):
    tokenized = tokenizer(examples["text"], truncation=True)
    tokenized["labels"] = [1 if elem == "sexist" else 0 for elem in examples["task1"]]
    return tokenized

encoded_dataset = dataset.map(preprocess_function, batched=True)

from transformers import AutoConfig, AutoModelForSequenceClassification, TrainingArguments, Trainer
import datasets

model_name = "bert-base-multilingual-cased"
config = AutoConfig.from_pretrained(
    model_name,
    num_labels=len(np.unique(encoded_dataset["train"]["labels"])),
)

model = AutoModelForSequenceClassification.from_pretrained(model_name, config=config)
training_arguments = TrainingArguments(
    output_dir=f"{model_name}",
    do_train=True,
    do_eval=True,
    evaluation_strategy="steps",
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    learning_rate=2e-5,
    num_train_epochs=5,
    label_names=["labels"],
    load_best_model_at_end=True,
    metric_for_best_model=datasets.load_metric("accuracy"),
    eval_steps=50,
)
def compute_metrics(eval_pred):
    predictions, labels = eval_pred
    predictions = np.argmax(predictions, axis=1)
    return metric.compute(predictions=predictions, references=labels)

trainer = Trainer(
    model,
    training_arguments,
    train_dataset=encoded_dataset["train"],
    eval_dataset=encoded_dataset["validation"],
    tokenizer=tokenizer,
    compute_metrics=compute_metrics,
)
trainer.train()
```

The error:

```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-44-974384763956> in <module>
     10     #callbacks=[EarlyStoppingCallback(early_stopping_patience=4)], # Por alguna razón, casca
     11 )
---> 12 trainer.train()

~/.local/lib/python3.8/site-packages/transformers/trainer.py in train(self, resume_from_checkpoint, trial, **kwargs)
    981                     self.control = self.callback_handler.on_step_end(self.args, self.state, self.control)
    982 
--> 983                     self._maybe_log_save_evaluate(tr_loss, model, trial, epoch)
    984 
    985                 if self.control.should_epoch_stop or self.control.should_training_stop:

~/.local/lib/python3.8/site-packages/transformers/trainer.py in _maybe_log_save_evaluate(self, tr_loss, model, trial, epoch)
   1060 
   1061         if self.control.should_save:
-> 1062             self._save_checkpoint(model, trial, metrics=metrics)
   1063             self.control = self.callback_handler.on_save(self.args, self.state, self.control)
   1064 

~/.local/lib/python3.8/site-packages/transformers/trainer.py in _save_checkpoint(self, model, trial, metrics)
   1085             self.store_flos()
   1086 
-> 1087         self.save_model(output_dir)
   1088         if self.deepspeed:
   1089             self.deepspeed.save_checkpoint(output_dir)

~/.local/lib/python3.8/site-packages/transformers/trainer.py in save_model(self, output_dir)
   1376             self._save_tpu(output_dir)
   1377         elif self.is_world_process_zero():
-> 1378             self._save(output_dir)
   1379 
   1380         # If on sagemaker and we are saving the main model (not a checkpoint so output_dir=None), save a copy to

~/.local/lib/python3.8/site-packages/transformers/trainer.py in _save(self, output_dir)
   1419 
   1420         # Good practice: save your training arguments together with the trained model
-> 1421         torch.save(self.args, os.path.join(output_dir, "training_args.bin"))
   1422 
   1423     def store_flos(self):

~/miniconda3/envs/transformers4/lib/python3.8/site-packages/torch/serialization.py in save(obj, f, pickle_module, pickle_protocol, _use_new_zipfile_serialization)
    370         if _use_new_zipfile_serialization:
    371             with _open_zipfile_writer(opened_file) as opened_zipfile:
--> 372                 _save(obj, opened_zipfile, pickle_module, pickle_protocol)
    373                 return
    374         _legacy_save(obj, opened_file, pickle_module, pickle_protocol)

~/miniconda3/envs/transformers4/lib/python3.8/site-packages/torch/serialization.py in _save(obj, zip_file, pickle_module, pickle_protocol)
    474     pickler = pickle_module.Pickler(data_buf, protocol=pickle_protocol)
    475     pickler.persistent_id = persistent_id
--> 476     pickler.dump(obj)
    477     data_value = data_buf.getvalue()
    478     zip_file.write_record('data.pkl', data_value, len(data_value))

TypeError: cannot pickle '_thread.lock' object
```
It doesn't matter the save steps, etc. When it tries to save the model, I get that error. I don't think that I can post the dataset here but it doesn't look like a dataset problem. I'm following the sequence classification notebook but I changed some things to use binary classification and load my own dataset.

I also get a different error when using the early stopping callback:

```python
from transformers import EarlyStoppingCallback

trainer = Trainer(
    model,
    training_arguments,
    train_dataset=encoded_dataset["train"],
    eval_dataset=encoded_dataset["validation"],
    tokenizer=tokenizer,
    compute_metrics=compute_metrics,
    callbacks=[EarlyStoppingCallback(early_stopping_patience=4)],
)
trainer.train()
```

```
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-46-17322e623d42> in <module>
     10     callbacks=[EarlyStoppingCallback(early_stopping_patience=4)], # Por alguna razón, casca
     11 )
---> 12 trainer.train()

~/.local/lib/python3.8/site-packages/transformers/trainer.py in train(self, resume_from_checkpoint, trial, **kwargs)
    981                     self.control = self.callback_handler.on_step_end(self.args, self.state, self.control)
    982 
--> 983                     self._maybe_log_save_evaluate(tr_loss, model, trial, epoch)
    984 
    985                 if self.control.should_epoch_stop or self.control.should_training_stop:

~/.local/lib/python3.8/site-packages/transformers/trainer.py in _maybe_log_save_evaluate(self, tr_loss, model, trial, epoch)
   1056         metrics = None
   1057         if self.control.should_evaluate:
-> 1058             metrics = self.evaluate()
   1059             self._report_to_hp_search(trial, epoch, metrics)
   1060 

~/.local/lib/python3.8/site-packages/transformers/trainer.py in evaluate(self, eval_dataset, ignore_keys, metric_key_prefix)
   1522             xm.master_print(met.metrics_report())
   1523 
-> 1524         self.control = self.callback_handler.on_evaluate(self.args, self.state, self.control, output.metrics)
   1525         return output.metrics
   1526 

~/.local/lib/python3.8/site-packages/transformers/trainer_callback.py in on_evaluate(self, args, state, control, metrics)
    360     def on_evaluate(self, args: TrainingArguments, state: TrainerState, control: TrainerControl, metrics):
    361         control.should_evaluate = False
--> 362         return self.call_event("on_evaluate", args, state, control, metrics=metrics)
    363 
    364     def on_save(self, args: TrainingArguments, state: TrainerState, control: TrainerControl):

~/.local/lib/python3.8/site-packages/transformers/trainer_callback.py in call_event(self, event, args, state, control, **kwargs)
    375     def call_event(self, event, args, state, control, **kwargs):
    376         for callback in self.callbacks:
--> 377             result = getattr(callback, event)(
    378                 args,
    379                 state,

~/.local/lib/python3.8/site-packages/transformers/trainer_callback.py in on_evaluate(self, args, state, control, metrics, **kwargs)
    527     def on_evaluate(self, args, state, control, metrics, **kwargs):
    528         metric_to_check = args.metric_for_best_model
--> 529         if not metric_to_check.startswith("eval_"):
    530             metric_to_check = f"eval_{metric_to_check}"
    531         metric_value = metrics.get(metric_to_check)

AttributeError: 'Accuracy' object has no attribute 'startswith'

```


Comments:
As indicated in the [documentation](https://huggingface.co/transformers/main_classes/trainer.html#trainingarguments) `metric_for_best_model` _Must be the name of a metric returned by the evaluation with or without the prefix "eval\_"_. You passed a Metric object to it instead of the name returned by your `compute_metric` function, which is what caused your error.

Thank you so much, I'm sorry for that