token-classification example looses ner_tags labels in trained model
Author: mpost
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.16.2
- Platform: Linux-5.10.81.1-microsoft-standard-WSL2-x86_64-with-debian-bullseye-sid
- Python version: 3.6.13
- PyTorch version (GPU?): 1.10.1+cu102 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help

 @sgugger, @patil-suraj

## Information

Model I am using (Bert, XLNet ...): distilbert-base-uncased

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Run script run_ner.py with for example the https://huggingface.co/datasets/darentang/sroie dataset
2. The generated model does not contain expected id2label mapping but the labels are also the id numbers. The correct names are available in the `'ner_tags'` column of the dataset.

The incorrect mapping happens when [setting up the config](https://github.com/huggingface/transformers/blob/master/examples/pytorch/token-classification/run_ner.py#L312). It only feeds the number of labels and not a proper label_to_id/id_to_label mapping.

Here is a rough patch to work around it and to illustrate the issue:

```patch
Index: examples/pytorch/token-classification/run_ner.py
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/examples/pytorch/token-classification/run_ner.py b/examples/pytorch/token-classification/run_ner.py
--- a/examples/pytorch/token-classification/run_ner.py	(revision 1d94d575461a76cb1dcb3ebe6e85f1c85d1dafcd)
+++ b/examples/pytorch/token-classification/run_ner.py	(date 1643797832215)
@@ -48,7 +48,7 @@
 
 
 # Will error if the minimal version of Transformers is not installed. Remove at your own risks.
-check_min_version("4.17.0.dev0")
+check_min_version("4.16.2.dev0")
 
 require_version("datasets>=1.8.0", "To fix: pip install -r examples/pytorch/token-classification/requirements.txt")
 
@@ -309,9 +309,13 @@
     # Distributed training:
     # The .from_pretrained methods guarantee that only one local process can concurrently
     # download model & vocab.
+    id_to_label = {str(i): label for i, label in enumerate(label_list)}
+    label_to_id = {v: k for k, v in id_to_label.items()}
     config = AutoConfig.from_pretrained(
         model_args.config_name if model_args.config_name else model_args.model_name_or_path,
         num_labels=num_labels,
+        id2label=id_to_label,
+        label2id=label_to_id,
         finetuning_task=data_args.task_name,
         cache_dir=model_args.cache_dir,
         revision=model_args.model_revision,
@@ -354,22 +358,6 @@
             "requirement"
         )
 
-    if model.config.label2id != PretrainedConfig(num_labels=num_labels).label2id:
-        label_name_to_id = {k: v for k, v in model.config.label2id.items()}
-        if list(sorted(label_name_to_id.keys())) == list(sorted(label_list)):
-            label_to_id = {k: int(label_name_to_id[k]) for k in label_keys}
-        else:
-            logger.warning(
-                "Your model seems to have been trained with labels, but they don't match the dataset: ",
-                f"model labels: {list(sorted(label_name_to_id.keys()))}, dataset labels: {list(sorted(label_list))}."
-                "\nIgnoring the model labels as a result.",
-            )
-    else:
-        label_to_id = {k: i for i, k in enumerate(label_keys)}
-
-    model.config.label2id = label_to_id
-    model.config.id2label = {i: l for l, i in label_to_id.items()}
-
     # Map that sends B-Xxx label to its I-Xxx counterpart
     b_to_i_label = []
     for idx, label in enumerate(label_list):
@@ -404,12 +392,12 @@
                     label_ids.append(-100)
                 # We set the label for the first token of each word.
                 elif word_idx != previous_word_idx:
-                    label_ids.append(label_to_id[label[word_idx]])
+                    label_ids.append(label[word_idx])
                 # For the other tokens in a word, we set the label to either the current label or -100, depending on
                 # the label_all_tokens flag.
                 else:
                     if data_args.label_all_tokens:
-                        label_ids.append(b_to_i_label[label_to_id[label[word_idx]]])
+                        label_ids.append(b_to_i_label[label[word_idx]])
                     else:
                         label_ids.append(-100)
                 previous_word_idx = word_idx

```
## Expected behavior

The label2id map should be properly populated by the `'ner_tags'`.


Comments:
I can reproduce and see the problem. The fix is a tiny bit more complex than what you're suggesting (to work in all settings), I'll work on that this morning.

Thanks for looking into it. I expected my solution to be a bit to focused. Great to see such a quick turnaround. :)

@sgugger While running the updated example it became apparent that it solves the issue correctly but leads to unexpected logs.

Since the initial [loading of the AutoConfig](https://github.com/huggingface/transformers/blob/45cac3fade34cb7134b080c5060c250f810db5e2/examples/pytorch/token-classification/run_ner.py#L315) is done without id2Labels/labelsToId the logs print a relation like `1: Label_1,...`. Only after the loading process is the `config.model.label2id/id2label` corrected. 

It would be great if the config could be correctly configured upfront, leading to correct logs. Atm it is a bit confusing since what you see is not what you get.

That's nothing something we can do as the log is generated by the config load, but we need to analyze the labels that may be present in that config in order to finalize the `label2id`/`id2label` fields.
We can add another log when we update the config however. Would you like to make a PR for that?