Repository: huggingface/transformers

Issue:
`label_to_id` in `run_glue.py` seems to have a wrong `if` statement
## Environment info

- `transformers` version: 4.3.0.dev0
- Platform: Linux-4.4.0-179-generic-x86_64-with-glibc2.10
- Python version: 3.8.0
- PyTorch version (GPU?): 1.7.1 (False)
- Tensorflow version (GPU?): 2.4.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj

## Information

Model I am using (Bert, XLNet ...): Bert, xlm-roberta-large

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

It seems the `if` statement for `label_to_id` seems to be strange.

https://github.com/huggingface/transformers/blob/eba418ac5df71d08927efb7e3b738833998162ff/examples/text-classification/run_glue.py#L316-L333

About `and is_regression` in L320, should not it be `and not is_regression`?

I inserted `logging.info` to check the True/False as below:

```python
    label_to_id = None
    logger.info("--- label_to_id if statement check ---")
    logger.info(f"{model.config.label2id != PretrainedConfig(num_labels=num_labels).label2id}")
    logger.info(f"{data_args.task_name is not None}")
    logger.info(f"{is_regression}")
    
    if (
        model.config.label2id != PretrainedConfig(num_labels=num_labels).label2id
        and data_args.task_name is not None
        and is_regression
    ):
        logger.info("loading model.config.label2id")
        # Some have all caps in their config, some don't.
        label_name_to_id = {k.lower(): v for k, v in model.config.label2id.items()}
```

Then I got:

```python
01/27/2021 03:23:24 - INFO - __main__ -   --- label_to_id if statement check ---
01/27/2021 03:23:24 - INFO - __main__ -   False
01/27/2021 03:23:24 - INFO - __main__ -   True
01/27/2021 03:23:24 - INFO - __main__ -   False
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00,  6.02ba/s]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 20.86ba/s]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 12.80ba/s]
01/27/2021 03:23:25 - INFO - __main__ -   Sample 2619 of the training set: {'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'idx': 2916, 'input_ids': [0, 581, 172337, 5180, 3542, 39958, 1257, 678, 117303, 1010, 22230, 1810, 150, 592, 2363, 7225, 26548, 2022, 112478, 6, 4, 16454, 3912, 37967, 111, 60525, 1810, 150, 592, 747, 125682, 7, 26548, 4049, 6, 5, 2, 2, 84607, 26420, 5180, 3542, 39958, 1257, 678, 117303, 1010, 22230, 1810, 150, 592, 2363, 7225, 26548, 2022, 112478, 6, 4, 16454, 10, 3912, 9, 22469, 94309, 1363, 31330, 47, 70, 29685, 6, 5, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'label': 1, 'sentence1': 'The proceedings were taken up with prosecutors outlining their case against Amrozi , reading 33 pages of documents outlining allegations against him .', 'sentence2': 'Proceedings were taken up with prosecutors outlining their case against Amrozi , reading a 33-page accusation letter to the court .'}.
```

When `is_regression` is False (when the task is `classification`), `model.config.label2id` is never used for `label_to_id`.
If I'm not mistaken, would not this behave differently than what is intended?

I am sorry that I could not find an appropriate task/model combination to show when all other conditions would be true.

Thank you in advance.


Discussion:
Yes, there should be a not here. Do you want to open a PR since you found the problem and its fix?
Thanks, I'd love to open a PR! Please wait a minute.

Resolving patch:
diff --git a/examples/text-classification/run_glue.py b/examples/text-classification/run_glue.py
--- a/examples/text-classification/run_glue.py
+++ b/examples/text-classification/run_glue.py
@@ -317,7 +317,7 @@ def main():
     if (
         model.config.label2id != PretrainedConfig(num_labels=num_labels).label2id
         and data_args.task_name is not None
-        and is_regression
+        and not is_regression
     ):
         # Some have all caps in their config, some don't.
         label_name_to_id = {k.lower(): v for k, v in model.config.label2id.items()}


Test patch: