Fail to run run_classification.py example
Author: pengwa
Pull request: no

### System Info

transformers: 4.35.0.dev0, 8211c59b9a8fe84d2861446b26542f89a0260e64

Name: torch： Version: 2.2.0.dev20231007+cu118


Repro command:
```
torchrun --nproc_per_node $num_gpus \
    examples/pytorch/text-classification/run_classification.py  \
        --model_name_or_path meta-llama/Llama-2-7b-hf \
        --dataset_name dair-ai/emotion \
        --dataset_config_name split \
        --shuffle_train_dataset --metric_name accuracy --text_column_name text --label_column_name label \
        --max_seq_length 512 \
        --per_device_train_batch_size 10 \
        --per_device_eval_batch_size 1 \
        --do_train \
        --fp16 \
        --output_dir /tmp/test-clssification --overwrite_output_dir \
        --report_to none \
        --max_steps 300 --logging_steps 1 \
        --token $HUGGINGFACE_TOKEN

```

Error messages:
```
[INFO|modeling_utils.py:3861] 2023-10-31 02:03:32,535 >> Some weights of the model checkpoint at meta-llama/Llama-2-7b-hf were not used when initializing LlamaForSequenceClassification: ['lm_head.weight']
- This IS expected if you are initializing LlamaForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing LlamaForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
[WARNING|modeling_utils.py:3873] 2023-10-31 02:03:32,535 >> Some weights of LlamaForSequenceClassification were not initialized from the model checkpoint at meta-llama/Llama-2-7b-hf and are newly initialized: ['score.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
2023-10-31 02:03:32,538 __main__ [WARNING] - The label2id key in the model config.json is not equal to the label2id key of this run. You can ignore this if you are doing finetuning.
Running tokenizer on dataset:   0%|                                                                                                                                                                                                                                                    | 0/16000 [00:00<?, ? examples/s]
Traceback (most recent call last):
  File "examples/pytorch/text-classification/run_classification.py", line 763, in <module>
    main()
  File "examples/pytorch/text-classification/run_classification.py", line 588, in main
    raw_datasets = raw_datasets.map(
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/datasets/dataset_dict.py", line 853, in map
    {
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/datasets/dataset_dict.py", line 854, in <dictcomp>
    k: dataset.map(
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 592, in wrapper
    out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 557, in wrapper
    out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 3097, in map
    for rank, done, content in Dataset._map_single(**dataset_kwargs):
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 3474, in _map_single
    batch = apply_function_on_filtered_inputs(
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 3353, in apply_function_on_filtered_inputs
    processed_inputs = function(*fn_args, *additional_args, **fn_kwargs)
  File "examples/pytorch/text-classification/run_classification.py", line 578, in preprocess_function
    result = tokenizer(examples["sentence"], padding=padding, max_length=max_seq_length, truncation=True)
  File "/tmp/transformers/src/transformers/tokenization_utils_base.py", line 2798, in __call__
    encodings = self._call_one(text=text, text_pair=text_pair, **all_kwargs)
  File "/tmp/transformers/src/transformers/tokenization_utils_base.py", line 2884, in _call_one
    return self.batch_encode_plus(
  File "/tmp/transformers/src/transformers/tokenization_utils_base.py", line 3066, in batch_encode_plus
    padding_strategy, truncation_strategy, max_length, kwargs = self._get_padding_truncation_strategies(
  File "/tmp/transformers/src/transformers/tokenization_utils_base.py", line 2703, in _get_padding_truncation_strategies
    raise ValueError(
ValueError: Asking to pad but the tokenizer does not have a padding token. Please select a token to use as `pad_token` `(tokenizer.pad_token = tokenizer.eos_token e.g.)` or add a new pad token via `tokenizer.add_special_tokens({'pad_token': '[PAD]'})`.
[2023-10-31 02:03:38,731] torch.distributed.elastic.multiprocessing.api: [ERROR] failed (exitcode: 1) local_rank: 0 (pid: 29880) of binary: /opt/conda/envs/ptca/bin/python
Traceback (most recent call last):
  File "/opt/conda/envs/ptca/bin/torchrun", line 8, in <module>
    sys.exit(main())
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/torch/distributed/run.py", line 806, in main
    run(args)
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/torch/distributed/run.py", line 797, in run
    elastic_launch(
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 134, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 264, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
============================================================
examples/pytorch/text-classification/run_classification.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2023-10-31_02:03:38
  host      : node-0
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 29880)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html


```

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Run the commandl above using 1 or more than 1 gpus, 
2. it failed with above messages. 

### Expected behavior

The expected behavior is to run successfully. 

Comments:
Hi @pengwa, thanks for raising this issue! 

From the error message you can see that an exception is raised here: 

```
  File "/tmp/transformers/src/transformers/tokenization_utils_base.py", line 2703, in _get_padding_truncation_strategies
    raise ValueError(
ValueError: Asking to pad but the tokenizer does not have a padding token. Please select a token to use as `pad_token` `(tokenizer.pad_token = tokenizer.eos_token e.g.)` or add a new pad token via `tokenizer.add_special_tokens({'pad_token': '[PAD]'})`.
```

This indicates that the tokenizer being loaded with `meta-llama/Llama-2-7b-hf` doesn't have a pad_token assigned and is required for processing the text. You'll need to modify the script to add a pad token to the tokenizer as per the example in the error message. 

I see, thank @amyeroberts for the quick response!

Do you mean something like this?

```diff

    tokenizer = AutoTokenizer.from_pretrained(
        model_args.tokenizer_name if model_args.tokenizer_name else model_args.model_name_or_path,
        cache_dir=model_args.cache_dir,
        use_fast=model_args.use_fast_tokenizer,
        revision=model_args.model_revision,
        token=model_args.token,
        trust_remote_code=model_args.trust_remote_code,
    )
+    tokenizer.pad_token = tokenizer.eos_token

+ model.config.pad_token_id = tokenizer.pad_token
    model = AutoModelForSequenceClassification.from_pretrained(
        model_args.model_name_or_path,
        from_tf=bool(".ckpt" in model_args.model_name_or_path),
        config=config,
        cache_dir=model_args.cache_dir,
        revision=model_args.model_revision,
        token=model_args.token,
        trust_remote_code=model_args.trust_remote_code,
        ignore_mismatched_sizes=model_args.ignore_mismatched_sizes,
        load_in_4bit=True,
        quantization_config=bnb_config,
    )
    
```

Sometimes I see  model.config.pad_token_id = tokenizer.pad_token is done after `model = AutoModelForSequenceClassification.from_pretrained`, for example "https://github.com/huggingface/transformers/blob/HEAD/src/transformers/generation/utils.py#L2691-L2692", is it too late to let model be aware of the token id ? 

@pengwa, yes, something like that. You'll need to do `model.config.pad_token = tokenizer.pad_token` after the `model = AutoModelForSequenceClassification.from_pretrained` call as the model object doesn't exist until after the from_pretrained call. 

> @pengwa, yes, something like that. You'll need to do `model.config.pad_token = tokenizer.pad_token` after the `model = AutoModelForSequenceClassification.from_pretrained` call as the model object doesn't exist until after the from_pretrained call.

Really appreciate your claficiation on this!!! @amyeroberts.

Another quick question, it seems doing in below two approaches are equivalent in model definition, while one difference is: in approach 2  when model is initialization nn.embedding in its constructor, it can pass padding token id in nn.Embedding apis, am I right?

Approach 1： 

```diff
    model = AutoModelForSequenceClassification.from_pretrained(
        model_args.model_name_or_path,
        from_tf=bool(".ckpt" in model_args.model_name_or_path),
        config=config,
        cache_dir=model_args.cache_dir,
        revision=model_args.model_revision,
        token=model_args.token,
        trust_remote_code=model_args.trust_remote_code,
        ignore_mismatched_sizes=model_args.ignore_mismatched_sizes,
        load_in_4bit=True,
        quantization_config=bnb_config,
    )
+    model.config.pad_token_id = tokenizer.pad_token
``` 

Approach 2： 


```diff
+    config.pad_token_id = tokenizer.pad_token
    model = AutoModelForSequenceClassification.from_pretrained(
        model_args.model_name_or_path,
        from_tf=bool(".ckpt" in model_args.model_name_or_path),
        config=config,
        cache_dir=model_args.cache_dir,
        revision=model_args.model_revision,
        token=model_args.token,
        trust_remote_code=model_args.trust_remote_code,
        ignore_mismatched_sizes=model_args.ignore_mismatched_sizes,
        load_in_4bit=True,
        quantization_config=bnb_config,
    )
``` 

Hi @pengwa, depending on the model, they're not necessarily equivalent. 

The model uses the config to construct itself. Therefore, changing the config input can change how the model is constructed, and logic dependant on the config input wouldn't be affected if the config values are created after model instantiation. For example `padding_idx` [set here](https://github.com/huggingface/transformers/blob/e9dbd3926317a4effb1d033d8454ff18280d0b7d/src/transformers/models/llama/modeling_llama.py#L819) for Llama. 

If possible, the config should be updated first and then passed to the model.  



> Hi @pengwa, depending on the model, they're not necessarily equivalent.
> 
> The model uses the config to construct itself. Therefore, changing the config input can change how the model is constructed, and logic dependant on the config input wouldn't be affected if the config values are created after model instantiation. For example `padding_idx` [set here](https://github.com/huggingface/transformers/blob/e9dbd3926317a4effb1d033d8454ff18280d0b7d/src/transformers/models/llama/modeling_llama.py#L819) for Llama.
> 
> If possible, the config should be updated first and then passed to the model.

Thank you @amyeroberts .  `If possible, the config should be updated first and then passed to the model.` , this is exactly what I am assuming. Thanks a lot for the help! Closing this issue.