TypeError: 'ModelMetaclass' object is not iterable 
Author: andysingal
Pull request: no

### System Info

RTX 3090

### Who can help?

@younesbelkada while working on:
```
import os
from dataclasses import dataclass, field
from typing import Optional
from datasets.arrow_dataset import Dataset
import torch
from datasets import load_dataset
from peft import LoraConfig
from peft import AutoPeftModelForCausalLM
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
    HfArgumentParser,
    AutoTokenizer,
    TrainingArguments,
)
from pydantic_settings import BaseSettings

from trl import SFTTrainer


torch.manual_seed(42)

# @dataclass
class ScriptArguments(BaseSettings):
    """
    These arguments vary depending on how many GPUs you have, what their capacity and features are, and what size model you want to train.
    """

    local_rank: Optional[int] = field(default=-1, metadata={"help": "Used for multi-gpu"})

    per_device_train_batch_size: Optional[int] = field(default=4)
    per_device_eval_batch_size: Optional[int] = field(default=4)
    gradient_accumulation_steps: Optional[int] = field(default=4)
    learning_rate: Optional[float] = field(default=2e-5)
    max_grad_norm: Optional[float] = field(default=0.3)
    weight_decay: Optional[int] = field(default=0.01)
    lora_alpha: Optional[int] = field(default=16)
    lora_dropout: Optional[float] = field(default=0.1)
    lora_r: Optional[int] = field(default=32)
    max_seq_length: Optional[int] = field(default=512)
    model_name: Optional[str] = field(
        default="mistralai/Mistral-7B-Instruct-v0.1",
        metadata={
            "help": "The model that you want to train from the Hugging Face hub. E.g. gpt2, gpt2-xl, bert, etc."
        }
    )
    dataset_name: Optional[str] = field(
        default="iamtarun/python_code_instructions_18k_alpaca",
        metadata={"help": "The preference dataset to use."},
    )

    use_4bit: Optional[bool] = field(
        default=True,
        metadata={"help": "Activate 4bit precision base model loading"},
    )
    use_nested_quant: Optional[bool] = field(
        default=False,
        metadata={"help": "Activate nested quantization for 4bit base models"},
    )
    bnb_4bit_compute_dtype: Optional[str] = field(
        default="float16",
        metadata={"help": "Compute dtype for 4bit base models"},
    )
    bnb_4bit_quant_type: Optional[str] = field(
        default="nf4",
        metadata={"help": "Quantization type fp4 or nf4"},
    )
    num_train_epochs: Optional[int] = field(
        default=100,
        metadata={"help": "The number of training epochs for the reward model."},
    )
    fp16: Optional[bool] = field(
        default=False,
        metadata={"help": "Enables fp16 training."},
    )
    bf16: Optional[bool] = field(
        default=True,
        metadata={"help": "Enables bf16 training."},
    )
    packing: Optional[bool] = field(
        default=False,
        metadata={"help": "Use packing dataset creating."},
    )
    gradient_checkpointing: Optional[bool] = field(
        default=True,
        metadata={"help": "Enables gradient checkpointing."},
    )
    optim: Optional[str] = field(
        default="paged_adamw_32bit",
        metadata={"help": "The optimizer to use."},
    )
    lr_scheduler_type: str = field(
        default="constant",
        metadata={"help": "Learning rate schedule. Constant a bit better than cosine, and has advantage for analysis"},
    )
    max_steps: int = field(default=1000000, metadata={"help": "How many optimizer update steps to take"})
    warmup_ratio: float = field(default=0.03, metadata={"help": "Fraction of steps to do a warmup for"})
    group_by_length: bool = field(
        default=True,
        metadata={
            "help": "Group sequences into batches with same length. Saves memory and speeds up training considerably."
        },
    )
    save_steps: int = field(default=50, metadata={"help": "Save checkpoint every X updates steps."})
    logging_steps: int = field(default=50, metadata={"help": "Log every X updates steps."})
    merge_and_push: Optional[bool] = field(
        default=False,
        metadata={"help": "Merge and push weights after training"},
    )
    output_dir: str = field(
        default="./results_packing",
        metadata={"help": "The output directory where the model predictions and checkpoints will be written."},
    )


parser = HfArgumentParser(ScriptArguments)
script_args = parser.parse_args_into_dataclasses()[0]
```
ERROR:
```
/usr/local/lib/python3.10/dist-packages/trl/trainer/ppo_config.py:141: UserWarning: The `optimize_cuda_cache` arguement will be deprecated soon, please use `optimize_device_cache` instead.
  warnings.warn(
/usr/local/lib/python3.10/dist-packages/pydantic/_internal/_fields.py:149: UserWarning: Field "model_name" has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ('settings_',)`.
  warnings.warn(
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[2], line 117
    107     merge_and_push: Optional[bool] = field(
    108         default=False,
    109         metadata={"help": "Merge and push weights after training"},
    110     )
    111     output_dir: str = field(
    112         default="./results_packing",
    113         metadata={"help": "The output directory where the model predictions and checkpoints will be written."},
    114     )
--> 117 parser = HfArgumentParser(ScriptArguments)
    118 script_args = parser.parse_args_into_dataclasses()[0]

File /usr/local/lib/python3.10/dist-packages/transformers/hf_argparser.py:134, in HfArgumentParser.__init__(self, dataclass_types, **kwargs)
    132 if dataclasses.is_dataclass(dataclass_types):
    133     dataclass_types = [dataclass_types]
--> 134 self.dataclass_types = list(dataclass_types)
    135 for dtype in self.dataclass_types:
    136     self._add_dataclass_arguments(dtype)

TypeError: 'ModelMetaclass' object is not iterable
```

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

shared about 
Addiitonally:
```
%pip install transformers peft bitsandbytes accelerate trl pydantic-settings --quiet
```

### Expected behavior

needs to run 
check: 
https://python.plainenglish.io/intruct-fine-tuning-mistral-7b-model-with-your-custom-data-7eb22921a483 

Comments:
Hi @andysingal 
Thanks for the issue! 
I think `HfArgumentParser` requires a `dataclass` - can you try to un-comment `# @dataclass` ?

> Hi @andysingal Thanks for the issue! I think `HfArgumentParser` requires a `dataclass` - can you try to un-comment `# @dataclass` ?

i tried, now getting error:
```
usage: ipykernel_launcher.py [-h] --local_rank LOCAL_RANK
                             --per_device_train_batch_size
                             PER_DEVICE_TRAIN_BATCH_SIZE
                             --per_device_eval_batch_size
                             PER_DEVICE_EVAL_BATCH_SIZE
                             --gradient_accumulation_steps
                             GRADIENT_ACCUMULATION_STEPS --learning_rate
                             LEARNING_RATE --max_grad_norm MAX_GRAD_NORM
                             --weight_decay WEIGHT_DECAY --lora_alpha
                             LORA_ALPHA --lora_dropout LORA_DROPOUT --lora_r
                             LORA_R --max_seq_length MAX_SEQ_LENGTH
                             --model_name MODEL_NAME --dataset_name
                             DATASET_NAME [--use_4bit USE_4BIT]
                             [--use_nested_quant USE_NESTED_QUANT]
                             --bnb_4bit_compute_dtype BNB_4BIT_COMPUTE_DTYPE
                             --bnb_4bit_quant_type BNB_4BIT_QUANT_TYPE
                             --num_train_epochs NUM_TRAIN_EPOCHS [--fp16 FP16]
                             [--bf16 BF16] [--packing PACKING]
                             [--gradient_checkpointing GRADIENT_CHECKPOINTING]
                             --optim OPTIM --lr_scheduler_type
                             LR_SCHEDULER_TYPE --max_steps MAX_STEPS
                             --warmup_ratio WARMUP_RATIO
                             [--group_by_length [GROUP_BY_LENGTH]]
                             --save_steps SAVE_STEPS --logging_steps
                             LOGGING_STEPS [--merge_and_push MERGE_AND_PUSH]
                             --output_dir OUTPUT_DIR
ipykernel_launcher.py: error: the following arguments are required: --local_rank, --per_device_train_batch_size, --per_device_eval_batch_size, --gradient_accumulation_steps, --learning_rate, --max_grad_norm, --weight_decay, --lora_alpha, --lora_dropout, --lora_r, --max_seq_length, --model_name, --dataset_name, --bnb_4bit_compute_dtype, --bnb_4bit_quant_type, --num_train_epochs, --optim, --lr_scheduler_type, --max_steps, --warmup_ratio, --save_steps, --logging_steps, --output_dir
An exception has occurred, use %tb to see the full traceback.

SystemExit: 2
/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py:3556: UserWarning: To exit: use 'exit', 'quit', or Ctrl-D.
  warn("To exit: use 'exit', 'quit', or Ctrl-D.", stacklevel=1)
```

Much better! Now this simply means that to run your script you need to provide the required arguments such as --local-rank, etc.

> Much better! Now this simply means that to run your script you need to provide the required arguments such as --local-rank, etc.

Thanks alot for instant reply, love to see how i can run the code in an efficient way... i was following: https://python.plainenglish.io/intruct-fine-tuning-mistral-7b-model-with-your-custom-data-7eb22921a483 

Thanks ! I guess the issue is solved now! Feel free to close it!

> > Much better! Now this simply means that to run your script you need to provide the required arguments such as --local-rank, etc.
> 
> Thanks alot for instant reply, love to see how i can run the code in an efficient way... i was following: https://python.plainenglish.io/intruct-fine-tuning-mistral-7b-model-with-your-custom-data-7eb22921a483

@younesbelkada if you find a workaround to resolve the issue, please keep me posted

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.