Repository: huggingface/trl

Issue:
sft_llama2 error
Hello! 

I made no changes to the sample code or the dataset. I just simply wanted to get it to run the stacked llama2 example. However, I get this error:

UserWarning: The passed formatting_func has more than one argument. Usually that function should have a single argument `example` which corresponds to the dictionary returned by each element of the dataset. Make sure you know what you are doing.

 in __init__
    raise ValueError("the `--group_by_length` option is only available for `Dataset`, not `IterableDataset")
ValueError: the `--group_by_length` option is only available for `Dataset`, not `IterableDataset

(Sorry, I must omit certain parts of the error message since I am working on an institution's server). 

Thank you!


Discussion:
Can someone please just follow the example in https://github.com/huggingface/trl/tree/main/examples/research_projects/stack_llama_2/scripts ? (and don't change the code or the dataset)

I am sure this error is reproducible. 

Thank you!
Are you using the constant length dataset? That's indeed incompatible with `--group_by_length` as this requires globally sorting the dataset whereas with the constant length dataset the dataset is generated on the fly and thus not globally sortable.
@lvwerra Thank you for your quick reply! Honestly, I did not change a single line of code in https://github.com/huggingface/trl/tree/main/examples/research_projects/stack_llama_2/scripts (which is a very new tutorial published 2 days ago).  
I am using the default "lvwerra/stack-exchange-paired" as the dataset. 
I checked sft_llama2.py and I see that it is indeed using constant length dataset (https://github.com/huggingface/trl/blob/3b2c820db68aecca36c2ff5d152de59deec8fe5c/examples/research_projects/stack_llama_2/scripts/sft_llama2.py#L118). However, the default flag for --group_by_length is also set to "True" (https://github.com/huggingface/trl/blob/3b2c820db68aecca36c2ff5d152de59deec8fe5c/examples/research_projects/stack_llama_2/scripts/sft_llama2.py#L38C5-L38C34)
Does it work if you set `--group_by_length` to `False`?
It is training now after I set --group_by_length to False. Hopefully this is how it is supposed to work! Thank you so much @lvwerra  . Sorry, I was working on another project recently, so I was not checking my email linked to my GitHub
@younesbelkada I think we should catch that combination of args and throw an error before we start training. What do you think?
Sounds good, will propose something!

Resolving patch:
diff --git a/examples/research_projects/stack_llama_2/scripts/sft_llama2.py b/examples/research_projects/stack_llama_2/scripts/sft_llama2.py
--- a/examples/research_projects/stack_llama_2/scripts/sft_llama2.py
+++ b/examples/research_projects/stack_llama_2/scripts/sft_llama2.py
@@ -35,7 +35,8 @@ class ScriptArguments:
     gradient_checkpointing: Optional[bool] = field(
         default=True, metadata={"help": "whether to use gradient checkpointing"}
     )
-    group_by_length: Optional[bool] = field(default=True, metadata={"help": "whether to group by length"})
+    group_by_length: Optional[bool] = field(default=False, metadata={"help": "whether to group by length"})
+    packing: Optional[bool] = field(default=True, metadata={"help": "whether to use packing for SFTTrainer"})
 
     lora_alpha: Optional[float] = field(default=16, metadata={"help": "the lora alpha parameter"})
     lora_dropout: Optional[float] = field(default=0.05, metadata={"help": "the lora dropout parameter"})
@@ -54,6 +55,9 @@ class ScriptArguments:
 parser = HfArgumentParser(ScriptArguments)
 script_args = parser.parse_args_into_dataclasses()[0]
 
+if script_args.group_by_length and script_args.packing:
+    raise ValueError("Cannot use both packing and group by length")
+
 
 def chars_token_ratio(dataset, tokenizer, nb_examples=400):
     """
@@ -189,7 +193,7 @@ def create_datasets(tokenizer, args):
     train_dataset=train_dataset,
     eval_dataset=eval_dataset,
     peft_config=peft_config,
-    packing=True,
+    packing=script_args.packing,
     max_seq_length=None,
     tokenizer=tokenizer,
     args=training_args,


Test patch: