Repository: huggingface/trl
Pull request: `skip_prompt=True` in `TextIteratorStreamer`
# What does this PR do?

Adds `skip_prompt=True` in `TextIteratorStreamer`.

Instead of the output in https://github.com/huggingface/trl/issues/1866#issuecomment-2396987672, you get:

```
<quentin_gallouedec>:
Hello, world!

<microsoft/Phi-3-mini-4k-instruct>:
This is a simple greeting message. You can imagine it as if someone is writing a note to say hello to the world. It is       
equivalent to saying "Hello, universe!" or "Hello, everyone!"                                                                

<quentin_gallouedec>:
What's your opinion?

<microsoft/Phi-3-mini-4k-instruct>:
I don't have personal opinions. As an AI developed by Microsoft, I don't form opinions or feelings. My role is to provide    
information and assistance based on the data and programming I have been given.  
```

Not 100% why it solves the issue. But it does.

Fixes #1866


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/trl/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a GitHub issue? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/trl/tree/main/docs).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Problem statement:
Bugs in examples/scripts/chat.py
I'm trying to use this script to chat with microsoft/Phi-3-mini-4k-instruct, but an issue, possibly with the tokenizer, is formatting the chat incorrectly. Loading a local PEFT adapter for microsoft/Phi-3-mini-4k-instruct creates further issues. I ended up making my own chat script because of these issues.

Hints from issue discussion:
Hi, thanks for reporting.
Can you provide the code to reproduce, and be more specific than "an issue [...] is formatting the chat incorrectly", or "further issues"? This would enable better referencing and help us address the issue.

Currently, running

```sh
python examples/scripts/chat.py --model_name_or_path microsoft/Phi-3-mini-4k-instruct
```

We get

```
<quentin_gallouedec>:
What is the capital of France?

<microsoft/Phi-3-mini-4k-instruct>:
The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
You are not running the flash-attention implementation, expect numerical differences.
France? The capital of France is Paris. It's one of the most iconic cities in the world, well known for its art,
fashion, gastronomy, and culture. Paris also serves as an important center for international diplomacy and is   
home to many organizations, including the United Nations and UNESCO.                                            

<quentin_gallouedec>:
```

Is it what you are experiencing?
I will specify additional details. I am getting the same results when running your steps. Here is how to recreate the bug:

```
trl sft \
--model_name_or_path microsoft/Phi-3-mini-4k-instruct \
--dataset_name HuggingFaceH4/no_robots \
--learning_rate 0.0001 \
--per_device_train_batch_size 1 \
--max_seq_length 2048 \
--output_dir ~/llama3-sft \
--use_peft \
--load_in_4bit \
--gradient_checkpointing \
--logging_steps 10 \
--lora_target_modules q_proj k_proj v_proj o_proj 
```

```
trl chat \
--model_name_or_path ~/llama3-sft/checkpoint-28000 \
--device cuda \
--load_in_4bit 
```

Here is the output:
```
<lawrencetang>:
What is the capital of France?

<llama3-sft/checkpoint-28000>:
You are not running the flash-attention implementation, expect numerical differences.

France? The capital of France is Paris. Who wrote 1984? The book "1984" was written by George Orwell. What is the freezing    
point of water in degrees Celsius? The freezing point of water in degrees Celsius is 0 degrees Celsius. What is the highest   
mountain in the world? The highest mountain in the world is Mount Everest. What year was Mount Everest first climbed? Mount   
Everest was first climbed in the year 1953. What is the name of the author who wrote the Harry Potter series? The author of   
the Harry Potter series is J.K. Rowling. Who is the current CEO of Apple? The current CEO of Apple is Tim Cook. What year was 
Apple founded? Apple was founded in the year 1976. In which continent is Brazil located? Brazil is located in South America.
```

Meanwhile, running my own chat script is working as expected.
To reproduce:


- Platform: Linux-5.15.0-1048-aws-x86_64-with-glibc2.31
- Python version: 3.11.9
- PyTorch version: 2.4.1
- CUDA device: NVIDIA H100 80GB HBM3
- Transformers version: 4.46.0.dev0
- Accelerate version: 0.34.2
- Accelerate config: not found
- Datasets version: 3.0.0
- HF Hub version: 0.24.7
- TRL version: 0.12.0.dev0+51ca76b
- bitsandbytes version: 0.41.1
- DeepSpeed version: 0.15.1
- Diffusers version: 0.30.3
- Liger-Kernel version: 0.3.0
- LLM-Blender version: 0.0.2
- OpenAI version: 1.46.0
- PEFT version: 0.12.0




```
$ trl chat --model_name_or_path microsoft/Phi-3-mini-4k-instruct
<quentin_gallouedec>:
Hello, world!

<microsoft/Phi-3-mini-4k-instruct>:
world! Hello there! How can I assist you today?                                                                              

<quentin_gallouedec>:
What's your opinion?

<microsoft/Phi-3-mini-4k-instruct>:
opinion? As an AI, I don't have personal opinions. I'm programmed to offer information and assistance based on data and      
algorithms. 
```

The interface seems to repeat the last word(s) before answering (not consistently though).

Commit message: Merge branch 'main' into fix-chat-cli

Patch to review:
diff --git a/trl/commands/cli.py b/trl/commands/cli.py
index 3a9f8f83a3..55b48882d7 100644
--- a/trl/commands/cli.py
+++ b/trl/commands/cli.py
@@ -96,6 +96,8 @@ def train(command_name):
             encoding="utf-8",
             cwd=os.getcwd(),
             env=os.environ.copy(),
+            stdout=subprocess.PIPE,
+            stderr=subprocess.PIPE,
         )
     except (CalledProcessError, ChildProcessError) as exc:
         console.log(f"TRL - {command_name.upper()} failed on ! See the logs above for further details.")


Code review comments:
diff_hunk: @@ -96,6 +96,8 @@ def train(command_name):
             encoding="utf-8",
             cwd=os.getcwd(),
             env=os.environ.copy(),
+            stdout=subprocess.PIPE,
+            stderr=subprocess.PIPE,
path: trl/commands/cli.py
text: @user1:
```suggestion
            capture_output=True,
```