Repository: hpcaitech/ColossalAI

Issue:
[tensor] fix some unittests

[tensor] fix some unittests

[tensor] fix some unittests

[BUG]: ChatGPT - There is a possibility that the generation is not working properly.
### 🐛 Describe the bug

`generate_kwargs` is not properly passed to `PPOTrainer`. So, generating sentences might be incorrect. (At leat in my case)

https://github.com/hpcaitech/ColossalAI/blob/5d5f475d758347b5e61dbb4b0ccb6108821e3e93/applications/ChatGPT/chatgpt/trainer/ppo.py#L64-L76

Like Huggingface GPT2 Model, `prepare_inputs_fn` and `update_model_kwargs_fn` should be passed to generate. But in that code, these functions are not applied because `_set_default_generate_kwargs()` is called after `super().__init__()`

So, I think the order of call should be changed.

### Environment

_No response_


Discussion:



Hi, `_set_default_generate_kwargs()` will modify `generate_kwargs` inplace, so this is OK. But I agree the order is bad and we will update this. 

Thanks for your suggestion!
But when `super().__init__()` is calling, all keyword arguments are unpacked and passed to the super class. So, I think modifying the arguments will not propagate to super class.

Like this:

```python
class Parent:
    def __init__(self, **kwargs):
        self.kwargs = kwargs

class Child(Parent):
    def __init__(self, **kwargs):
        super().__init__(**kwargs)

        kwargs['this_had_to_be_contained'] = 1

instance = Child(param1=0, param2=1)

# Print "{'param1': 0, 'param2': 1}"
print(instance.kwargs)
```
Can you create a PR to solve this problem? All contribution is welcome.

Resolving patch:
diff --git a/applications/ChatGPT/chatgpt/trainer/ppo.py b/applications/ChatGPT/chatgpt/trainer/ppo.py
--- a/applications/ChatGPT/chatgpt/trainer/ppo.py
+++ b/applications/ChatGPT/chatgpt/trainer/ppo.py
@@ -63,6 +63,7 @@ def __init__(self,
                  **generate_kwargs) -> None:
         experience_maker = NaiveExperienceMaker(actor, critic, reward_model, initial_model, kl_coef)
         replay_buffer = NaiveReplayBuffer(train_batch_size, buffer_limit, buffer_cpu_offload)
+        generate_kwargs = _set_default_generate_kwargs(strategy, generate_kwargs, actor)
         super().__init__(strategy, experience_maker, replay_buffer, experience_batch_size, max_epochs, tokenizer,
                          sample_replay_buffer, dataloader_pin_memory, callbacks, **generate_kwargs)
         self.actor = actor
@@ -73,7 +74,6 @@ def __init__(self,
 
         self.actor_optim = actor_optim
         self.critic_optim = critic_optim
-        self._set_default_generate_kwargs(generate_kwargs, actor)
 
     def training_step(self, experience: Experience) -> Dict[str, float]:
         self.actor.train()
@@ -102,11 +102,15 @@ def training_step(self, experience: Experience) -> Dict[str, float]:
 
         return {'actor_loss': actor_loss.item(), 'critic_loss': critic_loss.item()}
 
-    def _set_default_generate_kwargs(self, generate_kwargs: dict, actor: Actor) -> None:
-        origin_model = self.strategy._unwrap_actor(actor)
-        # use huggingface models method directly
-        if 'prepare_inputs_fn' not in generate_kwargs and hasattr(origin_model, 'prepare_inputs_for_generation'):
-            generate_kwargs['prepare_inputs_fn'] = origin_model.prepare_inputs_for_generation
 
-        if 'update_model_kwargs_fn' not in generate_kwargs:
-            generate_kwargs['update_model_kwargs_fn'] = update_model_kwargs_fn
+def _set_default_generate_kwargs(strategy: Strategy, generate_kwargs: dict, actor: Actor) -> None:
+    origin_model = strategy._unwrap_actor(actor)
+    new_kwargs = {**generate_kwargs}
+    # use huggingface models method directly
+    if 'prepare_inputs_fn' not in generate_kwargs and hasattr(origin_model, 'prepare_inputs_for_generation'):
+        new_kwargs['prepare_inputs_fn'] = origin_model.prepare_inputs_for_generation
+
+    if 'update_model_kwargs_fn' not in generate_kwargs:
+        new_kwargs['update_model_kwargs_fn'] = update_model_kwargs_fn
+
+    return new_kwargs


Test patch: