Repository: PaddlePaddle/PaddleNLP
Pull request: fix trainer during saving checkpoints
<!-- Demo: https://github.com/PaddlePaddle/PaddleNLP/pull/26 -->
### PR types
<!-- One of [ New features | Bug fixes | Function optimization | Performance optimization | Breaking changes | Others ] -->
Bug fixes

### PR changes
<!-- One of [ Models | APIs | Docs | Others ] -->
Others

### Description
<!-- Describe what this PR does -->
* fix trainer during saving checkpoints for single gpu.
* This resolves #6575


Problem statement:
[Bug]: 单卡微调gpt-3模型，出现GPTTrainer获取不到dp_group
### 软件环境

```Markdown
- paddlepaddle:2.5.0
- paddlepaddle-xpu: 2.5.0
- paddlenlp: 2.6.0rc0.post0
```


### 重复问题

- [X] I have searched the existing issues

### 错误描述

```Markdown
单卡微调时，通过paddle.distributed.get_world_size()获取world_size的值为1，
无法对dp_group属性进行赋值。只有当world_size大于1时，才会赋值。
导致保存模型参数时，会找不到属性报错。
[图片]
```


### 稳定复现步骤 & 代码

export PYTHONPATH="../../PaddleNLP/"

log_dir="log"
rm -rf $log_dir

python3 -m paddle.distributed.launch \
    --xpus "0" \
    --log_dir ${log_dir} \
    finetune_generation.py \
    --model_type "gpt" \
    --model_name_or_path gpt2-medium-en \
    --output_dir "output/$task_name" \
    --per_device_train_batch_size 2 \
    --per_device_eval_batch_size 1 \
    --tensor_parallel_degree 1 \
    --pipeline_parallel_degree 1 \
    --scale_loss 1024 \
    --learning_rate 0.00001 \
    --max_steps 10000 \
    --save_steps 5000 \
    --weight_decay 0.01 \
    --warmup_ratio 0.01 \
    --max_grad_norm 1.0 \
    --logging_steps 1\
    --dataloader_num_workers 1 \
    --sharding "stage2" \
    --max_evaluate_steps 1000 \
    --eval_steps 1000 \
    --report_to "visualdl" \
    --disable_tqdm true \
    --recompute 1 \
    --gradient_accumulation_steps 2 \
    --do_train \
    --do_eval \
    --device "xpu"
执行以上代码，问题必现。
解决方案：可以在获取属性之前，通过hasattr判断属性是否存在。

Hints from issue discussion:
你贴一下 报错吧。

Commit message: fix

Patch to review:
diff --git a/paddlenlp/trainer/training_args.py b/paddlenlp/trainer/training_args.py
index 39dc07a7512a..828288c1e3fc 100644
--- a/paddlenlp/trainer/training_args.py
+++ b/paddlenlp/trainer/training_args.py
@@ -766,8 +766,9 @@ def __post_init__(self):
         if len(self.sharding) == 0 and self.sharding_parallel_degree > 0:
             warnings.warn("`--sharding_parallel_degree` is useful only when `--sharding` is specified.")
 
-        if len(self.sharding) > 0 or self.tensor_parallel_degree > 1 or self.pipeline_parallel_degree > 1:
-            self.use_hybrid_parallel = True
+        if paddle.distributed.get_world_size() > 1:
+            if len(self.sharding) > 0 or self.tensor_parallel_degree > 1 or self.pipeline_parallel_degree > 1:
+                self.use_hybrid_parallel = True
 
         if self.amp_master_grad:
             if self.pipeline_parallel_degree <= 1 and self.tensor_parallel_degree <= 1:


Code review comments:
diff_hunk: @@ -766,8 +766,9 @@ def __post_init__(self):
         if len(self.sharding) == 0 and self.sharding_parallel_degree > 0:
             warnings.warn("`--sharding_parallel_degree` is useful only when `--sharding` is specified.")
 
-        if len(self.sharding) > 0 or self.tensor_parallel_degree > 1 or self.pipeline_parallel_degree > 1:
-            self.use_hybrid_parallel = True
+        if paddle.distributed.get_world_size() > 1:
+            if len(self.sharding) > 0 or self.tensor_parallel_degree > 1 or self.pipeline_parallel_degree > 1:
path: paddlenlp/trainer/training_args.py
text: @user1:
```suggestion
            if len(self.sharding) > 0 or self.tensor_parallel_degree > 1 or self.pipeline_parallel_degree > 1 and paddle.distributed.get_world_size() > 1:
```