Should I configure FP16, optimizers, batch_size in DeepSpeed config of Pytorch-Lightning?
My deepspeed_zero2_config.json:
{
    "zero_optimization": {
        "stage": 2,
        "offload_optimizer": {
            "device": "cpu",
            "pin_memory": true
        },
        "allgather_partitions": true,
        "allgather_bucket_size": 2e8,
        "overlap_comm": true,
        "reduce_scatter": true,
        "reduce_bucket_size": 2e8,
        "contiguous_gradients": true
    },
    "gradient_accumulation_steps": "auto",
    "gradient_clipping": "auto",
    "steps_per_print": 2000,
    "train_batch_size": "auto",
    "train_micro_batch_size_per_gpu": "auto",
    "wall_clock_breakdown": false
}
I have some questions about how to configure DeepSpeed in Pytorch-Lightning:

I see that the custom deepspeed config includes optimizer and scheduler. Should I add them in my config even I have configured in Model.configure_optimizers?
You have not specified an optimizer or scheduler within the DeepSpeed config. Using `configure_optimizers` to define optimizer and scheduler.


Should I add fp16 config into deepspeed config json even I have passed precision="bf16" in pl.Trainer?
Should I pass logging_batch_size_per_gpu to pl.plugins.DeepSpeedPlugin even I have configured batch_size in data loader?
[2022-03-24 12:42:11,529] [WARNING] [deepspeed.py:630:_auto_select_batch_size] Tried to infer the batch size for internal deepspeed logging from the `train_dataloader()`. To ensure DeepSpeed logging remains correct, please manually pass the plugin with the batch size, `Trainer(strategy=DeepSpeedPlugin(logging_batch_size_per_gpu=batch_size))`.


It appears in log before training every time. Is that okey?
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.



Thanks a lot!😊
yes, you don't need to set them inside config since this is done by Lightning already here if you set them in trainer and lightning module: https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pytorch_lightning/strategies/deepspeed.py