diff --git a/docs/source/usage_guides/deepspeed.mdx b/docs/source/usage_guides/deepspeed.mdx
index 29561c77b..0377296c1 100644
--- a/docs/source/usage_guides/deepspeed.mdx
+++ b/docs/source/usage_guides/deepspeed.mdx
@@ -395,6 +395,196 @@ We will look at the changes needed in the code when using these.
 based on model, dataloaders, dummy optimizer and dummy schedulers provided to `prepare` method. 
 Only the `auto` fields specified in above examples are handled by `prepare` method and the rest have to be explicitly specified by the user.
 
+**Things to note when using DeepSpeed Config File**
+
+Below is a sample script using `deepspeed_config_file` in different scenarios.
+
+Code `test.py`:
+
+```python
+from accelerate import Accelerator
+from accelerate.state import AcceleratorState
+
+
+def main():
+    accelerator = Accelerator()
+    accelerator.print(f"{AcceleratorState()}")
+
+
+if __name__ == "__main__":
+    main()
+```
+
+**Scenario 1**: Manually tampered accelerate config file having `deepspeed_config_file` along with other entries.
+
+1. Content of the `accelerate` config:
+
+```yaml
+command_file: null
+commands: null
+compute_environment: LOCAL_MACHINE
+deepspeed_config:
+  gradient_accumulation_steps: 1
+  gradient_clipping: 1.0
+  offload_optimizer_device: 'cpu'
+  offload_param_device: 'cpu'
+  zero3_init_flag: true
+  zero3_save_16bit_model: true
+  zero_stage: 3
+  deepspeed_config_file: 'ds_config.json'
+distributed_type: DEEPSPEED
+downcast_bf16: 'no'
+dynamo_backend: 'NO'
+fsdp_config: {}
+gpu_ids: null
+machine_rank: 0
+main_process_ip: null
+main_process_port: null
+main_training_function: main
+megatron_lm_config: {}
+num_machines: 1
+num_processes: 2
+rdzv_backend: static
+same_network: true
+tpu_name: null
+tpu_zone: null
+use_cpu: false
+```
+
+2. `ds_config.json`:
+
+```json
+{
+    "bf16": {
+        "enabled": true
+    },
+    "zero_optimization": {
+        "stage": 3,
+        "stage3_gather_16bit_weights_on_model_save": false,
+        "offload_optimizer": {
+            "device": "none"
+        },
+        "offload_param": {
+            "device": "none"
+        }
+    },
+    "gradient_clipping": 1.0,
+    "train_batch_size": "auto",
+    "train_micro_batch_size_per_gpu": "auto",
+    "gradient_accumulation_steps": 10,
+    "steps_per_print": 2000000
+}
+```
+
+3. Output of `accelerate launch test.py`:
+
+```bash
+ValueError: When using `deepspeed_config_file`, the following accelerate config variables will be ignored: 
+['gradient_accumulation_steps', 'gradient_clipping', 'zero_stage', 'offload_optimizer_device', 'offload_param_device', 
+'zero3_save_16bit_model', 'mixed_precision'].
+Please specify them appropriately in the DeepSpeed config file.
+If you are using an accelerate config file, remove others config variables mentioned in the above specified list.
+The easiest method is to create a new config following the questionnaire via `accelerate config`.
+It will only ask for the necessary config variables when using `deepspeed_config_file`.
+```
+
+**Scenario 2**: Use the solution of the error to create new accelerate config and check that no ambiguity error is now thrown.
+
+1. Run `accelerate config`:
+
+```bash
+$ accelerate config
+-------------------------------------------------------------------------------------------------------------------------------
+In which compute environment are you running?
+This machine                                                                                                                   
+-------------------------------------------------------------------------------------------------------------------------------
+Which type of machine are you using?                                                                                           
+multi-GPU                                                                                                                      
+How many different machines will you use (use more than 1 for multi-node training)? [1]:                                       
+Do you wish to optimize your script with torch dynamo?[yes/NO]:                                                                
+Do you want to use DeepSpeed? [yes/NO]: yes                                                                                    
+Do you want to specify a json file to a DeepSpeed config? [yes/NO]: yes                                                        
+Please enter the path to the json DeepSpeed config file: ds_config.json                                                        
+Do you want to enable `deepspeed.zero.Init` when using ZeRO Stage-3 for constructing massive models? [yes/NO]: yes
+How many GPU(s) should be used for distributed training? [1]:4
+accelerate configuration saved at ds_config_sample.yaml
+```
+
+2. Content of the `accelerate` config:
+
+```yaml
+compute_environment: LOCAL_MACHINE
+deepspeed_config:
+  deepspeed_config_file: ds_config.json
+  zero3_init_flag: true
+distributed_type: DEEPSPEED
+downcast_bf16: 'no'
+dynamo_backend: 'NO'
+fsdp_config: {}
+machine_rank: 0
+main_training_function: main
+megatron_lm_config: {}
+num_machines: 1
+num_processes: 4
+rdzv_backend: static
+same_network: true
+use_cpu: false
+```
+
+3. Output of `accelerate launch test.py`:
+
+```bash
+Distributed environment: DEEPSPEED  Backend: nccl
+Num processes: 4
+Process index: 0
+Local process index: 0
+Device: cuda:0
+Mixed precision type: bf16
+ds_config: {'bf16': {'enabled': True}, 'zero_optimization': {'stage': 3, 'stage3_gather_16bit_weights_on_model_save': False, 'offload_optimizer': {'device': 'none'}, 'offload_param': {'device': 'none'}}, 'gradient_clipping': 1.0, 'train_batch_size': 'auto', 'train_micro_batch_size_per_gpu': 'auto', 'gradient_accumulation_steps': 10, 'steps_per_print': inf, 'fp16': {'enabled': False}}
+```
+
+**Scenario 3**: Setting the `accelerate launch` command arguments related to DeepSpeed as `"auto"` in the DeepSpeed` configuration file and check that things work as expected.
+
+1. New `ds_config.json` with `"auto"` for the `accelerate launch` DeepSpeed command arguments:
+
+```json
+{
+    "bf16": {
+        "enabled": "auto"
+    },
+    "zero_optimization": {
+        "stage": "auto",
+        "stage3_gather_16bit_weights_on_model_save": "auto",
+        "offload_optimizer": {
+            "device": "auto"
+        },
+        "offload_param": {
+            "device": "auto"
+        }
+    },
+    "gradient_clipping": "auto",
+    "train_batch_size": "auto",
+    "train_micro_batch_size_per_gpu": "auto",
+    "gradient_accumulation_steps": "auto",
+    "steps_per_print": 2000000
+}
+```
+
+2. Output of `accelerate launch --mixed_precision="fp16" --zero_stage=3 --gradient_accumulation_steps=5 --gradient_clipping=1.0 --offload_param_device="cpu" --offload_optimizer_device="nvme" --zero3_save_16bit_model="true" test.py`:
+
+```bash
+Distributed environment: DEEPSPEED  Backend: nccl
+Num processes: 4
+Process index: 0
+Local process index: 0
+Device: cuda:0
+Mixed precision type: fp16
+ds_config: {'bf16': {'enabled': False}, 'zero_optimization': {'stage': 3, 'stage3_gather_16bit_weights_on_model_save': True, 'offload_optimizer': {'device': 'nvme'}, 'offload_param': {'device': 'cpu'}}, 'gradient_clipping': 1.0, 'train_batch_size': 'auto', 'train_micro_batch_size_per_gpu': 'auto', 'gradient_accumulation_steps': 5, 'steps_per_print': inf, 'fp16': {'enabled': True, 'auto_cast': True}}
+```
+
+**Note**: Remaining `"auto"` values are handled in `accelerator.prepare()` call as explained in point 2 of 
+`Important code changes when using DeepSpeed Config File`.
+
 ## Saving and loading
 
 1. Saving and loading of models is unchanged for ZeRO Stage-1 and Stage-2.
diff --git a/src/accelerate/commands/config/cluster.py b/src/accelerate/commands/config/cluster.py
index a3dcd2dcb..50acf55a1 100644
--- a/src/accelerate/commands/config/cluster.py
+++ b/src/accelerate/commands/config/cluster.py
@@ -460,7 +460,7 @@ def get_cluster_input():
 
     if distributed_type != DistributedType.TPU:
         if distributed_type == DistributedType.DEEPSPEED and use_deepspeed_config:
-            mixed_precision = "no"
+            mixed_precision = None
         else:
             mixed_precision = _ask_options(
                 "Do you wish to use FP16 or BF16 (mixed precision)?",
diff --git a/src/accelerate/commands/config/config_args.py b/src/accelerate/commands/config/config_args.py
index ba492802e..8b4a28292 100644
--- a/src/accelerate/commands/config/config_args.py
+++ b/src/accelerate/commands/config/config_args.py
@@ -78,6 +78,7 @@ def to_dict(self):
         for key, value in result.items():
             if isinstance(value, Enum):
                 result[key] = value.value
+        result = {k: v for k, v in result.items() if v is not None}
         return result
 
     @classmethod
@@ -88,7 +89,7 @@ def from_json_file(cls, json_file=None):
         if "compute_environment" not in config_dict:
             config_dict["compute_environment"] = ComputeEnvironment.LOCAL_MACHINE
         if "mixed_precision" not in config_dict:
-            config_dict["mixed_precision"] = "fp16" if ("fp16" in config_dict and config_dict["fp16"]) else "no"
+            config_dict["mixed_precision"] = "fp16" if ("fp16" in config_dict and config_dict["fp16"]) else None
         if "fp16" in config_dict:  # Convert the config to the new format.
             del config_dict["fp16"]
         if "use_cpu" not in config_dict:
@@ -111,7 +112,7 @@ def from_yaml_file(cls, yaml_file=None):
             config_dict["compute_environment"] = ComputeEnvironment.LOCAL_MACHINE
 
         if "mixed_precision" not in config_dict:
-            config_dict["mixed_precision"] = "fp16" if ("fp16" in config_dict and config_dict["fp16"]) else "no"
+            config_dict["mixed_precision"] = "fp16" if ("fp16" in config_dict and config_dict["fp16"]) else None
         if "fp16" in config_dict:  # Convert the config to the new format.
             del config_dict["fp16"]
         if "use_cpu" not in config_dict:
diff --git a/src/accelerate/commands/launch.py b/src/accelerate/commands/launch.py
index 91d4427ac..b5f831b47 100644
--- a/src/accelerate/commands/launch.py
+++ b/src/accelerate/commands/launch.py
@@ -301,45 +301,50 @@ def launch_command_parser(subparsers=None):
         "--zero_stage",
         default=None,
         type=int,
-        help="DeepSpeed's ZeRO optimization stage (useful only when `use_deepspeed` flag is passed).",
+        help="DeepSpeed's ZeRO optimization stage (useful only when `use_deepspeed` flag is passed). "
+        "If unspecified, will default to `2`.",
     )
     deepspeed_args.add_argument(
         "--offload_optimizer_device",
         default=None,
         type=str,
-        help="Decides where (none|cpu|nvme) to offload optimizer states (useful only when `use_deepspeed` flag is passed).",
+        help="Decides where (none|cpu|nvme) to offload optimizer states (useful only when `use_deepspeed` flag is passed). "
+        "If unspecified, will default to 'none'.",
     )
     deepspeed_args.add_argument(
         "--offload_param_device",
         default=None,
         type=str,
-        help="Decides where (none|cpu|nvme) to offload parameters (useful only when `use_deepspeed` flag is passed).",
+        help="Decides where (none|cpu|nvme) to offload parameters (useful only when `use_deepspeed` flag is passed). "
+        "If unspecified, will default to 'none'.",
     )
     deepspeed_args.add_argument(
         "--gradient_accumulation_steps",
         default=None,
         type=int,
-        help="No of gradient_accumulation_steps used in your training script (useful only when `use_deepspeed` flag is passed).",
+        help="No of gradient_accumulation_steps used in your training script (useful only when `use_deepspeed` flag is passed). "
+        "If unspecified, will default to `1`.",
     )
     deepspeed_args.add_argument(
         "--gradient_clipping",
         default=None,
         type=float,
-        help="gradient clipping value used in your training script (useful only when `use_deepspeed` flag is passed).",
+        help="gradient clipping value used in your training script (useful only when `use_deepspeed` flag is passed). "
+        "If unspecified, will default to `1.0`.",
     )
     deepspeed_args.add_argument(
         "--zero3_init_flag",
-        default="true",
+        default=None,
         type=str,
         help="Decides Whether (true|false) to enable `deepspeed.zero.Init` for constructing massive models. "
-        "Only applicable with DeepSpeed ZeRO Stage-3.",
+        "Only applicable with DeepSpeed ZeRO Stage-3. If unspecified, will default to `true`.",
     )
     deepspeed_args.add_argument(
         "--zero3_save_16bit_model",
-        default="false",
+        default=None,
         type=str,
         help="Decides Whether (true|false) to save 16-bit model weights when using ZeRO Stage-3. "
-        "Only applicable with DeepSpeed ZeRO Stage-3.",
+        "Only applicable with DeepSpeed ZeRO Stage-3. If unspecified, will default to `false`.",
     )
     deepspeed_args.add_argument(
         "--deepspeed_hostfile",
@@ -363,7 +368,7 @@ def launch_command_parser(subparsers=None):
         "--deepspeed_multinode_launcher",
         default=None,
         type=str,
-        help="DeepSpeed multi-node launcher to use.",
+        help="DeepSpeed multi-node launcher to use. If unspecified, will default to `pdsh`.",
     )
 
     # fsdp arguments
@@ -717,14 +722,22 @@ def deepspeed_launcher(args):
 
     current_env["PYTHONPATH"] = env_var_path_add("PYTHONPATH", os.path.abspath("."))
     current_env["ACCELERATE_MIXED_PRECISION"] = str(mixed_precision)
+    current_env["ACCELERATE_CONFIG_DS_FIELDS"] = str(args.deepspeed_fields_from_accelerate_config).lower()
     current_env["ACCELERATE_USE_DEEPSPEED"] = "true"
-    current_env["ACCELERATE_DEEPSPEED_ZERO_STAGE"] = str(args.zero_stage)
-    current_env["ACCELERATE_GRADIENT_ACCUMULATION_STEPS"] = str(args.gradient_accumulation_steps)
-    current_env["ACCELERATE_GRADIENT_CLIPPING"] = str(args.gradient_clipping).lower()
-    current_env["ACCELERATE_DEEPSPEED_OFFLOAD_OPTIMIZER_DEVICE"] = str(args.offload_optimizer_device).lower()
-    current_env["ACCELERATE_DEEPSPEED_OFFLOAD_PARAM_DEVICE"] = str(args.offload_param_device).lower()
-    current_env["ACCELERATE_DEEPSPEED_ZERO3_INIT"] = str(args.zero3_init_flag).lower()
-    current_env["ACCELERATE_DEEPSPEED_ZERO3_SAVE_16BIT_MODEL"] = str(args.zero3_save_16bit_model).lower()
+    if args.zero_stage is not None:
+        current_env["ACCELERATE_DEEPSPEED_ZERO_STAGE"] = str(args.zero_stage)
+    if args.gradient_accumulation_steps is not None:
+        current_env["ACCELERATE_GRADIENT_ACCUMULATION_STEPS"] = str(args.gradient_accumulation_steps)
+    if args.gradient_clipping is not None:
+        current_env["ACCELERATE_GRADIENT_CLIPPING"] = str(args.gradient_clipping).lower()
+    if args.offload_optimizer_device is not None:
+        current_env["ACCELERATE_DEEPSPEED_OFFLOAD_OPTIMIZER_DEVICE"] = str(args.offload_optimizer_device).lower()
+    if args.offload_param_device is not None:
+        current_env["ACCELERATE_DEEPSPEED_OFFLOAD_PARAM_DEVICE"] = str(args.offload_param_device).lower()
+    if args.zero3_init_flag is not None:
+        current_env["ACCELERATE_DEEPSPEED_ZERO3_INIT"] = str(args.zero3_init_flag).lower()
+    if args.zero3_save_16bit_model is not None:
+        current_env["ACCELERATE_DEEPSPEED_ZERO3_SAVE_16BIT_MODEL"] = str(args.zero3_save_16bit_model).lower()
     if args.deepspeed_config_file is not None:
         current_env["ACCELERATE_DEEPSPEED_CONFIG_FILE"] = str(args.deepspeed_config_file)
 
@@ -966,6 +979,7 @@ def launch_command(args):
 
     defaults = None
     warned = []
+    mp_from_config_flag = False
     # Get the default from the config file.
     if args.config_file is not None or os.path.isfile(default_config_file) and not args.cpu:
         defaults = load_config_from_file(args.config_file)
@@ -1013,7 +1027,12 @@ def launch_command(args):
                 ):
                     setattr(args, name, attr)
         if not args.mixed_precision:
-            args.mixed_precision = defaults.mixed_precision
+            if defaults.mixed_precision is None:
+                args.mixed_precision = "no"
+            else:
+                args.mixed_precision = defaults.mixed_precision
+                mp_from_config_flag = True
+
         if args.dynamo_backend is None:
             warned.append("\t`--dynamo_backend` was set to a value of `'no'`")
             args.dynamo_backend = "no"
@@ -1056,6 +1075,10 @@ def launch_command(args):
 
     # Use the proper launcher
     if args.use_deepspeed and not args.cpu:
+        args.deepspeed_fields_from_accelerate_config = list(defaults.deepspeed_config.keys()) if defaults else []
+        if mp_from_config_flag:
+            args.deepspeed_fields_from_accelerate_config.append("mixed_precision")
+        args.deepspeed_fields_from_accelerate_config = ",".join(args.deepspeed_fields_from_accelerate_config)
         deepspeed_launcher(args)
     elif args.use_fsdp and not args.cpu:
         multi_gpu_launcher(args)
diff --git a/src/accelerate/utils/dataclasses.py b/src/accelerate/utils/dataclasses.py
index 5a87d7860..01f174a10 100644
--- a/src/accelerate/utils/dataclasses.py
+++ b/src/accelerate/utils/dataclasses.py
@@ -394,6 +394,28 @@ class DeepSpeedPlugin:
     def __post_init__(self):
         from .deepspeed import HfDeepSpeedConfig
 
+        if self.gradient_accumulation_steps is None:
+            self.gradient_accumulation_steps = int(os.environ.get("ACCELERATE_GRADIENT_ACCUMULATION_STEPS", 1))
+
+        if self.gradient_clipping is None:
+            gradient_clipping = os.environ.get("ACCELERATE_GRADIENT_CLIPPING", "none")
+            if gradient_clipping != "none":
+                self.gradient_clipping = float(gradient_clipping)
+
+        if self.zero_stage is None:
+            self.zero_stage = int(os.environ.get("ACCELERATE_DEEPSPEED_ZERO_STAGE", 2))
+
+        if self.offload_optimizer_device is None:
+            self.offload_optimizer_device = os.environ.get("ACCELERATE_DEEPSPEED_OFFLOAD_OPTIMIZER_DEVICE", "none")
+
+        if self.offload_param_device is None:
+            self.offload_param_device = os.environ.get("ACCELERATE_DEEPSPEED_OFFLOAD_PARAM_DEVICE", "none")
+
+        if self.zero3_save_16bit_model is None:
+            self.zero3_save_16bit_model = (
+                os.environ.get("ACCELERATE_DEEPSPEED_ZERO3_SAVE_16BIT_MODEL", "false") == "true"
+            )
+
         if self.hf_ds_config is None:
             self.hf_ds_config = os.environ.get("ACCELERATE_DEEPSPEED_CONFIG_FILE", "none")
         if (
@@ -405,33 +427,22 @@ def __post_init__(self):
                 self.hf_ds_config = HfDeepSpeedConfig(self.hf_ds_config)
             if "gradient_accumulation_steps" not in self.hf_ds_config.config:
                 self.hf_ds_config.config["gradient_accumulation_steps"] = 1
-            elif self.hf_ds_config.config["gradient_accumulation_steps"] == "auto":
-                raise ValueError("gradient_accumulation_steps cannot be set to 'auto' in the DeepSpeed config.")
             if "zero_optimization" not in self.hf_ds_config.config:
                 raise ValueError("Please specify the ZeRO optimization config in the DeepSpeed config.")
-        else:
-            if self.gradient_accumulation_steps is None:
-                self.gradient_accumulation_steps = int(os.environ.get("ACCELERATE_GRADIENT_ACCUMULATION_STEPS", 1))
-
-            if self.gradient_clipping is None:
-                gradient_clipping = os.environ.get("ACCELERATE_GRADIENT_CLIPPING", "none")
-                if gradient_clipping != "none":
-                    self.gradient_clipping = float(gradient_clipping)
-
-            if self.zero_stage is None:
-                self.zero_stage = int(os.environ.get("ACCELERATE_DEEPSPEED_ZERO_STAGE", 2))
-
-            if self.offload_optimizer_device is None:
-                self.offload_optimizer_device = os.environ.get("ACCELERATE_DEEPSPEED_OFFLOAD_OPTIMIZER_DEVICE", "none")
-
-            if self.offload_param_device is None:
-                self.offload_param_device = os.environ.get("ACCELERATE_DEEPSPEED_OFFLOAD_PARAM_DEVICE", "none")
-
-            if self.zero3_save_16bit_model is None:
-                self.zero3_save_16bit_model = (
-                    os.environ.get("ACCELERATE_DEEPSPEED_ZERO3_SAVE_16BIT_MODEL", "false") == "true"
-                )
 
+            self._deepspeed_config_checks()
+            kwargs = {
+                "gradient_accumulation_steps": self.gradient_accumulation_steps,
+                "gradient_clipping": self.gradient_clipping if self.gradient_clipping else 1.0,
+                "zero_optimization.stage": self.zero_stage,
+                "zero_optimization.offload_optimizer.device": self.offload_optimizer_device,
+                "zero_optimization.offload_param.device": self.offload_param_device,
+                "zero_optimization.stage3_gather_16bit_weights_on_model_save": self.zero3_save_16bit_model,
+            }
+            for key in kwargs.keys():
+                self.fill_match(key, **kwargs, must_match=False)
+            self.hf_ds_config.set_stage_and_offload()
+        else:
             config = {
                 "train_batch_size": "auto",
                 "train_micro_batch_size_per_gpu": "auto",
@@ -450,15 +461,19 @@ def __post_init__(self):
             if self.gradient_clipping:
                 config["gradient_clipping"] = self.gradient_clipping
             self.hf_ds_config = HfDeepSpeedConfig(config)
+
         self.deepspeed_config = self.hf_ds_config.config
         self.deepspeed_config["steps_per_print"] = float("inf")  # this will stop deepspeed from logging @ stdout
         if self.zero3_init_flag is None:
-            self.zero3_init_flag = os.environ.get("ACCELERATE_DEEPSPEED_ZERO3_INIT", "false") == "true"
+            self.zero3_init_flag = (
+                strtobool(os.environ.get("ACCELERATE_DEEPSPEED_ZERO3_INIT", str(self.hf_ds_config.is_zero3()))) == 1
+            )
         if self.zero3_init_flag and not self.hf_ds_config.is_zero3():
             warnings.warn("DeepSpeed Zero3 Init flag is only applicable for ZeRO Stage 3. Setting it to False.")
             self.zero3_init_flag = False
 
-    def fill_match(self, ds_key_long, mismatches, must_match=True, **kwargs):
+    def fill_match(self, ds_key_long, mismatches=None, must_match=True, **kwargs):
+        mismatches = [] if mismatches is None else mismatches
         config, ds_key = self.hf_ds_config.find_config_node(ds_key_long)
         if config is None:
             return
@@ -503,10 +518,28 @@ def deepspeed_config_process(self, prefix="", mismatches=None, config=None, must
 
     def set_mixed_precision(self, mixed_precision):
         ds_config = self.deepspeed_config
-        if mixed_precision == "fp16" and "fp16" not in ds_config and "bf16" not in ds_config:
-            ds_config.update({"fp16": {"enabled": True, "auto_cast": True}})
-        elif mixed_precision == "bf16" and "fp16" not in ds_config and "bf16" not in ds_config:
-            ds_config.update({"bf16": {"enabled": True}})
+        kwargs = {
+            "fp16.enabled": mixed_precision == "fp16",
+            "bf16.enabled": mixed_precision == "bf16",
+        }
+        if mixed_precision == "fp16":
+            if "fp16" not in ds_config:
+                ds_config["fp16"] = {"enabled": True, "auto_cast": True}
+        elif mixed_precision == "bf16":
+            if "bf16" not in ds_config:
+                ds_config["bf16"] = {"enabled": True}
+
+        if mixed_precision != "no":
+            diff_dtype = "bf16" if mixed_precision == "fp16" else "fp16"
+            if str(ds_config.get(diff_dtype, {}).get("enabled", "False")).lower() == "true":
+                raise ValueError(
+                    f"`--mixed_precision` arg cannot be set to `{mixed_precision}` when `{diff_dtype}` is set in the DeepSpeed config file."
+                )
+        for dtype in ["fp16", "bf16"]:
+            if dtype not in ds_config:
+                ds_config[dtype] = {"enabled": False}
+        self.fill_match("fp16.enabled", must_match=False, **kwargs)
+        self.fill_match("bf16.enabled", must_match=False, **kwargs)
 
     def set_deepspeed_weakref(self):
         from .imports import is_transformers_available
@@ -549,6 +582,31 @@ def zero3_init_context_manager(self, enable=False):
             self.dschf = None
             self.set_deepspeed_weakref()
 
+    def _deepspeed_config_checks(self):
+        env_variable_names_to_ignore = [
+            "ACCELERATE_GRADIENT_ACCUMULATION_STEPS",
+            "ACCELERATE_GRADIENT_CLIPPING",
+            "ACCELERATE_DEEPSPEED_ZERO_STAGE",
+            "ACCELERATE_DEEPSPEED_OFFLOAD_OPTIMIZER_DEVICE",
+            "ACCELERATE_DEEPSPEED_OFFLOAD_PARAM_DEVICE",
+            "ACCELERATE_DEEPSPEED_ZERO3_SAVE_16BIT_MODEL",
+            "ACCELERATE_MIXED_PRECISION",
+        ]
+        env_variable_names_to_ignore = [
+            name.replace("ACCELERATE_", "").replace("DEEPSPEED_", "").lower() for name in env_variable_names_to_ignore
+        ]
+
+        deepspeed_fields_from_accelerate_config = os.environ.get("ACCELERATE_CONFIG_DS_FIELDS", "").split(",")
+
+        if any(name in env_variable_names_to_ignore for name in deepspeed_fields_from_accelerate_config):
+            raise ValueError(
+                f"When using `deepspeed_config_file`, the following accelerate config variables will be ignored: {env_variable_names_to_ignore}.\n"
+                "Please specify them appropriately in the DeepSpeed config file.\n"
+                "If you are using an accelerate config file, remove others config variables mentioned in the above specified list.\n"
+                "The easiest method is to create a new config following the questionnaire via `accelerate config`.\n"
+                "It will only ask for the necessary config variables when using `deepspeed_config_file`."
+            )
+
 
 @dataclass
 class FullyShardedDataParallelPlugin:
diff --git a/src/accelerate/utils/deepspeed.py b/src/accelerate/utils/deepspeed.py
index 02d1ab8bc..69dc5c7f8 100644
--- a/src/accelerate/utils/deepspeed.py
+++ b/src/accelerate/utils/deepspeed.py
@@ -50,6 +50,9 @@ def __init__(self, config_file_or_dict):
             raise ValueError("expecting either a path to a DeepSpeed config file or a pre-populated dict")
         self.config = config
 
+        self.set_stage_and_offload()
+
+    def set_stage_and_offload(self):
         # zero stage - this is done as early as possible, before model is created, to allow
         # ``is_deepspeed_zero3_enabled`` query and getting to the early deepspeed config object
         # during ``zero.Init()`` which needs to know the dtype, and some other hparams.
diff --git a/tests/deepspeed/test_deepspeed.py b/tests/deepspeed/test_deepspeed.py
index eca75c1ed..fceb0bd23 100644
--- a/tests/deepspeed/test_deepspeed.py
+++ b/tests/deepspeed/test_deepspeed.py
@@ -285,8 +285,6 @@ def test_prepare_deepspeed(self, optim_type, scheduler_type):
         from deepspeed.runtime.engine import DeepSpeedEngine
 
         kwargs = {
-            "fp16.enabled": True,
-            "bf16.enabled": False,
             "optimizer.params.lr": 5e-5,
             "optimizer.params.weight_decay": 0.0,
             "scheduler.params.warmup_min_lr": 0.0,
@@ -370,7 +368,7 @@ def test_prepare_deepspeed(self, optim_type, scheduler_type):
             # Test DeepSpeed optimizer + DeepSpeed scheduler
             deepspeed_plugin = DeepSpeedPlugin(hf_ds_config=self.ds_config_file[ZERO2])
             with mockenv_context(**self.dist_env):
-                accelerator = Accelerator(deepspeed_plugin=deepspeed_plugin)
+                accelerator = Accelerator(deepspeed_plugin=deepspeed_plugin, mixed_precision="fp16")
                 train_set = RegressionDataset(length=80)
                 eval_set = RegressionDataset(length=20)
                 train_dataloader = DataLoader(train_set, batch_size=10, shuffle=True)
@@ -430,7 +428,7 @@ def test_prepare_deepspeed(self, optim_type, scheduler_type):
             # Test custom optimizer + DeepSpeed scheduler
             deepspeed_plugin = DeepSpeedPlugin(hf_ds_config=self.ds_config_file[ZERO2])
             with mockenv_context(**self.dist_env):
-                accelerator = Accelerator(deepspeed_plugin=deepspeed_plugin)
+                accelerator = Accelerator(deepspeed_plugin=deepspeed_plugin, mixed_precision="fp16")
                 train_set = RegressionDataset(length=80)
                 eval_set = RegressionDataset(length=20)
                 train_dataloader = DataLoader(train_set, batch_size=10, shuffle=True)
@@ -463,7 +461,7 @@ def test_prepare_deepspeed(self, optim_type, scheduler_type):
             # Test deepspeed optimizer + custom scheduler
             deepspeed_plugin = DeepSpeedPlugin(hf_ds_config=self.ds_config_file[ZERO2])
             with mockenv_context(**self.dist_env):
-                accelerator = Accelerator(deepspeed_plugin=deepspeed_plugin)
+                accelerator = Accelerator(deepspeed_plugin=deepspeed_plugin, mixed_precision="fp16")
                 train_set = RegressionDataset(length=80)
                 eval_set = RegressionDataset(length=20)
                 train_dataloader = DataLoader(train_set, batch_size=10, shuffle=True)
@@ -501,8 +499,6 @@ def test_save_checkpoints(self):
         )
         del deepspeed_plugin.deepspeed_config["bf16"]
         kwargs = {
-            "fp16.enabled": True,
-            "bf16.enabled": False,
             "optimizer.params.lr": 5e-5,
             "optimizer.params.weight_decay": 0.0,
             "scheduler.params.warmup_min_lr": 0.0,
@@ -518,7 +514,7 @@ def test_save_checkpoints(self):
         }
 
         with mockenv_context(**self.dist_env):
-            accelerator = Accelerator(deepspeed_plugin=deepspeed_plugin)
+            accelerator = Accelerator(deepspeed_plugin=deepspeed_plugin, mixed_precision="fp16")
             kwargs["train_batch_size"] = (
                 kwargs["train_micro_batch_size_per_gpu"]
                 * deepspeed_plugin.deepspeed_config["gradient_accumulation_steps"]
@@ -594,6 +590,81 @@ def test_autofill_dsconfig(self):
                 accelerator.deepspeed_config["zero_optimization"]["stage3_gather_16bit_weights_on_model_save"]
             )
 
+    @parameterized.expand([FP16, BF16], name_func=parameterized_custom_name_func)
+    def test_autofill_dsconfig_from_ds_plugin(self, dtype):
+        ds_config = self.ds_config_dict["zero3"]
+        if dtype == BF16:
+            del ds_config["fp16"]
+        else:
+            del ds_config["bf16"]
+        ds_config[dtype]["enabled"] = "auto"
+        ds_config["zero_optimization"]["stage"] = "auto"
+        ds_config["zero_optimization"]["stage3_gather_16bit_weights_on_model_save"] = "auto"
+        ds_config["zero_optimization"]["offload_optimizer"]["device"] = "auto"
+        ds_config["zero_optimization"]["offload_param"]["device"] = "auto"
+        ds_config["gradient_accumulation_steps"] = "auto"
+        ds_config["gradient_clipping"] = "auto"
+
+        deepspeed_plugin = DeepSpeedPlugin(
+            hf_ds_config=ds_config,
+            zero3_init_flag=True,
+            gradient_accumulation_steps=1,
+            gradient_clipping=1.0,
+            zero_stage=2,
+            offload_optimizer_device="cpu",
+            offload_param_device="cpu",
+            zero3_save_16bit_model=True,
+        )
+
+        with mockenv_context(**self.dist_env):
+            accelerator = Accelerator(deepspeed_plugin=deepspeed_plugin, mixed_precision=dtype)
+            deepspeed_plugin = accelerator.state.deepspeed_plugin
+            self.assertEqual(deepspeed_plugin.deepspeed_config["gradient_clipping"], 1.0)
+            self.assertEqual(deepspeed_plugin.deepspeed_config["gradient_accumulation_steps"], 1)
+            self.assertEqual(deepspeed_plugin.deepspeed_config["zero_optimization"]["stage"], 2)
+            self.assertEqual(
+                deepspeed_plugin.deepspeed_config["zero_optimization"]["offload_optimizer"]["device"], "cpu"
+            )
+            self.assertEqual(deepspeed_plugin.deepspeed_config["zero_optimization"]["offload_param"]["device"], "cpu")
+            self.assertTrue(
+                deepspeed_plugin.deepspeed_config["zero_optimization"]["stage3_gather_16bit_weights_on_model_save"]
+            )
+            self.assertTrue(deepspeed_plugin.deepspeed_config[dtype]["enabled"])
+
+        AcceleratorState._reset_state()
+        diff_dtype = "bf16" if dtype == "fp16" else "fp16"
+        with mockenv_context(**self.dist_env):
+            with self.assertRaises(ValueError) as cm:
+                accelerator = Accelerator(deepspeed_plugin=deepspeed_plugin, mixed_precision=diff_dtype)
+            self.assertTrue(
+                f"`--mixed_precision` arg cannot be set to `{diff_dtype}` when `{dtype}` is set in the DeepSpeed config file."
+                in str(cm.exception)
+            )
+
+    def test_ds_config_assertions(self):
+        ambiguous_env = self.dist_env.copy()
+        ambiguous_env[
+            "ACCELERATE_CONFIG_DS_FIELDS"
+        ] = "gradient_accumulation_steps,gradient_clipping,zero_stage,offload_optimizer_device,offload_param_device,zero3_save_16bit_model,mixed_precision"
+
+        with mockenv_context(**ambiguous_env):
+            with self.assertRaises(ValueError) as cm:
+                deepspeed_plugin = DeepSpeedPlugin(
+                    hf_ds_config=self.ds_config_file[ZERO3],
+                    zero3_init_flag=True,
+                    gradient_accumulation_steps=1,
+                    gradient_clipping=1.0,
+                    zero_stage=ZERO2,
+                    offload_optimizer_device="cpu",
+                    offload_param_device="cpu",
+                    zero3_save_16bit_model=True,
+                )
+                _ = Accelerator(deepspeed_plugin=deepspeed_plugin, mixed_precision=FP16)
+            self.assertTrue(
+                "If you are using an accelerate config file, remove others config variables mentioned in the above specified list."
+                in str(cm.exception)
+            )
+
     def test_basic_run(self):
         mod_file = inspect.getfile(accelerate.test_utils)
         test_file_path = os.path.sep.join(


Review comments: body: Nit: all `none`->`None`
diff_hunk: @@ -301,45 +301,50 @@ def launch_command_parser(subparsers=None):
         "--zero_stage",
         default=None,
         type=int,
-        help="DeepSpeed's ZeRO optimization stage (useful only when `use_deepspeed` flag is passed).",
+        help="DeepSpeed's ZeRO optimization stage (useful only when `use_deepspeed` flag is passed). "
+        "If unspecified, will default to `2`.",
     )
     deepspeed_args.add_argument(
         "--offload_optimizer_device",
         default=None,
         type=str,
-        help="Decides where (none|cpu|nvme) to offload optimizer states (useful only when `use_deepspeed` flag is passed).",
+        help="Decides where (none|cpu|nvme) to offload optimizer states (useful only when `use_deepspeed` flag is passed). "
+        "If unspecified, will default to `none`.",
body: ```suggestion
                "If you are using an accelerate config file, set `mixed_precision=no` "
```
diff_hunk: @@ -549,6 +551,40 @@ def zero3_init_context_manager(self, enable=False):
             self.dschf = None
             self.set_deepspeed_weakref()
 
+    def _deepspeed_config_checks(self):
+        env_variable_names_to_ignore = [
+            "ACCELERATE_GRADIENT_ACCUMULATION_STEPS",
+            "ACCELERATE_GRADIENT_CLIPPING",
+            "ACCELERATE_DEEPSPEED_ZERO_STAGE",
+            "ACCELERATE_DEEPSPEED_OFFLOAD_OPTIMIZER_DEVICE",
+            "ACCELERATE_DEEPSPEED_OFFLOAD_PARAM_DEVICE",
+            "ACCELERATE_DEEPSPEED_ZERO3_SAVE_16BIT_MODEL",
+            "ACCELERATE_MIXED_PRECISION",
+        ]
+        duplicate_values_flag = False
+        for name in env_variable_names_to_ignore:
+            if name != "ACCELERATE_MIXED_PRECISION" and os.environ.get(name, None) is not None:
+                duplicate_values_flag = True
+            elif name == "ACCELERATE_MIXED_PRECISION" and os.environ.get(name, "no") != "no":
+                duplicate_values_flag = True
+            if duplicate_values_flag:
+                break
+
+        if duplicate_values_flag:
+            env_variable_names_to_ignore = [
+                name.replace("ACCELERATE_", "").replace("DEEPSPEED_", "").lower()
+                for name in env_variable_names_to_ignore
+            ]
+            raise ValueError(
+                f"When using `deepspeed_config_file`, the following accelerate config variables will be ignored: {env_variable_names_to_ignore}.\n"
+                "Please specify them appropriately in the DeepSpeed config file.\n"
+                "If you are using accelerate config file, set `mixed_precision=no` "
body: ```suggestion
                "and make sure to not specify these config variables in `accelerate launch` command. \n"
```
diff_hunk: @@ -549,6 +551,40 @@ def zero3_init_context_manager(self, enable=False):
             self.dschf = None
             self.set_deepspeed_weakref()
 
+    def _deepspeed_config_checks(self):
+        env_variable_names_to_ignore = [
+            "ACCELERATE_GRADIENT_ACCUMULATION_STEPS",
+            "ACCELERATE_GRADIENT_CLIPPING",
+            "ACCELERATE_DEEPSPEED_ZERO_STAGE",
+            "ACCELERATE_DEEPSPEED_OFFLOAD_OPTIMIZER_DEVICE",
+            "ACCELERATE_DEEPSPEED_OFFLOAD_PARAM_DEVICE",
+            "ACCELERATE_DEEPSPEED_ZERO3_SAVE_16BIT_MODEL",
+            "ACCELERATE_MIXED_PRECISION",
+        ]
+        duplicate_values_flag = False
+        for name in env_variable_names_to_ignore:
+            if name != "ACCELERATE_MIXED_PRECISION" and os.environ.get(name, None) is not None:
+                duplicate_values_flag = True
+            elif name == "ACCELERATE_MIXED_PRECISION" and os.environ.get(name, "no") != "no":
+                duplicate_values_flag = True
+            if duplicate_values_flag:
+                break
+
+        if duplicate_values_flag:
+            env_variable_names_to_ignore = [
+                name.replace("ACCELERATE_", "").replace("DEEPSPEED_", "").lower()
+                for name in env_variable_names_to_ignore
+            ]
+            raise ValueError(
+                f"When using `deepspeed_config_file`, the following accelerate config variables will be ignored: {env_variable_names_to_ignore}.\n"
+                "Please specify them appropriately in the DeepSpeed config file.\n"
+                "If you are using accelerate config file, set `mixed_precision=no` "
+                "and remove others config variables mentioned in the above specified list; "
+                "else don't specify these config variables in `accelerate launch` command. \n"
body: Hello, `none` is the string option as possible values are `cpu`|`nvme`l`none`
diff_hunk: @@ -301,45 +301,50 @@ def launch_command_parser(subparsers=None):
         "--zero_stage",
         default=None,
         type=int,
-        help="DeepSpeed's ZeRO optimization stage (useful only when `use_deepspeed` flag is passed).",
+        help="DeepSpeed's ZeRO optimization stage (useful only when `use_deepspeed` flag is passed). "
+        "If unspecified, will default to `2`.",
     )
     deepspeed_args.add_argument(
         "--offload_optimizer_device",
         default=None,
         type=str,
-        help="Decides where (none|cpu|nvme) to offload optimizer states (useful only when `use_deepspeed` flag is passed).",
+        help="Decides where (none|cpu|nvme) to offload optimizer states (useful only when `use_deepspeed` flag is passed). "
+        "If unspecified, will default to `none`.",
body: Oh then add quotes?
diff_hunk: @@ -301,45 +301,50 @@ def launch_command_parser(subparsers=None):
         "--zero_stage",
         default=None,
         type=int,
-        help="DeepSpeed's ZeRO optimization stage (useful only when `use_deepspeed` flag is passed).",
+        help="DeepSpeed's ZeRO optimization stage (useful only when `use_deepspeed` flag is passed). "
+        "If unspecified, will default to `2`.",
     )
     deepspeed_args.add_argument(
         "--offload_optimizer_device",
         default=None,
         type=str,
-        help="Decides where (none|cpu|nvme) to offload optimizer states (useful only when `use_deepspeed` flag is passed).",
+        help="Decides where (none|cpu|nvme) to offload optimizer states (useful only when `use_deepspeed` flag is passed). "
+        "If unspecified, will default to `none`.",
body: Done.
diff_hunk: @@ -301,45 +301,50 @@ def launch_command_parser(subparsers=None):
         "--zero_stage",
         default=None,
         type=int,
-        help="DeepSpeed's ZeRO optimization stage (useful only when `use_deepspeed` flag is passed).",
+        help="DeepSpeed's ZeRO optimization stage (useful only when `use_deepspeed` flag is passed). "
+        "If unspecified, will default to `2`.",
     )
     deepspeed_args.add_argument(
         "--offload_optimizer_device",
         default=None,
         type=str,
-        help="Decides where (none|cpu|nvme) to offload optimizer states (useful only when `use_deepspeed` flag is passed).",
+        help="Decides where (none|cpu|nvme) to offload optimizer states (useful only when `use_deepspeed` flag is passed). "
+        "If unspecified, will default to `none`.",
body: Did I understand it right that here you suggest to the user to set `mixed_precision=no` even if they use mixed precision in ds_config.json? If so this now is worse than before, as now you're proposing to force 2 opposite values in the same config entry. As in `mixed_precision=no` and `fp16 { enabled: true}`.

Unless I'm misunderstanding the wording that is. 

I think it should be either `mixed_precision=xyz` or `ds_config.json` (with fp16 or bf16 blocks or no block at all as fp16 is the default), but not both.
diff_hunk: @@ -549,6 +551,40 @@ def zero3_init_context_manager(self, enable=False):
             self.dschf = None
             self.set_deepspeed_weakref()
 
+    def _deepspeed_config_checks(self):
+        env_variable_names_to_ignore = [
+            "ACCELERATE_GRADIENT_ACCUMULATION_STEPS",
+            "ACCELERATE_GRADIENT_CLIPPING",
+            "ACCELERATE_DEEPSPEED_ZERO_STAGE",
+            "ACCELERATE_DEEPSPEED_OFFLOAD_OPTIMIZER_DEVICE",
+            "ACCELERATE_DEEPSPEED_OFFLOAD_PARAM_DEVICE",
+            "ACCELERATE_DEEPSPEED_ZERO3_SAVE_16BIT_MODEL",
+            "ACCELERATE_MIXED_PRECISION",
+        ]
+        duplicate_values_flag = False
+        for name in env_variable_names_to_ignore:
+            if name != "ACCELERATE_MIXED_PRECISION" and os.environ.get(name, None) is not None:
+                duplicate_values_flag = True
+            elif name == "ACCELERATE_MIXED_PRECISION" and os.environ.get(name, "no") != "no":
+                duplicate_values_flag = True
+            if duplicate_values_flag:
+                break
+
+        if duplicate_values_flag:
+            env_variable_names_to_ignore = [
+                name.replace("ACCELERATE_", "").replace("DEEPSPEED_", "").lower()
+                for name in env_variable_names_to_ignore
+            ]
+            raise ValueError(
+                f"When using `deepspeed_config_file`, the following accelerate config variables will be ignored: {env_variable_names_to_ignore}.\n"
+                "Please specify them appropriately in the DeepSpeed config file.\n"
+                "If you are using an accelerate config file, set `mixed_precision=no` "
body: ```suggestion
                "The easiest method is to create a new config following the questionnaire via `accelerate config`.\n"
```
diff_hunk: @@ -578,10 +578,9 @@ def _deepspeed_config_checks(self):
             raise ValueError(
                 f"When using `deepspeed_config_file`, the following accelerate config variables will be ignored: {env_variable_names_to_ignore}.\n"
                 "Please specify them appropriately in the DeepSpeed config file.\n"
-                "If you are using an accelerate config file, set `mixed_precision=no` "
-                "and remove others config variables mentioned in the above specified list; "
+                "If you are using an accelerate config file, remove others config variables mentioned in the above specified list; "
                 "and make sure to not specify these config variables in `accelerate launch` command. \n"
-                "The easiest method is to create new config following the questionnaire via  `accelerate config`.\n"
+                "The easiest method is to create new config following the questionnaire via `accelerate config`.\n"
body: Done. I've simplified the error message wherein user can just remove all the ambiguous entries from  accelerate config file and not specify them in launch command
diff_hunk: @@ -549,6 +551,40 @@ def zero3_init_context_manager(self, enable=False):
             self.dschf = None
             self.set_deepspeed_weakref()
 
+    def _deepspeed_config_checks(self):
+        env_variable_names_to_ignore = [
+            "ACCELERATE_GRADIENT_ACCUMULATION_STEPS",
+            "ACCELERATE_GRADIENT_CLIPPING",
+            "ACCELERATE_DEEPSPEED_ZERO_STAGE",
+            "ACCELERATE_DEEPSPEED_OFFLOAD_OPTIMIZER_DEVICE",
+            "ACCELERATE_DEEPSPEED_OFFLOAD_PARAM_DEVICE",
+            "ACCELERATE_DEEPSPEED_ZERO3_SAVE_16BIT_MODEL",
+            "ACCELERATE_MIXED_PRECISION",
+        ]
+        duplicate_values_flag = False
+        for name in env_variable_names_to_ignore:
+            if name != "ACCELERATE_MIXED_PRECISION" and os.environ.get(name, None) is not None:
+                duplicate_values_flag = True
+            elif name == "ACCELERATE_MIXED_PRECISION" and os.environ.get(name, "no") != "no":
+                duplicate_values_flag = True
+            if duplicate_values_flag:
+                break
+
+        if duplicate_values_flag:
+            env_variable_names_to_ignore = [
+                name.replace("ACCELERATE_", "").replace("DEEPSPEED_", "").lower()
+                for name in env_variable_names_to_ignore
+            ]
+            raise ValueError(
+                f"When using `deepspeed_config_file`, the following accelerate config variables will be ignored: {env_variable_names_to_ignore}.\n"
+                "Please specify them appropriately in the DeepSpeed config file.\n"
+                "If you are using an accelerate config file, set `mixed_precision=no` "
body: to focus we are dealing with `mixed_precision` duplicity here:

I've just tried:

```
deepspeed_config:
  deepspeed_multinode_launcher: standard
  deepspeed_config_file: ./configs/vopt-large-z3/ds_config.json
  zero3_init_flag: true
distributed_type: DEEPSPEED
mixed_precision: 'no'
```

```
{
    "fp16": {
        "enabled": true,
        [...]
```
and it doesn't assert.

with the latest commit of `dd2d57b6a1`


diff_hunk: @@ -549,6 +551,40 @@ def zero3_init_context_manager(self, enable=False):
             self.dschf = None
             self.set_deepspeed_weakref()
 
+    def _deepspeed_config_checks(self):
+        env_variable_names_to_ignore = [
+            "ACCELERATE_GRADIENT_ACCUMULATION_STEPS",
+            "ACCELERATE_GRADIENT_CLIPPING",
+            "ACCELERATE_DEEPSPEED_ZERO_STAGE",
+            "ACCELERATE_DEEPSPEED_OFFLOAD_OPTIMIZER_DEVICE",
+            "ACCELERATE_DEEPSPEED_OFFLOAD_PARAM_DEVICE",
+            "ACCELERATE_DEEPSPEED_ZERO3_SAVE_16BIT_MODEL",
+            "ACCELERATE_MIXED_PRECISION",
+        ]
+        duplicate_values_flag = False
+        for name in env_variable_names_to_ignore:
+            if name != "ACCELERATE_MIXED_PRECISION" and os.environ.get(name, None) is not None:
+                duplicate_values_flag = True
+            elif name == "ACCELERATE_MIXED_PRECISION" and os.environ.get(name, "no") != "no":
+                duplicate_values_flag = True
+            if duplicate_values_flag:
+                break
+
+        if duplicate_values_flag:
+            env_variable_names_to_ignore = [
+                name.replace("ACCELERATE_", "").replace("DEEPSPEED_", "").lower()
+                for name in env_variable_names_to_ignore
+            ]
+            raise ValueError(
+                f"When using `deepspeed_config_file`, the following accelerate config variables will be ignored: {env_variable_names_to_ignore}.\n"
+                "Please specify them appropriately in the DeepSpeed config file.\n"
+                "If you are using an accelerate config file, set `mixed_precision=no` "
body: (incidentally `mixed_precision: fp16|bf16|'no'` is odd - why do different values don't follow the same style - i.e 'no' in quotes but not the other values)
diff_hunk: @@ -549,6 +551,40 @@ def zero3_init_context_manager(self, enable=False):
             self.dschf = None
             self.set_deepspeed_weakref()
 
+    def _deepspeed_config_checks(self):
+        env_variable_names_to_ignore = [
+            "ACCELERATE_GRADIENT_ACCUMULATION_STEPS",
+            "ACCELERATE_GRADIENT_CLIPPING",
+            "ACCELERATE_DEEPSPEED_ZERO_STAGE",
+            "ACCELERATE_DEEPSPEED_OFFLOAD_OPTIMIZER_DEVICE",
+            "ACCELERATE_DEEPSPEED_OFFLOAD_PARAM_DEVICE",
+            "ACCELERATE_DEEPSPEED_ZERO3_SAVE_16BIT_MODEL",
+            "ACCELERATE_MIXED_PRECISION",
+        ]
+        duplicate_values_flag = False
+        for name in env_variable_names_to_ignore:
+            if name != "ACCELERATE_MIXED_PRECISION" and os.environ.get(name, None) is not None:
+                duplicate_values_flag = True
+            elif name == "ACCELERATE_MIXED_PRECISION" and os.environ.get(name, "no") != "no":
+                duplicate_values_flag = True
+            if duplicate_values_flag:
+                break
+
+        if duplicate_values_flag:
+            env_variable_names_to_ignore = [
+                name.replace("ACCELERATE_", "").replace("DEEPSPEED_", "").lower()
+                for name in env_variable_names_to_ignore
+            ]
+            raise ValueError(
+                f"When using `deepspeed_config_file`, the following accelerate config variables will be ignored: {env_variable_names_to_ignore}.\n"
+                "Please specify them appropriately in the DeepSpeed config file.\n"
+                "If you are using an accelerate config file, set `mixed_precision=no` "
body: Hello @stas, this case can't be helped as 'no' is the default value and if one doesn't specify the entry in accelerate config file, it will get default value of 'no' and thereby we have no way to check if user has given the default in config file or was it set to default when reading the config file. It would mean a lot of code rewriting as `mixed_precision` is used in various other non DeepSpeed parts.

The default value of `mixed_precision` would be overriden by that in `deepspeed_config_file`. Now, if you answer the questionnaire via `accelerate config`, the entry of `mixed_precision` won't be there when using `deepspeed_config_file`. 


@sgugger, what are your thoughts around this. 
diff_hunk: @@ -549,6 +551,40 @@ def zero3_init_context_manager(self, enable=False):
             self.dschf = None
             self.set_deepspeed_weakref()
 
+    def _deepspeed_config_checks(self):
+        env_variable_names_to_ignore = [
+            "ACCELERATE_GRADIENT_ACCUMULATION_STEPS",
+            "ACCELERATE_GRADIENT_CLIPPING",
+            "ACCELERATE_DEEPSPEED_ZERO_STAGE",
+            "ACCELERATE_DEEPSPEED_OFFLOAD_OPTIMIZER_DEVICE",
+            "ACCELERATE_DEEPSPEED_OFFLOAD_PARAM_DEVICE",
+            "ACCELERATE_DEEPSPEED_ZERO3_SAVE_16BIT_MODEL",
+            "ACCELERATE_MIXED_PRECISION",
+        ]
+        duplicate_values_flag = False
+        for name in env_variable_names_to_ignore:
+            if name != "ACCELERATE_MIXED_PRECISION" and os.environ.get(name, None) is not None:
+                duplicate_values_flag = True
+            elif name == "ACCELERATE_MIXED_PRECISION" and os.environ.get(name, "no") != "no":
+                duplicate_values_flag = True
+            if duplicate_values_flag:
+                break
+
+        if duplicate_values_flag:
+            env_variable_names_to_ignore = [
+                name.replace("ACCELERATE_", "").replace("DEEPSPEED_", "").lower()
+                for name in env_variable_names_to_ignore
+            ]
+            raise ValueError(
+                f"When using `deepspeed_config_file`, the following accelerate config variables will be ignored: {env_variable_names_to_ignore}.\n"
+                "Please specify them appropriately in the DeepSpeed config file.\n"
+                "If you are using an accelerate config file, set `mixed_precision=no` "
body: understood! in which case I think this should work:

1. if the `mixed_precision` config isn't in the accelerate config file all is good
2. if the `mixed_precision` config is in the accelerate config file and ds_config file is used, the value of the former must match the value of the latter or assert.
diff_hunk: @@ -549,6 +551,40 @@ def zero3_init_context_manager(self, enable=False):
             self.dschf = None
             self.set_deepspeed_weakref()
 
+    def _deepspeed_config_checks(self):
+        env_variable_names_to_ignore = [
+            "ACCELERATE_GRADIENT_ACCUMULATION_STEPS",
+            "ACCELERATE_GRADIENT_CLIPPING",
+            "ACCELERATE_DEEPSPEED_ZERO_STAGE",
+            "ACCELERATE_DEEPSPEED_OFFLOAD_OPTIMIZER_DEVICE",
+            "ACCELERATE_DEEPSPEED_OFFLOAD_PARAM_DEVICE",
+            "ACCELERATE_DEEPSPEED_ZERO3_SAVE_16BIT_MODEL",
+            "ACCELERATE_MIXED_PRECISION",
+        ]
+        duplicate_values_flag = False
+        for name in env_variable_names_to_ignore:
+            if name != "ACCELERATE_MIXED_PRECISION" and os.environ.get(name, None) is not None:
+                duplicate_values_flag = True
+            elif name == "ACCELERATE_MIXED_PRECISION" and os.environ.get(name, "no") != "no":
+                duplicate_values_flag = True
+            if duplicate_values_flag:
+                break
+
+        if duplicate_values_flag:
+            env_variable_names_to_ignore = [
+                name.replace("ACCELERATE_", "").replace("DEEPSPEED_", "").lower()
+                for name in env_variable_names_to_ignore
+            ]
+            raise ValueError(
+                f"When using `deepspeed_config_file`, the following accelerate config variables will be ignored: {env_variable_names_to_ignore}.\n"
+                "Please specify them appropriately in the DeepSpeed config file.\n"
+                "If you are using an accelerate config file, set `mixed_precision=no` "
body: ```suggestion
Below is a sample script using `deepspeed_config_file` in different scenarios.
```
diff_hunk: @@ -395,6 +395,194 @@ We will look at the changes needed in the code when using these.
 based on model, dataloaders, dummy optimizer and dummy schedulers provided to `prepare` method. 
 Only the `auto` fields specified in above examples are handled by `prepare` method and the rest have to be explicitly specified by the user.
 
+**Things to note when using DeepSpeed Config File**
+
+Below is a smaple script using `deepspeed_config_file` in different scenarios.
body: Are the lines of `-` intended? Should there be a new line before the text?
diff_hunk: @@ -395,6 +395,194 @@ We will look at the changes needed in the code when using these.
 based on model, dataloaders, dummy optimizer and dummy schedulers provided to `prepare` method. 
 Only the `auto` fields specified in above examples are handled by `prepare` method and the rest have to be explicitly specified by the user.
 
+**Things to note when using DeepSpeed Config File**
+
+Below is a smaple script using `deepspeed_config_file` in different scenarios.
+
+Code `test.py`:
+
+```python
+from accelerate import Accelerator
+from accelerate.state import AcceleratorState
+
+
+def main():
+    accelerator = Accelerator()
+    accelerator.print(f"{AcceleratorState()}")
+
+
+if __name__ == "__main__":
+    main()
+```
+
+**Scenario 1**: Manually tampered accelerate config file having `deepspeed_config_file` along with other entries.
+
+1. `accelerate config`:
+
+```yaml
+command_file: null
+commands: null
+compute_environment: LOCAL_MACHINE
+deepspeed_config:
+  gradient_accumulation_steps: 1
+  gradient_clipping: 1.0
+  offload_optimizer_device: 'cpu'
+  offload_param_device: 'cpu'
+  zero3_init_flag: true
+  zero3_save_16bit_model: true
+  zero_stage: 3
+  deepspeed_config_file: 'ds_config.json'
+distributed_type: DEEPSPEED
+downcast_bf16: 'no'
+dynamo_backend: 'NO'
+fsdp_config: {}
+gpu_ids: null
+machine_rank: 0
+main_process_ip: null
+main_process_port: null
+main_training_function: main
+megatron_lm_config: {}
+num_machines: 1
+num_processes: 2
+rdzv_backend: static
+same_network: true
+tpu_name: null
+tpu_zone: null
+use_cpu: false
+```
+
+2. `ds_config.json`:
+
+```json
+{
+    "bf16": {
+        "enabled": true
+    },
+    "zero_optimization": {
+        "stage": 3,
+        "stage3_gather_16bit_weights_on_model_save": false,
+        "offload_optimizer": {
+            "device": "none"
+        },
+        "offload_param": {
+            "device": "none"
+        }
+    },
+    "gradient_clipping": 1.0,
+    "train_batch_size": "auto",
+    "train_micro_batch_size_per_gpu": "auto",
+    "gradient_accumulation_steps": 10,
+    "steps_per_print": 2000000
+}
+```
+
+3. Output of `accelerate launch test.py`:
+
+```bash
+ValueError: When using `deepspeed_config_file`, the following accelerate config variables will be ignored: 
+['gradient_accumulation_steps', 'gradient_clipping', 'zero_stage', 'offload_optimizer_device', 'offload_param_device', 
+'zero3_save_16bit_model', 'mixed_precision'].
+Please specify them appropriately in the DeepSpeed config file.
+If you are using an accelerate config file, remove others config variables mentioned in the above specified list.
+The easiest method is to create a new config following the questionnaire via `accelerate config`.
+It will only ask for the necessary config variables when using `deepspeed_config_file`.
+```
+
+**Scenario 2**: Use the solution of the error to create new accelerate config and check that no ambiguity error is now thrown.
+
+1. Run `accelerate config`:
+
+```bash
+$ accelerate config
+-------------------------------------------------------------------------------------------------------------------------------In which compute environment are you running?
+This machine                                                                                                                   
+-------------------------------------------------------------------------------------------------------------------------------Which type of machine are you using?                                                                                           
body: ```suggestion
1. Content of the `accelerate` config:
```
(to not mistake this with the command `accelerate config`
diff_hunk: @@ -395,6 +395,194 @@ We will look at the changes needed in the code when using these.
 based on model, dataloaders, dummy optimizer and dummy schedulers provided to `prepare` method. 
 Only the `auto` fields specified in above examples are handled by `prepare` method and the rest have to be explicitly specified by the user.
 
+**Things to note when using DeepSpeed Config File**
+
+Below is a smaple script using `deepspeed_config_file` in different scenarios.
+
+Code `test.py`:
+
+```python
+from accelerate import Accelerator
+from accelerate.state import AcceleratorState
+
+
+def main():
+    accelerator = Accelerator()
+    accelerator.print(f"{AcceleratorState()}")
+
+
+if __name__ == "__main__":
+    main()
+```
+
+**Scenario 1**: Manually tampered accelerate config file having `deepspeed_config_file` along with other entries.
+
+1. `accelerate config`:
body: ```suggestion
2. Content of the `accelerate` config:
```
diff_hunk: @@ -395,6 +395,194 @@ We will look at the changes needed in the code when using these.
 based on model, dataloaders, dummy optimizer and dummy schedulers provided to `prepare` method. 
 Only the `auto` fields specified in above examples are handled by `prepare` method and the rest have to be explicitly specified by the user.
 
+**Things to note when using DeepSpeed Config File**
+
+Below is a smaple script using `deepspeed_config_file` in different scenarios.
+
+Code `test.py`:
+
+```python
+from accelerate import Accelerator
+from accelerate.state import AcceleratorState
+
+
+def main():
+    accelerator = Accelerator()
+    accelerator.print(f"{AcceleratorState()}")
+
+
+if __name__ == "__main__":
+    main()
+```
+
+**Scenario 1**: Manually tampered accelerate config file having `deepspeed_config_file` along with other entries.
+
+1. `accelerate config`:
+
+```yaml
+command_file: null
+commands: null
+compute_environment: LOCAL_MACHINE
+deepspeed_config:
+  gradient_accumulation_steps: 1
+  gradient_clipping: 1.0
+  offload_optimizer_device: 'cpu'
+  offload_param_device: 'cpu'
+  zero3_init_flag: true
+  zero3_save_16bit_model: true
+  zero_stage: 3
+  deepspeed_config_file: 'ds_config.json'
+distributed_type: DEEPSPEED
+downcast_bf16: 'no'
+dynamo_backend: 'NO'
+fsdp_config: {}
+gpu_ids: null
+machine_rank: 0
+main_process_ip: null
+main_process_port: null
+main_training_function: main
+megatron_lm_config: {}
+num_machines: 1
+num_processes: 2
+rdzv_backend: static
+same_network: true
+tpu_name: null
+tpu_zone: null
+use_cpu: false
+```
+
+2. `ds_config.json`:
+
+```json
+{
+    "bf16": {
+        "enabled": true
+    },
+    "zero_optimization": {
+        "stage": 3,
+        "stage3_gather_16bit_weights_on_model_save": false,
+        "offload_optimizer": {
+            "device": "none"
+        },
+        "offload_param": {
+            "device": "none"
+        }
+    },
+    "gradient_clipping": 1.0,
+    "train_batch_size": "auto",
+    "train_micro_batch_size_per_gpu": "auto",
+    "gradient_accumulation_steps": 10,
+    "steps_per_print": 2000000
+}
+```
+
+3. Output of `accelerate launch test.py`:
+
+```bash
+ValueError: When using `deepspeed_config_file`, the following accelerate config variables will be ignored: 
+['gradient_accumulation_steps', 'gradient_clipping', 'zero_stage', 'offload_optimizer_device', 'offload_param_device', 
+'zero3_save_16bit_model', 'mixed_precision'].
+Please specify them appropriately in the DeepSpeed config file.
+If you are using an accelerate config file, remove others config variables mentioned in the above specified list.
+The easiest method is to create a new config following the questionnaire via `accelerate config`.
+It will only ask for the necessary config variables when using `deepspeed_config_file`.
+```
+
+**Scenario 2**: Use the solution of the error to create new accelerate config and check that no ambiguity error is now thrown.
+
+1. Run `accelerate config`:
+
+```bash
+$ accelerate config
+-------------------------------------------------------------------------------------------------------------------------------In which compute environment are you running?
+This machine                                                                                                                   
+-------------------------------------------------------------------------------------------------------------------------------Which type of machine are you using?                                                                                           
+multi-GPU                                                                                                                      
+How many different machines will you use (use more than 1 for multi-node training)? [1]:                                       
+Do you wish to optimize your script with torch dynamo?[yes/NO]:                                                                
+Do you want to use DeepSpeed? [yes/NO]: yes                                                                                    
+Do you want to specify a json file to a DeepSpeed config? [yes/NO]: yes                                                        
+Please enter the path to the json DeepSpeed config file: ds_config.json                                                        
+Do you want to enable `deepspeed.zero.Init` when using ZeRO Stage-3 for constructing massive models? [yes/NO]: yes
+How many GPU(s) should be used for distributed training? [1]:4
+accelerate configuration saved at ds_config_sample.yaml
+```
+
+2. `accelerate config`:
body: ```suggestion
**Scenario 3**: Setting the `accelerate launch` command arguments related to DeepSpeed as `"auto"` in the DeepSpeed` configuration file and check that things work as expected.
```
diff_hunk: @@ -395,6 +395,194 @@ We will look at the changes needed in the code when using these.
 based on model, dataloaders, dummy optimizer and dummy schedulers provided to `prepare` method. 
 Only the `auto` fields specified in above examples are handled by `prepare` method and the rest have to be explicitly specified by the user.
 
+**Things to note when using DeepSpeed Config File**
+
+Below is a smaple script using `deepspeed_config_file` in different scenarios.
+
+Code `test.py`:
+
+```python
+from accelerate import Accelerator
+from accelerate.state import AcceleratorState
+
+
+def main():
+    accelerator = Accelerator()
+    accelerator.print(f"{AcceleratorState()}")
+
+
+if __name__ == "__main__":
+    main()
+```
+
+**Scenario 1**: Manually tampered accelerate config file having `deepspeed_config_file` along with other entries.
+
+1. `accelerate config`:
+
+```yaml
+command_file: null
+commands: null
+compute_environment: LOCAL_MACHINE
+deepspeed_config:
+  gradient_accumulation_steps: 1
+  gradient_clipping: 1.0
+  offload_optimizer_device: 'cpu'
+  offload_param_device: 'cpu'
+  zero3_init_flag: true
+  zero3_save_16bit_model: true
+  zero_stage: 3
+  deepspeed_config_file: 'ds_config.json'
+distributed_type: DEEPSPEED
+downcast_bf16: 'no'
+dynamo_backend: 'NO'
+fsdp_config: {}
+gpu_ids: null
+machine_rank: 0
+main_process_ip: null
+main_process_port: null
+main_training_function: main
+megatron_lm_config: {}
+num_machines: 1
+num_processes: 2
+rdzv_backend: static
+same_network: true
+tpu_name: null
+tpu_zone: null
+use_cpu: false
+```
+
+2. `ds_config.json`:
+
+```json
+{
+    "bf16": {
+        "enabled": true
+    },
+    "zero_optimization": {
+        "stage": 3,
+        "stage3_gather_16bit_weights_on_model_save": false,
+        "offload_optimizer": {
+            "device": "none"
+        },
+        "offload_param": {
+            "device": "none"
+        }
+    },
+    "gradient_clipping": 1.0,
+    "train_batch_size": "auto",
+    "train_micro_batch_size_per_gpu": "auto",
+    "gradient_accumulation_steps": 10,
+    "steps_per_print": 2000000
+}
+```
+
+3. Output of `accelerate launch test.py`:
+
+```bash
+ValueError: When using `deepspeed_config_file`, the following accelerate config variables will be ignored: 
+['gradient_accumulation_steps', 'gradient_clipping', 'zero_stage', 'offload_optimizer_device', 'offload_param_device', 
+'zero3_save_16bit_model', 'mixed_precision'].
+Please specify them appropriately in the DeepSpeed config file.
+If you are using an accelerate config file, remove others config variables mentioned in the above specified list.
+The easiest method is to create a new config following the questionnaire via `accelerate config`.
+It will only ask for the necessary config variables when using `deepspeed_config_file`.
+```
+
+**Scenario 2**: Use the solution of the error to create new accelerate config and check that no ambiguity error is now thrown.
+
+1. Run `accelerate config`:
+
+```bash
+$ accelerate config
+-------------------------------------------------------------------------------------------------------------------------------In which compute environment are you running?
+This machine                                                                                                                   
+-------------------------------------------------------------------------------------------------------------------------------Which type of machine are you using?                                                                                           
+multi-GPU                                                                                                                      
+How many different machines will you use (use more than 1 for multi-node training)? [1]:                                       
+Do you wish to optimize your script with torch dynamo?[yes/NO]:                                                                
+Do you want to use DeepSpeed? [yes/NO]: yes                                                                                    
+Do you want to specify a json file to a DeepSpeed config? [yes/NO]: yes                                                        
+Please enter the path to the json DeepSpeed config file: ds_config.json                                                        
+Do you want to enable `deepspeed.zero.Init` when using ZeRO Stage-3 for constructing massive models? [yes/NO]: yes
+How many GPU(s) should be used for distributed training? [1]:4
+accelerate configuration saved at ds_config_sample.yaml
+```
+
+2. `accelerate config`:
+
+```yaml
+compute_environment: LOCAL_MACHINE
+deepspeed_config:
+  deepspeed_config_file: ds_config.json
+  zero3_init_flag: true
+distributed_type: DEEPSPEED
+downcast_bf16: 'no'
+dynamo_backend: 'NO'
+fsdp_config: {}
+machine_rank: 0
+main_training_function: main
+megatron_lm_config: {}
+num_machines: 1
+num_processes: 4
+rdzv_backend: static
+same_network: true
+use_cpu: false
+```
+
+3. Output of `accelerate launch test.py`:
+
+```bash
+Distributed environment: DEEPSPEED  Backend: nccl
+Num processes: 4
+Process index: 0
+Local process index: 0
+Device: cuda:0
+Mixed precision type: bf16
+ds_config: {'bf16': {'enabled': True}, 'zero_optimization': {'stage': 3, 'stage3_gather_16bit_weights_on_model_save': False, 'offload_optimizer': {'device': 'none'}, 'offload_param': {'device': 'none'}}, 'gradient_clipping': 1.0, 'train_batch_size': 'auto', 'train_micro_batch_size_per_gpu': 'auto', 'gradient_accumulation_steps': 10, 'steps_per_print': inf, 'fp16': {'enabled': False}}
+```
+
+**Scenario 3**: Setting the `accelerate launch` cmd args related to deepspeed as `auto` in  `deepspeed_config_file` and check that things work as expected.
body: ```suggestion
1. New `ds_config.json` with `"auto"` for the `accelerate launch` DeepSpeed command arguments:
```
diff_hunk: @@ -395,6 +395,194 @@ We will look at the changes needed in the code when using these.
 based on model, dataloaders, dummy optimizer and dummy schedulers provided to `prepare` method. 
 Only the `auto` fields specified in above examples are handled by `prepare` method and the rest have to be explicitly specified by the user.
 
+**Things to note when using DeepSpeed Config File**
+
+Below is a smaple script using `deepspeed_config_file` in different scenarios.
+
+Code `test.py`:
+
+```python
+from accelerate import Accelerator
+from accelerate.state import AcceleratorState
+
+
+def main():
+    accelerator = Accelerator()
+    accelerator.print(f"{AcceleratorState()}")
+
+
+if __name__ == "__main__":
+    main()
+```
+
+**Scenario 1**: Manually tampered accelerate config file having `deepspeed_config_file` along with other entries.
+
+1. `accelerate config`:
+
+```yaml
+command_file: null
+commands: null
+compute_environment: LOCAL_MACHINE
+deepspeed_config:
+  gradient_accumulation_steps: 1
+  gradient_clipping: 1.0
+  offload_optimizer_device: 'cpu'
+  offload_param_device: 'cpu'
+  zero3_init_flag: true
+  zero3_save_16bit_model: true
+  zero_stage: 3
+  deepspeed_config_file: 'ds_config.json'
+distributed_type: DEEPSPEED
+downcast_bf16: 'no'
+dynamo_backend: 'NO'
+fsdp_config: {}
+gpu_ids: null
+machine_rank: 0
+main_process_ip: null
+main_process_port: null
+main_training_function: main
+megatron_lm_config: {}
+num_machines: 1
+num_processes: 2
+rdzv_backend: static
+same_network: true
+tpu_name: null
+tpu_zone: null
+use_cpu: false
+```
+
+2. `ds_config.json`:
+
+```json
+{
+    "bf16": {
+        "enabled": true
+    },
+    "zero_optimization": {
+        "stage": 3,
+        "stage3_gather_16bit_weights_on_model_save": false,
+        "offload_optimizer": {
+            "device": "none"
+        },
+        "offload_param": {
+            "device": "none"
+        }
+    },
+    "gradient_clipping": 1.0,
+    "train_batch_size": "auto",
+    "train_micro_batch_size_per_gpu": "auto",
+    "gradient_accumulation_steps": 10,
+    "steps_per_print": 2000000
+}
+```
+
+3. Output of `accelerate launch test.py`:
+
+```bash
+ValueError: When using `deepspeed_config_file`, the following accelerate config variables will be ignored: 
+['gradient_accumulation_steps', 'gradient_clipping', 'zero_stage', 'offload_optimizer_device', 'offload_param_device', 
+'zero3_save_16bit_model', 'mixed_precision'].
+Please specify them appropriately in the DeepSpeed config file.
+If you are using an accelerate config file, remove others config variables mentioned in the above specified list.
+The easiest method is to create a new config following the questionnaire via `accelerate config`.
+It will only ask for the necessary config variables when using `deepspeed_config_file`.
+```
+
+**Scenario 2**: Use the solution of the error to create new accelerate config and check that no ambiguity error is now thrown.
+
+1. Run `accelerate config`:
+
+```bash
+$ accelerate config
+-------------------------------------------------------------------------------------------------------------------------------In which compute environment are you running?
+This machine                                                                                                                   
+-------------------------------------------------------------------------------------------------------------------------------Which type of machine are you using?                                                                                           
+multi-GPU                                                                                                                      
+How many different machines will you use (use more than 1 for multi-node training)? [1]:                                       
+Do you wish to optimize your script with torch dynamo?[yes/NO]:                                                                
+Do you want to use DeepSpeed? [yes/NO]: yes                                                                                    
+Do you want to specify a json file to a DeepSpeed config? [yes/NO]: yes                                                        
+Please enter the path to the json DeepSpeed config file: ds_config.json                                                        
+Do you want to enable `deepspeed.zero.Init` when using ZeRO Stage-3 for constructing massive models? [yes/NO]: yes
+How many GPU(s) should be used for distributed training? [1]:4
+accelerate configuration saved at ds_config_sample.yaml
+```
+
+2. `accelerate config`:
+
+```yaml
+compute_environment: LOCAL_MACHINE
+deepspeed_config:
+  deepspeed_config_file: ds_config.json
+  zero3_init_flag: true
+distributed_type: DEEPSPEED
+downcast_bf16: 'no'
+dynamo_backend: 'NO'
+fsdp_config: {}
+machine_rank: 0
+main_training_function: main
+megatron_lm_config: {}
+num_machines: 1
+num_processes: 4
+rdzv_backend: static
+same_network: true
+use_cpu: false
+```
+
+3. Output of `accelerate launch test.py`:
+
+```bash
+Distributed environment: DEEPSPEED  Backend: nccl
+Num processes: 4
+Process index: 0
+Local process index: 0
+Device: cuda:0
+Mixed precision type: bf16
+ds_config: {'bf16': {'enabled': True}, 'zero_optimization': {'stage': 3, 'stage3_gather_16bit_weights_on_model_save': False, 'offload_optimizer': {'device': 'none'}, 'offload_param': {'device': 'none'}}, 'gradient_clipping': 1.0, 'train_batch_size': 'auto', 'train_micro_batch_size_per_gpu': 'auto', 'gradient_accumulation_steps': 10, 'steps_per_print': inf, 'fp16': {'enabled': False}}
+```
+
+**Scenario 3**: Setting the `accelerate launch` cmd args related to deepspeed as `auto` in  `deepspeed_config_file` and check that things work as expected.
+
+1. New `ds_config.json` with `auto` for the `accelerate launch` deepspeed cmd args:
body: ```suggestion
**Note**: Remaining `"auto"` values are handled in `accelerator.prepare()` call as explained in point 2 of 
```
diff_hunk: @@ -395,6 +395,194 @@ We will look at the changes needed in the code when using these.
 based on model, dataloaders, dummy optimizer and dummy schedulers provided to `prepare` method. 
 Only the `auto` fields specified in above examples are handled by `prepare` method and the rest have to be explicitly specified by the user.
 
+**Things to note when using DeepSpeed Config File**
+
+Below is a smaple script using `deepspeed_config_file` in different scenarios.
+
+Code `test.py`:
+
+```python
+from accelerate import Accelerator
+from accelerate.state import AcceleratorState
+
+
+def main():
+    accelerator = Accelerator()
+    accelerator.print(f"{AcceleratorState()}")
+
+
+if __name__ == "__main__":
+    main()
+```
+
+**Scenario 1**: Manually tampered accelerate config file having `deepspeed_config_file` along with other entries.
+
+1. `accelerate config`:
+
+```yaml
+command_file: null
+commands: null
+compute_environment: LOCAL_MACHINE
+deepspeed_config:
+  gradient_accumulation_steps: 1
+  gradient_clipping: 1.0
+  offload_optimizer_device: 'cpu'
+  offload_param_device: 'cpu'
+  zero3_init_flag: true
+  zero3_save_16bit_model: true
+  zero_stage: 3
+  deepspeed_config_file: 'ds_config.json'
+distributed_type: DEEPSPEED
+downcast_bf16: 'no'
+dynamo_backend: 'NO'
+fsdp_config: {}
+gpu_ids: null
+machine_rank: 0
+main_process_ip: null
+main_process_port: null
+main_training_function: main
+megatron_lm_config: {}
+num_machines: 1
+num_processes: 2
+rdzv_backend: static
+same_network: true
+tpu_name: null
+tpu_zone: null
+use_cpu: false
+```
+
+2. `ds_config.json`:
+
+```json
+{
+    "bf16": {
+        "enabled": true
+    },
+    "zero_optimization": {
+        "stage": 3,
+        "stage3_gather_16bit_weights_on_model_save": false,
+        "offload_optimizer": {
+            "device": "none"
+        },
+        "offload_param": {
+            "device": "none"
+        }
+    },
+    "gradient_clipping": 1.0,
+    "train_batch_size": "auto",
+    "train_micro_batch_size_per_gpu": "auto",
+    "gradient_accumulation_steps": 10,
+    "steps_per_print": 2000000
+}
+```
+
+3. Output of `accelerate launch test.py`:
+
+```bash
+ValueError: When using `deepspeed_config_file`, the following accelerate config variables will be ignored: 
+['gradient_accumulation_steps', 'gradient_clipping', 'zero_stage', 'offload_optimizer_device', 'offload_param_device', 
+'zero3_save_16bit_model', 'mixed_precision'].
+Please specify them appropriately in the DeepSpeed config file.
+If you are using an accelerate config file, remove others config variables mentioned in the above specified list.
+The easiest method is to create a new config following the questionnaire via `accelerate config`.
+It will only ask for the necessary config variables when using `deepspeed_config_file`.
+```
+
+**Scenario 2**: Use the solution of the error to create new accelerate config and check that no ambiguity error is now thrown.
+
+1. Run `accelerate config`:
+
+```bash
+$ accelerate config
+-------------------------------------------------------------------------------------------------------------------------------In which compute environment are you running?
+This machine                                                                                                                   
+-------------------------------------------------------------------------------------------------------------------------------Which type of machine are you using?                                                                                           
+multi-GPU                                                                                                                      
+How many different machines will you use (use more than 1 for multi-node training)? [1]:                                       
+Do you wish to optimize your script with torch dynamo?[yes/NO]:                                                                
+Do you want to use DeepSpeed? [yes/NO]: yes                                                                                    
+Do you want to specify a json file to a DeepSpeed config? [yes/NO]: yes                                                        
+Please enter the path to the json DeepSpeed config file: ds_config.json                                                        
+Do you want to enable `deepspeed.zero.Init` when using ZeRO Stage-3 for constructing massive models? [yes/NO]: yes
+How many GPU(s) should be used for distributed training? [1]:4
+accelerate configuration saved at ds_config_sample.yaml
+```
+
+2. `accelerate config`:
+
+```yaml
+compute_environment: LOCAL_MACHINE
+deepspeed_config:
+  deepspeed_config_file: ds_config.json
+  zero3_init_flag: true
+distributed_type: DEEPSPEED
+downcast_bf16: 'no'
+dynamo_backend: 'NO'
+fsdp_config: {}
+machine_rank: 0
+main_training_function: main
+megatron_lm_config: {}
+num_machines: 1
+num_processes: 4
+rdzv_backend: static
+same_network: true
+use_cpu: false
+```
+
+3. Output of `accelerate launch test.py`:
+
+```bash
+Distributed environment: DEEPSPEED  Backend: nccl
+Num processes: 4
+Process index: 0
+Local process index: 0
+Device: cuda:0
+Mixed precision type: bf16
+ds_config: {'bf16': {'enabled': True}, 'zero_optimization': {'stage': 3, 'stage3_gather_16bit_weights_on_model_save': False, 'offload_optimizer': {'device': 'none'}, 'offload_param': {'device': 'none'}}, 'gradient_clipping': 1.0, 'train_batch_size': 'auto', 'train_micro_batch_size_per_gpu': 'auto', 'gradient_accumulation_steps': 10, 'steps_per_print': inf, 'fp16': {'enabled': False}}
+```
+
+**Scenario 3**: Setting the `accelerate launch` cmd args related to deepspeed as `auto` in  `deepspeed_config_file` and check that things work as expected.
+
+1. New `ds_config.json` with `auto` for the `accelerate launch` deepspeed cmd args:
+
+```json
+{
+    "bf16": {
+        "enabled": "auto"
+    },
+    "zero_optimization": {
+        "stage": "auto",
+        "stage3_gather_16bit_weights_on_model_save": "auto",
+        "offload_optimizer": {
+            "device": "auto"
+        },
+        "offload_param": {
+            "device": "auto"
+        }
+    },
+    "gradient_clipping": "auto",
+    "train_batch_size": "auto",
+    "train_micro_batch_size_per_gpu": "auto",
+    "gradient_accumulation_steps": "auto",
+    "steps_per_print": 2000000
+}
+```
+
+2. Output of `accelerate launch --mixed_precision="fp16" --zero_stage=3 --gradient_accumulation_steps=5 --gradient_clipping=1.0 --offload_param_device="cpu" --offload_optimizer_device="nvme" --zero3_save_16bit_model="true" test.py`:
+
+```bash
+Distributed environment: DEEPSPEED  Backend: nccl
+Num processes: 4
+Process index: 0
+Local process index: 0
+Device: cuda:0
+Mixed precision type: fp16
+ds_config: {'bf16': {'enabled': False}, 'zero_optimization': {'stage': 3, 'stage3_gather_16bit_weights_on_model_save': True, 'offload_optimizer': {'device': 'nvme'}, 'offload_param': {'device': 'cpu'}}, 'gradient_clipping': 1.0, 'train_batch_size': 'auto', 'train_micro_batch_size_per_gpu': 'auto', 'gradient_accumulation_steps': 5, 'steps_per_print': inf, 'fp16': {'enabled': True, 'auto_cast': True}}
+```
+
+**Note**: Remaining `auto` values are handled in `accelerator.prepare()` call as explained in point 2 of 
body: Why is this set to True by default now?
diff_hunk: @@ -450,15 +461,17 @@ def __post_init__(self):
             if self.gradient_clipping:
                 config["gradient_clipping"] = self.gradient_clipping
             self.hf_ds_config = HfDeepSpeedConfig(config)
+
         self.deepspeed_config = self.hf_ds_config.config
         self.deepspeed_config["steps_per_print"] = float("inf")  # this will stop deepspeed from logging @ stdout
         if self.zero3_init_flag is None:
-            self.zero3_init_flag = os.environ.get("ACCELERATE_DEEPSPEED_ZERO3_INIT", "false") == "true"
+            self.zero3_init_flag = os.environ.get("ACCELERATE_DEEPSPEED_ZERO3_INIT", "true") == "true"
body: these are the exact outputs from the new config #830. Added new lines.  
diff_hunk: @@ -395,6 +395,194 @@ We will look at the changes needed in the code when using these.
 based on model, dataloaders, dummy optimizer and dummy schedulers provided to `prepare` method. 
 Only the `auto` fields specified in above examples are handled by `prepare` method and the rest have to be explicitly specified by the user.
 
+**Things to note when using DeepSpeed Config File**
+
+Below is a smaple script using `deepspeed_config_file` in different scenarios.
+
+Code `test.py`:
+
+```python
+from accelerate import Accelerator
+from accelerate.state import AcceleratorState
+
+
+def main():
+    accelerator = Accelerator()
+    accelerator.print(f"{AcceleratorState()}")
+
+
+if __name__ == "__main__":
+    main()
+```
+
+**Scenario 1**: Manually tampered accelerate config file having `deepspeed_config_file` along with other entries.
+
+1. `accelerate config`:
+
+```yaml
+command_file: null
+commands: null
+compute_environment: LOCAL_MACHINE
+deepspeed_config:
+  gradient_accumulation_steps: 1
+  gradient_clipping: 1.0
+  offload_optimizer_device: 'cpu'
+  offload_param_device: 'cpu'
+  zero3_init_flag: true
+  zero3_save_16bit_model: true
+  zero_stage: 3
+  deepspeed_config_file: 'ds_config.json'
+distributed_type: DEEPSPEED
+downcast_bf16: 'no'
+dynamo_backend: 'NO'
+fsdp_config: {}
+gpu_ids: null
+machine_rank: 0
+main_process_ip: null
+main_process_port: null
+main_training_function: main
+megatron_lm_config: {}
+num_machines: 1
+num_processes: 2
+rdzv_backend: static
+same_network: true
+tpu_name: null
+tpu_zone: null
+use_cpu: false
+```
+
+2. `ds_config.json`:
+
+```json
+{
+    "bf16": {
+        "enabled": true
+    },
+    "zero_optimization": {
+        "stage": 3,
+        "stage3_gather_16bit_weights_on_model_save": false,
+        "offload_optimizer": {
+            "device": "none"
+        },
+        "offload_param": {
+            "device": "none"
+        }
+    },
+    "gradient_clipping": 1.0,
+    "train_batch_size": "auto",
+    "train_micro_batch_size_per_gpu": "auto",
+    "gradient_accumulation_steps": 10,
+    "steps_per_print": 2000000
+}
+```
+
+3. Output of `accelerate launch test.py`:
+
+```bash
+ValueError: When using `deepspeed_config_file`, the following accelerate config variables will be ignored: 
+['gradient_accumulation_steps', 'gradient_clipping', 'zero_stage', 'offload_optimizer_device', 'offload_param_device', 
+'zero3_save_16bit_model', 'mixed_precision'].
+Please specify them appropriately in the DeepSpeed config file.
+If you are using an accelerate config file, remove others config variables mentioned in the above specified list.
+The easiest method is to create a new config following the questionnaire via `accelerate config`.
+It will only ask for the necessary config variables when using `deepspeed_config_file`.
+```
+
+**Scenario 2**: Use the solution of the error to create new accelerate config and check that no ambiguity error is now thrown.
+
+1. Run `accelerate config`:
+
+```bash
+$ accelerate config
+-------------------------------------------------------------------------------------------------------------------------------In which compute environment are you running?
+This machine                                                                                                                   
+-------------------------------------------------------------------------------------------------------------------------------Which type of machine are you using?                                                                                           
body: Hello, as stas mentioned in Issue #922 

> this flag should be True by default as zero stage 3 is for large models, it's very unlikely the user will be able to load those models w/o zero.Init
diff_hunk: @@ -450,15 +461,17 @@ def __post_init__(self):
             if self.gradient_clipping:
                 config["gradient_clipping"] = self.gradient_clipping
             self.hf_ds_config = HfDeepSpeedConfig(config)
+
         self.deepspeed_config = self.hf_ds_config.config
         self.deepspeed_config["steps_per_print"] = float("inf")  # this will stop deepspeed from logging @ stdout
         if self.zero3_init_flag is None:
-            self.zero3_init_flag = os.environ.get("ACCELERATE_DEEPSPEED_ZERO3_INIT", "false") == "true"
+            self.zero3_init_flag = os.environ.get("ACCELERATE_DEEPSPEED_ZERO3_INIT", "true") == "true"
body: Is this only used if zero3 is enabled in the rest of the code? It should at least only default to True when Zero-3 is enabled otherwise.
diff_hunk: @@ -450,15 +461,17 @@ def __post_init__(self):
             if self.gradient_clipping:
                 config["gradient_clipping"] = self.gradient_clipping
             self.hf_ds_config = HfDeepSpeedConfig(config)
+
         self.deepspeed_config = self.hf_ds_config.config
         self.deepspeed_config["steps_per_print"] = float("inf")  # this will stop deepspeed from logging @ stdout
         if self.zero3_init_flag is None:
-            self.zero3_init_flag = os.environ.get("ACCELERATE_DEEPSPEED_ZERO3_INIT", "false") == "true"
+            self.zero3_init_flag = os.environ.get("ACCELERATE_DEEPSPEED_ZERO3_INIT", "true") == "true"
body: Yes, only used when zero3 is enabled else will give a warning and gets set to False as mentioned in on of the above threads. Done, defaults to True only for stage 3.
diff_hunk: @@ -450,15 +461,17 @@ def __post_init__(self):
             if self.gradient_clipping:
                 config["gradient_clipping"] = self.gradient_clipping
             self.hf_ds_config = HfDeepSpeedConfig(config)
+
         self.deepspeed_config = self.hf_ds_config.config
         self.deepspeed_config["steps_per_print"] = float("inf")  # this will stop deepspeed from logging @ stdout
         if self.zero3_init_flag is None:
-            self.zero3_init_flag = os.environ.get("ACCELERATE_DEEPSPEED_ZERO3_INIT", "false") == "true"
+            self.zero3_init_flag = os.environ.get("ACCELERATE_DEEPSPEED_ZERO3_INIT", "true") == "true"

Discussion: body: _The documentation is not available anymore as the PR was closed or merged._
body: I tried this branch, getting:

```
Traceback (most recent call last):
  File "m4/training/main.py", line 47, in <module>
    accelerator = Accelerator(
  File "/mnt/nvme0/code/huggingface/accelerate-master/src/accelerate/accelerator.py", line 246, in __init__
    DeepSpeedPlugin() if os.environ.get("ACCELERATE_USE_DEEPSPEED", "false") == "true" else None
  File "<string>", line 12, in __init__
  File "/mnt/nvme0/code/huggingface/accelerate-master/src/accelerate/utils/dataclasses.py", line 412, in __post_init__
    self._deepspeed_config_checks()
  File "/mnt/nvme0/code/huggingface/accelerate-master/src/accelerate/utils/dataclasses.py", line 560, in _deepspeed_config_checks
    if ds_gradient_accumulation_steps != int(accelerate_gradient_accumulation_steps):
ValueError: invalid literal for int() with base 10: 'None'
```

configs:

```
{
    "fp16": {
        "enabled": true,
        "auto_cast": true,
        "loss_scale": 0.0,
        "initial_scale_power": 10,
        "loss_scale_window": 1000,
        "hysteresis": 2,
        "min_loss_scale": 1
    },
    "zero_optimization": {
        "stage": 3,
        "allgather_partitions": true,
        "allgather_bucket_size": 5e8,
        "overlap_comm": false,
        "reduce_scatter": true,
        "reduce_bucket_size": "auto",
        "contiguous_gradients": true,
        "stage3_gather_16bit_weights_on_model_save": false,
        "offload_optimizer": {
            "device": "none"
        },
        "offload_param": {
            "device": "none"
        }
    },
    "gradient_clipping": 1.0,
    "gradient_accumulation_steps": 2,
    "train_batch_size": "auto",
    "train_micro_batch_size_per_gpu": "auto",
    "steps_per_print": 2000000
}

compute_environment: LOCAL_MACHINE
deepspeed_config:
  deepspeed_multinode_launcher: standard
  deepspeed_config_file: ./configs/vopt-large-z3/ds_config.json
  zero3_init_flag: true
distributed_type: DEEPSPEED
fsdp_config: {}
machine_rank: 0
main_process_ip: null
main_process_port: null
main_training_function: main
num_machines: 1
num_processes: 1
#num_machines: 20
#num_processes: 80
use_cpu: false


```
body: @pacman100 default needs to be 1 instead of `None` :)
body: Also clarifying defaults for args in this PR. Now, `accelerate launch --use_deepspeed --help` shows below output wherein default value info is specified `If unspecified, will default to *`:
```
...

DeepSpeed Arguments:
  Arguments related to DeepSpeed.

  --deepspeed_config_file DEEPSPEED_CONFIG_FILE
                        DeepSpeed config file.
  --zero_stage ZERO_STAGE
                        DeepSpeed's ZeRO optimization stage (useful only when `use_deepspeed` flag
                        is passed). If unspecified, will default to `2`.
  --offload_optimizer_device OFFLOAD_OPTIMIZER_DEVICE
                        Decides where (none|cpu|nvme) to offload optimizer states (useful only
                        when `use_deepspeed` flag is passed). If unspecified, will default to
                        `none`.
  --offload_param_device OFFLOAD_PARAM_DEVICE
                        Decides where (none|cpu|nvme) to offload parameters (useful only when
                        `use_deepspeed` flag is passed). If unspecified, will default to `none`.
  --gradient_accumulation_steps GRADIENT_ACCUMULATION_STEPS
                        No of gradient_accumulation_steps used in your training script (useful
                        only when `use_deepspeed` flag is passed). If unspecified, will default to
                        `1`.
  --gradient_clipping GRADIENT_CLIPPING
                        gradient clipping value used in your training script (useful only when
                        `use_deepspeed` flag is passed). If unspecified, will default to `1.0`.
  --zero3_init_flag ZERO3_INIT_FLAG
                        Decides Whether (true|false) to enable `deepspeed.zero.Init` for
                        constructing massive models. Only applicable with DeepSpeed ZeRO Stage-3.
                        If unspecified, will default to `true`.
  --zero3_save_16bit_model ZERO3_SAVE_16BIT_MODEL
                        Decides Whether (true|false) to save 16-bit model weights when using ZeRO
                        Stage-3. Only applicable with DeepSpeed ZeRO Stage-3. If unspecified, will
                        default to `false`.
  --deepspeed_hostfile DEEPSPEED_HOSTFILE
                        DeepSpeed hostfile for configuring multi-node compute resources.
  --deepspeed_exclusion_filter DEEPSPEED_EXCLUSION_FILTER
                        DeepSpeed exclusion filter string when using mutli-node setup.
  --deepspeed_inclusion_filter DEEPSPEED_INCLUSION_FILTER
                        DeepSpeed inclusion filter string when using mutli-node setup.
  --deepspeed_multinode_launcher DEEPSPEED_MULTINODE_LAUNCHER
                        DeepSpeed multi-node launcher to use. If unspecified, will default to
                        `pdsh`.

```
body: > ```shell
> ValueError: When using `deepspeed_config_file`, the following accelerate config variables will be 
> ignored: ['gradient_accumulation_steps', 'gradient_clipping', 'zero_stage', 
> 'offload_optimizer_device', 'offload_param_device', 'zero3_save_16bit_model', 'mixed_precision'].
> Please specify them appropriately in the DeepSpeed config file.
> If you are using accelerate config file, set `mixed_precision=no` and remove others config variables
> mentioned in the above specified list; else don't specify these config variables in `accelerate 
> launch` command. 
> The easiest method is to create new config following the questionnaire via  `accelerate config`.
> It will only ask for the necessary config variables when using `deepspeed_config_file`.
> ```

Now, the error will be this
body: > but really what we want is one definitive source of information.

Hello @stas00, let us know if this addresses the issue. When `deepspeed_config_file` is specified, it is the single definitive source of information and error is raised when duplicates are found either in accelerate config file or through the arguments of `accelerate launch` command
body: > Will this still work with `auto` entries in `ds_config.json`

Hello Stas, they will work as before because nothing is changing other than the fact that an error is raised if their is any duplicate entry in `accelerate config` with things that are in `deepspeed_config_file`. Here, these fields which can be duplicates are never `auto` in either case and hence fields that had `auto` support won't be impacted.
body: Thank you for confirming that, @pacman100 - let me try it out as it's easier to see things in action.
body: But this shows otherwise:
```
  File "/mnt/nvme0/code/huggingface/accelerate-master/src/accelerate/utils/dataclasses.py", line 409, in __post_init__
    raise ValueError("gradient_accumulation_steps cannot be set to 'auto' in the DeepSpeed config.")
ValueError: gradient_accumulation_steps cannot be set to 'auto' in the DeepSpeed config.
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid
```

why can't it be `auto`? I'd expect it to be set either (1) to the default value or (2) the value passed via the `--gradient_accumulation_steps` cmd line arg.

I wonder if we are having a miscommunication here. I brought up the issue of duplicity of the 2 styles of settings the ds config, since accelerate used its own config files from the beginning, but I have never suggested that setting values via cmd line args support should be dropped.
body: Regarding `gradient_accumulation_steps` and `auto`, that piece of code wasn't changed at all and behaviour is the same as before. If one doesn't specify that entry in the config file, it is set to default of `1`, if it is `auto` error is raised else thale given value is used. When not using `deepspeed_config_file`, it is asked when using `accelerate config` cmd.
body: > Regarding `gradient_accumulation_steps` and `auto`, that piece of code wasn't changed at all and behaviour is the same as before. If one doesn't specify that entry in the config file, it is set to default of `1`, if it is `auto` error is raised else thale given value is used. When not using `deepspeed_config_file`, it is asked when using `accelerate config` cmd.

OK, so your logic is different from HF Trainer then.

The HF Trainer was directing cmd args into the config file's `auto` values so that the user could override them via cmd line args.

I'm not saying the 2 logics have to match. If I am not mistaken the accelerate logic is less flexible, but it's ok if you prefer it that way. 

In HF Trainer the `auto` feature was designed to be used:
1. when the value can't be known before running - "boot"-time calculated configs
2. values to be set via cmd line args and defaults
body: As accelerate is meant to work with all models apart from Transformers and user being in control of the training loop, they are in charge of all the arguments and the naming convention of arguments will be different across different users. On the other hand, in Trainer, users are restricted to a given args set and as such those can be used to fill the DeepSpeed config due to clear mapping between args and DS config params. The idea is that artifacts sent to `accelerator.prepare` have the params required by DS config and we exactly know the mapping between them and makes filling of the params independent of the user's training loop and their arguments naming convention.

In accelerate, the `auto` values are those that can be filled via artifacts being sent to `accelerator.prepare` as all the other places,  the  user has complete control over the training loop, argument naming and filling. The user still has flexibility to fill in all the `auto` values themselves as mentioned here https://github.com/huggingface/accelerate/pull/676#issuecomment-1318139353
body: > As accelerate is meant to work with all models apart from Transformers and user being in control of the training loop, they are in charge of all the arguments and the naming convention of arguments will be different across different users. On the other hand, in Trainer, users are restricted to a given args set and as such those can be used to fill the DeepSpeed config due to clear mapping between args and DS config params. The idea is that artifacts sent to `accelerator.prepare` have the params required by DS config and we exactly know the mapping between them and makes filling of the params independent of the user's training loop and their arguments naming convention.

Thank you for explaining this to me, Sourab, but I'm having a hard time following how Accelerate is any different from HF Trainer wrt sending cmd line arg values to the unfilled out config values in ds_config. e.g.  the Accelerate launcher provides an explicit list of cmd line args for the deepspeed use. There is a 1:1 mapping here as well. Could you please explain how is this different from the HF Trainer?

But as I said above it's totally fine if you prefer to do it this way, Sourab. This doesn't prevent users from doing what they need.

> In accelerate, the auto values are those that can be filled via artifacts being sent to accelerator.prepare as all the other places, the user has complete control over the training loop, argument naming and filling. The user still has flexibility to fill in all the auto values themselves as mentioned here https://github.com/huggingface/accelerate/pull/676#issuecomment-1318139353

Understood. more work, but doable. Thank you for the explanations.


body: > Could you please explain how is this different from the HF Trainer?

Users can have `bs` or `batch_size` as cmd arguments in their code and as such we can't fill ds config's `train_micro_batch_size_per_gpu` whereas Trainer always maps `args.per_device_train_batch_size` to it. The same reason can go for other configs. 

Please note that `accelerate launch` cmd args are primarily used for setting accelerate config's deespseed fields rather than setting `auto` values of `deepspeed_config_file`. Now I understood that you meant using `accelerate launch` cmd args for filling in `auto` values of `deepspeed_config_file` and I've made respective changes.

Latest changes:

Code `test.py`:
```python
from accelerate import Accelerator
from accelerate.state import AcceleratorState

def main():
    accelerator = Accelerator()
    accelerator.print(f"{AcceleratorState()}")
if __name__ == "__main__":
    main()
```

Scenario 1: manually tampered accelerate config file having `deepspeed_config_file` along with other entries.

1. `accelerate config`:
```yaml
command_file: null
commands: null
compute_environment: LOCAL_MACHINE
deepspeed_config:
  gradient_accumulation_steps: 1
  gradient_clipping: 1.0
  offload_optimizer_device: 'cpu'
  offload_param_device: 'cpu'
  zero3_init_flag: true
  zero3_save_16bit_model: true
  zero_stage: 3
  deepspeed_config_file: 'ds_config.json'
distributed_type: DEEPSPEED
downcast_bf16: 'no'
dynamo_backend: 'NO'
fsdp_config: {}
gpu_ids: null
machine_rank: 0
main_process_ip: null
main_process_port: null
main_training_function: main
megatron_lm_config: {}
num_machines: 1
num_processes: 2
rdzv_backend: static
same_network: true
tpu_name: null
tpu_zone: null
use_cpu: false
```

2. `ds_config.json`:
```json
{
    "bf16": {
        "enabled": true
    },
    "zero_optimization": {
        "stage": 3,
        "stage3_gather_16bit_weights_on_model_save": false,
        "offload_optimizer": {
            "device": "none"
        },
        "offload_param": {
            "device": "none"
        }
    },
    "gradient_clipping": 1.0,
    "train_batch_size": "auto",
    "train_micro_batch_size_per_gpu": "auto",
    "gradient_accumulation_steps": 10,
    "steps_per_print": 2000000
}
```

3. Output of `accelerate launch test.py`:
```
ValueError: When using `deepspeed_config_file`, the following accelerate config variables will be ignored: 
['gradient_accumulation_steps', 'gradient_clipping', 'zero_stage', 'offload_optimizer_device', 'offload_param_device', 
'zero3_save_16bit_model', 'mixed_precision'].
Please specify them appropriately in the DeepSpeed config file.
If you are using an accelerate config file, remove others config variables mentioned in the above specified list.
The easiest method is to create a new config following the questionnaire via `accelerate config`.
It will only ask for the necessary config variables when using `deepspeed_config_file`.
```

Scenario 2: Use the solution of the error to create new accelerate config and check that no ambiguity error is now thrown.
1. Run `accelerate config`:
```
$ accelerate config
-------------------------------------------------------------------------------------------------------------------------------In which compute environment are you running?
This machine                                                                                                                   
-------------------------------------------------------------------------------------------------------------------------------Which type of machine are you using?                                                                                           
multi-GPU                                                                                                                      
How many different machines will you use (use more than 1 for multi-node training)? [1]:                                       
Do you wish to optimize your script with torch dynamo?[yes/NO]:                                                                
Do you want to use DeepSpeed? [yes/NO]: yes                                                                                    
Do you want to specify a json file to a DeepSpeed config? [yes/NO]: yes                                                        
Please enter the path to the json DeepSpeed config file: ds_config.json                                                        
Do you want to enable `deepspeed.zero.Init` when using ZeRO Stage-3 for constructing massive models? [yes/NO]: yes
How many GPU(s) should be used for distributed training? [1]:4
accelerate configuration saved at ds_config_sample.yaml
```

2. `accelerate config`:
```yaml
compute_environment: LOCAL_MACHINE
deepspeed_config:
  deepspeed_config_file: ds_config.json
  zero3_init_flag: true
distributed_type: DEEPSPEED
downcast_bf16: 'no'
dynamo_backend: 'NO'
fsdp_config: {}
machine_rank: 0
main_training_function: main
megatron_lm_config: {}
num_machines: 1
num_processes: 4
rdzv_backend: static
same_network: true
use_cpu: false
```

3. Output of `accelerate launch test.py`:
```
Distributed environment: DEEPSPEED  Backend: nccl
Num processes: 4
Process index: 0
Local process index: 0
Device: cuda:0
Mixed precision type: bf16
ds_config: {'bf16': {'enabled': True}, 'zero_optimization': {'stage': 3, 'stage3_gather_16bit_weights_on_model_save': False, 'offload_optimizer': {'device': 'none'}, 'offload_param': {'device': 'none'}}, 'gradient_clipping': 1.0, 'train_batch_size': 'auto', 'train_micro_batch_size_per_gpu': 'auto', 'gradient_accumulation_steps': 10, 'steps_per_print': inf, 'fp16': {'enabled': False}}
```

Scenario 3: Setting the `accelerate launch` cmd args related to deepspeed as `auto` in  `deepspeed_config_file` and check that things work as expected.
1. new `ds_config.json` with `auto` for the `accelerate launch` deepspeed cmd args:
```json
{
    "bf16": {
        "enabled": "auto"
    },
    "zero_optimization": {
        "stage": "auto",
        "stage3_gather_16bit_weights_on_model_save": "auto",
        "offload_optimizer": {
            "device": "auto"
        },
        "offload_param": {
            "device": "auto"
        }
    },
    "gradient_clipping": "auto",
    "train_batch_size": "auto",
    "train_micro_batch_size_per_gpu": "auto",
    "gradient_accumulation_steps": "auto",
    "steps_per_print": 2000000
}
```

2. Output of `accelerate launch --mixed_precision="fp16" --zero_stage=3 --gradient_accumulation_steps=5 --gradient_clipping=1.0 --offload_param_device="cpu" --offload_optimizer_device="nvme" --zero3_save_16bit_model="true" test.py`:
```bash
Distributed environment: DEEPSPEED  Backend: nccl
Num processes: 4
Process index: 0
Local process index: 0
Device: cuda:0
Mixed precision type: fp16
ds_config: {'bf16': {'enabled': False}, 'zero_optimization': {'stage': 3, 'stage3_gather_16bit_weights_on_model_save': True, 'offload_optimizer': {'device': 'nvme'}, 'offload_param': {'device': 'cpu'}}, 'gradient_clipping': 1.0, 'train_batch_size': 'auto', 'train_micro_batch_size_per_gpu': 'auto', 'gradient_accumulation_steps': 5, 'steps_per_print': inf, 'fp16': {'enabled': True, 'auto_cast': True}}
```

**Note**: Remaining `auto` values are handled in `accelerator.prepare()` call.



body: Looks fantastic, Sourab! Thank you for the improvements and taking the time to layout out the different scenarios - if I'm not mistaken those would make for perfect additions to the documentation if it resonates. (at the very least the last one to demo how `auto` values work and why would one want to use those).

BTW, the config generates things like:
```
fsdp_config: {}
megatron_lm_config: {}
```
why not just skip parts that the user hasn't asked for? It just makes the config scarier than it is, no? I'm asking since when I first looked at it I wasn't a all sure which of the empty placeholders were safe to remove and which aren't. My personal preference is for active config - that is to only ever list config entries that I work with and any defaults should be just that defaults and not be listed at all. Which I suppose isn't the case with typical configs where everything is listed out whether it's being used or not.

And I can of course remove all those, so definitely it's not an issue, I'm just asking if my thinking resonates with you.
body: Sourab, I found one more ambiguous combo in one of our tests:

```
  zero3_init_flag: true
  zero_stage: 2
```

This combo is quietly getting accepted. I'm concerned that a developer may see `zero3_init_flag: true` and think it's zero3?

Do you think accelerate should assert when `zero3_init_flag==True` is used with `stage < 3`?
body: Hello @stas00, with current setup below warning is given which I think is fine:
```bash
UserWarning: DeepSpeed Zero3 Init flag is only applicable for ZeRO Stage 3. Setting it to False.
```
body: oh boy. I didn't see it. :(

If a tree falls in a forest and no one is around to hear it, does it make a sound?

I guess I need to start using this pragma to turn warnings into errors, but then some warnings can't be acted upon :(

```
import warnings
warnings.filterwarnings("error")
```

This is for example an even larger issue for tests, where distributed setup hides most warnings or again there are too many of warnings to see anything.