Repository: Lightning-AI/pytorch-lightning
File: pytorch_lightning/plugins/training_type/deepspeed.py (Python)

Code before:
                See `deepspeed tutorial
                <https://www.deepspeed.ai/tutorials/megatron/#deepspeed-activation-checkpoints-optional>`_

            cpu_checkpointing: Offloads partitioned activations to CPU if ``partition_activations`` is enabled

            contiguous_memory_optimization: Copies partitioned activations so that they are contiguous in memory.
                Not supported by all models

            synchronize_checkpoint_boundary: Insert :func:`torch.cuda.synchronize` at each checkpoint boundary.

            save_full_weights: Gathers weights across all processes before saving to disk
                when using ZeRO Stage 3. This allows a single weight file to contain the entire model,
                rather than individual sharded weight files.
                Disable to save sharded states individually. (Default: True)
        """
        if not _DEEPSPEED_AVAILABLE:
            raise MisconfigurationException(
                "To use the DeepSpeed plugin, you must have DeepSpeed installed."
                " pip install deepspeed"
            )

        if cpu_offload or cpu_offload_params or cpu_offload_use_pin_memory:
            warnings.warn(
                "The usage of ``cpu_offload`` ``cpu_offload_params`` and ``cpu_offload_use_pin_memory`` "
                "is deprecated since v1.3 and will be removed in v1.4."
                " From now on use ``offload_optimizer``, ``offload_parameters`` and ``pin_memory``.", DeprecationWarning
            )
            offload_optimizer = cpu_offload
            offload_parameters = cpu_offload_params
            pin_memory = cpu_offload_use_pin_memory

        super().__init__(
            parallel_devices=parallel_devices, num_nodes=num_nodes, cluster_environment=cluster_environment
        )
        self.config = self._load_config(config)
        if self.config is None:
            # User has not overridden config, set defaults
            self.config = self._create_default_config(
                zero_optimization,
                zero_allow_untested_optimizer,
                offload_optimizer=offload_optimizer,
                offload_parameters=offload_parameters,
                nvme_path=nvme_path,
                offload_params_device=offload_params_device,
                params_buffer_count=params_buffer_count,
                params_buffer_size=params_buffer_size,
                max_in_cpu=max_in_cpu,
                pin_memory=pin_memory,
                offload_optimizer_device=offload_optimizer_device,
                optimizer_buffer_count=optimizer_buffer_count,
                block_size=block_size,

Review comment (performance):
Double backticks make sense for docs (as they render monospace), not for warnings

```suggestion
                "The usage of `cpu_offload`, `cpu_offload_params`, and `cpu_offload_use_pin_memory` "
                "is deprecated since v1.3 and will be removed in v1.4."
                " From now on use `offload_optimizer`, `offload_parameters` and `pin_memory`.", DeprecationWarning
```

Code after:
                Still requires you to wrap your forward functions in deepspeed.checkpointing.checkpoint.
                See `deepspeed tutorial
                <https://www.deepspeed.ai/tutorials/megatron/#deepspeed-activation-checkpoints-optional>`_.

            cpu_checkpointing: Offloads partitioned activations to CPU if ``partition_activations`` is enabled.

            contiguous_memory_optimization: Copies partitioned activations so that they are contiguous in memory.
                Not supported by all models.

            synchronize_checkpoint_boundary: Insert :func:`torch.cuda.synchronize` at each checkpoint boundary.

            save_full_weights: Gathers weights across all processes before saving to disk
                when using ZeRO Stage 3. This allows a single weight file to contain the entire model,
                rather than individual sharded weight files.
                Disable to save sharded states individually.
        """
        if not _DEEPSPEED_AVAILABLE:
            raise MisconfigurationException(
                "To use the DeepSpeed plugin, you must have DeepSpeed installed."
                " pip install deepspeed"
            )

        if cpu_offload or cpu_offload_params or cpu_offload_use_pin_memory:
            warnings.warn(
                "The usage of `cpu_offload`, `cpu_offload_params`, and `cpu_offload_use_pin_memory` "
                "is deprecated since v1.4 and will be removed in v1.5."
                " From now on use `offload_optimizer`, `offload_parameters` and `pin_memory`.", DeprecationWarning
            )
            offload_optimizer = cpu_offload
            offload_parameters = cpu_offload_params
            pin_memory = cpu_offload_use_pin_memory

        super().__init__(
            parallel_devices=parallel_devices, num_nodes=num_nodes, cluster_environment=cluster_environment
        )
        self.config = self._load_config(config)
        if self.config is None:
            # User has not overridden config, set defaults
            self.config = self._create_default_config(
                zero_optimization,
                zero_allow_untested_optimizer,
                logging_batch_size_per_gpu,
                offload_optimizer=offload_optimizer,
                offload_parameters=offload_parameters,
                nvme_path=nvme_path,
                offload_params_device=offload_params_device,
                params_buffer_count=params_buffer_count,
                params_buffer_size=params_buffer_size,
                max_in_cpu=max_in_cpu,
                pin_memory=pin_memory,
                offload_optimizer_device=offload_optimizer_device,