Repository: Lightning-AI/pytorch-lightning
File: pytorch_lightning/core/lightning.py (Python)

Code before:
                    )

        Note:

            Some things to know:

            - Lightning calls ``.backward()`` and ``.step()`` on each optimizer
              and learning rate scheduler as needed.

            - If you use 16-bit precision (``precision=16``), Lightning will automatically
              handle the optimizers for you.

            - If you use multiple optimizers, :meth:`training_step` will have an additional
              ``optimizer_idx`` parameter.

            - If you use LBFGS Lightning handles the closure function automatically for you.

            - If you use multiple optimizers, gradients will be calculated only
              for the parameters of current optimizer at each training step.

            - If you need to control how often those optimizers step or override the
              default ``.step()`` schedule, override the :meth:`optimizer_step` hook.

            - If you only want to call a learning rate scheduler every ``x`` step or epoch,
              or want to monitor a custom metric, you can specify these in a lr_dict:

              .. code-block:: python

                  {
                      'scheduler': lr_scheduler,
                      'interval': 'step',  # or 'epoch'
                      'monitor': 'val_f1',
                      'frequency': x,
                  }

        """
        rank_zero_warn("`configure_optimizers` must be implemented to be used with the Lightning Trainer")

    def manual_backward(self, loss: Tensor, optimizer: Optimizer, *args, **kwargs) -> None:
        """
        Call this directly from your training_step when doing optimizations manually.
        By using this we can ensure that all the proper scaling when using 16-bit etc has been done for you

        This function forwards all args to the .backward() call as well.

        .. tip:: In manual mode we still automatically clip grads if Trainer(gradient_clip_val=x) is set

        .. tip:: In manual mode we still automatically accumulate grad over batches if
           Trainer(accumulate_grad_batches=x) is set and you use `optimizer.step()`

        Example::

Review comment (performance):
```suggestion
            It should be given either to none or to all of the optimizers.
```

Code after:
            Certain callback methods like :meth:`~pytorch_lightning.callbacks.base.Callback.on_init_start`
            will never be invoked on the new callbacks returned here.
        """
        return []

    def configure_optimizers(self):
        r"""
        Choose what optimizers and learning-rate schedulers to use in your optimization.
        Normally you'd need one. But in the case of GANs or similar you might have multiple.

        Return:
            Any of these 6 options.

            - **Single optimizer**.
            - **List or Tuple** of optimizers.
            - **Two lists** - The first list has multiple optimizers, and the second has multiple LR schedulers (or
              multiple lr_dict).
            - **Dictionary**, with an ``"optimizer"`` key, and (optionally) a ``"lr_scheduler"``
              key whose value is a single LR scheduler or lr_dict.
            - **Tuple of dictionaries** as described above, with an optional ``"frequency"`` key.
            - **None** - Fit will run without any optimizer.

        Note:
            The ``frequency`` value specified in a dict along with the ``optimizer`` key is an int corresponding
            to the number of sequential batches optimized with the specific optimizer.
            It should be given to none or to all of the optimizers.
            There is a difference between passing multiple optimizers in a list,
            and passing multiple optimizers in dictionaries with a frequency of 1:
            In the former case, all optimizers will operate on the given batch in each optimization step.
            In the latter, only one optimizer will operate on the given batch at every step.
            This is different from the ``frequency`` value specified in the lr_dict mentioned below.

            .. code-block:: python

                def configure_optimizers(self):
                    optimizer_one = torch.optim.SGD(self.model.parameters(), lr=0.01)
                    optimizer_two = torch.optim.SGD(self.model.parameters(), lr=0.01)
                    return [
                        {'optimizer': optimizer_one, 'frequency': 5},
                        {'optimizer': optimizer_two, 'frequency': 10},
                    ]

            In this example, the first optimizer will be used for the first 5 steps,
            the second optimizer for the next 10 steps and that cycle will continue.
            If an LR scheduler is specified for an optimizer using the ``lr_scheduler`` key in the above dict,
            the scheduler will only be updated when its optimizer is being used.

        Note:
            The lr_dict is a dictionary which contains the scheduler and its associated configuration.
            The default configuration is shown below.