Repository: Lightning-AI/pytorch-lightning
File: pytorch_lightning/core/lightning.py (Python)

Code before:
            to the number of sequential batches optimized with the specific optimizer.
            It should be given to none or to all of the optimizers.
            There is a difference between passing multiple optimizers in a list,
            and passing multiple optimizers in dictionaries with a frequency of 1:
            In the former case, all optimizers will operate on the given batch in each optimization step.
            In the latter, only one optimizer will operate on the given batch at every step.
            This is different from the 'frequency' value specified in the lr_dict mentioned below.

            .. code-block:: python

                def configure_optimizers(self):
                    optimizer_one = torch.optim.SGD(self.model.parameters(), lr=0.01)
                    optimizer_two = torch.optim.SGD(self.model.parameters(), lr=0.01)
                    return [
                        {
                            'optimizer': optimizer_one,
                            'frequency': 5
                        },
                        {
                            'optimizer': optimizer_two,
                            'frequency': 10
                    }]

            In this example, the first optimizer will be used for the first 5 steps,
            the second optimizer for the next 10 steps and that cycle will continue.
            If an LR scheduler is specified for an optimizer using the `lr_scheduler` key in the above dict,
            the scheduler will only be updated when its optimizer is being used.

        Note:
            The lr_dict is a dictionary which contains scheduler and its associated configuration.
            It has five keys. The default configuration is shown below.

            .. code-block:: python

                {
                    'scheduler': lr_scheduler, # The LR schduler
                    'interval': 'epoch', # The unit of the scheduler's step size
                    'frequency': 1, # The frequency of the scheduler
                    'reduce_on_plateau': False, # For ReduceLROnPlateau scheduler
                    'monitor': 'val_loss', # Metric for ReduceLROnPlateau to monitor
                    'strict': True # Whether to crash the training if `monitor` is not found
                }

            If user only provides LR schedulers, then their configuration will set to default as shown above.

        Examples:
            .. code-block:: python

                # most cases
                def configure_optimizers(self):
                    opt = Adam(self.parameters(), lr=1e-3)

Review comment (performance):
```suggestion
            In this example, the first optimizer will be used for the first 5 steps,
            the second optimizer for the next 10 steps and that cycle will continue.
            If an LR scheduler is specified for an optimizer using the ``lr_scheduler`` key in the above dict,
            the scheduler will only be updated when its optimizer is being used.
```

Code after:
                # batch is 1/num_gpus big
                x, y = batch

                out = self.encoder(x)
                return out

            def test_step_end(self, output_results):
                # this out is now the full size of the batch
                all_test_step_outs = output_results.out
                loss = nce_loss(all_test_step_outs)
                self.log('test_loss', loss)

        See Also:
            See the :ref:`advanced/multi_gpu:Multi-GPU training` guide for more details.
        """

    def test_epoch_end(self, outputs: EPOCH_OUTPUT) -> None:
        """
        Called at the end of a test epoch with the output of all test steps.

        .. code-block:: python

            # the pseudocode for these calls
            test_outs = []
            for test_batch in test_data:
                out = test_step(test_batch)
                test_outs.append(out)
            test_epoch_end(test_outs)

        Args:
            outputs: List of outputs you defined in :meth:`test_step_end`, or if there
                are multiple dataloaders, a list containing a list of outputs for each dataloader

        Return:
            None

        Note:
            If you didn't define a :meth:`test_step`, this won't be called.

        Examples:
            With a single dataloader:

            .. code-block:: python

                def test_epoch_end(self, outputs):
                    # do something with the outputs of all test batches
                    all_test_preds = test_step_outputs.predictions

                    some_result = calc_all_results(all_test_preds)
                    self.log(some_result)