Repository: huggingface/diffusers
Issue category: Performance Issue

Post processing performance can be improved
## Problem

Images generated in batches pay a performance penalty in the post-processing step of the diffusion pipeline.

A lot of calls to image_processor.denormalize are made instead of batching the computation.

### Suggested Improvements

#### Using multiplication instead of division

This is a freebie, numerically the same but much cheaper on all compute platforms

```python
def denormalize(images: Union[np.ndarray, torch.Tensor]) -> Union[np.ndarray, torch.Tensor]:
    return (images * 0.5 + 0.5).clamp(0, 1)
```

#### Adding Fast path in an all-or-nothing denormalization scenario

Instead of calling `torch.stack` on a batch where all (or none) of the images require denormalization, apply the operation on the full tensor directly

```python
def post_process(...):
    # ... 
    if do_denormalize is None:
        return self.denormalize(images) if self.config.do_normalize else images
    # ...
```

#### Denormalize first, stack later

Invoking the `denormalize` call multiple times incurs a lot of overhead on the compute pipeline, more so than "wasting" compute of denormalizing everything and calling `torch.stack` on the results.

```python
def post_process(...):
    # ...
    denormalized = self.denormalize(image)
    image = torch.stack([
        denormalized[i] if do_denormalize[i] else image[i] for i in range(image.shape[0])
    ])
    # ...
```

## Benchmarks

https://colab.research.google.com/drive/1H1SKUlyEZduUeU50V8SVEZgmFrdl6Ego?usp=sharing

These were ran in a Colab T4 instance on CUDA. The batch is of the shape `[1024,3,512,512]` and dtype `fp16`. Further combinations can be made if requested.

### Baseline

<img width="745" alt="image" src="https://github.com/user-attachments/assets/3bd79c84-bce0-47f4-aae9-5ba86b77a2de">

### Suggested Improvements

<img width="729" alt="image" src="https://github.com/user-attachments/assets/3c739ea0-71e6-467d-b36a-e07e42593003">



Functions to edit:
src/diffusers/image_processor.py:VaeImageProcessor.denormalize
src/diffusers/image_processor.py:VaeImageProcessor.postprocess
src/diffusers/image_processor.py:VaeImageProcessorLDM3D.postprocess
Functions added: src/diffusers/image_processor.py:VaeImageProcessor._denormalize_conditionally

Patch:
diff --git a/src/diffusers/image_processor.py b/src/diffusers/image_processor.py
index 00d8588d5a2a..d6913f045ad2 100644
--- a/src/diffusers/image_processor.py
+++ b/src/diffusers/image_processor.py
@@ -236,7 +236,7 @@ def denormalize(images: Union[np.ndarray, torch.Tensor]) -> Union[np.ndarray, to
             `np.ndarray` or `torch.Tensor`:
                 The denormalized image array.
         """
-        return (images / 2 + 0.5).clamp(0, 1)
+        return (images * 0.5 + 0.5).clamp(0, 1)
 
     @staticmethod
     def convert_to_rgb(image: PIL.Image.Image) -> PIL.Image.Image:
@@ -537,6 +537,26 @@ def binarize(self, image: PIL.Image.Image) -> PIL.Image.Image:
 
         return image
 
+    def _denormalize_conditionally(
+        self, images: torch.Tensor, do_denormalize: Optional[List[bool]] = None
+    ) -> torch.Tensor:
+        r"""
+        Denormalize a batch of images based on a condition list.
+
+        Args:
+            images (`torch.Tensor`):
+                The input image tensor.
+            do_denormalize (`Optional[List[bool]`, *optional*, defaults to `None`):
+                A list of booleans indicating whether to denormalize each image in the batch. If `None`, will use the
+                value of `do_normalize` in the `VaeImageProcessor` config.
+        """
+        if do_denormalize is None:
+            return self.denormalize(images) if self.config.do_normalize else images
+
+        return torch.stack(
+            [self.denormalize(images[i]) if do_denormalize[i] else images[i] for i in range(images.shape[0])]
+        )
+
     def get_default_height_width(
         self,
         image: Union[PIL.Image.Image, np.ndarray, torch.Tensor],
@@ -752,12 +772,7 @@ def postprocess(
         if output_type == "latent":
             return image
 
-        if do_denormalize is None:
-            do_denormalize = [self.config.do_normalize] * image.shape[0]
-
-        image = torch.stack(
-            [self.denormalize(image[i]) if do_denormalize[i] else image[i] for i in range(image.shape[0])]
-        )
+        image = self._denormalize_conditionally(image, do_denormalize)
 
         if output_type == "pt":
             return image
@@ -966,12 +981,7 @@ def postprocess(
             deprecate("Unsupported output_type", "1.0.0", deprecation_message, standard_warn=False)
             output_type = "np"
 
-        if do_denormalize is None:
-            do_denormalize = [self.config.do_normalize] * image.shape[0]
-
-        image = torch.stack(
-            [self.denormalize(image[i]) if do_denormalize[i] else image[i] for i in range(image.shape[0])]
-        )
+        image = self._denormalize_conditionally(image, do_denormalize)
 
         image = self.pt_to_numpy(image)