Repository: huggingface/diffusers

Issue:
Remove attention slicing from docs
### Describe the bug

The attention slicing function as described in:
**Sliced attention for additional memory savings**
under https://huggingface.co/docs/diffusers/optimization/fp16 

Is very outdated and we should probably remove the section from the optimizations docs. 

90% of users use either xformers or Torch 2.0 with SDPA. If this is used then the attention computation is dispatched to flash attention in which case the attention function is **very** memory efficient and **attention slicing**:
a) makes no sense because the attention computation won't be the memory bottleneck and
b) can lead to serious slow downs

Only in edge cases does it make sense to keep using attention slicing.

### Reproduction

N/A

### Logs

_No response_

### System Info

N/A

### Who can help?

@stevhliu @sayakpaul can we remove attention slicing from this highly read doc page and in general add a disclaimer to the function that it should **not** be used in combination with SDPA or xformers? 

I'm opening this issue because a user on Discord stumbled upon it


Discussion:

Resolving patch:
diff --git a/src/diffusers/pipelines/pipeline_utils.py b/src/diffusers/pipelines/pipeline_utils.py
--- a/src/diffusers/pipelines/pipeline_utils.py
+++ b/src/diffusers/pipelines/pipeline_utils.py
@@ -1669,8 +1669,16 @@ def fn_recursive_set_mem_eff(module: torch.nn.Module):
     def enable_attention_slicing(self, slice_size: Optional[Union[str, int]] = "auto"):
         r"""
         Enable sliced attention computation. When this option is enabled, the attention module splits the input tensor
-        in slices to compute attention in several steps. This is useful to save some memory in exchange for a small
-        speed decrease.
+        in slices to compute attention in several steps. For more than one attention head, the computation is performed
+        sequentially over each head. This is useful to save some memory in exchange for a small speed decrease.
+
+        <Tip warning={true}>
+
+        ⚠️ Don't enable attention slicing if you're already using `scaled_dot_product_attention` (SDPA) from PyTorch
+        2.0 or xFormers. These attention computations are already very memory efficient so you won't need to enable
+        this function. If you enable attention slicing with SDPA or xFormers, it can lead to serious slow downs!
+
+        </Tip>
 
         Args:
             slice_size (`str` or `int`, *optional*, defaults to `"auto"`):
@@ -1678,6 +1686,23 @@ def enable_attention_slicing(self, slice_size: Optional[Union[str, int]] = "auto
                 `"max"`, maximum amount of memory will be saved by running only one slice at a time. If a number is
                 provided, uses as many slices as `attention_head_dim // slice_size`. In this case, `attention_head_dim`
                 must be a multiple of `slice_size`.
+
+        Examples:
+
+        ```py
+        >>> import torch
+        >>> from diffusers import StableDiffusionPipeline
+
+        >>> pipe = StableDiffusionPipeline.from_pretrained(
+        ...     "runwayml/stable-diffusion-v1-5",
+        ...     torch_dtype=torch.float16,
+        ...     use_safetensors=True,
+        ... )
+
+        >>> prompt = "a photo of an astronaut riding a horse on mars"
+        >>> pipe.enable_attention_slicing()
+        >>> image = pipe(prompt).images[0]
+        ```
         """
         self.set_attention_slice(slice_size)
 


Test patch: