Repository: huggingface/diffusers

Issue:
numpy_to_pil fails on flax pipeline output
### Describe the bug

When I try to `numpy_to_pil` the output of FlaxStableDiffusionPipeline, it crashes with
> 'DeviceArray' object has no attribute '__array_interface__'

(maybe possibly related to safety_checker=None)

### Reproduction

```py
pipe, params = diffusers.FlaxStableDiffusionPipeline.from_pretrained(
    "CompVis/stable-diffusion-v1-4",
    revision="flax",
    safety_checker=None,
    device_map='auto',
    dtype=jnp.float16,
)

prompt="A delightful child enjoying a hearty bowl of JAX cereal."
prompt_inputs = pipe.prepare_inputs(prompt)
result = pipe(
    prompt_ids=prompt_inputs, 
    params=params,
    num_inference_steps=12,
    prng_seed=jax.random.PRNGKey(0)
)

images = pipe.numpy_to_pil(result.images)
```

### Logs

```shell
diffusers/pipeline_flax_utils.py:447 in <listcomp>                                                          
│   444 │   │   if images.ndim == 3:                                                               
│   445 │   │   │   images = images[None, ...]                                                     
│   446 │   │   images = (images * 255).round().astype("uint8")                                    
│ ❱ 447 │   │   pil_images = [Image.fromarray(image) for image in images]                          
│   448 │   │                                                                                      
│   449 │   │   return pil_images                                          

Image.py:2803 in fromarray
│ ❱ 2803 │   arr = obj.__array_interface__                                                         
AttributeError: 'DeviceArray' object has no attribute '__array_interface__'
```


### System Info

- `diffusers` version: 0.5.1
- Platform: Linux-5.15.0-50-generic-x86_64-with-glibc2.35
- Python version: 3.10.6
- PyTorch version (GPU?): 1.12.1+cu116 (True)
- Huggingface_hub version: 0.10.1
- Transformers version: 4.23.1
- Using GPU in script?: unsure? CPU utilization is high, vRAM allocation is high, GPU utilization is low. `result.images.device()` is CPU.
- Using distributed or parallel set-up in script?: no


Discussion:
This works to get a PIL image:
```py
PIL.Image.fromarray(np.asarray(result.images[0] * 0xFF, dtype=np.ubyte))
```
@keturn Thanks for reporting this, I can reproduce in TPU hardware too.

As a matter of fact, we are ignoring `output_type` in the Flax pipeline. We should be returning PIL by default.
IMO we should update the `numpy_to_pil` funciton here to make it work correctly. 

I'm actually against returning PIL images directly from the pipeline as this won't be compatible with `pmap(...)` etc... IMO the output of pipe in Flax should for now always be `Union[np.ndarray, jnp.ndarray]`
...hence #458
Either way I think we should return the same type whether the safety checker is used or not.
I'd advocate to just return numpy images only in Flax no matter what -> it's different from PyTorch because we want to be able to compile the whole function end-to-end which is not possible with PIL
Assigning to myself to take a look soon.
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/diffusers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Resolving patch:
diff --git a/src/diffusers/pipelines/stable_diffusion/__init__.py b/src/diffusers/pipelines/stable_diffusion/__init__.py
--- a/src/diffusers/pipelines/stable_diffusion/__init__.py
+++ b/src/diffusers/pipelines/stable_diffusion/__init__.py
@@ -63,15 +63,14 @@ class FlaxStableDiffusionPipelineOutput(BaseOutput):
         Output class for Stable Diffusion pipelines.
 
         Args:
-            images (`List[PIL.Image.Image]` or `np.ndarray`)
-                List of denoised PIL images of length `batch_size` or numpy array of shape `(batch_size, height, width,
-                num_channels)`. PIL images or numpy array present the denoised images of the diffusion pipeline.
+            images (`np.ndarray`)
+                Array of shape `(batch_size, height, width, num_channels)` with images from the diffusion pipeline.
             nsfw_content_detected (`List[bool]`)
                 List of flags denoting whether the corresponding generated image likely represents "not-safe-for-work"
                 (nsfw) content.
         """
 
-        images: Union[List[PIL.Image.Image], np.ndarray]
+        images: np.ndarray
         nsfw_content_detected: List[bool]
 
     from ...schedulers.scheduling_pndm_flax import PNDMSchedulerState
diff --git a/src/diffusers/pipelines/stable_diffusion/pipeline_flax_stable_diffusion.py b/src/diffusers/pipelines/stable_diffusion/pipeline_flax_stable_diffusion.py
--- a/src/diffusers/pipelines/stable_diffusion/pipeline_flax_stable_diffusion.py
+++ b/src/diffusers/pipelines/stable_diffusion/pipeline_flax_stable_diffusion.py
@@ -317,9 +317,6 @@ def __call__(
                 Pre-generated noisy latents, sampled from a Gaussian distribution, to be used as inputs for image
                 generation. Can be used to tweak the same generation with different prompts. If not provided, a latents
                 tensor will ge generated by sampling using the supplied random `generator`.
-            output_type (`str`, *optional*, defaults to `"pil"`):
-                The output format of the generate image. Choose between
-                [PIL](https://pillow.readthedocs.io/en/stable/): `PIL.Image.Image` or `np.array`.
             jit (`bool`, defaults to `False`):
                 Whether to run `pmap` versions of the generation and safety scoring functions. NOTE: This argument
                 exists because `__call__` is not yet end-to-end pmap-able. It will be removed in a future release.
@@ -383,6 +380,7 @@ def __call__(
 
             images = images.reshape(num_devices, batch_size, height, width, 3)
         else:
+            images = np.asarray(images)
             has_nsfw_concept = False
 
         if not return_dict:


Test patch: