Repository: huggingface/diffusers

Issue:
AttributeError: 'numpy.ndarray' object has no attribute 'device' [OnnxPipeline] [Schedulers DDPM, Euler, Euler-A]
### Describe the bug

[PR 636](https://github.com/huggingface/diffusers/pull/636) and [PR 944](https://github.com/huggingface/diffusers/pull/944) added the [Euler ](https://moon-ci-docs.huggingface.co/docs/diffusers/pr_1244/en/api/schedulers#diffusers.EulerDiscreteScheduler) and [Euler-A](https://moon-ci-docs.huggingface.co/docs/diffusers/pr_1244/en/api/schedulers#diffusers.EulerAncestralDiscreteScheduler) schedulers. When using the DDPM, Euler or Euler-A [schedulers ](https://moon-ci-docs.huggingface.co/docs/diffusers/pr_1244/en/api/schedulers) with the [Onnx pipeline](https://moon-ci-docs.huggingface.co/docs/diffusers/pr_1244/en/optimization/onnx) w/DirectML the following error occurs:

AttributeError: 'numpy.ndarray' object has no attribute 'device'

### Reproduction

Test environment setup following [Stable Diffusion for AMD GPUs on Windows using DirectML](https://gist.github.com/averad/256c507baa3dcc9464203dc14610d674)

```
from diffusers import OnnxStableDiffusionPipeline, DDPMScheduler, EulerDiscreteScheduler, EulerAncestralDiscreteScheduler
height=512
width=512
num_inference_steps=30
guidance_scale=7.5
prompt = "a photo of an astronaut riding a horse on mars"
negative_prompt="bad hands, blurry"
#scheduler = DDPMScheduler()
#scheduler = EulerDiscreteScheduler()
scheduler = EulerAncestralDiscreteScheduler()
pipe = OnnxStableDiffusionPipeline.from_pretrained("./models/stable_diffusion_v1-5-vae_mse_onnx", provider="DmlExecutionProvider", scheduler=scheduler, safety_checker=None)
image = pipe(prompt, height, width, num_inference_steps, guidance_scale, negative_prompt).images[0] 
image.save("astronaut_rides_horse.png")
```

### Logs

```shell
Traceback (most recent call last):
  File "D:\ai\1.py", line 19, in <module>
    image = pipe(prompt, height, width, num_inference_steps, guidance_scale, negative_prompt).images[0]
  File "D:\ai\sd_env\lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_onnx_stable_diffusion.py", line 262, in __call__
    latents = self.scheduler.step(noise_pred, t, torch.from_numpy(latents), **extra_step_kwargs).prev_sample
  File "D:\ai\sd_env\lib\site-packages\diffusers\schedulers\scheduling_ddpm.py", line 295, in step
    device = model_output.device
AttributeError: 'numpy.ndarray' object has no attribute 'device'
```

```shell
Traceback (most recent call last):
  File "D:\ai\1.py", line 17, in <module>
    image = pipe(prompt, height, width, num_inference_steps, guidance_scale, negative_prompt).images[0]
  File "D:\ai\sd_env\lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_onnx_stable_diffusion.py", line 262, in __call__
    latents = self.scheduler.step(noise_pred, t, torch.from_numpy(latents), **extra_step_kwargs).prev_sample
  File "D:\ai\sd_env\lib\site-packages\diffusers\schedulers\scheduling_euler_discrete.py", line 221, in step
    device = model_output.device
AttributeError: 'numpy.ndarray' object has no attribute 'device'
```

```shell
Traceback (most recent call last):
  File "D:\ai\1.py", line 14, in <module>
    image = pipe(prompt, height, width, num_inference_steps, guidance_scale, negative_prompt).images[0]
  File "D:\ai\sd_env\lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_onnx_stable_diffusion.py", line 262, in __call__
    latents = self.scheduler.step(noise_pred, t, torch.from_numpy(latents), **extra_step_kwargs).prev_sample
  File "D:\ai\sd_env\lib\site-packages\diffusers\schedulers\scheduling_euler_ancestral_discrete.py", line 224, in step
    device = model_output.device
AttributeError: 'numpy.ndarray' object has no attribute 'device'
```

### System Info

- `diffusers` version: 0.8.0.dev0
- Platform: Windows-11
- Python version: 3.10.8
- PyTorch version (GPU?): 1.13.0+cpu (False)
- Huggingface_hub version: 0.10.1
- Transformers version: 4.24.0
- Using GPU in script?: RX580


Discussion:
Gently pinging @anton-l here

Resolving patch:
diff --git a/src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion.py b/src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion.py
--- a/src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion.py
+++ b/src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion.py
@@ -261,8 +261,10 @@ def __call__(
                 noise_pred = noise_pred_uncond + guidance_scale * (noise_pred_text - noise_pred_uncond)
 
             # compute the previous noisy sample x_t -> x_t-1
-            latents = self.scheduler.step(noise_pred, t, torch.from_numpy(latents), **extra_step_kwargs).prev_sample
-            latents = np.array(latents)
+            scheduler_output = self.scheduler.step(
+                torch.from_numpy(noise_pred), t, torch.from_numpy(latents), **extra_step_kwargs
+            )
+            latents = scheduler_output.prev_sample.numpy()
 
             # call the callback, if provided
             if callback is not None and i % callback_steps == 0:
diff --git a/src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion_img2img.py b/src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion_img2img.py
--- a/src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion_img2img.py
+++ b/src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion_img2img.py
@@ -401,8 +401,10 @@ def __call__(
                 noise_pred = noise_pred_uncond + guidance_scale * (noise_pred_text - noise_pred_uncond)
 
             # compute the previous noisy sample x_t -> x_t-1
-            latents = self.scheduler.step(noise_pred, t, torch.from_numpy(latents), **extra_step_kwargs).prev_sample
-            latents = latents.numpy()
+            scheduler_output = self.scheduler.step(
+                torch.from_numpy(noise_pred), t, torch.from_numpy(latents), **extra_step_kwargs
+            )
+            latents = scheduler_output.prev_sample.numpy()
 
             # call the callback, if provided
             if callback is not None and i % callback_steps == 0:
diff --git a/src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion_inpaint.py b/src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion_inpaint.py
--- a/src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion_inpaint.py
+++ b/src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion_inpaint.py
@@ -424,8 +424,10 @@ def __call__(
                 noise_pred = noise_pred_uncond + guidance_scale * (noise_pred_text - noise_pred_uncond)
 
             # compute the previous noisy sample x_t -> x_t-1
-            latents = self.scheduler.step(noise_pred, t, torch.from_numpy(latents), **extra_step_kwargs).prev_sample
-            latents = latents.numpy()
+            scheduler_output = self.scheduler.step(
+                torch.from_numpy(noise_pred), t, torch.from_numpy(latents), **extra_step_kwargs
+            )
+            latents = scheduler_output.prev_sample.numpy()
 
             # call the callback, if provided
             if callback is not None and i % callback_steps == 0:


Test patch: