Repository: stable-diffusion-webui
Issue: [Bug]: progress bar and interupt/skip button dissapear when doing x/y/z plot after the first checkpoint is finnished generating. 
Labels: bug-report

### Is there an existing issue for this?

- [X] I have searched the existing issues and checked the recent builds/commits

### What happened?

whenever I do an x/y/z plot and choose seed and checkpoint, after the first set of images for the first checkpoint are loaded an error happens and the images stop showing, the progress bar stops, and the interrupt/skip button goes back to the yellow generate button. but in the command prompt window, it shows progress still happening, and eventually after everything is finished it will appear.  

the error happens after the first set. and this is the error given in the command window at that time. 

X/Y plot will create 4 images on 1 4x1 grid. (Total steps to process: 200)
100%|██████████████████████████████████████████████████████████████████████████████████| 50/50 [00:03<00:00, 13.71it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 50/50 [00:03<00:00, 14.14it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 50/50 [00:03<00:00, 14.26it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 50/50 [00:03<00:00, 14.19it/s]
Total progress: 100%|████████████████████████████████████████████████████████████████| 200/200 [00:15<00:00, 12.75it/s]
X/Y plot will create 176 images on 1 4x44 grid. (Total steps to process: 8800)
                                  Loading weights [8712e20a5d] from D:\SD\stable-diffusion-webui\models\Stable-diffusion\Anything-V3.0.ckpt [00:00, ?it/s]
Loading VAE weights specified in settings: D:\SD\stable-diffusion-webui\models\VAE\vae-ft-mse-840000-ema-pruned.ckpt
Applying xformers cross attention optimization.
Weights loaded in 8.7s (load weights from disk: 7.7s, apply weights to model: 0.3s, load VAE: 0.2s, move model to device: 0.6s).
100%|██████████████████████████████████████████████████████████████████████████████████| 50/50 [00:03<00:00, 14.24it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 50/50 [00:03<00:00, 14.23it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 50/50 [00:03<00:00, 14.24it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 50/50 [00:03<00:00, 14.20it/s]
Loading weights [0fc198c490] from D:\SD\stable-diffusion-webui\models\Stable-diffusion\AbyssOrangeMix2_hard.safetensors
Loading VAE weights specified in settings: D:\SD\stable-diffusion-webui\models\VAE\vae-ft-mse-840000-ema-pruned.ckpt
Applying xformers cross attention optimization.
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 407, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\fastapi\applications.py", line 270, in __call__
    await super().__call__(scope, receive, send)
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\starlette\applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 184, in __call__
    raise exc
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\gzip.py", line 24, in __call__
    await responder(scope, receive, send)
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\gzip.py", line 44, in __call__
    await self.app(scope, receive, self.send_with_gzip)
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
    raise exc
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in __call__
    raise e
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 706, in __call__
    await route.handle(scope, receive, send)
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 66, in app
    response = await func(request)
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\fastapi\routing.py", line 235, in app
    raw_response = await run_endpoint_function(
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\fastapi\routing.py", line 163, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\starlette\concurrency.py", line 41, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "D:\SD\stable-diffusion-webui\modules\progress.py", line 85, in progressapi
    shared.state.set_current_image()
  File "D:\SD\stable-diffusion-webui\modules\shared.py", line 239, in set_current_image
    self.do_set_current_image()
  File "D:\SD\stable-diffusion-webui\modules\shared.py", line 247, in do_set_current_image
    self.assign_current_image(modules.sd_samplers.samples_to_image_grid(self.current_latent))
  File "D:\SD\stable-diffusion-webui\modules\sd_samplers.py", line 135, in samples_to_image_grid
    return images.image_grid([single_sample_to_image(sample, approximation) for sample in samples])
  File "D:\SD\stable-diffusion-webui\modules\sd_samplers.py", line 135, in <listcomp>
    return images.image_grid([single_sample_to_image(sample, approximation) for sample in samples])
  File "D:\SD\stable-diffusion-webui\modules\sd_samplers.py", line 122, in single_sample_to_image
    x_sample = processing.decode_first_stage(shared.sd_model, sample.unsqueeze(0))[0]
  File "D:\SD\stable-diffusion-webui\modules\processing.py", line 420, in decode_first_stage
    x = model.decode_first_stage(x.to(devices.dtype_vae) if devices.unet_needs_upcast else x)
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "D:\SD\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 826, in decode_first_stage
    return self.first_stage_model.decode(z)
  File "D:\SD\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\autoencoder.py", line 90, in decode
    dec = self.decoder(z)
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "D:\SD\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\model.py", line 637, in forward
    h = self.up[i_level].block[i_block](h, temb)
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "D:\SD\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\model.py", line 131, in forward
    h = self.norm1(h)
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\normalization.py", line 273, in forward
    return F.group_norm(
  File "D:\SD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\functional.py", line 2528, in group_norm
    return torch.group_norm(input, num_groups, weight, bias, eps, torch.backends.cudnn.enabled)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument weight in method wrapper__native_group_norm)
Weights loaded in 0.9s (apply weights to model: 0.3s, load VAE: 0.2s, move model to device: 0.3s).
100%|██████████████████████████████████████████████████████████████████████████████████| 50/50 [00:03<00:00, 15.39it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 50/50 [00:03<00:00, 15.31it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 50/50 [00:03<00:00, 15.38it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 50/50 [00:03<00:00, 15.38it/s]
Loading weights [90d9ab4eff] from D:\SD\stable-diffusion-webui\models\Stable-diffusion\dantionrealmix_10.safetensors/s]
Loading VAE weights specified in settings: D:\SD\stable-diffusion-webui\models\VAE\vae-ft-mse-840000-ema-pruned.ckpt
Applying xformers cross attention optimization.

### Steps to reproduce the problem

1. make xyz plot,
2. for x, random seed -1--1 [4]
3. for y checkpoint, add more than one 
4. run. 


### What should have happened?

the progress bar should be visible the entire time, and the button to interrupt should be visible. 

### Commit where the problem happens

Commit hash: cc8c9b7474d917888a0bd069fcd59a458c67ae4b

### What platforms do you use to access UI ?

Windows

### What browsers do you use to access the UI ?

Google Chrome

### Command Line Arguments

```Shell
--xformers
```


### Additional information, context and logs

Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Commit hash: cc8c9b7474d917888a0bd069fcd59a458c67ae4b