Repository: stable-diffusion-webui
Issue: Python high CPU load
Labels: bug-report

### Is there an existing issue for this?

- [X] I have searched the existing issues and checked the recent builds/commits

### What happened?

[Bug]: Hi, I'm seeing very high cpu usage simultaneously with the gpu during img2img upscale with controlnet and Ultimate SD upscale. ~50% constant usage on a 5900x alongside ~80-90% on a rtx 4070. Taskmanager shows Python as the cpu hog. Is this normal?

### Steps to reproduce the problem

1. Go to img2img module and drop an image
2. Enable Controlnet and drop same image, control type Tile, Contronet is more important
3. Enable Ultimate SD Upscale, scale from image size, Upscaler 4x-UltraSharp


### What should have happened?

I remember seeing mild cpu usage before when upscaling but not a constant 50%+. I usually multitask, doing 3d rendering in the same time and I would have noticed it

### Sysinfo

[sysinfo-2023-10-05-16-13.txt](https://github.com/AUTOMATIC1111/stable-diffusion-webui/files/12820687/sysinfo-2023-10-05-16-13.txt)


### What browsers do you use to access the UI ?

Microsoft Edge

### Console logs

```Shell
From https://github.com/AUTOMATIC1111/stable-diffusion-webui
 * branch              master     -> FETCH_HEAD
Already up to date.
venv "venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.6.0
Commit hash: 5ef669de080814067961f28357256e8fe27544f4
Launching Web UI with arguments: --xformers --autolaunch
2023-10-05 19:00:00,125 - ControlNet - INFO - ControlNet v1.1.410
ControlNet preprocessor location: X:\AI\Stable Diffusion\extensions\sd-webui-controlnet\annotator\downloads
2023-10-05 19:00:00,354 - ControlNet - INFO - ControlNet v1.1.410
Loading weights [aadddd3d75] from X:\AI\Stable Diffusion\models\Stable-diffusion\deliberate_v3.safetensors
Creating model from config: X:\AI\Stable Diffusion\configs\v1-inference.yaml
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 15.0s (prepare environment: 4.0s, import torch: 3.3s, import gradio: 1.8s, setup paths: 1.6s, initialize shared: 0.2s, other imports: 1.2s, setup codeformer: 0.1s, list SD models: 0.2s, load scripts: 1.6s, create ui: 0.6s, gradio launch: 0.3s, app_started_callback: 0.2s).
Applying attention optimization: xformers... done.
Model loaded in 5.7s (load weights from disk: 0.4s, create model: 0.7s, apply weights to model: 2.5s, apply half(): 1.0s, load VAE: 0.1s, calculate empty prompt: 0.8s).
Canva size: 2048x2048
Image size: 1024x1024
Scale factor: 2
Upscaling iteration 1 with scale factor 2
Tile size: 512x512
Tiles amount: 16
Grid: 4x4
Redraw enabled: True
Seams fix mode: NONE
2023-10-05 19:02:51,830 - ControlNet - INFO - Loading model: control_v11f1e_sd15_tile [a371b31b]
2023-10-05 19:02:52,176 - ControlNet - INFO - Loaded state_dict from [X:\AI\Stable Diffusion\extensions\sd-webui-controlnet\models\control_v11f1e_sd15_tile.pth]
2023-10-05 19:02:52,177 - ControlNet - INFO - controlnet_default_config
2023-10-05 19:02:53,862 - ControlNet - INFO - ControlNet model control_v11f1e_sd15_tile [a371b31b] loaded.
2023-10-05 19:03:01,452 - ControlNet - INFO - Loading preprocessor: tile_resample
2023-10-05 19:03:01,452 - ControlNet - INFO - preprocessor resolution = -1
2023-10-05 19:03:01,529 - ControlNet - INFO - ControlNet Hooked - Time = 9.845678329467773
100%|██████████████████████████████████████████████████████████████████████████████████| 16/16 [00:02<00:00,  6.68it/s]
2023-10-05 19:03:05,893 - ControlNet - INFO - Loading model from cache: control_v11f1e_sd15_tile [a371b31b],  7.72it/s]
2023-10-05 19:03:06,032 - ControlNet - INFO - Loading preprocessor: tile_resample
2023-10-05 19:03:06,032 - ControlNet - INFO - preprocessor resolution = -1
2023-10-05 19:03:06,057 - ControlNet - INFO - ControlNet Hooked - Time = 0.16493797302246094
100%|██████████████████████████████████████████████████████████████████████████████████| 16/16 [00:02<00:00,  7.68it/s]
2023-10-05 19:03:11,115 - ControlNet - INFO - Loading model from cache: control_v11f1e_sd15_tile [a371b31b],  7.61it/s]
2023-10-05 19:03:11,257 - ControlNet - INFO - Loading preprocessor: tile_resample
2023-10-05 19:03:11,257 - ControlNet - INFO - preprocessor resolution = -1
2023-10-05 19:03:11,277 - ControlNet - INFO - ControlNet Hooked - Time = 0.1612386703491211
100%|██████████████████████████████████████████████████████████████████████████████████| 16/16 [00:02<00:00,  7.66it/s]
2023-10-05 19:03:16,332 - ControlNet - INFO - Loading model from cache: control_v11f1e_sd15_tile [a371b31b],  7.42it/s]
2023-10-05 19:03:23,751 - ControlNet - INFO - Loading preprocessor: tile_resample
2023-10-05 19:03:23,751 - ControlNet - INFO - preprocessor resolution = -1
2023-10-05 19:03:23,778 - ControlNet - INFO - ControlNet Hooked - Time = 7.44639778137207
100%|██████████████████████████████████████████████████████████████████████████████████| 16/16 [00:02<00:00,  7.66it/s]
2023-10-05 19:03:27,749 - ControlNet - INFO - Loading model from cache: control_v11f1e_sd15_tile [a371b31b],  6.84it/s]
2023-10-05 19:03:27,885 - ControlNet - INFO - Loading preprocessor: tile_resample
2023-10-05 19:03:27,885 - ControlNet - INFO - preprocessor resolution = -1
2023-10-05 19:03:27,904 - ControlNet - INFO - ControlNet Hooked - Time = 0.15589118003845215
100%|██████████████████████████████████████████████████████████████████████████████████| 16/16 [00:02<00:00,  7.66it/s]
2023-10-05 19:03:32,969 - ControlNet - INFO - Loading model from cache: control_v11f1e_sd15_tile [a371b31b],  7.61it/s]
2023-10-05 19:03:33,107 - ControlNet - INFO - Loading preprocessor: tile_resample
2023-10-05 19:03:33,107 - ControlNet - INFO - preprocessor resolution = -1
2023-10-05 19:03:33,127 - ControlNet - INFO - ControlNet Hooked - Time = 0.15788793563842773
100%|██████████████████████████████████████████████████████████████████████████████████| 16/16 [00:02<00:00,  7.67it/s]
2023-10-05 19:03:38,179 - ControlNet - INFO - Loading model from cache: control_v11f1e_sd15_tile [a371b31b],  7.48it/s]
2023-10-05 19:03:38,319 - ControlNet - INFO - Loading preprocessor: tile_resample
2023-10-05 19:03:38,319 - ControlNet - INFO - preprocessor resolution = -1
2023-10-05 19:03:38,341 - ControlNet - INFO - ControlNet Hooked - Time = 0.1610698699951172
100%|██████████████████████████████████████████████████████████████████████████████████| 16/16 [00:02<00:00,  7.66it/s]
2023-10-05 19:03:43,376 - ControlNet - INFO - Loading model from cache: control_v11f1e_sd15_tile [a371b31b],  7.46it/s]
2023-10-05 19:03:43,514 - ControlNet - INFO - Loading preprocessor: tile_resample
2023-10-05 19:03:43,514 - ControlNet - INFO - preprocessor resolution = -1
2023-10-05 19:03:43,540 - ControlNet - INFO - ControlNet Hooked - Time = 0.1641395092010498
100%|██████████████████████████████████████████████████████████████████████████████████| 16/16 [00:02<00:00,  7.68it/s]
2023-10-05 19:03:48,578 - ControlNet - INFO - Loading model from cache: control_v11f1e_sd15_tile [a371b31b],  7.46it/s]
2023-10-05 19:03:48,717 - ControlNet - INFO - Loading preprocessor: tile_resample
2023-10-05 19:03:48,717 - ControlNet - INFO - preprocessor resolution = -1
2023-10-05 19:03:48,742 - ControlNet - INFO - ControlNet Hooked - Time = 0.1645052433013916
100%|██████████████████████████████████████████████████████████████████████████████████| 16/16 [00:02<00:00,  7.67it/s]
2023-10-05 19:03:53,807 - ControlNet - INFO - Loading model from cache: control_v11f1e_sd15_tile [a371b31b],  7.45it/s]
2023-10-05 19:03:53,950 - ControlNet - INFO - Loading preprocessor: tile_resample
2023-10-05 19:03:53,951 - ControlNet - INFO - preprocessor resolution = -1
2023-10-05 19:03:53,980 - ControlNet - INFO - ControlNet Hooked - Time = 0.17217564582824707
100%|██████████████████████████████████████████████████████████████████████████████████| 16/16 [00:02<00:00,  7.70it/s]
2023-10-05 19:03:59,008 - ControlNet - INFO - Loading model from cache: control_v11f1e_sd15_tile [a371b31b],  7.44it/s]
2023-10-05 19:03:59,148 - ControlNet - INFO - Loading preprocessor: tile_resample
2023-10-05 19:03:59,149 - ControlNet - INFO - preprocessor resolution = -1
2023-10-05 19:03:59,169 - ControlNet - INFO - ControlNet Hooked - Time = 0.16080760955810547
100%|██████████████████████████████████████████████████████████████████████████████████| 16/16 [00:02<00:00,  7.63it/s]
2023-10-05 19:04:04,223 - ControlNet - INFO - Loading model from cache: control_v11f1e_sd15_tile [a371b31b],  7.42it/s]
2023-10-05 19:04:04,362 - ControlNet - INFO - Loading preprocessor: tile_resample
2023-10-05 19:04:04,362 - ControlNet - INFO - preprocessor resolution = -1
2023-10-05 19:04:04,384 - ControlNet - INFO - ControlNet Hooked - Time = 0.1615910530090332
100%|██████████████████████████████████████████████████████████████████████████████████| 16/16 [00:02<00:00,  7.67it/s]
2023-10-05 19:04:09,427 - ControlNet - INFO - Loading model from cache: control_v11f1e_sd15_tile [a371b31b],  7.45it/s]
2023-10-05 19:04:16,851 - ControlNet - INFO - Loading preprocessor: tile_resample
2023-10-05 19:04:16,851 - ControlNet - INFO - preprocessor resolution = -1
2023-10-05 19:04:16,874 - ControlNet - INFO - ControlNet Hooked - Time = 7.4478209018707275
100%|██████████████████████████████████████████████████████████████████████████████████| 16/16 [00:02<00:00,  7.68it/s]
2023-10-05 19:04:20,839 - ControlNet - INFO - Loading model from cache: control_v11f1e_sd15_tile [a371b31b],  6.81it/s]
2023-10-05 19:04:20,978 - ControlNet - INFO - Loading preprocessor: tile_resample
2023-10-05 19:04:20,978 - ControlNet - INFO - preprocessor resolution = -1
2023-10-05 19:04:20,997 - ControlNet - INFO - ControlNet Hooked - Time = 0.1578214168548584
100%|██████████████████████████████████████████████████████████████████████████████████| 16/16 [00:02<00:00,  7.66it/s]
2023-10-05 19:04:26,042 - ControlNet - INFO - Loading model from cache: control_v11f1e_sd15_tile [a371b31b],  7.65it/s]
2023-10-05 19:04:26,183 - ControlNet - INFO - Loading preprocessor: tile_resample
2023-10-05 19:04:26,183 - ControlNet - INFO - preprocessor resolution = -1
2023-10-05 19:04:26,207 - ControlNet - INFO - ControlNet Hooked - Time = 0.16570448875427246
100%|██████████████████████████████████████████████████████████████████████████████████| 16/16 [00:02<00:00,  7.67it/s]
2023-10-05 19:04:31,232 - ControlNet - INFO - Loading model from cache: control_v11f1e_sd15_tile [a371b31b],  7.46it/s]
2023-10-05 19:04:38,655 - ControlNet - INFO - Loading preprocessor: tile_resample
2023-10-05 19:04:38,655 - ControlNet - INFO - preprocessor resolution = -1
2023-10-05 19:04:38,675 - ControlNet - INFO - ControlNet Hooked - Time = 7.443018198013306
100%|██████████████████████████████████████████████████████████████████████████████████| 16/16 [00:02<00:00,  7.60it/s]
Total progress: 100%|████████████████████████████████████████████████████████████████| 256/256 [01:39<00:00,  2.58it/s]
```


### Additional information

_No response_