Repository: stable-diffusion-webui
Issue: [Bug]: Error upon loading SD3.5 medium
Labels: bug-report

### Checklist

- [x] The issue exists after disabling all extensions
- [x] The issue exists on a clean installation of webui
- [ ] The issue is caused by an extension, but I believe it is caused by a bug in the webui
- [X] The issue exists in the current version of the webui
- [ ] The issue has not been reported before recently
- [ ] The issue has been reported before but has not been fixed yet

### What happened?

Tried to load \stableDiffusion35_medium_912387.safetensors, failed

### Steps to reproduce the problem

Load webui
Select model stableDiffusion35_medium_912387.safetensors


### What should have happened?

WebUI should load correctly the model

### What browsers do you use to access the UI ?

Google Chrome

### Sysinfo

[sysinfo-2024-11-01-23-02.json](https://github.com/user-attachments/files/17604740/sysinfo-2024-11-01-23-02.json)


### Console logs

```Shell
venv "C:\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.10.1
Commit hash: 82a973c04367123ae98bd9abdf80d9eda9b910e2
Installing sd-webui-controlnet requirement: changing opencv-python version from 4.10.0.84 to 4.8.0
removing nvidia-cudnn-cu11
Launching Web UI with arguments:
C:\stable-diffusion-webui\venv\lib\site-packages\timm\models\layers\__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
  warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
No module 'xformers'. Proceeding without it.
CivitAI Browser+: Aria2 RPC started
2024-11-02 00:00:45,972 - ControlNet - INFO - ControlNet v1.1.415
ControlNet preprocessor location: C:\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads
2024-11-02 00:00:46,062 - ControlNet - INFO - ControlNet v1.1.415
Loading weights [ee6a527295] from C:\stable-diffusion-webui\models\Stable-diffusion\********************.safetensors
Creating model from config: C:\stable-diffusion-webui\configs\v1-inference.yaml
C:\stable-diffusion-webui\venv\lib\site-packages\huggingface_hub\file_download.py:797: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
[ERROR]: Config states C:\stable-diffusion-webui\config_states\civitai_subfolders.json, "created_at" does not exist
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 16.0s (prepare environment: 5.2s, import torch: 3.5s, import gradio: 1.1s, setup paths: 0.8s, initialize shared: 0.2s, other imports: 0.5s, load scripts: 3.6s, create ui: 0.9s, gradio launch: 0.2s).
Applying attention optimization: Doggettx... done.
Model loaded in 4.8s (load weights from disk: 0.5s, create model: 0.3s, apply weights to model: 3.7s, calculate empty prompt: 0.2s).
Reusing loaded model kizukiAnimeHentai_animeHentaiV4.safetensors [ee6a527295] to load stableDiffusion35_medium_912387.safetensors [11fe06e223]
Loading weights [11fe06e223] from C:\stable-diffusion-webui\models\Stable-diffusion\stableDiffusion35_medium_912387.safetensors
Creating model from config: C:\stable-diffusion-webui\configs\sd3-inference.yaml
C:\stable-diffusion-webui\venv\lib\site-packages\huggingface_hub\file_download.py:797: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
changing setting sd_model_checkpoint to stableDiffusion35_medium_912387.safetensors [11fe06e223]: RuntimeError
Traceback (most recent call last):
  File "C:\stable-diffusion-webui\modules\options.py", line 165, in set
    option.onchange()
  File "C:\stable-diffusion-webui\modules\call_queue.py", line 14, in f
    res = func(*args, **kwargs)
  File "C:\stable-diffusion-webui\modules\initialize_util.py", line 181, in <lambda>
    shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: sd_models.reload_model_weights()), call=False)
  File "C:\stable-diffusion-webui\modules\sd_models.py", line 977, in reload_model_weights
    load_model(checkpoint_info, already_loaded_state_dict=state_dict)
  File "C:\stable-diffusion-webui\modules\sd_models.py", line 845, in load_model
    load_model_weights(sd_model, checkpoint_info, state_dict, timer)
  File "C:\stable-diffusion-webui\modules\sd_models.py", line 440, in load_model_weights
    model.load_state_dict(state_dict, strict=False)
  File "C:\stable-diffusion-webui\modules\sd_disable_initialization.py", line 223, in <lambda>
    module_load_state_dict = self.replace(torch.nn.Module, 'load_state_dict', lambda *args, **kwargs: load_state_dict(module_load_state_dict, *args, **kwargs))
  File "C:\stable-diffusion-webui\modules\sd_disable_initialization.py", line 221, in load_state_dict
    original(module, state_dict, strict=strict)
  File "C:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 2152, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for SD3Inferencer:
        size mismatch for model.diffusion_model.joint_blocks.0.x_block.adaLN_modulation.1.weight: copying a param with shape torch.Size([13824, 1536]) from checkpoint, the shape in current model is torch.Size([9216, 1536]).
        size mismatch for model.diffusion_model.joint_blocks.0.x_block.adaLN_modulation.1.bias: copying a param with shape torch.Size([13824]) from checkpoint, the shape in current model is torch.Size([9216]).
        size mismatch for model.diffusion_model.joint_blocks.1.x_block.adaLN_modulation.1.weight: copying a param with shape torch.Size([13824, 1536]) from checkpoint, the shape in current model is torch.Size([9216, 1536]).
        size mismatch for model.diffusion_model.joint_blocks.1.x_block.adaLN_modulation.1.bias: copying a param with shape torch.Size([13824]) from checkpoint, the shape in current model is torch.Size([9216]).
        size mismatch for model.diffusion_model.joint_blocks.2.x_block.adaLN_modulation.1.weight: copying a param with shape torch.Size([13824, 1536]) from checkpoint, the shape in current model is torch.Size([9216, 1536]).
        size mismatch for model.diffusion_model.joint_blocks.2.x_block.adaLN_modulation.1.bias: copying a param with shape torch.Size([13824]) from checkpoint, the shape in current model is torch.Size([9216]).
        size mismatch for model.diffusion_model.joint_blocks.3.x_block.adaLN_modulation.1.weight: copying a param with shape torch.Size([13824, 1536]) from checkpoint, the shape in current model is torch.Size([9216, 1536]).
        size mismatch for model.diffusion_model.joint_blocks.3.x_block.adaLN_modulation.1.bias: copying a param with shape torch.Size([13824]) from checkpoint, the shape in current model is torch.Size([9216]).
        size mismatch for model.diffusion_model.joint_blocks.4.x_block.adaLN_modulation.1.weight: copying a param with shape torch.Size([13824, 1536]) from checkpoint, the shape in current model is torch.Size([9216, 1536]).
        size mismatch for model.diffusion_model.joint_blocks.4.x_block.adaLN_modulation.1.bias: copying a param with shape torch.Size([13824]) from checkpoint, the shape in current model is torch.Size([9216]).
        size mismatch for model.diffusion_model.joint_blocks.5.x_block.adaLN_modulation.1.weight: copying a param with shape torch.Size([13824, 1536]) from checkpoint, the shape in current model is torch.Size([9216, 1536]).
        size mismatch for model.diffusion_model.joint_blocks.5.x_block.adaLN_modulation.1.bias: copying a param with shape torch.Size([13824]) from checkpoint, the shape in current model is torch.Size([9216]).
        size mismatch for model.diffusion_model.joint_blocks.6.x_block.adaLN_modulation.1.weight: copying a param with shape torch.Size([13824, 1536]) from checkpoint, the shape in current model is torch.Size([9216, 1536]).
        size mismatch for model.diffusion_model.joint_blocks.6.x_block.adaLN_modulation.1.bias: copying a param with shape torch.Size([13824]) from checkpoint, the shape in current model is torch.Size([9216]).
        size mismatch for model.diffusion_model.joint_blocks.7.x_block.adaLN_modulation.1.weight: copying a param with shape torch.Size([13824, 1536]) from checkpoint, the shape in current model is torch.Size([9216, 1536]).
        size mismatch for model.diffusion_model.joint_blocks.7.x_block.adaLN_modulation.1.bias: copying a param with shape torch.Size([13824]) from checkpoint, the shape in current model is torch.Size([9216]).
        size mismatch for model.diffusion_model.joint_blocks.8.x_block.adaLN_modulation.1.weight: copying a param with shape torch.Size([13824, 1536]) from checkpoint, the shape in current model is torch.Size([9216, 1536]).
        size mismatch for model.diffusion_model.joint_blocks.8.x_block.adaLN_modulation.1.bias: copying a param with shape torch.Size([13824]) from checkpoint, the shape in current model is torch.Size([9216]).
        size mismatch for model.diffusion_model.joint_blocks.9.x_block.adaLN_modulation.1.weight: copying a param with shape torch.Size([13824, 1536]) from checkpoint, the shape in current model is torch.Size([9216, 1536]).
        size mismatch for model.diffusion_model.joint_blocks.9.x_block.adaLN_modulation.1.bias: copying a param with shape torch.Size([13824]) from checkpoint, the shape in current model is torch.Size([9216]).
        size mismatch for model.diffusion_model.joint_blocks.10.x_block.adaLN_modulation.1.weight: copying a param with shape torch.Size([13824, 1536]) from checkpoint, the shape in current model is torch.Size([9216, 1536]).
        size mismatch for model.diffusion_model.joint_blocks.10.x_block.adaLN_modulation.1.bias: copying a param with shape torch.Size([13824]) from checkpoint, the shape in current model is torch.Size([9216]).
        size mismatch for model.diffusion_model.joint_blocks.11.x_block.adaLN_modulation.1.weight: copying a param with shape torch.Size([13824, 1536]) from checkpoint, the shape in current model is torch.Size([9216, 1536]).
        size mismatch for model.diffusion_model.joint_blocks.11.x_block.adaLN_modulation.1.bias: copying a param with shape torch.Size([13824]) from checkpoint, the shape in current model is torch.Size([9216]).
        size mismatch for model.diffusion_model.joint_blocks.12.x_block.adaLN_modulation.1.weight: copying a param with shape torch.Size([13824, 1536]) from checkpoint, the shape in current model is torch.Size([9216, 1536]).
        size mismatch for model.diffusion_model.joint_blocks.12.x_block.adaLN_modulation.1.bias: copying a param with shape torch.Size([13824]) from checkpoint, the shape in current model is torch.Size([9216]).
```


### Additional information

The model is SD3.5 medium, T5 text encoder had been disabled
+--------------------------------------------------------------------------------------------+
| NVIDIA-SMI 566.03                 Driver Version: 566.03         CUDA Version: 12.7             |
|-----------------------------------------+------------------------+-------------------------+
| GPU  Name                 Driver-Model  | Bus-Id               Disp.A | Volatile Uncorr. ECC     |
| Fan  Temp   Perf         Pwr:Usage/Cap|            Memory-Usage | GPU-Util  Compute M. |
|                                                            |                                      |                         MIG M. |
|========================+==============+===============|
|  0  NVIDIA GeForce RTX 4060      WDDM  |   00000000:01:00.0  On |                 v    N/A |
|  0%   47C    P2               N/A /  120W |    6700MiB /   8188MiB |             0%      Default |
|                                                             |                                      |                             N/A |
+-----------------------------------------+------------------------+------------------------+