Repository: stable-diffusion-webui
Issue: [Bug]: Loading an fp16 SDXL model fails on MPS when model loading RAM optimization is enabled
Labels: bug-report

### Checklist

- [ ] The issue exists after disabling all extensions
- [ ] The issue exists on a clean installation of webui
- [ ] The issue is caused by an extension, but I believe it is caused by a bug in the webui
- [X] The issue exists in the current version of the webui
- [ ] The issue has not been reported before recently
- [ ] The issue has been reported before but has not been fixed yet

### What happened?

Loading an fp16 model on MPS fails when `--disable-model-loading-ram-optimization` is not enabled. I suspect this is because fp32 is natively supported by MPS, but something is causing an attempted cast to fp64 from fp16, which is not supported.



### Steps to reproduce the problem

1. Try to load an fp16 model on MPS (Apple Silicon).

### What should have happened?

The model loading RAM optimization hack shouldn't interfere with this.

### What browsers do you use to access the UI ?

_No response_

### Sysinfo

```
{
    "Platform": "macOS-14.2.1-arm64-arm-64bit",
    "Python": "3.10.13",
    "Version": "v1.9.0-RC-12-gac8ffb34",
    "Commit": "ac8ffb34e3d1196319c8dcfcf0f5d3eded713176",
    "Commandline": [
        "webui.py",
        "--skip-torch-cuda-test",
        "--upcast-sampling",
        "--no-half-vae",
        "--use-cpu",
        "interrogate",
        "--skip-load-model-at-start"
    ],
}
```

### Console logs

```Shell
Traceback (most recent call last):
  File "stable-diffusion-webui/modules/options.py", line 165, in set
    option.onchange()
  File "stable-diffusion-webui/modules/call_queue.py", line 13, in f
    res = func(*args, **kwargs)
  File "stable-diffusion-webui/modules/initialize_util.py", line 181, in <lambda>
    shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: sd_models.reload_model_weights()), call=False)
  File "stable-diffusion-webui/modules/sd_models.py", line 860, in reload_model_weights
    sd_model = reuse_model_from_already_loaded(sd_model, checkpoint_info, timer)
  File "stable-diffusion-webui/modules/sd_models.py", line 826, in reuse_model_from_already_loaded
    load_model(checkpoint_info)
  File "stable-diffusion-webui/modules/sd_models.py", line 748, in load_model
    load_model_weights(sd_model, checkpoint_info, state_dict, timer)
  File "stable-diffusion-webui/modules/sd_models.py", line 393, in load_model_weights
    model.load_state_dict(state_dict, strict=False)
  File "stable-diffusion-webui/modules/sd_disable_initialization.py", line 223, in <lambda>
    module_load_state_dict = self.replace(torch.nn.Module, 'load_state_dict', lambda *args, **kwargs: load_state_dict(module_load_state_dict, *args, **kwargs))
  File "stable-diffusion-webui/modules/sd_disable_initialization.py", line 221, in load_state_dict
    original(module, state_dict, strict=strict)
  File "stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2139, in load_state_dict
    load(self, state_dict)
  File "stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2127, in load
    load(child, child_state_dict, child_prefix)
  File "stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2127, in load
    load(child, child_state_dict, child_prefix)
  File "stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2127, in load
    load(child, child_state_dict, child_prefix)
  [Previous line repeated 3 more times]
  File "stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2121, in load
    module._load_from_state_dict(
  File "stable-diffusion-webui/modules/sd_disable_initialization.py", line 226, in <lambda>
    conv2d_load_from_state_dict = self.replace(torch.nn.Conv2d, '_load_from_state_dict', lambda *args, **kwargs: load_from_state_dict(conv2d_load_from_state_dict, *args, **kwargs))
  File "stable-diffusion-webui/modules/sd_disable_initialization.py", line 191, in load_from_state_dict
    module._parameters[name] = torch.nn.parameter.Parameter(torch.zeros_like(param, device=device, dtype=dtype), requires_grad=param.requires_grad)
  File "stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/_meta_registrations.py", line 4806, in zeros_like
    res = aten.empty_like.default(
  File "stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/_ops.py", line 513, in __call__
    return self._op(*args, **(kwargs or {}))
  File "stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/_prims_common/wrappers.py", line 250, in _fn
    result = fn(*args, **kwargs)
  File "stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/_refs/__init__.py", line 4755, in empty_like
    return torch.empty_permuted(
TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead
```


### Additional information

_No response_