Repository: stable-diffusion-webui
Issue: [Bug]: Input type (torch.cuda.HalfTensor) and weight type (torch.HalfTensor) should be the same
Labels: bug-report

### Is there an existing issue for this?

- [X] I have searched the existing issues and checked the recent builds/commits

### What happened?

Textual inversion training failed

### Steps to reproduce the problem

Training as per wiki


### What should have happened?

It should have trained

### Commit where the problem happens

commit: 5ab7f213

### What platforms do you use to access the UI ?

Windows

### What browsers do you use to access the UI ?

Google Chrome

### Command Line Arguments

```Shell
set COMMANDLINE_ARGS= --xformers --api --cors-allow-origins=
```


### List of extensions

Only built-in

### Console logs

```Shell
E:\stable-diffusion\stable-diffusion_May23\stable-diffusion-webui>git pull
Already up to date.
venv "E:\stable-diffusion\stable-diffusion_May23\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Commit hash: 5ab7f213bec2f816f9c5644becb32eb72c8ffb89
Installing requirements

Launching Web UI with arguments: --xformers --api --cors-allow-origins=
ControlNet v1.1.132
ControlNet v1.1.132
Loading weights [c6bbc15e32] from E:\stable-diffusion\stable-diffusion_May23\stable-diffusion-webui\models\Stable-diffusion\sd-v1-5-inpainting.ckpt
Creating model from config: E:\stable-diffusion\stable-diffusion_May23\stable-diffusion-webui\configs\v1-inpainting-inference.yaml
LatentInpaintDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.54 M params.
Applying xformers cross attention optimization.
Textual inversion embeddings loaded(25): guillm, guillm-1656, lndscp1, lndscp1-1032, lndscp1-464, lndscp1-568, lndscp1-744, lndscp1-896, lndscp2, lndscp2-1100, lndscp2-836, lndscp2-968, retzquang, retzquang2, retzquang2-174, retzquang2-297, retzquang2-351, retzquang2-432, retzquang2-621, spranza-1-140, spranza-1-196, spranza-1-378, spranza-1-574, spranza3, testoil
Textual inversion embeddings skipped(3): classipeint, laxpeint, laxpeintV2
Model loaded in 6.6s (load weights from disk: 1.9s, create model: 0.7s, apply weights to model: 0.5s, apply half(): 0.8s, move model to device: 1.2s, load textual inversion embeddings: 1.4s).
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 56.1s (import torch: 15.5s, import gradio: 5.9s, import ldm: 3.0s, other imports: 5.3s, list SD models: 0.1s, setup codeformer: 0.4s, load scripts: 14.8s, load SD checkpoint: 6.9s, create ui: 3.5s, gradio launch: 0.4s, scripts app_started_callback: 0.1s).
Training at rate of 0.005 until step 3000
Preparing dataset...
100%|██████████████████████████████████████████████████████████████████████████████████| 60/60 [00:06<00:00,  9.88it/s]
  0%|                                                                                         | 0/3000 [00:00<?, ?it/s]Traceback (most recent call last):
  File "E:\stable-diffusion\stable-diffusion_May23\stable-diffusion-webui\modules\textual_inversion\textual_inversion.py", line 499, in train_embedding
    img_c = processing.txt2img_image_conditioning(shared.sd_model, c, training_width, training_height)
  File "E:\stable-diffusion\stable-diffusion_May23\stable-diffusion-webui\modules\processing.py", line 86, in txt2img_image_conditioning
    image_conditioning = sd_model.get_first_stage_encoding(sd_model.encode_first_stage(image_conditioning))
  File "E:\stable-diffusion\stable-diffusion_May23\stable-diffusion-webui\modules\sd_hijack_utils.py", line 17, in <lambda>
    setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
  File "E:\stable-diffusion\stable-diffusion_May23\stable-diffusion-webui\modules\sd_hijack_utils.py", line 28, in __call__
    return self.__orig_func(*args, **kwargs)
  File "E:\stable-diffusion\stable-diffusion_May23\stable-diffusion-webui\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "E:\stable-diffusion\stable-diffusion_May23\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 830, in encode_first_stage
    return self.first_stage_model.encode(x)
  File "E:\stable-diffusion\stable-diffusion_May23\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\autoencoder.py", line 83, in encode
    h = self.encoder(x)
  File "E:\stable-diffusion\stable-diffusion_May23\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable-diffusion\stable-diffusion_May23\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\model.py", line 523, in forward
    hs = [self.conv_in(x)]
  File "E:\stable-diffusion\stable-diffusion_May23\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable-diffusion\stable-diffusion_May23\stable-diffusion-webui\extensions-builtin\Lora\lora.py", line 323, in lora_Conv2d_forward
    return torch.nn.Conv2d_forward_before_lora(self, input)
  File "E:\stable-diffusion\stable-diffusion_May23\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\conv.py", line 463, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "E:\stable-diffusion\stable-diffusion_May23\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\conv.py", line 459, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Input type (torch.cuda.HalfTensor) and weight type (torch.HalfTensor) should be the same

Applying xformers cross attention optimization.
```


### Additional information

_No response_