Repository: stable-diffusion-webui
Issue: [Bug]: transformers 4.25.1 causing me issues
Labels: bug-report

### Is there an existing issue for this?

- [x] I have searched the existing issues and checked the recent builds/commits

### What happened?

I updated to a recent commit and was getting the error 'OSError: Can't load tokenizer for 'openai/clip-vit-large-patch14'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'openai/clip-vit-large-patch14' is the correct path to a directory containing all relevant files for a CLIPTokenizer tokenizer.' I did a fresh install of the webui. Same problem. I changed my transformers version to 4.19.2 and it worked. Dreambooth broke. I need it for a project. The kohya ss scripts don't work either. My guess is something is wrong with models--openai--clip-vit-large-patch14 in my .cache folder. There is nothing in there, and I'm pretty sure that there is supposed to be a snapshot folder.

### Steps to reproduce the problem

1. Install webui
2. Press webui-user
3. No profit I guess


### What should have happened?

No errors, same as with transformers==4.19.2

### Commit where the problem happens

0cc0ee1bcb4c24a8c9715f66cede06601bfc00c8

### What platforms do you use to access the UI ?

Windows

### What browsers do you use to access the UI ?

Google Chrome

### Command Line Arguments

```Shell
--xformers --medvram --no-half-vae
```


### List of extensions

Dreambooth, controlnet, smartprocess and a few more minor ones

### Console logs

```Shell
venv "D:\Stable Diffusion\SD New\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.7 (tags/v3.10.7:6cc6b13, Sep  5 2022, 14:08:36) [MSC v.1933 64 bit (AMD64)]
Commit hash: 0cc0ee1bcb4c24a8c9715f66cede06601bfc00c8
Installing requirements for Web UI


#######################################################################################################
Initializing Dreambooth
If submitting an issue on github, please provide the below text for debugging purposes:

Python revision: 3.10.7 (tags/v3.10.7:6cc6b13, Sep  5 2022, 14:08:36) [MSC v.1933 64 bit (AMD64)]
Dreambooth revision: 43ae9d55531004f1dedaea7ac2443e9b16739913
SD-WebUI revision: 0cc0ee1bcb4c24a8c9715f66cede06601bfc00c8

Checking Dreambooth requirements...
[+] bitsandbytes version 0.35.0 installed.
[+] diffusers version 0.10.2 installed.
[+] transformers version 4.25.1 installed.
[+] xformers version 0.0.17.dev461 installed.
[+] torch version 1.13.1+cu117 installed.
[+] torchvision version 0.14.1+cu117 installed.

#######################################################################################################

loading Smart Crop reqs from D:\Stable Diffusion\SD New\stable-diffusion-webui\extensions\sd_smartprocess\requirements.txt
Checking Smart Crop requirements.



Launching Web UI with arguments: --xformers --medvram --no-half-vae
SD-Webui API layer loaded
Loading weights [e911995387] from D:\Stable Diffusion\SD New\stable-diffusion-webui\models\Stable-diffusion\models\cetusMix_cetusVersion2.safetensors
Creating model from config: D:\Stable Diffusion\SD New\stable-diffusion-webui\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Failed to create model quickly; will retry using slow method.
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
loading stable diffusion model: OSError
Traceback (most recent call last):
  File "D:\Stable Diffusion\SD New\stable-diffusion-webui\webui.py", line 111, in initialize
    modules.sd_models.load_model()
  File "D:\Stable Diffusion\SD New\stable-diffusion-webui\modules\sd_models.py", line 406, in load_model
    sd_model = instantiate_from_config(sd_config.model)
  File "D:\Stable Diffusion\SD New\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\util.py", line 79, in instantiate_from_config
    return get_obj_from_str(config["target"])(**config.get("params", dict()))
  File "D:\Stable Diffusion\SD New\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 563, in __init__
    self.instantiate_cond_stage(cond_stage_config)
  File "D:\Stable Diffusion\SD New\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 630, in instantiate_cond_stage
    model = instantiate_from_config(config)
  File "D:\Stable Diffusion\SD New\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\util.py", line 79, in instantiate_from_config
    return get_obj_from_str(config["target"])(**config.get("params", dict()))
  File "D:\Stable Diffusion\SD New\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\encoders\modules.py", line 99, in __init__
    self.tokenizer = CLIPTokenizer.from_pretrained(version)
  File "D:\Stable Diffusion\SD New\stable-diffusion-webui\venv\lib\site-packages\transformers\tokenization_utils_base.py", line 1785, in from_pretrained
    raise EnvironmentError(
OSError: Can't load tokenizer for 'openai/clip-vit-large-patch14'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'openai/clip-vit-large-patch14' is the correct path to a directory containing all relevant files for a CLIPTokenizer tokenizer.


Stable diffusion model failed to load, exiting
Press any key to continue . . .
```


### Additional information

_No response_