Repository: huggingface/diffusers

Issue:
OSError: h94/IP-Adapter does not appear to have a file named models\image_encoder\config.json
### Describe the bug

I'm getting this error:

`OSError: h94/IP-Adapter does not appear to have a file named models\image_encoder\config.json. Checkout 'https://huggingface.co/h94/IP-Adapter/main' for available files.`

When running the IP-Adapter example from here: https://huggingface.co/docs/diffusers/main/en/using-diffusers/loading_adapters

### Reproduction

Run the IP-Adapter example from here: https://huggingface.co/docs/diffusers/main/en/using-diffusers/loading_adapters

```
import torch
from diffusers import StableDiffusionPipeline, DDIMScheduler
from diffusers.utils import load_image

noise_scheduler = DDIMScheduler(
    num_train_timesteps=1000,
    beta_start=0.00085,
    beta_end=0.012,
    beta_schedule="scaled_linear",
    clip_sample=False,
    set_alpha_to_one=False,
    steps_offset=1
)

pipeline = StableDiffusionPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    torch_dtype=torch.float16,
    scheduler=noise_scheduler,
).to("cuda")

pipeline.load_ip_adapter("h94/IP-Adapter", subfolder="models", weight_name="ip-adapter-full-face_sd15.bin")

pipeline.set_ip_adapter_scale(0.7)

image = load_image("https://huggingface.co/datasets/YiYiXu/testing-images/resolve/main/ai_face2.png")

generator = torch.Generator(device="cpu").manual_seed(33)

image = pipeline(
    prompt="A photo of a girl wearing a black dress, holding red roses in hand, upper body, behind is the Eiffel Tower",
    ip_adapter_image=image,
    negative_prompt="monochrome, lowres, bad anatomy, worst quality, low quality", 
    num_inference_steps=50, num_images_per_prompt=1, width=512, height=704,
    generator=generator,
).images[0]
```

OSError: h94/IP-Adapter does not appear to have a file named models\image_encoder\config.json. Checkout 'https://huggingface.co/h94/IP-Adapter/main' for available files.

### Logs

```shell
OSError: h94/IP-Adapter does not appear to have a file named models\image_encoder\config.json. Checkout 'https://huggingface.co/h94/IP-Adapter/main' for available files.
```


### System Info

Win 11
Diffusers 0.25.0

### Who can help?

@sayakpaul @yiyixuxu @DN6 @patrickvonplaten


Discussion:
hi @tin2tin, I encountered the same issue on **Windows**, it is caused by the different convention in path generation.

To solve this, I separately load the `CLIPVisionModelWithProjection` and pass it to the pipeline as it follows:

```
from transformers import CLIPVisionModelWithProjection
image_encoder = CLIPVisionModelWithProjection.from_pretrained(
    "h94/IP-Adapter",
    subfolder="models/image_encoder",
    torch_dtype=torch.float16,
).to("cuda")

pipeline = StableDiffusionPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    torch_dtype=torch.float16,
    scheduler=noise_scheduler,
    image_encoder=image_encoder,
).to("cuda")
```

(Or you also can set `pipeline.image_encoder = image_encoder` after creating the pipeline.)

Then you can load the IP Adapter

Resolving patch:
diff --git a/src/diffusers/loaders/ip_adapter.py b/src/diffusers/loaders/ip_adapter.py
--- a/src/diffusers/loaders/ip_adapter.py
+++ b/src/diffusers/loaders/ip_adapter.py
@@ -11,7 +11,7 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-import os
+from pathlib import Path
 from typing import Dict, Union
 
 import torch
@@ -138,7 +138,7 @@ def load_ip_adapter(
                 logger.info(f"loading image_encoder from {pretrained_model_name_or_path_or_dict}")
                 image_encoder = CLIPVisionModelWithProjection.from_pretrained(
                     pretrained_model_name_or_path_or_dict,
-                    subfolder=os.path.join(subfolder, "image_encoder"),
+                    subfolder=Path(subfolder, "image_encoder").as_posix(),
                 ).to(self.device, dtype=self.dtype)
                 self.image_encoder = image_encoder
                 self.register_to_config(image_encoder=["transformers", "CLIPVisionModelWithProjection"])


Test patch: