fofr commited on
Commit
37f9210
1 Parent(s): bcc0ad7

Upload folder using huggingface_hub

Browse files
Files changed (4) hide show
  1. README.md +77 -0
  2. embeddings.pti +0 -0
  3. lora.safetensors +3 -0
  4. special_params.json +1 -0
README.md ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: creativeml-openrail-m
3
+ tags:
4
+ - text-to-image
5
+ - stable-diffusion
6
+ - lora
7
+ - diffusers
8
+ base_model: stabilityai/stable-diffusion-xl-base-1.0
9
+ pivotal_tuning: true
10
+ textual_embeddings: embeddings.pti
11
+ instance_prompt: <s0><s1>
12
+ inference: false
13
+ ---
14
+ # sdxl-tng-interior LoRA by [fofr](https://replicate.com/fofr)
15
+ ### SDXL fine-tune of Star Trek Next Generation interiors
16
+
17
+ ![lora_image](https://replicate.delivery/pbxt/CGxE1DgG675SMNPUH8NAuTdHkEh3Cw3l78ze4XbR52f1wWeiA/out-0.png)
18
+ >
19
+
20
+ ## Inference with Replicate API
21
+ Grab your replicate token [here](https://replicate.com/account)
22
+ ```bash
23
+ pip install replicate
24
+ export REPLICATE_API_TOKEN=r8_*************************************
25
+ ```
26
+
27
+ ```py
28
+ import replicate
29
+
30
+ output = replicate.run(
31
+ "sdxl-tng-interior@sha256:45f1d0cf3445f54d4b19a2a03e53b15abd7237ea72e2fb4824b193ffa429e31f",
32
+ input={"prompt": "A photo in the style of TOK, interior, house - sustainable, minimalist, organic, light-filled, dynamic, efficient, autonomous, connected, harmonious, innovative, detailed, 8k, high resolution, sharp focus"}
33
+ )
34
+ print(output)
35
+ ```
36
+ You may also do inference via the API with Node.js or curl, and locally with COG and Docker, [check out the Replicate API page for this model](https://replicate.com/fofr/sdxl-tng-interior/api)
37
+
38
+ ## Inference with 🧨 diffusers
39
+ Replicate SDXL LoRAs are trained with Pivotal Tuning, which combines training a concept via Dreambooth LoRA with training a new token with Textual Inversion.
40
+ As `diffusers` doesn't yet support textual inversion for SDXL, we will use cog-sdxl `TokenEmbeddingsHandler` class.
41
+
42
+ The trigger tokens for your prompt will be `<s0><s1>`
43
+
44
+ ```shell
45
+ pip install diffusers transformers accelerate safetensors huggingface_hub
46
+ git clone https://github.com/replicate/cog-sdxl cog_sdxl
47
+ ```
48
+
49
+ ```py
50
+ import torch
51
+ from huggingface_hub import hf_hub_download
52
+ from diffusers import DiffusionPipeline
53
+ from cog_sdxl.dataset_and_utils import TokenEmbeddingsHandler
54
+ from diffusers.models import AutoencoderKL
55
+
56
+ pipe = DiffusionPipeline.from_pretrained(
57
+ "stabilityai/stable-diffusion-xl-base-1.0",
58
+ torch_dtype=torch.float16,
59
+ variant="fp16",
60
+ ).to("cuda")
61
+
62
+ pipe.load_lora_weights("fofr/sdxl-tng-interior", weight_name="lora.safetensors")
63
+
64
+ text_encoders = [pipe.text_encoder, pipe.text_encoder_2]
65
+ tokenizers = [pipe.tokenizer, pipe.tokenizer_2]
66
+
67
+ embedding_path = hf_hub_download(repo_id="fofr/sdxl-tng-interior", filename="embeddings.pti", repo_type="model")
68
+ embhandler = TokenEmbeddingsHandler(text_encoders, tokenizers)
69
+ embhandler.load_embeddings(embedding_path)
70
+ prompt="A photo in the style of <s0><s1>, interior, house - sustainable, minimalist, organic, light-filled, dynamic, efficient, autonomous, connected, harmonious, innovative, detailed, 8k, high resolution, sharp focus"
71
+ images = pipe(
72
+ prompt,
73
+ cross_attention_kwargs={"scale": 0.8},
74
+ ).images
75
+ #your output image
76
+ images[0]
77
+ ```
embeddings.pti ADDED
lora.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d6b781d7c181e8c00c70d45b9f507a088ccbe0f48d6fabcb2bcf274d8e6d693b
3
+ size 185968776
special_params.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"TOK": "<s0><s1>"}