Update README.md

README.md

@@ -12,24 +12,104 @@ tags:
 inference: true
 ---
     
-# LoRA text2image fine-tuning - NYUAD-ComNets/Middle_Eastern_Male_Profession_Model
 
-These are LoRA adaption weights for stabilityai/stable-diffusion-xl-base-1.0. The weights were fine-tuned on the NYUAD-ComNets/Middle_Eastern_Male_Profession dataset. You can find some example images in the following. 
+# Model description
+LoRA text2image fine-tuning - NYUAD-ComNets/Middle_Eastern_Male_Profession_Model
 
-![img_0](./image_0.png)
-![img_1](./image_1.png)
-![img_2](./image_2.png)
-![img_3](./image_3.png)
-![img_4](./image_4.png)
-![img_5](./image_5.png)
-![img_6](./image_6.png)
-![img_7](./image_7.png)
-![img_8](./image_8.png)
-![img_9](./image_9.png)
-![img_10](./image_10.png)
-![img_11](./image_11.png)
+These are LoRA adaption weights for stabilityai/stable-diffusion-xl-base-1.0. The weights were fine-tuned on the NYUAD-ComNets/Middle_Eastern_Male_Profession dataset. 
+You can find some example images.
 
+prompt: a photo of a {profession}, looking at the camera, closeup headshot facing forward, ultra quality, sharp focus
+
+# How to use this model:
+
+``` python
+
+
+import torch
+from compel import Compel, ReturnedEmbeddingsType
+from diffusers import DiffusionPipeline
+
+import random
+
+
+negative_prompt = "cartoon, anime, 3d, painting, b&w, low quality" 
+
+
+models=["NYUAD-ComNets/Asian_Female_Profession_Model","NYUAD-ComNets/Black_Female_Profession_Model","NYUAD-ComNets/White_Female_Profession_Model",
+"NYUAD-ComNets/Indian_Female_Profession_Model","NYUAD-ComNets/Latino_Hispanic_Female_Profession_Model","NYUAD-ComNets/Middle_Eastern_Female_Profession_Model",
+"NYUAD-ComNets/Asian_Male_Profession_Model","NYUAD-ComNets/Black_Male_Profession_Model","NYUAD-ComNets/White_Male_Profession_Model",
+"NYUAD-ComNets/Indian_Male_Profession_Model","NYUAD-ComNets/Latino_Hispanic_Male_Profession_Model","NYUAD-ComNets/Middle_Eastern_Male_Profession_Model"]
+
+adapters=["asian_female","black_female","white_female","indian_female","latino_female","middle_east_female",
+"asian_male","black_male","white_male","indian_male","latino_male","middle_east_male"]
+
+pipeline = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", variant="fp16", use_safetensors=True, torch_dtype=torch.float16).to("cuda")
+
+
+for i,j in zip(models,adapters):
+    pipeline.load_lora_weights(i, weight_name="pytorch_lora_weights.safetensors",adapter_name=j) 
+
+
+prof='doctor'
+    
+
+pipeline.set_adapters(random.choice(adapters))
+
+
+compel = Compel(tokenizer=[pipeline.tokenizer, pipeline.tokenizer_2] ,
+                    text_encoder=[pipeline.text_encoder, pipeline.text_encoder_2],
+                    returned_embeddings_type=ReturnedEmbeddingsType.PENULTIMATE_HIDDEN_STATES_NON_NORMALIZED, 
+                    requires_pooled=[False, True],truncate_long_prompts=False)
+
+    
+conditioning, pooled = compel("a photo of a {}, looking at the camera, closeup headshot facing forward, ultra quality, sharp focus".format(prof)) 
+
+negative_conditioning, negative_pooled = compel(negative_prompt)
+[conditioning, negative_conditioning] = compel.pad_conditioning_tensors_to_same_length([conditioning, negative_conditioning])
+
+image = pipeline(prompt_embeds=conditioning, negative_prompt_embeds=negative_conditioning,
+                     pooled_prompt_embeds=pooled, negative_pooled_prompt_embeds=negative_pooled,
+                     num_inference_steps=40).images[0]
+
+image.save('/../../x.jpg')
+
+```
+
+
+# Examples
+
+| | | |
+|:-------------------------:|:-------------------------:|:-------------------------:|
+|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./image_0.png"> |  <img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./image_1.png">|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./image_2.png">|
+|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./image_3.png"> |  <img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./image_4.png">|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./image_5.png">|
+|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./image_6.png"> |  <img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./image_7.png">|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./image_8.png">|
+|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./image_9.png"> |  <img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./image_10.png">|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./image_11.png">|
+
+
+
+
+# Training data
+
+NYUAD-ComNets/Middle_Eastern_Male_Profession dataset was used to fine-tune stabilityai/stable-diffusion-xl-base-1.0
+
+
+
+# Configurations
 
 LoRA for the text encoder was enabled: False.
 
 Special VAE used for training: madebyollin/sdxl-vae-fp16-fix.
+
+
+
+# BibTeX entry and citation info
+
+```
+@misc{ComNets,
+      url={[https://huggingface.co/NYUAD-ComNets/Middle_Eastern_Male_Profession_Model](https://huggingface.co/NYUAD-ComNets/Middle_Eastern_Male_Profession_Model)},
+      title={Middle_Eastern_Male_Profession_Model},
+      author={Nouar AlDahoul, Talal Rahwan, Yasir Zaki}
+}
+```
+