Why don't any of these work with pony based XL?
#2
by
jackboot
- opened
I have tried all your clips and they work well in flux or normal models but for some reason ponyXL tunes produce static.
According to my testing, the Pony Diffusion V6 XL model has a highly modified CLIP compared to the original SDXL_Base model.
Please refer to my blog post for more details:
Even when combining the Pony Diffusion V6 XL model with the CLIP from the original SDXL_Base model, it fails to generate images properly.
Left: CLIP from Pony Diffusion V6 XL Right: CLIP from SDXL_Base
As Zer0int’s model is based on an improved version of the original CLIP-L, it seems to be incompatible with the customized Pony Diffusion V6 XL model.
Although I haven't tested this directly with the Pony Diffusion V6 XL model, I did try merging the improved CLIP-L with the highly customized NoobAI-XL model. In this case, illustrations collapsed when the ratio of the improved CLIP-L exceeded 10%.
Since many SDXL custom models involve significant modifications, various compatibility issues seem to arise.
I don't use / am not familiar with Pony, I'd ask about that / e.g. open an issue on the respective GitHub repo. It's possible that this is due to an unexpected format (HuggingFace vs. OpenAI/CLIP), which can affect the outcome. I've seen multiple research repos that said "the results will be different for safetensors models; to reproduce the results from the paper, please use the original PyTorch model!".
It is also possible that Pony uses e.g. the embeddings obtained BEFORE the final layer_norm, and expects the CLIP Text Encoder model to not even have that final layer norm - and if the Text Encoder has a final layer norm, it ends up using that after all, which leads to unexpected results. That's speculation / depends on the code; however, what you can do is: Compare the original Pony CLIP-L to my CLIP-L, and remove anything that is not present in Pony's CLIP-L from my CLIP-L. Then retry using this "clipped CLIP" version of my CLIP. =)
Here's a simple script that will compare two safetensors files with regard to the present keys, and if any excess keys are found in Model 2, it will prompt you to remove these and save the model.
Note that this only works if the Pony keys follow the same [HuggingFace] naming scheme as my CLIP; if the keys have different names, ALL will show up as being not present in Model 1.
However, the code also prints all keys (green for matches, red for mismatches), so you can review that upfront (requires colorama; pip install colorama if you don't have it).
Save this as compare.py and use like so: python compare.py <model1> <model2>, e.g.:python compare.py "C:/Users/zer0int/models/clip/CLIP_L_00001.safetensors" "C:/Users/zer0int/models/clip/ViT-L-14-SAE-TE-only.safetensors"
from safetensors.torch import load_file, save_file
from colorama import Fore, Style
import sys
import os
def compare_safetensors_keys(file1, file2):
try:
# Load safetensors models
model1 = load_file(file1)
model2 = load_file(file2)
# Get the keys from each model
keys1 = set(model1.keys())
keys2 = set(model2.keys())
# Find matching keys, unique to each model
matching_keys = keys1 & keys2
unique_keys1 = keys1 - keys2
unique_keys2 = keys2 - keys1
# Print results with colors
print(f"Model {file1}\t\t\t\tModel {file2}")
print("=" * 50)
for key in sorted(keys1 | keys2):
if key in matching_keys:
print(f"{Fore.GREEN}{key}\t\t\t\t{key}{Style.RESET_ALL}")
elif key in unique_keys1:
print(f"{Fore.YELLOW}[Only in {file1}] {key}\t\t\t\t{Style.RESET_ALL}")
elif key in unique_keys2:
print(f"\t\t\t\t{Fore.YELLOW}[Only in {file2}] {key}{Style.RESET_ALL}")
# Handle excess keys in Model 2
if unique_keys2:
response = input(f"{Fore.YELLOW}Model 2 has keys not present in Model 1. Remove excess keys and save adjusted model? (y/N): {Style.RESET_ALL}")
if response.lower() == 'y':
adjusted_model2 = {key: model2[key] for key in matching_keys}
save_path = os.path.join(os.path.dirname(file2), os.path.splitext(os.path.basename(file2))[0] + "-adjusted.safetensors")
save_file(adjusted_model2, save_path)
print(f"{Fore.GREEN}Adjusted model saved to {save_path}{Style.RESET_ALL}")
else:
print(f"{Fore.RED}Excess keys were not removed. Exiting.{Style.RESET_ALL}")
except Exception as e:
print(f"{Fore.RED}Error: {e}{Style.RESET_ALL}")
if __name__ == "__main__":
if len(sys.argv) != 3:
print(f"Usage: python {sys.argv[0]} <model1.safetensors> <model2.safetensors>")
sys.exit(1)
file1 = sys.argv[1]
file2 = sys.argv[2]
compare_safetensors_keys(file1, file2)
If that still doesn't work, I'd open an issue on the Pony GitHub repo, or whereever you get the models from. If anything, the creator of Pony etc. should know what's up with this CLIP guidance.
python compare.py PonyXL_clip_l.safetensors ViT-L-14-GmP-SAE-TE-only.safetensors
Model PonyXL_clip_l.safetensors Model ViT-L-14-GmP-SAE-TE-only.safetensors
==================================================
logit_scale logit_scale
text_model.embeddings.position_embedding.weight text_model.embeddings.position_embedding.weight
text_model.embeddings.token_embedding.weight text_model.embeddings.token_embedding.weight
text_model.encoder.layers.0.layer_norm1.bias text_model.encoder.layers.0.layer_norm1.bias
text_model.encoder.layers.0.layer_norm1.weight text_model.encoder.layers.0.layer_norm1.weight
text_model.encoder.layers.0.layer_norm2.bias text_model.encoder.layers.0.layer_norm2.bias
text_model.encoder.layers.0.layer_norm2.weight text_model.encoder.layers.0.layer_norm2.weight
text_model.encoder.layers.0.mlp.fc1.bias text_model.encoder.layers.0.mlp.fc1.bias
text_model.encoder.layers.0.mlp.fc1.weight text_model.encoder.layers.0.mlp.fc1.weight
text_model.encoder.layers.0.mlp.fc2.bias text_model.encoder.layers.0.mlp.fc2.bias
text_model.encoder.layers.0.mlp.fc2.weight text_model.encoder.layers.0.mlp.fc2.weight
text_model.encoder.layers.0.self_attn.k_proj.bias text_model.encoder.layers.0.self_attn.k_proj.bias
text_model.encoder.layers.0.self_attn.k_proj.weight text_model.encoder.layers.0.self_attn.k_proj.weight
text_model.encoder.layers.0.self_attn.out_proj.bias text_model.encoder.layers.0.self_attn.out_proj.bias
text_model.encoder.layers.0.self_attn.out_proj.weight text_model.encoder.layers.0.self_attn.out_proj.weight
text_model.encoder.layers.0.self_attn.q_proj.bias text_model.encoder.layers.0.self_attn.q_proj.bias
text_model.encoder.layers.0.self_attn.q_proj.weight text_model.encoder.layers.0.self_attn.q_proj.weight
text_model.encoder.layers.0.self_attn.v_proj.bias text_model.encoder.layers.0.self_attn.v_proj.bias
text_model.encoder.layers.0.self_attn.v_proj.weight text_model.encoder.layers.0.self_attn.v_proj.weight
text_model.encoder.layers.1.layer_norm1.bias text_model.encoder.layers.1.layer_norm1.bias
text_model.encoder.layers.1.layer_norm1.weight text_model.encoder.layers.1.layer_norm1.weight
text_model.encoder.layers.1.layer_norm2.bias text_model.encoder.layers.1.layer_norm2.bias
text_model.encoder.layers.1.layer_norm2.weight text_model.encoder.layers.1.layer_norm2.weight
text_model.encoder.layers.1.mlp.fc1.bias text_model.encoder.layers.1.mlp.fc1.bias
text_model.encoder.layers.1.mlp.fc1.weight text_model.encoder.layers.1.mlp.fc1.weight
text_model.encoder.layers.1.mlp.fc2.bias text_model.encoder.layers.1.mlp.fc2.bias
text_model.encoder.layers.1.mlp.fc2.weight text_model.encoder.layers.1.mlp.fc2.weight
text_model.encoder.layers.1.self_attn.k_proj.bias text_model.encoder.layers.1.self_attn.k_proj.bias
text_model.encoder.layers.1.self_attn.k_proj.weight text_model.encoder.layers.1.self_attn.k_proj.weight
text_model.encoder.layers.1.self_attn.out_proj.bias text_model.encoder.layers.1.self_attn.out_proj.bias
text_model.encoder.layers.1.self_attn.out_proj.weight text_model.encoder.layers.1.self_attn.out_proj.weight
text_model.encoder.layers.1.self_attn.q_proj.bias text_model.encoder.layers.1.self_attn.q_proj.bias
text_model.encoder.layers.1.self_attn.q_proj.weight text_model.encoder.layers.1.self_attn.q_proj.weight
text_model.encoder.layers.1.self_attn.v_proj.bias text_model.encoder.layers.1.self_attn.v_proj.bias
text_model.encoder.layers.1.self_attn.v_proj.weight text_model.encoder.layers.1.self_attn.v_proj.weight
text_model.encoder.layers.10.layer_norm1.bias text_model.encoder.layers.10.layer_norm1.bias
text_model.encoder.layers.10.layer_norm1.weight text_model.encoder.layers.10.layer_norm1.weight
text_model.encoder.layers.10.layer_norm2.bias text_model.encoder.layers.10.layer_norm2.bias
text_model.encoder.layers.10.layer_norm2.weight text_model.encoder.layers.10.layer_norm2.weight
text_model.encoder.layers.10.mlp.fc1.bias text_model.encoder.layers.10.mlp.fc1.bias
text_model.encoder.layers.10.mlp.fc1.weight text_model.encoder.layers.10.mlp.fc1.weight
text_model.encoder.layers.10.mlp.fc2.bias text_model.encoder.layers.10.mlp.fc2.bias
text_model.encoder.layers.10.mlp.fc2.weight text_model.encoder.layers.10.mlp.fc2.weight
text_model.encoder.layers.10.self_attn.k_proj.bias text_model.encoder.layers.10.self_attn.k_proj.bias
text_model.encoder.layers.10.self_attn.k_proj.weight text_model.encoder.layers.10.self_attn.k_proj.weight
text_model.encoder.layers.10.self_attn.out_proj.bias text_model.encoder.layers.10.self_attn.out_proj.bias
text_model.encoder.layers.10.self_attn.out_proj.weight text_model.encoder.layers.10.self_attn.out_proj.weight
text_model.encoder.layers.10.self_attn.q_proj.bias text_model.encoder.layers.10.self_attn.q_proj.bias
text_model.encoder.layers.10.self_attn.q_proj.weight text_model.encoder.layers.10.self_attn.q_proj.weight
text_model.encoder.layers.10.self_attn.v_proj.bias text_model.encoder.layers.10.self_attn.v_proj.bias
text_model.encoder.layers.10.self_attn.v_proj.weight text_model.encoder.layers.10.self_attn.v_proj.weight
text_model.encoder.layers.11.layer_norm1.bias text_model.encoder.layers.11.layer_norm1.bias
text_model.encoder.layers.11.layer_norm1.weight text_model.encoder.layers.11.layer_norm1.weight
text_model.encoder.layers.11.layer_norm2.bias text_model.encoder.layers.11.layer_norm2.bias
text_model.encoder.layers.11.layer_norm2.weight text_model.encoder.layers.11.layer_norm2.weight
text_model.encoder.layers.11.mlp.fc1.bias text_model.encoder.layers.11.mlp.fc1.bias
text_model.encoder.layers.11.mlp.fc1.weight text_model.encoder.layers.11.mlp.fc1.weight
text_model.encoder.layers.11.mlp.fc2.bias text_model.encoder.layers.11.mlp.fc2.bias
text_model.encoder.layers.11.mlp.fc2.weight text_model.encoder.layers.11.mlp.fc2.weight
text_model.encoder.layers.11.self_attn.k_proj.bias text_model.encoder.layers.11.self_attn.k_proj.bias
text_model.encoder.layers.11.self_attn.k_proj.weight text_model.encoder.layers.11.self_attn.k_proj.weight
text_model.encoder.layers.11.self_attn.out_proj.bias text_model.encoder.layers.11.self_attn.out_proj.bias
text_model.encoder.layers.11.self_attn.out_proj.weight text_model.encoder.layers.11.self_attn.out_proj.weight
text_model.encoder.layers.11.self_attn.q_proj.bias text_model.encoder.layers.11.self_attn.q_proj.bias
text_model.encoder.layers.11.self_attn.q_proj.weight text_model.encoder.layers.11.self_attn.q_proj.weight
text_model.encoder.layers.11.self_attn.v_proj.bias text_model.encoder.layers.11.self_attn.v_proj.bias
text_model.encoder.layers.11.self_attn.v_proj.weight text_model.encoder.layers.11.self_attn.v_proj.weight
text_model.encoder.layers.2.layer_norm1.bias text_model.encoder.layers.2.layer_norm1.bias
text_model.encoder.layers.2.layer_norm1.weight text_model.encoder.layers.2.layer_norm1.weight
text_model.encoder.layers.2.layer_norm2.bias text_model.encoder.layers.2.layer_norm2.bias
text_model.encoder.layers.2.layer_norm2.weight text_model.encoder.layers.2.layer_norm2.weight
text_model.encoder.layers.2.mlp.fc1.bias text_model.encoder.layers.2.mlp.fc1.bias
text_model.encoder.layers.2.mlp.fc1.weight text_model.encoder.layers.2.mlp.fc1.weight
text_model.encoder.layers.2.mlp.fc2.bias text_model.encoder.layers.2.mlp.fc2.bias
text_model.encoder.layers.2.mlp.fc2.weight text_model.encoder.layers.2.mlp.fc2.weight
text_model.encoder.layers.2.self_attn.k_proj.bias text_model.encoder.layers.2.self_attn.k_proj.bias
text_model.encoder.layers.2.self_attn.k_proj.weight text_model.encoder.layers.2.self_attn.k_proj.weight
text_model.encoder.layers.2.self_attn.out_proj.bias text_model.encoder.layers.2.self_attn.out_proj.bias
text_model.encoder.layers.2.self_attn.out_proj.weight text_model.encoder.layers.2.self_attn.out_proj.weight
text_model.encoder.layers.2.self_attn.q_proj.bias text_model.encoder.layers.2.self_attn.q_proj.bias
text_model.encoder.layers.2.self_attn.q_proj.weight text_model.encoder.layers.2.self_attn.q_proj.weight
text_model.encoder.layers.2.self_attn.v_proj.bias text_model.encoder.layers.2.self_attn.v_proj.bias
text_model.encoder.layers.2.self_attn.v_proj.weight text_model.encoder.layers.2.self_attn.v_proj.weight
text_model.encoder.layers.3.layer_norm1.bias text_model.encoder.layers.3.layer_norm1.bias
text_model.encoder.layers.3.layer_norm1.weight text_model.encoder.layers.3.layer_norm1.weight
text_model.encoder.layers.3.layer_norm2.bias text_model.encoder.layers.3.layer_norm2.bias
text_model.encoder.layers.3.layer_norm2.weight text_model.encoder.layers.3.layer_norm2.weight
text_model.encoder.layers.3.mlp.fc1.bias text_model.encoder.layers.3.mlp.fc1.bias
text_model.encoder.layers.3.mlp.fc1.weight text_model.encoder.layers.3.mlp.fc1.weight
text_model.encoder.layers.3.mlp.fc2.bias text_model.encoder.layers.3.mlp.fc2.bias
text_model.encoder.layers.3.mlp.fc2.weight text_model.encoder.layers.3.mlp.fc2.weight
text_model.encoder.layers.3.self_attn.k_proj.bias text_model.encoder.layers.3.self_attn.k_proj.bias
text_model.encoder.layers.3.self_attn.k_proj.weight text_model.encoder.layers.3.self_attn.k_proj.weight
text_model.encoder.layers.3.self_attn.out_proj.bias text_model.encoder.layers.3.self_attn.out_proj.bias
text_model.encoder.layers.3.self_attn.out_proj.weight text_model.encoder.layers.3.self_attn.out_proj.weight
text_model.encoder.layers.3.self_attn.q_proj.bias text_model.encoder.layers.3.self_attn.q_proj.bias
text_model.encoder.layers.3.self_attn.q_proj.weight text_model.encoder.layers.3.self_attn.q_proj.weight
text_model.encoder.layers.3.self_attn.v_proj.bias text_model.encoder.layers.3.self_attn.v_proj.bias
text_model.encoder.layers.3.self_attn.v_proj.weight text_model.encoder.layers.3.self_attn.v_proj.weight
text_model.encoder.layers.4.layer_norm1.bias text_model.encoder.layers.4.layer_norm1.bias
text_model.encoder.layers.4.layer_norm1.weight text_model.encoder.layers.4.layer_norm1.weight
text_model.encoder.layers.4.layer_norm2.bias text_model.encoder.layers.4.layer_norm2.bias
text_model.encoder.layers.4.layer_norm2.weight text_model.encoder.layers.4.layer_norm2.weight
text_model.encoder.layers.4.mlp.fc1.bias text_model.encoder.layers.4.mlp.fc1.bias
text_model.encoder.layers.4.mlp.fc1.weight text_model.encoder.layers.4.mlp.fc1.weight
text_model.encoder.layers.4.mlp.fc2.bias text_model.encoder.layers.4.mlp.fc2.bias
text_model.encoder.layers.4.mlp.fc2.weight text_model.encoder.layers.4.mlp.fc2.weight
text_model.encoder.layers.4.self_attn.k_proj.bias text_model.encoder.layers.4.self_attn.k_proj.bias
text_model.encoder.layers.4.self_attn.k_proj.weight text_model.encoder.layers.4.self_attn.k_proj.weight
text_model.encoder.layers.4.self_attn.out_proj.bias text_model.encoder.layers.4.self_attn.out_proj.bias
text_model.encoder.layers.4.self_attn.out_proj.weight text_model.encoder.layers.4.self_attn.out_proj.weight
text_model.encoder.layers.4.self_attn.q_proj.bias text_model.encoder.layers.4.self_attn.q_proj.bias
text_model.encoder.layers.4.self_attn.q_proj.weight text_model.encoder.layers.4.self_attn.q_proj.weight
text_model.encoder.layers.4.self_attn.v_proj.bias text_model.encoder.layers.4.self_attn.v_proj.bias
text_model.encoder.layers.4.self_attn.v_proj.weight text_model.encoder.layers.4.self_attn.v_proj.weight
text_model.encoder.layers.5.layer_norm1.bias text_model.encoder.layers.5.layer_norm1.bias
text_model.encoder.layers.5.layer_norm1.weight text_model.encoder.layers.5.layer_norm1.weight
text_model.encoder.layers.5.layer_norm2.bias text_model.encoder.layers.5.layer_norm2.bias
text_model.encoder.layers.5.layer_norm2.weight text_model.encoder.layers.5.layer_norm2.weight
text_model.encoder.layers.5.mlp.fc1.bias text_model.encoder.layers.5.mlp.fc1.bias
text_model.encoder.layers.5.mlp.fc1.weight text_model.encoder.layers.5.mlp.fc1.weight
text_model.encoder.layers.5.mlp.fc2.bias text_model.encoder.layers.5.mlp.fc2.bias
text_model.encoder.layers.5.mlp.fc2.weight text_model.encoder.layers.5.mlp.fc2.weight
text_model.encoder.layers.5.self_attn.k_proj.bias text_model.encoder.layers.5.self_attn.k_proj.bias
text_model.encoder.layers.5.self_attn.k_proj.weight text_model.encoder.layers.5.self_attn.k_proj.weight
text_model.encoder.layers.5.self_attn.out_proj.bias text_model.encoder.layers.5.self_attn.out_proj.bias
text_model.encoder.layers.5.self_attn.out_proj.weight text_model.encoder.layers.5.self_attn.out_proj.weight
text_model.encoder.layers.5.self_attn.q_proj.bias text_model.encoder.layers.5.self_attn.q_proj.bias
text_model.encoder.layers.5.self_attn.q_proj.weight text_model.encoder.layers.5.self_attn.q_proj.weight
text_model.encoder.layers.5.self_attn.v_proj.bias text_model.encoder.layers.5.self_attn.v_proj.bias
text_model.encoder.layers.5.self_attn.v_proj.weight text_model.encoder.layers.5.self_attn.v_proj.weight
text_model.encoder.layers.6.layer_norm1.bias text_model.encoder.layers.6.layer_norm1.bias
text_model.encoder.layers.6.layer_norm1.weight text_model.encoder.layers.6.layer_norm1.weight
text_model.encoder.layers.6.layer_norm2.bias text_model.encoder.layers.6.layer_norm2.bias
text_model.encoder.layers.6.layer_norm2.weight text_model.encoder.layers.6.layer_norm2.weight
text_model.encoder.layers.6.mlp.fc1.bias text_model.encoder.layers.6.mlp.fc1.bias
text_model.encoder.layers.6.mlp.fc1.weight text_model.encoder.layers.6.mlp.fc1.weight
text_model.encoder.layers.6.mlp.fc2.bias text_model.encoder.layers.6.mlp.fc2.bias
text_model.encoder.layers.6.mlp.fc2.weight text_model.encoder.layers.6.mlp.fc2.weight
text_model.encoder.layers.6.self_attn.k_proj.bias text_model.encoder.layers.6.self_attn.k_proj.bias
text_model.encoder.layers.6.self_attn.k_proj.weight text_model.encoder.layers.6.self_attn.k_proj.weight
text_model.encoder.layers.6.self_attn.out_proj.bias text_model.encoder.layers.6.self_attn.out_proj.bias
text_model.encoder.layers.6.self_attn.out_proj.weight text_model.encoder.layers.6.self_attn.out_proj.weight
text_model.encoder.layers.6.self_attn.q_proj.bias text_model.encoder.layers.6.self_attn.q_proj.bias
text_model.encoder.layers.6.self_attn.q_proj.weight text_model.encoder.layers.6.self_attn.q_proj.weight
text_model.encoder.layers.6.self_attn.v_proj.bias text_model.encoder.layers.6.self_attn.v_proj.bias
text_model.encoder.layers.6.self_attn.v_proj.weight text_model.encoder.layers.6.self_attn.v_proj.weight
text_model.encoder.layers.7.layer_norm1.bias text_model.encoder.layers.7.layer_norm1.bias
text_model.encoder.layers.7.layer_norm1.weight text_model.encoder.layers.7.layer_norm1.weight
text_model.encoder.layers.7.layer_norm2.bias text_model.encoder.layers.7.layer_norm2.bias
text_model.encoder.layers.7.layer_norm2.weight text_model.encoder.layers.7.layer_norm2.weight
text_model.encoder.layers.7.mlp.fc1.bias text_model.encoder.layers.7.mlp.fc1.bias
text_model.encoder.layers.7.mlp.fc1.weight text_model.encoder.layers.7.mlp.fc1.weight
text_model.encoder.layers.7.mlp.fc2.bias text_model.encoder.layers.7.mlp.fc2.bias
text_model.encoder.layers.7.mlp.fc2.weight text_model.encoder.layers.7.mlp.fc2.weight
text_model.encoder.layers.7.self_attn.k_proj.bias text_model.encoder.layers.7.self_attn.k_proj.bias
text_model.encoder.layers.7.self_attn.k_proj.weight text_model.encoder.layers.7.self_attn.k_proj.weight
text_model.encoder.layers.7.self_attn.out_proj.bias text_model.encoder.layers.7.self_attn.out_proj.bias
text_model.encoder.layers.7.self_attn.out_proj.weight text_model.encoder.layers.7.self_attn.out_proj.weight
text_model.encoder.layers.7.self_attn.q_proj.bias text_model.encoder.layers.7.self_attn.q_proj.bias
text_model.encoder.layers.7.self_attn.q_proj.weight text_model.encoder.layers.7.self_attn.q_proj.weight
text_model.encoder.layers.7.self_attn.v_proj.bias text_model.encoder.layers.7.self_attn.v_proj.bias
text_model.encoder.layers.7.self_attn.v_proj.weight text_model.encoder.layers.7.self_attn.v_proj.weight
text_model.encoder.layers.8.layer_norm1.bias text_model.encoder.layers.8.layer_norm1.bias
text_model.encoder.layers.8.layer_norm1.weight text_model.encoder.layers.8.layer_norm1.weight
text_model.encoder.layers.8.layer_norm2.bias text_model.encoder.layers.8.layer_norm2.bias
text_model.encoder.layers.8.layer_norm2.weight text_model.encoder.layers.8.layer_norm2.weight
text_model.encoder.layers.8.mlp.fc1.bias text_model.encoder.layers.8.mlp.fc1.bias
text_model.encoder.layers.8.mlp.fc1.weight text_model.encoder.layers.8.mlp.fc1.weight
text_model.encoder.layers.8.mlp.fc2.bias text_model.encoder.layers.8.mlp.fc2.bias
text_model.encoder.layers.8.mlp.fc2.weight text_model.encoder.layers.8.mlp.fc2.weight
text_model.encoder.layers.8.self_attn.k_proj.bias text_model.encoder.layers.8.self_attn.k_proj.bias
text_model.encoder.layers.8.self_attn.k_proj.weight text_model.encoder.layers.8.self_attn.k_proj.weight
text_model.encoder.layers.8.self_attn.out_proj.bias text_model.encoder.layers.8.self_attn.out_proj.bias
text_model.encoder.layers.8.self_attn.out_proj.weight text_model.encoder.layers.8.self_attn.out_proj.weight
text_model.encoder.layers.8.self_attn.q_proj.bias text_model.encoder.layers.8.self_attn.q_proj.bias
text_model.encoder.layers.8.self_attn.q_proj.weight text_model.encoder.layers.8.self_attn.q_proj.weight
text_model.encoder.layers.8.self_attn.v_proj.bias text_model.encoder.layers.8.self_attn.v_proj.bias
text_model.encoder.layers.8.self_attn.v_proj.weight text_model.encoder.layers.8.self_attn.v_proj.weight
text_model.encoder.layers.9.layer_norm1.bias text_model.encoder.layers.9.layer_norm1.bias
text_model.encoder.layers.9.layer_norm1.weight text_model.encoder.layers.9.layer_norm1.weight
text_model.encoder.layers.9.layer_norm2.bias text_model.encoder.layers.9.layer_norm2.bias
text_model.encoder.layers.9.layer_norm2.weight text_model.encoder.layers.9.layer_norm2.weight
text_model.encoder.layers.9.mlp.fc1.bias text_model.encoder.layers.9.mlp.fc1.bias
text_model.encoder.layers.9.mlp.fc1.weight text_model.encoder.layers.9.mlp.fc1.weight
text_model.encoder.layers.9.mlp.fc2.bias text_model.encoder.layers.9.mlp.fc2.bias
text_model.encoder.layers.9.mlp.fc2.weight text_model.encoder.layers.9.mlp.fc2.weight
text_model.encoder.layers.9.self_attn.k_proj.bias text_model.encoder.layers.9.self_attn.k_proj.bias
text_model.encoder.layers.9.self_attn.k_proj.weight text_model.encoder.layers.9.self_attn.k_proj.weight
text_model.encoder.layers.9.self_attn.out_proj.bias text_model.encoder.layers.9.self_attn.out_proj.bias
text_model.encoder.layers.9.self_attn.out_proj.weight text_model.encoder.layers.9.self_attn.out_proj.weight
text_model.encoder.layers.9.self_attn.q_proj.bias text_model.encoder.layers.9.self_attn.q_proj.bias
text_model.encoder.layers.9.self_attn.q_proj.weight text_model.encoder.layers.9.self_attn.q_proj.weight
text_model.encoder.layers.9.self_attn.v_proj.bias text_model.encoder.layers.9.self_attn.v_proj.bias
text_model.encoder.layers.9.self_attn.v_proj.weight text_model.encoder.layers.9.self_attn.v_proj.weight
text_model.final_layer_norm.bias text_model.final_layer_norm.bias
text_model.final_layer_norm.weight text_model.final_layer_norm.weight
[Only in ViT-L-14-GmP-SAE-TE-only.safetensors] text_projection
[Only in PonyXL_clip_l.safetensors] text_projection.weight
Model 2 has keys not present in Model 1. Remove excess keys and save adjusted model? (y/N):
I ran it against clip ripped from ponyxl. It should all be HF format. Sadly it still didn't work. The image is cooked and mostly noise.