Spaces:
Running
Running
| import gradio as gr | |
| from sklearn.metrics.pairwise import cosine_similarity | |
| from scipy.sparse import csr_matrix | |
| import numpy as np | |
| import joblib | |
| from joblib import load | |
| import h5py | |
| from io import BytesIO | |
| import csv | |
| import re | |
| import random | |
| import compress_fasttext | |
| from collections import OrderedDict | |
| from lark import Lark, Tree, Token | |
| from lark.exceptions import ParseError | |
| import json | |
| import zipfile | |
| from PIL import Image | |
| import io | |
| import os | |
| import glob | |
| import itertools | |
| from itertools import islice | |
| from pathlib import Path | |
| import logging | |
| # Set up logging | |
| logging.basicConfig(filename='error.log', level=logging.DEBUG, format='%(asctime)s %(levelname)s:%(message)s') | |
| faq_content=""" | |
| # Questions: | |
| ## What is the purpose of this tool? | |
| Since Stable Diffusion's initial release in 2022, users have developed a myriad of fine-tuned text to image models, each with unique "linguistic" preferences depending on the data from which it was fine-tuned. | |
| Some models react best when prompted with verbose scene descriptions akin to DALL-E, while others fine-tuned on images scraped from popular image boards understand those boards' tag sets. | |
| This tool serves as a linguistic bridge to the e621 image board tag lexicon, on which many popular models such as Fluffyrock, Fluffusion, and Pony Diffusion v6 were trained. | |
| When you enter a txt2img prompt and press the "submit" button, Prompt Squirrel parses your prompt and checks that all your tags are valid e621 tags. | |
| If it finds any that are not, it recommends some valid e621 tags you can use to replace them in the "Unknown Tags" section. | |
| Additionally, in the "Top Artists" text box, it lists the artists who would most likely draw an image having the set of tags you provided. | |
| This is useful to align your prompt with the expected input to an e621-trained model. | |
| ## Does input order matter? | |
| No | |
| ## Should I use underscores or spaces in the input tags? | |
| As a rule, e621-trained models replace underscores in tags with spaces, so spaces are preferred. | |
| ## Can I use parentheses or weights as in the Stable Diffusion Automatic1111 WebUI? | |
| Yes, but only '(' and ')' and numerical weights, and all of these things are ignored in all calculations. The main benefit of this is that you can copy/paste prompts from one program to another with minimal editing. | |
| An example that illustrates acceptable parentheses and weight formatting is: | |
| ((sunset over the mountains)), (clear sky:1.5), ((eagle flying high:2.0)), river, (fish swimming in the river:1.2), (campfire, (marshmallows:2.1):1.3), stars in the sky, ((full moon:1.8)), (wolf howling:1.7) | |
| ## Why are some valid tags marked as "unknown", and why don't some artists ever get returned? | |
| Some data is excluded from consideration if it did not occur frequently enough in the sample from which the application makes its calculations. | |
| If an artist or tag is too infrequent, we might not think we have enough data to make predictions about it. Additionally, Prompt Squirrel gathers information from several sources, and the sources are not always consistent about things like exact tag names or counts, which vary over time. | |
| ## Why do some suggested tags not have summaries or wiki links, and of those that do, why do some look truncated? | |
| Both of these features are extracted from the tag wiki pages, but some valid e621 tags do not have wiki pages. Additionally, the summaries are heuristically extracted from the beginning of the wiki pages, and this extraction process is prone to some amount of error. | |
| ## Are there any special tags? | |
| Yes. We normalized the favorite counts of each image to a range of 0-9, with 0 being the lowest favcount, and 9 being the highest. | |
| You can include any of these special tags: "score:0", "score:1", "score:2", "score:3", "score:4", "score:5", "score:6", "score:7", "score:8", "score:9" | |
| in your list to bias the output toward artists with higher or lower scoring images. | |
| ## Are there any other special tricks? | |
| Yes. If you want to more strongly bias the artist output toward a specific tag, you can just list it multiple times. | |
| So for example, the query "red fox, red fox, red fox, score:7" will yield a list of artists who are more strongly associated with the tag "red fox" | |
| than the query "red fox, score:7". | |
| ## Why is this space tagged "not-for-all-audience" | |
| The "not-for-all-audience" tag informs users that this tool's text output is derived from e621.net data for tag prediction and completion. | |
| The app will try not to display nsfw tags unless the "Allow NSFW Tags" is checked, but the filter is not perfect. | |
| ## How is the artist list calculated? | |
| Each artist is represented by a "pseudo-document" composed of all the tags from their uploaded images, treating these tags similarly to words in a text document. | |
| Similarly, when you input a set of tags, the system creates a pseudo-document for your query out of all the tags. | |
| It then compares your tags against each artist's collection, essentially finding which artist's tags are most "similar" to yours. | |
| This method helps identify artists whose work is closely aligned with the themes or elements you're interested in. | |
| For those curious about the underlying mechanics of comparing text-like data, we employ the TF-IDF (Term Frequency-Inverse Document Frequency) method, a standard approach in information retrieval, and reduce the TF-IDF matrix to a reasonable size using Singular Value Decomposition. | |
| You can read more about TF-IDF on its [Wikipedia page](https://en.wikipedia.org/wiki/Tf%E2%80%93idf) and Singular Value Decomposition on its [Wikipedia page](https://en.wikipedia.org/wiki/Singular_value_decomposition). | |
| ## How does the tag corrector work? | |
| We collect the tag sets from over 4 million e621 posts, treating the tag set from each image as an individual document. | |
| We then randomly replace about 10% of the tags in each document with a randomly selected alias from e621's list of aliases for the tag | |
| (e.g. "canine" gets replaced with one of {k9,canines,mongrel,cannine,cnaine,feral_canine,anthro_canine}). | |
| We then train a FastText (https://fasttext.cc/) model on the documents. The result of this training is a function that maps arbitrary words to vectors such that | |
| the vector for a tag and the vectors for its aliases are all close together (because the model has seen them in similar contexts). | |
| Since the lists of aliases contain misspellings and rephrasings of tags, the model should be robust to these kinds of problems as long as they are not too dissimilar from the alias lists. | |
| To enhance the tag corrector further, we employ the same TF-IDF method we used for artist tags to calculate a separate, context-sensitive similarity score for each of the top 100 tags selected via the FastText method. | |
| By considering the context in which tags are used, we can now not only correct misspellings and rephrasings but also make more contextually relevant suggestions. | |
| The "similarity weight" slider controls how much weight these TF-IDF scores are given vs how much weight the FastText similarity model is given when suggesting replacements for invalid tags. | |
| A similarity weight slider value of 0 means that only the FastText model's predictions will be used to calculate similarity scores, and a value of 1 means only the TF-IDF scores are used (although the FastText model is still used to trim the list of candidates). | |
| ## How do the sample images work? | |
| In the first row of galleries, for each artist in the dataset, we generated a sample image with the model Fluffyrock Unleashed using the prompt "by artist, soyjak, anthro, male, bust portrait, meme, grin" where "artist" is the name of an artist. | |
| The simplicity of the prompt, the the simplicty of the default style, and the recognizability of the character make it easier to understand how artist names affect generated image styles. | |
| The image on the left captioned "No Artist" was generated with the same prompt, but with no artist name. | |
| You should compare all the images to the first to see how the artist names affect the output. | |
| Each subsequent row of images was generated using the same process, but with a different prompt. | |
| See SamplePrompts.csv for the list of prompts used and their descriptions. | |
| """ | |
| nsfw_threshold = 0.95 # Assuming the threshold value is defined here | |
| css = """ | |
| .scrollable-content { | |
| max-height: 500px; | |
| overflow-y: auto; | |
| } | |
| """ | |
| grammar=r""" | |
| !start: (prompt | /[][():]/+)* | |
| prompt: (emphasized | plain | comma | WHITESPACE)* | |
| !emphasized: "(" prompt ")" | |
| | "(" prompt ":" [WHITESPACE] NUMBER [WHITESPACE] ")" | |
| comma: "," | |
| WHITESPACE: /\s+/ | |
| plain: /([^,\\\[\]():|]|\\.)+/ | |
| %import common.SIGNED_NUMBER -> NUMBER | |
| """ | |
| # Initialize the parser | |
| parser = Lark(grammar, start='start') | |
| # Function to extract tags | |
| def extract_tags(tree): | |
| tags_with_positions = [] | |
| def _traverse(node): | |
| if isinstance(node, Token) and node.type == '__ANON_1': | |
| tag_position = node.start_pos | |
| tag_text = node.value | |
| tags_with_positions.append((tag_text, tag_position, "tag")) | |
| elif not isinstance(node, Token): | |
| for child in node.children: | |
| _traverse(child) | |
| _traverse(tree) | |
| return tags_with_positions | |
| special_tags = ["score:0", "score:1", "score:2", "score:3", "score:4", "score:5", "score:6", "score:7", "score:8", "score:9", "rating:s", "rating:q", "rating:e"] | |
| def remove_special_tags(original_string): | |
| tags = [tag.strip() for tag in original_string.split(",")] | |
| remaining_tags = [tag for tag in tags if tag not in special_tags] | |
| removed_tags = [tag for tag in tags if tag in special_tags] | |
| return ", ".join(remaining_tags), removed_tags | |
| # Define a function to load all necessary components | |
| def load_model_components(file_path): | |
| # Ensure the file path is a Path object for robust path handling | |
| file_path = Path(file_path) | |
| # Check if the file exists | |
| if not file_path.is_file(): | |
| raise FileNotFoundError(f"The specified joblib file was not found: {file_path}") | |
| # Load all the model components from the joblib file | |
| model_components = joblib.load(file_path) | |
| # Create a reverse mapping from row index to tag | |
| if 'tag_to_row_index' in model_components: | |
| model_components['row_to_tag'] = {idx: tag for tag, idx in model_components['tag_to_row_index'].items()} | |
| return model_components | |
| # Load all components at the start | |
| tf_idf_components = load_model_components('tf_idf_files_420.joblib') | |
| nsfw_tags = set() # Initialize an empty set to store words meeting the threshold | |
| # Open and read the CSV file | |
| with open("word_rating_probabilities.csv", 'r', newline='', encoding='utf-8') as csvfile: | |
| reader = csv.reader(csvfile) | |
| next(reader, None) # Skip the header row | |
| for row in reader: | |
| word = row[0] # The word is in the first column | |
| probability_sum = float(row[1]) # The sum of probabilities is in the second column, convert to float for comparison | |
| # Check if the probability sum meets the threshold and add the word to the set if it does | |
| if probability_sum >= nsfw_threshold: | |
| nsfw_tags.add(word) | |
| # Read the set of valid artists into memory. | |
| artist_set = set() | |
| with open("fluffyrock_3m.csv", 'r', newline='', encoding='utf-8') as csvfile: | |
| """ | |
| Load artist names from a CSV file and store them in the global set. | |
| Artist tags start with 'by_' and the prefix will be removed. | |
| """ | |
| reader = csv.reader(csvfile) | |
| for row in reader: | |
| tag_name = row[0] # Assuming the first column contains the tag names | |
| if tag_name.startswith('by_'): | |
| # Strip 'by_' from the start of the tag name and add to the set | |
| artist_name = tag_name[3:] # Remove the first three characters 'by_' | |
| artist_set.add(artist_name) | |
| def is_artist(name): | |
| return name in artist_set | |
| sample_images_directory_path = 'sampleimages' | |
| def generate_artist_image_tuples(top_artists, image_directory): | |
| json_files = glob.glob(f'{image_directory}/*.json') | |
| json_file_path = json_files[0] if json_files else None | |
| with open(json_file_path, 'r') as json_file: | |
| artist_to_file_map = json.load(json_file) | |
| filename = artist_to_file_map.get("") | |
| image_path = os.path.join(image_directory, filename) | |
| if os.path.exists(image_path): | |
| baseline_tuple = [(image_path, "No Artist")] | |
| artist_image_tuples = [] | |
| for artist in top_artists: | |
| filename = artist_to_file_map.get(artist) | |
| if filename: | |
| image_path = os.path.join(image_directory, filename) | |
| if os.path.exists(image_path): | |
| artist_image_tuples.append((image_path, artist if artist else "No Artist")) | |
| return baseline_tuple, artist_image_tuples | |
| def clean_tag(tag): | |
| return ''.join(char for char in tag if ord(char) < 128) | |
| #Normally returns tag to aliases, but when reverse=True, returns alias to tags | |
| def build_aliases_dict(filename, reverse=False): | |
| aliases_dict = {} | |
| with open(filename, 'r', newline='', encoding='utf-8') as csvfile: | |
| reader = csv.reader(csvfile) | |
| for row in reader: | |
| tag = clean_tag(row[0]) | |
| alias_list = [] if row[3] == "null" else [clean_tag(alias) for alias in row[3].split(',')] | |
| if reverse: | |
| for alias in alias_list: | |
| aliases_dict.setdefault(alias, []).append(tag) | |
| else: | |
| aliases_dict[tag] = alias_list | |
| return aliases_dict | |
| def build_tag_count_dict(filename): | |
| with open(filename, 'r', newline='', encoding='utf-8') as csvfile: | |
| reader = csv.reader(csvfile) | |
| result_dict = {} | |
| for row in reader: | |
| key = row[0] | |
| value = int(row[2]) if row[2].isdigit() else None | |
| if value is not None: | |
| result_dict[key] = value | |
| return result_dict | |
| import csv | |
| def build_tag_id_wiki_dict(filename='wiki_pages-2023-08-08.csv'): | |
| """ | |
| Reads a CSV file and returns a dictionary mapping tag names to tuples of | |
| (number, most relevant line from the wiki entry). Rows with a non-integer in the first column are ignored. | |
| The most relevant line is the first line that does not start with "thumb" and is not blank. | |
| Parameters: | |
| - filename: The path to the CSV file. | |
| Returns: | |
| - A dictionary where each key is a tag name and each value is a tuple (number, most relevant wiki entry line). | |
| """ | |
| tag_data = {} | |
| with open(filename, 'r', encoding='utf-8') as csvfile: | |
| reader = csv.reader(csvfile) | |
| # Skip the header row | |
| next(reader) | |
| for row in reader: | |
| try: | |
| # Attempt to convert the first column to an integer | |
| number = int(row[0]) | |
| except ValueError: | |
| # If conversion fails, skip this row | |
| continue | |
| tag = row[3] | |
| wiki_entry_full = row[4] | |
| # Process the wiki_entry to find the most relevant line | |
| relevant_line = '' | |
| for line in wiki_entry_full.split('\n'): | |
| if line.strip() and not line.startswith("thumb"): | |
| relevant_line = line | |
| break | |
| # Map the tag to a tuple of (number, relevant_line) | |
| tag_data[tag] = (number, relevant_line) | |
| return tag_data | |
| def create_html_tables_for_tags(subtable_heading, item_heading, word_similarity_tuples, tag2count, tag2idwiki): | |
| # Wrap the tag part in a <span> with styles for bold and larger font | |
| html_str = f"<div style='display: inline-block; margin: 20px; vertical-align: top;'><table><thead><tr><th colspan='3' style='text-align: center; padding-bottom: 10px;'><span style='font-weight: bold; font-size: 20px;'>{subtable_heading}</span></th></tr></thead><tbody><tr style='border-bottom: 1px solid #000;'><th>{item_heading}</th><th>Similarity</th><th>Count</th></tr>" | |
| # Loop through the results and add table rows for each | |
| for word, sim in word_similarity_tuples: | |
| word_with_underscores = word.replace(' ', '_') | |
| word_with_escaped_parentheses = word.replace("\\(", "(").replace("\\)", ")").replace("(", "\\(").replace(")", "\\)") | |
| count = tag2count.get(word_with_underscores.replace("\\(", "(").replace("\\)", ")"), 0) # Get the count if available, otherwise default to 0 | |
| tag_id, wiki_entry = tag2idwiki.get(word_with_underscores, (None, '')) | |
| # Check if tag_id and wiki_entry are valid | |
| if tag_id is not None and wiki_entry: | |
| # Construct the URL for the tag's wiki page | |
| wiki_url = f"https://e621.net/wiki_pages/{tag_id}" | |
| # Make the tag a hyperlink with a tooltip | |
| tag_element = f"<a href='{wiki_url}' target='_blank' title='{wiki_entry}'>{word_with_escaped_parentheses}</a>" | |
| else: | |
| # Display the word without any hyperlink or tooltip | |
| tag_element = word_with_escaped_parentheses | |
| # Include the tag element in the table row | |
| html_str += f"<tr><td style='border: none; padding: 5px; height: 20px;'>{tag_element}</td><td style='border: none; padding: 5px; height: 20px;'>{round(sim, 3)}</td><td style='border: none; padding: 5px; height: 20px;'>{count}</td></tr>" | |
| html_str += "</tbody></table></div>" | |
| return html_str | |
| def create_top_artists_table(top_artists): | |
| # Add a heading above the table | |
| html_str = "<div class=\"scrollable-content\" style='display: inline-block; margin: 20px; text-align: center;'>" | |
| html_str += "<h1>Top Artists</h1>" # Heading for the table | |
| # Start the table with increased font size and no borders between rows | |
| html_str += "<table style='font-size: 20px; border-collapse: collapse;'>" | |
| html_str += "<thead><tr><th>Artist</th><th>Similarity</th></tr></thead><tbody>" | |
| # Loop through the top artists and add a row for each without the rank and without borders between rows | |
| for artist, score in top_artists: | |
| artist_name = artist[3:] if artist.startswith("by ") else artist # Remove "by " prefix | |
| similarity_percentage = "{:.1f}%".format(score * 100) # Convert score to percentage string with one decimal | |
| html_str += f"<td style='padding: 3px 20px; border: none;'>{artist_name}</td><td style='padding: 3px 20px; border: none;'>{similarity_percentage}</td></tr>" | |
| # Close the table HTML | |
| html_str += "</tbody></table></div>" | |
| return html_str | |
| def construct_pseudo_vector(pseudo_doc_terms, idf_loaded, tag_to_row_loaded): | |
| # Initialize a vector of zeros with the length of the term_to_index mapping | |
| pseudo_vector = np.zeros(len(tag_to_row_loaded)) | |
| # Fill in the vector for terms in the pseudo document | |
| for term in pseudo_doc_terms: | |
| if term in tag_to_row_loaded: | |
| index = tag_to_row_loaded[term] | |
| pseudo_vector[index] = idf_loaded.get(term, 0) | |
| # Return the vector as a 2D array for compatibility with SVD transform | |
| return pseudo_vector.reshape(1, -1) | |
| def get_top_indices(reduced_pseudo_vector, reduced_matrix): | |
| # Compute cosine similarities | |
| similarities = cosine_similarity(reduced_pseudo_vector, reduced_matrix).flatten() | |
| # Get sorted tag indices based on similarities, in descending order | |
| sorted_indices = np.argsort(-similarities) | |
| # Return the top N indices | |
| return sorted_indices | |
| def get_tfidf_reduced_similar_tags(pseudo_doc_terms, allow_nsfw_tags): | |
| idf = tf_idf_components['idf'] | |
| term_to_column_index = tf_idf_components['tag_to_column_index'] | |
| row_to_tag = tf_idf_components['row_to_tag'] | |
| reduced_matrix = tf_idf_components['reduced_matrix'] | |
| svd = tf_idf_components['svd_model'] | |
| # Construct the TF-IDF vector | |
| pseudo_tfidf_vector = construct_pseudo_vector(pseudo_doc_terms, idf, term_to_column_index) | |
| # Reduce the dimensionality of the pseudo-document vector for the reduced matrix | |
| reduced_pseudo_vector = svd.transform(pseudo_tfidf_vector) | |
| # Compute cosine similarities in the reduced space | |
| cosine_similarities_reduced = cosine_similarity(reduced_pseudo_vector, reduced_matrix).flatten() | |
| # Sort the indices by descending cosine similarity | |
| top_indices_reduced = np.argsort(cosine_similarities_reduced) | |
| # Map indices to tags with their similarities | |
| tag_similarity_dict = {row_to_tag[i]: cosine_similarities_reduced[i] for i in top_indices_reduced if i in row_to_tag} | |
| if not allow_nsfw_tags: | |
| tag_similarity_dict = {tag: sim for tag, sim in tag_similarity_dict.items() if tag not in nsfw_tags} | |
| tag_similarity_dict = {"by " + tag if is_artist(tag) else tag: sim for tag, sim in tag_similarity_dict.items()} | |
| # Sort and transform tag names | |
| sorted_tag_similarity_dict = OrderedDict(sorted(tag_similarity_dict.items(), key=lambda x: x[1], reverse=True)) | |
| transformed_sorted_tag_similarity_dict = OrderedDict( | |
| (key.replace('_', ' ').replace('(', '\\(').replace(')', '\\)'), value) | |
| for key, value in sorted_tag_similarity_dict.items() | |
| ) | |
| return transformed_sorted_tag_similarity_dict | |
| def create_html_placeholder(title="", content="", placeholder_height=400, placeholder_width="100%"): | |
| # Include a title in the same style as the top artists table heading | |
| html_placeholder = f"<div class=\"scrollable-content\" style='text-align: center;'><h1>{title}</h1></div>" | |
| # Conditionally add content if present | |
| if content: | |
| html_placeholder += f"<div style='text-align: center; margin-bottom: 20px;'><p>{content}</p></div>" | |
| # Add the placeholder div with specified height and width | |
| html_placeholder += f"<div style='height: {placeholder_height}px; width: {placeholder_width}; margin: 20px auto; background: transparent;'></div>" | |
| return html_placeholder | |
| def find_similar_tags(test_tags, tag_to_context_similarity, context_similarity_weight, allow_nsfw_tags): | |
| #Initialize stuff | |
| if not hasattr(find_similar_tags, "fasttext_small_model"): | |
| find_similar_tags.fasttext_small_model = compress_fasttext.models.CompressedFastTextKeyedVectors.load('e621FastTextModel010Replacement_small.bin') | |
| tag_aliases_file = 'fluffyrock_3m.csv' | |
| if not hasattr(find_similar_tags, "tag2aliases"): | |
| find_similar_tags.tag2aliases = build_aliases_dict(tag_aliases_file) | |
| if not hasattr(find_similar_tags, "alias2tags"): | |
| find_similar_tags.alias2tags = build_aliases_dict(tag_aliases_file, reverse=True) | |
| if not hasattr(find_similar_tags, "tag2count"): | |
| find_similar_tags.tag2count = build_tag_count_dict(tag_aliases_file) | |
| if not hasattr(find_similar_tags, "tag2idwiki"): | |
| find_similar_tags.tag2idwiki = build_tag_id_wiki_dict() | |
| modified_tags = [tag_info['modified_tag'] for tag_info in test_tags] | |
| transformed_tags = [tag.replace(' ', '_') for tag in modified_tags] | |
| # Find similar tags and prepare data for tables | |
| html_content = "<div class=\"scrollable-content\" style='display: inline-block; margin: 20px; text-align: center;'>" | |
| html_content += "<h1>Unknown Tags</h1>" # Heading for the table | |
| tags_added = False | |
| bad_entities = [] | |
| known_entities_in_prompt = [] | |
| encountered_modified_tags = set() | |
| for tag_info in test_tags: | |
| original_tag = tag_info['original_tag'] | |
| modified_tag = tag_info['modified_tag'] | |
| start_pos = tag_info['start_pos'] | |
| end_pos = tag_info['end_pos'] | |
| node_type = tag_info['node_type'] | |
| if modified_tag in special_tags: | |
| bad_entities.append({"entity":"Special", "start":start_pos, "end":end_pos}) | |
| continue | |
| if modified_tag in encountered_modified_tags: | |
| bad_entities.append({"entity":"Duplicate", "start":start_pos, "end":end_pos}) | |
| continue | |
| encountered_modified_tags.add(modified_tag) | |
| modified_tag_for_search = modified_tag.replace(' ','_') | |
| similar_words = find_similar_tags.fasttext_small_model.most_similar(modified_tag_for_search, topn = 100) | |
| result, seen = [], set(transformed_tags) | |
| if modified_tag_for_search in find_similar_tags.tag2aliases: | |
| if modified_tag in find_similar_tags.tag2aliases and "_" in modified_tag: #Implicitly tell the user that they should get rid of the underscore | |
| result.append(modified_tag_for_search.replace('_',' '), 1) | |
| seen.add(modified_tag) | |
| else: #The user correctly did not put underscores in their tag | |
| count = find_similar_tags.tag2count.get(modified_tag_for_search, 0) # Get the count if available, otherwise default to 0 | |
| tag_id, wiki_entry = find_similar_tags.tag2idwiki.get(modified_tag_for_search, (None, '')) | |
| # Check if tag_id and wiki_entry are valid | |
| wiki_url = "" | |
| if tag_id is not None and wiki_entry: | |
| # Construct the URL for the tag's wiki page | |
| wiki_url = f"https://e621.net/wiki_pages/{tag_id}" | |
| known_entities_in_prompt.append({"entity":"Known Tag", "start":start_pos, "end":end_pos, "count":count, "wiki_url":wiki_url, "wiki_entry":wiki_entry}) | |
| continue | |
| else: | |
| for item in similar_words: | |
| similar_word, similarity = item | |
| if similar_word not in seen: | |
| if similar_word in find_similar_tags.tag2aliases: | |
| result.append((similar_word.replace('_', ' '), round(similarity, 3))) | |
| seen.add(similar_word) | |
| else: | |
| for similar_tag in find_similar_tags.alias2tags.get(similar_word, []): | |
| if similar_tag not in seen: | |
| result.append((similar_tag.replace('_', ' '), round(similarity, 3))) | |
| seen.add(similar_tag) | |
| #Remove NSFW tags if appropriate. | |
| if not allow_nsfw_tags: | |
| result = [(word, score) for word, score in result if word.replace(' ','_') not in nsfw_tags] | |
| #Adjust score based on context | |
| for i in range(len(result)): | |
| word, score = result[i] # Unpack the tuple | |
| context_score = tag_to_context_similarity.get(word,0) | |
| result[i] = (word, .5 * ((context_similarity_weight * context_score) + ((1 - context_similarity_weight) * score))) | |
| result = sorted(result, key=lambda x: x[1], reverse=True)[:10] | |
| html_content += create_html_tables_for_tags(modified_tag, "Corrected Tag", result, find_similar_tags.tag2count, find_similar_tags.tag2idwiki) | |
| bad_entities.append({"entity":"Unknown Tag", "start":start_pos, "end":end_pos}) | |
| tags_added=True | |
| # If no tags were processed, add a message | |
| if not tags_added: | |
| html_content = create_html_placeholder(title="Unknown Tags", content="No Unknown Tags Found") | |
| return html_content, bad_entities, known_entities_in_prompt # Return list of lists for Dataframe | |
| def build_tag_offsets_dicts(new_image_tags_with_positions): | |
| # Structure the data for HighlightedText | |
| tag_data = [] | |
| for tag_text, start_pos, nodetype in new_image_tags_with_positions: | |
| # Modify the tag | |
| modified_tag = tag_text.replace('_', ' ').replace('\\(', '(').replace('\\)', ')').strip() | |
| artist_matrix_tag = tag_text.replace('_', ' ').replace('\\(', '\(').replace('\\)', '\)').strip() | |
| tf_idf_matrix_tag = re.sub(r'\\([()])', r'\1', re.sub(r' ', '_', tag_text.strip().removeprefix('by ').removeprefix('by_'))) | |
| # Calculate the end position based on the original tag length | |
| end_pos = start_pos + len(tag_text) | |
| # Append the structured data for each tag | |
| tag_data.append({ | |
| "original_tag": tag_text, | |
| "start_pos": start_pos, | |
| "end_pos": end_pos, | |
| "modified_tag": modified_tag, | |
| "artist_matrix_tag": artist_matrix_tag, | |
| "tf_idf_matrix_tag": tf_idf_matrix_tag, | |
| "node_type": nodetype | |
| }) | |
| return tag_data | |
| def augment_bad_entities_with_regex(text): | |
| bad_entities = [] | |
| #comma at end | |
| match = re.search(r',(?=\s*$)', text) | |
| if match: | |
| index = match.start() | |
| bad_entities.append({"entity":"Remove Final Comma", "start":index, "end":index+1}) | |
| match = re.search(r'\([^()]*(,)\s*\)\s*$', text) | |
| if match: | |
| index = match.start(1) | |
| bad_entities.append({"entity":"Remove Final Comma", "start":index, "end":index+1}) | |
| match = re.search(r'\([^()]*(,)\s*:\s*\d+(\.\d+)?\s*\)\s*$', text) | |
| if match: | |
| index = match.start(1) | |
| bad_entities.append({"entity":"Remove Final Comma", "start":index, "end":index+1}) | |
| # Comma after parentheses, multiple occurrences | |
| for match in re.finditer(r'(?<!\\)\)\s*(,)\s*[^\s]', text): | |
| index = match.start(1) | |
| bad_entities.append({"entity": "Move Comma Inside Parentheses", "start": index, "end": index + 1}) | |
| # Double Comma detection | |
| for match in re.finditer(r',\s*,', text): | |
| index = match.start() | |
| bad_entities.append({"entity": "Double Comma", "start": index, "end": index + match.end() - match.start()}) | |
| return bad_entities | |
| def escape_html(text): | |
| return text.replace("&", "&").replace("<", "<").replace(">", ">").replace('"', """).replace("'", "'") | |
| def format_annotated_html(bad_entities, known_entities, text): | |
| tooltip_map = { | |
| "Unknown Tag": "This may not be a valid e621 tag. Consider removing or replacing it with tag(s) from the \"Unknown Tags\" section.", | |
| "Duplicate": "This tag has appeared multiple times in your prompt. Consider removing the copies.", | |
| "Remove Final Comma": "There should be no comma at the end of your prompt. Consider removing it.", | |
| "Move Comma Inside Parentheses": "In most e621-based models, the comma following a tag functions as an "attention anchor", carrying most of the tag's information. It should therefore be assigned the same weight as the rest of the tag. So instead of "(lineless:1.1),", consider "(lineless,:1.1)" or "(lineless,)"", | |
| "Double Comma": "One comma between tags is considered ample." | |
| } | |
| color_map = { | |
| "Unknown Tag": ("white", "red"), # White text on red background | |
| "Duplicate": ("black", "yellow"), # Black text on yellow background | |
| "Remove Final Comma": ("white", "blue"), # White text on blue background | |
| "Move Comma Inside Parentheses": ("white", "green"), # White text on green background | |
| "Double Comma": ("white","orange") | |
| } | |
| # Combine and sort entities | |
| combined_entities = bad_entities + known_entities | |
| combined_entities = sorted(combined_entities, key=lambda x: x['start'],reverse=True) | |
| # Generate HTML for the main text | |
| html_text = text | |
| for entity in combined_entities: | |
| start = entity['start'] | |
| end = entity['end'] | |
| label = entity['entity'] | |
| if label == "Known Tag": | |
| wiki_url = entity.get('wiki_url', '') | |
| count = entity['count'] | |
| wiki_entry = entity.get('wiki_entry', '') | |
| sanitized_wiki_entry = escape_html(wiki_entry) if wiki_entry else 'Unavailable' | |
| if wiki_url: # Check if wiki_url is not empty | |
| html_part = f'<a href="{wiki_url}" target="_blank" title="Count: {count}\tWiki: {sanitized_wiki_entry}" style="text-decoration: none; cursor: pointer; font-style: italic;">{text[start:end]}</a>' | |
| else: | |
| html_part = f'<span title="Count: {count}\tWiki: {sanitized_wiki_entry}" style="text-decoration: none; cursor: help; font-style: italic;">{text[start:end]}</span>' | |
| else: | |
| color = color_map.get(label, ("black", "white")) | |
| html_part = f'<span style="background-color: {color[1]}; color: {color[0]};">{text[start:end]}</span>' | |
| html_text = html_text[:start] + html_part + html_text[end:] | |
| # Generate HTML for the color key | |
| color_key_html = "<div style='text-align: right; margin-top: 20px;'>Key:" | |
| used_labels = set(entity['entity'] for entity in bad_entities) | |
| for label, colors in color_map.items(): | |
| if label in used_labels: | |
| tooltip = tooltip_map.get(label, "") | |
| # Adding margin-right for spacing between items | |
| color_key_html += f" <span style='background-color: {colors[1]}; color: {colors[0]}; margin-right: 10px;' title='{tooltip}'>{label}</span>" | |
| color_key_html += "</div>" | |
| return f'<div style="padding: 10px; font-size: 16px;">{html_text}</div>{color_key_html}' | |
| def find_similar_artists(original_tags_string, top_n, context_similarity_weight, allow_nsfw_tags): | |
| try: | |
| new_tags_string = original_tags_string.lower() | |
| new_tags_string, removed_tags = remove_special_tags(new_tags_string) | |
| # Parse the prompt | |
| parsed = parser.parse(new_tags_string) | |
| # Extract tags from the parsed tree | |
| new_image_tags = extract_tags(parsed) | |
| tag_data = build_tag_offsets_dicts(new_image_tags) | |
| #Suggested tags stuff | |
| suggested_tags_html_content = "<div class=\"scrollable-content\" style='display: inline-block; margin: 20px; text-align: center;'>" | |
| suggested_tags_html_content += "<h1>Suggested Tags</h1>" # Heading for the table | |
| suggested_tags = get_tfidf_reduced_similar_tags([item["tf_idf_matrix_tag"] for item in tag_data] + removed_tags, allow_nsfw_tags) | |
| unseen_tags_data, bad_entities, known_entities = find_similar_tags(tag_data, suggested_tags, context_similarity_weight, allow_nsfw_tags) | |
| #Bad tags stuff | |
| bad_entities.extend(augment_bad_entities_with_regex(new_tags_string)) | |
| bad_entities.sort(key=lambda x: x['start']) | |
| #bad_tags_illustrated_string = {"text":new_tags_string, "entities":bad_entities} | |
| bad_tags_illustrated_html = format_annotated_html(bad_entities, known_entities, new_tags_string) | |
| # Create a set of tags that should be filtered out | |
| filter_tags = {entry["original_tag"].strip() for entry in tag_data} | |
| # Use this set to filter suggested_tags | |
| suggested_tags_filtered = OrderedDict((k, v) for k, v in suggested_tags.items() if k not in filter_tags) | |
| # Splitting the dictionary into two based on the condition | |
| suggested_artist_tags_filtered = OrderedDict((k, v) for k, v in suggested_tags_filtered.items() if k.startswith("by ")) | |
| suggested_non_artist_tags_filtered = OrderedDict((k, v) for k, v in suggested_tags_filtered.items() if not k.startswith("by ") and k not in special_tags) | |
| topnsuggestions = list(islice(suggested_non_artist_tags_filtered.items(), 100)) | |
| suggested_tags_html_content += create_html_tables_for_tags("-", "Suggested Tag", topnsuggestions, find_similar_tags.tag2count, find_similar_tags.tag2idwiki) | |
| #Artist stuff | |
| excluded_artists = ["by conditional dnp", "by unknown artist"] | |
| top_artists = [(key, value) for key, value in suggested_artist_tags_filtered.items() if key.lower() not in excluded_artists][:top_n] | |
| top_artists_str = create_top_artists_table(top_artists) | |
| dynamic_prompts_formatted_artists = "{" + "|".join([artist for artist, _ in top_artists]) + "}" | |
| image_galleries = [] | |
| for root, dirs, files in os.walk(sample_images_directory_path): | |
| for name in dirs: | |
| baseline, artists = generate_artist_image_tuples([name[3:] for name, _ in top_artists], os.path.join(root, name)) | |
| image_galleries.append(baseline) # Add baseline as its own gallery item | |
| image_galleries.append(artists) # Extend the list with artist tuples | |
| return (unseen_tags_data, bad_tags_illustrated_html, suggested_tags_html_content, top_artists_str, dynamic_prompts_formatted_artists, *image_galleries) | |
| except ParseError as e: | |
| return [], "Parse Error: Check for mismatched parentheses or something", "", "", None, None | |
| with gr.Blocks(css=css) as app: | |
| with gr.Group(): | |
| with gr.Row(): | |
| with gr.Column(scale=3): | |
| image_tags = gr.Textbox(label="Enter Prompt", placeholder="e.g. fox, outside, detailed background, ...") | |
| #bad_tags_illustrated_string = gr.HighlightedText(show_legend=True, color_map={"Unknown Tag":"red","Duplicate":"yellow","Remove Final Comma":"purple","Move Comma Inside Parentheses":"green"}, label="Annotated Prompt") | |
| bad_tags_illustrated_string = gr.HTML() | |
| with gr.Column(scale=1): | |
| #image_path = os.path.join("https://huggingface.co/spaces/FoodDesert/Prompt_Squirrel/resolve/main", "transparentsquirrel.png") | |
| #gr.Image(label=" ", value=image_path, height=155, width=140) | |
| gr.HTML('<div style="text-align: center;"><img src="https://huggingface.co/spaces/FoodDesert/Prompt_Squirrel/resolve/main/mascotimages/transparentsquirrel.png" alt="Cute Mascot" style="height: 220px; width: auto; background: transparent;"></div><br>') | |
| #gr.HTML("<br>" * 2) # Adjust the number of line breaks ("<br>") as needed to push the button down | |
| #image_path = os.path.join('mascotimages', "transparentsquirrel.png") | |
| #random_image_path = os.path.join('mascotimages', random.choice([f for f in os.listdir('mascotimages') if os.path.isfile(os.path.join('mascotimages', f))])) | |
| #with Image.open(random_image_path) as img: | |
| # gr.Image(value=img,show_label=False, show_download_button=False, show_share_button=False, height=200) | |
| #gr.Image(value="https://huggingface.co/spaces/FoodDesert/Prompt_Squirrel/resolve/main/mascotimages/transparentsquirrel.png",show_label=False, show_download_button=False, show_share_button=False, height=200) | |
| #I posted the image to discord, and that's where this link came from. This is a very ugly way to do this, but I could not, no matter what I tried, get it to display an image from within the space itself. The galleries work fine for some reason, but not this. | |
| #gr.Image(value="https://res.cloudinary.com/dnse84ol6/image/upload/v1713538125/transparentsquirrel_zhou7f.png",show_label=False, show_download_button=False, show_share_button=False, height=200) | |
| submit_button = gr.Button(variant="primary") | |
| with gr.Row(): | |
| with gr.Column(scale=3): | |
| with gr.Group(): | |
| with gr.Row(): | |
| context_similarity_weight = gr.Slider(minimum=0, maximum=1, value=0.5, step=0.1, label="Context Similarity Weight") | |
| allow_nsfw = gr.Checkbox(label="Allow NSFW Tags", value=False) | |
| with gr.Row(): | |
| with gr.Column(scale=2): | |
| unseen_tags = gr.HTML(label="Unknown Tags", value=create_html_placeholder(title="Unknown Tags")) | |
| with gr.Column(scale=1): | |
| suggested_tags = gr.HTML(label="Suggested Tags", value=create_html_placeholder(title="Suggested Tags")) | |
| with gr.Column(scale=1): | |
| with gr.Group(): | |
| num_artists = gr.Slider(minimum=1, maximum=100, value=10, step=1, label="Number of artists") | |
| top_artists = gr.HTML(label="Top Artists", value=create_html_placeholder(title="Top Artists")) | |
| dynamic_prompts = gr.Textbox(label="Dynamic Prompts Format", info="For if you're using the Automatic1111 webui (https://github.com/AUTOMATIC1111/stable-diffusion-webui) with the Dynamic Prompts extension activated (https://github.com/adieyal/sd-dynamic-prompts) and want to try them all individually.") | |
| galleries = [] | |
| for root, dirs, files in os.walk(sample_images_directory_path): | |
| for name in dirs: | |
| with gr.Row(): | |
| baseline = gr.Gallery(allow_preview=False, rows=1, columns=1, height=420, scale=3) | |
| styles = gr.Gallery(preview=False, rows=2, columns=5, height=420, scale=8) | |
| galleries.extend([baseline, styles]) | |
| submit_button.click( | |
| find_similar_artists, | |
| inputs=[image_tags, num_artists, context_similarity_weight, allow_nsfw], | |
| outputs=[unseen_tags, bad_tags_illustrated_string, suggested_tags, top_artists, dynamic_prompts] + galleries | |
| ) | |
| gr.Markdown(faq_content) | |
| app.launch() | |