MichalMlodawski/negative-prompt-generator-large

✨ Flan-T5 Negative Prompt Generator

🌟 Model Overview

This model is a fine-tuned version of Google's Flan-T5 specifically designed for generating negative prompts for image generation. It takes a positive prompt as input and produces a corresponding negative prompt to guide image generation models away from unwanted elements.

🔍 Model Details

• Base Model: Flan-T5 (Google)
• Task: Negative Prompt Generation
• Language: English (primarily, but can handle multilingual inputs)
• License: cc-by-nc-nd-4.0

🎯 Use Cases

This model is particularly useful for:

1. 🖼️ Enhancing AI image generation by providing tailored negative prompts
2. 🛠️ Assisting artists and designers in refining their image generation process
3. 🔒 Content moderation and safety in AI-generated visual content
4. 🧪 Experimenting with prompt engineering techniques

💡 How It Works

The model takes a positive image generation prompt as input and generates a corresponding negative prompt. This negative prompt can be used alongside the positive prompt in image generation models to avoid unwanted elements or styles in the generated image.

🚀 Getting Started

To use the Flan-T5 Negative Prompt Generator, follow these steps:

Installation

pip install transformers torch

Usage

from transformers import T5Tokenizer, T5ForConditionalGeneration
import torch

# Load the model and tokenizer
model_path = "MichalMlodawski/negative-prompt-generator-large" 
tokenizer = T5Tokenizer.from_pretrained(model_path)
model = T5ForConditionalGeneration.from_pretrained(model_path)

def generate_negative_prompt(prompt, min_length=10, length_penalty=2.0, num_beams=4):
    input_ids = tokenizer(f"Generate negative prompt using this: {prompt}", return_tensors="pt").input_ids
    
    # Calculate dynamic max_length
    input_length = input_ids.shape[1]
    max_length = min(input_length * 2, 512)  # Adjust this formula as needed
    
    outputs = model.generate(
        input_ids,
        max_length=max_length,
        min_length=min_length,
        length_penalty=length_penalty,
        num_beams=num_beams,
        early_stopping=True
    )
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

# Example usage
positive_prompt = "A beautiful sunset over a calm ocean"
negative_prompt = generate_negative_prompt(positive_prompt)
print(f"Positive Prompt: {positive_prompt}")
print(f"Generated Negative Prompt: {negative_prompt}")

🎨 Examples

Here are a few examples of how the model generates negative prompts:

1. Positive Prompt: "Generate negative prompt using this: A serene forest landscape with a clear stream"

   Negative Prompt: "low res, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality"

2. Positive Prompt: "Generate negative prompt using this: side view, Surreal photograph of a figure of one male kneeling in a foggy forest."

   Negative Prompt: "low res, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry"

3. Positive Prompt: "Generate negative prompt using this: A photorealistic 4K image of a magical red crystal with sharp, jagged edges, dripping dark, crimson blood. The scene is dramatic with intense, focused lighting that highlights the intricate facets of the crystal. The blood appears thick, slowly flowing down from the crystal, creating a stark contrast against the sharp, glittering surface. The overall atmosphere is dark and intense, with a sense of power and mystery surrounding the crystal. Masterpiece quality, highly detailed."

   Negative Prompt: "low res, text, error, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, artist name, censored, shirt"

🛠️ Fine-tuning and Customization

If you want to fine-tune this model on your own dataset or customize it for specific use cases, you can use the Hugging Face Transformers library. Here's a basic example of how to start the fine-tuning process:

from transformers import T5ForConditionalGeneration, T5Tokenizer, Trainer, TrainingArguments

# Load the model and tokenizer
model = T5ForConditionalGeneration.from_pretrained("google/flan-t5-large")
tokenizer = T5Tokenizer.from_pretrained("google/flan-t5-large")

# Prepare your dataset
# ... (code to load and preprocess your dataset)

# Set up training arguments
training_args = TrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=8,
    per_device_eval_batch_size=8,
    warmup_steps=500,
    weight_decay=0.01,
    logging_dir="./logs",
)

# Create Trainer instance
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    tokenizer=tokenizer,
)

# Start training
trainer.train()

📊 Limitations

While this model is powerful for generating negative prompts, it's important to note its limitations:

• The model may sometimes generate inconsistent or contradictory negative prompts.
• Performance may vary depending on the complexity and specificity of the input prompt.
• The model's output is based on its training data and may reflect biases present in that data.

📚 Citation

If you use this model in your research or project, please cite it as follows:

@misc{flant5_negative_prompt_generator,
  author       = { Michał Młodawski},
  title        = {Flan-T5 Negative Prompt Generator},
  year         = {2024},
  howpublished = {\url{https://huggingface.co/MichalMlodawski/negative-prompt-generator-large}},
  note         = {Based on Google's Flan-T5 model},
  keywords     = {natural language processing, prompt engineering, image generation}
}