Repository: stable-diffusion-webui
Issue: [FEATURE] Automatic prompt correction, tracking and sharing
Labels: enhancement

There are countless threads on facebook and reddit of people sharing prompts they can't get to work.
Horses are twice the length of regular horses or whatever. Someone thinks they have a fix, but it doesn't apply in that user's specific context. Someone chimes in to say this is just a crude tool that will always need constant human oversight.
This is laborious, inefficient, imprecise and relies on the right people seeing it that just so happened to have the same problem and find a solution.
I propose we add optional UI elements to intelligently track changes between prompts (for our own sake also), rate changes that were positive or negative, add a checkbox to automatically share them online (not sure which API to use, but it needs to be free) and add inline suggestions when previously shared fixes occur in your prompt.

When you replace it with a prompt that's entirely different to start an entirely new image, there are regex similarity tests that should catch most occurrences automatically, but then there should be a button to manually tell it that you've started a new scene incase that has a false negative. (this has the added bonus of making folder management easier since we can lump all prompt attempts into 1 scene folder and manage them intelligently via history, so it's worth the user's time if they use it)
Not sure the best way to implement the UI. Perhaps it could all be overlayed inline in the prompt box.
e.g. If you changed the order of words in the same prompt, it would show arrows to where they've moved just above it. If you replace a word it highlights it a certain colour etc. A generic diff comparison and some regex is already so much better than a vanilla prompt.
We'll need a way to also rate other users' fixes. Or maybe something like a git structure would work. This is all adding extra clutter to the UI, so I suggest adding a button below the prompts that drops these new features down to a whole new row and hiding it by default to make it entirely optional.

While we're at it, we should also include all other parameters when it sends it to the database so we can compare relative differences.
e.g. If a prompt fix only applies with a certain cfg scale or sampler for a certain model, we don't have to suggest it as a fix.
This has the added bonus that we can use all this data to improve the dataset.
Either for the official stability training (let's just ignore how awkward collaboration with this repo and emad would be for now) or for some sort of model we can all train ourselves in a cluster, like say a giant "fix everything" hypernetwork or set of inter-related embeddings or alternate dreambooth that uses multiple class regularizers.
For a single person this would be an insurmountable effort, but if we share the effort we could all end up with a model that has all the sensible fixes like distorted limbs and extra fingers included by default.
It might turn out that isn't even necessary, as say if "distorted limbs" as a negative prompt weighs too much in some context, "mutilated" might be a fix, and it could turn out we can do this for everything if we actually have enough user input.

Going forward we can maybe use something like [DAAM](https://arxiv.org/pdf/2210.04885.pdf) [(github)](https://github.com/castorini/daam) to track relative changes visually, and maybe feed the database of corrections into a large language model to extrapolate the pattern we can't see as humans and replace this whole system with something like an autocorrect for plain english to prompt engineering language.
Maybe these are unrealistic goals, but I think it's important we at least start collecting the data first to even see how attainable it is.
![horses1](https://user-images.githubusercontent.com/6998259/196050739-26e80e5a-87da-4dcc-875c-858072524d4a.jpg)
![horses2](https://user-images.githubusercontent.com/6998259/196611010-0d6e5bc8-583a-45bc-a863-448321991d6b.jpg)