Repository: stable-diffusion-webui
Issue: [Feature Request]: Model Interleaving
Labels: enhancement

### Is there an existing issue for this?

- [X] I have searched the existing issues and checked the recent builds/commits

### What would your feature do ?

This image generation works by starting with noise or a reference image and running it through the model multiple times for some given number of steps.  What if we could have it alternate between two models?  Or maybe have it run 10 steps on one and 10 on another?  Or 15 and 5?

Basically, the idea is I want to be able to load multiple models, and then define a path for the initial image/noise to take through those models, rather than just some number of steps through only one model.

This could really useful in cases where one model produces desirable stylistic effects but cannot generate the desired content correctly.  So instead, you run the first ~15 steps on another model that generates the desired content, and then you run 5 finishing steps on the one that produces the desired stylistic effects.

Another place this could be useful is cleaning up artifacts that a particular model tends to create, like extra limbs, by alternating it with another model that does a much better job of eliminating those artifacts.  In this case you might want to just alternate every other step.

I can even imagine some use cases for doing this with up to three models, and in the long run, it might have even greater utility when used with highly specialized models, rotating between models specialized for different elements or effects, to get a final image where each component is generated by a model designed to do that one thing extremely well.

### Proposed workflow

I can imagine several ways to build this into the UI, but here's the one that I think offers the most flexibility:

1. In the txt2img or img2img tab,
2. Go to the "Sampling Steps" box,
3. The box by default contains what it currently contains, but with a '+' button at the bottom right and the model dropdown there instead at the top left of the page,
4. Set the model (it loads), set the steps and sampling method, then click the '+' button,
5. A second instance of the "Sampling Steps" box appears below the first, maybe in a scroll panel with the first,
6. Set the model for the second "Sampling Steps" box (it loads) and the other settings (steps and method),
7. Continue clicking the '+' button to add new boxes, but if an already loaded model is selected use the current instance instead of loading a new one,
8. Once the full pipeline is defined, make sure the prompts and the other settings are specified and click "Generate"

Two additional suggestions:

Keep the '+' button in all "Sampling Steps" boxes, and add the new box directly below the one clicked, allowing new steps to be inserted between existing ones.

Allow boxes to be reordered by clicking and dragging the "Sampling Steps" text (or whatever label text the box itself is given, as it may be better to give the boxes a new label reflecting that they are now defining the model, sampling steps, and sampling method).  (If clicking and dragging is too much of a pain, maybe instead have up and down buttons on the left or right side of each box, and clicking these moves the box's position in the order by one in specified direction.)


### Additional information

I actually have a very direct use case for this currently (though I realize this won't magically be available instantly merely because I have an immediate need).  SD2 isn't great for doing certain kinds of fantasy style art, and MidJourney is much better at that style.  MidJourney, however, has some biases for the prompts I am using that very consistently favor particular hair, eye, and background colors, regardless of what prompts I give it to the contrary.  SD2 produces much more variation in background colors and honors hair and eye color prompts far better, but the art styles it produces are less appealing, and it struggles a lot with certain aspects of the content.  What I would _like_ to do is run 5 to 10 steps in MidJourney to solidify some of the content that SD2 can't do well, run 10 to 20 steps in SD2 to nail down the things MidJourney doesn't handle well, and then run a final 5 to 10 steps in MidJourney to get the desired art style.

I also have a potential future use case starting with the Fantasy Card Diffusion model to get character types other models can't do well, do a few SD2 steps to remove some artifacts that Fantasy Card Diffusion has issues with, and then finish it in MidJourney to smooth out the print dithering artifacts produced by Fantasy Card Diffusion.