Plans on adding a normal model?
Have been absolutely blown away by the results when using the depth model, as using it with a 2.1 model has resulted in some of the best SD images I've ever generated.
That being said, one advantage that 1.5 models still have is the ability to use Normal Maps, which preserve a ton more detail than depth maps tend to. Since both normal and depth maps can be very easily generated in a program like Blender and used in conjunction, this is definitely a use case I am personally interested in. Are there currently any plans to create a Normal model?
I'm working on a Normal Map model. For now, results are not good enough.
How about style model? Any plans/experiments?
Thanks for your work!
the issue for the style model is to find/generate the dataset. If you have some links/ref/ideas about it, I can take a look!
I'm working on a Normal Map model. For now, results are not good enough.
Glad to hear it, and great job thus far! I noticed you added a "color" model recently, but I'm not entirely sure how it works based on the example you gave, and haven't seen any other "color" ControlNet models before, nor documentation on how it works. Could you possibly give a description of it, or link to the documentation you based it off? I normally prefer to use ControlNet without a preprocessor if I can, but based on the example I don't know if that's possible, nor the workflow that would go into using it. Thanks!
Very interesting, though I should note that I am struggling to get good results like the Discord attachment you provided shows. In txt2img the results are often far too noisy at any weight lower than 2, and don't have much of a change in color at anything 1 or lower. A weight of 2 can produce an image closest to what I am going for without too much visual noise, but the issue is that the image then starts to look like a rough, washed out oil painting, which is not at all like the style I am going for. Whether a weight of 0 or 2 produces better results is up to personal preference, and I am as of yet undecided, as the details appear better with a weight of 2 than a weight of 0, but the art style I am going for is not being adhered to.
Conversely, any weight between 0 and 2 on img2img just doesn't seem to have any effect on the image at all unless used with a denoising strength above 0.75, and even then I just can't seem to get results that are good enough to use.
It should be noted that I am not using the color preprocessor, and am instead using a custom pixelated color map in 16x16 of the color output I would like to see relative to an unweighted generation, since the preprocessor was giving what was virtually the exact same output I was going for with an annotator resolution of 1024 anyways. This might just be an issue the SD model I am using it on, or with the variety of generation settings in both SD and ControlNet I have, but I thought it might be good feedback for you to have.
Anywho, I won't derail from the original topic any further. Great job thus far, and thanks a ton for your work! I greatly look forward to seeing more from you :)