config.json and dataset
Thank you for this model. This ControlNet works well with sdxs-512-dreamshaper.
I have questions about config.json and dataset for this ControlNet.
In config.json,
down_block_types is not all CrossAttnDownBlock2D.
"down_block_types": [
"DownBlock2D",
"CrossAttnDownBlock2D",
"CrossAttnDownBlock2D"
],
Is there any reason to use DownBlock2D, and this has any effects on functionality of this ControlNet?
Another question is what kind of dataset is used for training this ControlNet.
Is the dataset is available?
Because self attention is O(n^2) overhead, it consumes a lot of memory and computation in the highest resolution stage.
We tried removing only self attention in SDXS-512-v0.9, but in subsequent training, we found that removing cross attention also worked well, so we removed all transformers at the highest resolution.
Thank you for response. I understand that block type difference is for performance reason.
In huggingface space demo of sketch ControlNet. Sketches using thick line and thin line seems to work for generating images. Is this ControlNet trained on images with thick lines images, or images with thin lines like canny image?
We used morphological operations (erosion and expansion) when generating the sketch, with different parameters corresponding to different thicknesses of the line.