Spaces:
Building
on
A10G
Building
on
A10G
| # ComfyUI-layerdiffuse | |
| ComfyUI implementation of https://github.com/layerdiffusion/LayerDiffuse. | |
| ## Installation | |
| Download the repository and unpack into the custom_nodes folder in the ComfyUI installation directory. | |
| Or clone via GIT, starting from ComfyUI installation directory: | |
| ```bash | |
| cd custom_nodes | |
| git clone git@github.com:huchenlei/ComfyUI-layerdiffuse.git | |
| ``` | |
| Run `pip install -r requirements.txt` to install python dependencies. You might experience version conflict on diffusers if you have other extensions | |
| that depends on other versions of diffusers. In this case, it is recommended to setup separate Python venvs. | |
| ## Workflows | |
| ### [Generate foreground](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_fg_example_rgba.json) | |
|  | |
| ### [Generate foreground (RGB + alpha)](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_fg_example.json) | |
| If you want more control of getting RGB image and alpha channel mask separately, you can use this workflow. | |
|  | |
| ### [Blending (FG/BG)](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_cond_example.json) | |
| Blending given FG | |
|  | |
| Blending given BG | |
|  | |
| ### [Extract FG from Blended + BG](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_diff_fg.json) | |
|  | |
| ### [Extract BG from Blended + FG](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_diff_bg.json) | |
| [Forge impl's sanity check](https://github.com/layerdiffuse/sd-forge-layerdiffuse#sanity-check) sets `Stop at` to 0.5 to get better quality BG. | |
| This workflow might be inferior comparing to other object removal workflows. | |
|  | |
| ### [Extract BG from Blended + FG (Stop at 0.5)](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_diff_bg_stop_at.json) | |
| In [SD Forge impl](https://github.com/layerdiffuse/sd-forge-layerdiffuse), there is a `stop at` param that determines when | |
| layer diffuse should stop in the denosing process. In the background, what this param does is unapply the LoRA and c_concat cond after a certain step | |
| threshold. This is hard/risky to implement directly in ComfyUI as it requires manually load a model that has every changes except the layer diffusion | |
| change applied. A workaround in ComfyUI is to have another img2img pass on the layer diffuse result to simulate the effect of `stop at` param. | |
|  | |
| ### [Generate FG from BG combined](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_cond_fg_all.json) | |
| Combines previous workflows to generate blended and FG given BG. We found that there are some color variations in the extracted FG. Need to confirm | |
| with layer diffusion authors on whether this is expected. | |
|  | |
| ### [2024-3-9] [Generate FG + Blended given BG](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_cond_joint_bg.json) | |
| Need batch size = 2N. Currently only for SD15. | |
|  | |
| ### [2024-3-9] [Generate BG + Blended given FG](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_cond_joint_fg.json) | |
| Need batch size = 2N. Currently only for SD15. | |
|  | |
| ### [2024-3-9] [Generate BG + FG + Blended together](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_joint.json) | |
| Need batch size = 3N. Currently only for SD15. | |
|  | |
| ## Note | |
| - Currently only SDXL/SD15 are supported. See https://github.com/layerdiffuse/sd-forge-layerdiffuse#model-notes for more details. | |
| - To decode RGBA result, the generation dimension must be multiple of 64. Otherwise, you will get decode error:  | |