MWAI-Interior-Design / explanation.py
BertChristiaens's picture
add docs
5981c2d
import streamlit as st
def make_inpainting_explanation():
with st.expander("Explanation inpainting", expanded=False):
st.write("In the inpainting mode, you can draw regions on the input image that you want to regenerate. "
"This can be useful to remove unwanted objects from the image or to improve the consistency of the image."
)
st.image("content/inpainting_sidebar.png", caption="Image before inpainting, note the ornaments on the wall", width=500)
st.write("You can find drawing options in the sidebar. There are two modes: freedraw and polygon. Freedraw allows the user to draw with a pencil of a certain width. "
"Polygon allows the user to draw a polygon by clicking on the image to add a point. The polygon is closed by right clicking.")
st.write("### Example inpainting")
st.write("In the example below, the ornaments on the wall are removed. The inpainting is done by drawing a mask on the image.")
st.image("content/inpainting_before.jpg", caption="Image before inpainting, note the ornaments on the wall")
st.image("content/inpainting_after.png", caption="Image before inpainting, note the ornaments on the wall")
def make_regeneration_explanation():
with st.expander("Explanation object regeneration"):
st.write("In this object regeneration mode, the model calculates which objects occur in the image. "
"The user can then select which objects can be regenerated by the controlnet model by adding them in the multiselect box. "
"All the object classes that are not selected will remain the same as in the original image."
)
st.write("### Example object regeneration")
st.write("In the example below, the room consists of various objects such as wall, ceiling, floor, lamp, bed, ... "
"In the multiselect box, all the objects except for 'lamp', 'bed and 'table' are selected to be regenerated. "
)
st.image("content/regen_example.png", caption="Room where all concepts except for 'bed', 'lamp', 'table' are regenerated")
def make_segmentation_explanation():
with st.expander("Segmentation mode", expanded=False):
st.write("In the segmentation mode, the user can use his imagination and the paint brush to place concepts in the image. "
"In the left sidebar, you can first find the high level category of the concept you want to add, such as 'lighting', 'floor', .. "
"After selecting the category, you can select the specific concept you want to add in the 'Choose a color' dropdown. "
"This will change the color of the paint brush, which you can then use to draw on the input image. "
"The model will then regenerate the image with the concepts you have drawn and leave the rest of the image unchanged. "
)
st.image("content/sidebar segmentation.png", caption="Sidebar with segmentation options", width=300)
st.write("You can choose the freedraw mode which gives you a pencil of a certain (chosen) width or the polygon mode. With the polygon mode you can click to add a point to the polygon and close the polygon by right clicking. ")
st.write("Important: "
"it's not easy to draw a good segmentation mask. This is because you need to keep in mind the perspective of the room and the exact "
"shape of the object you want to draw within this perspective. Controlnet will follow your segmentation mask pretty well, so "
"a non-natural object shape will sometimes result in weird outputs. However, give it a try and see what you can do! "
)
st.image("content/segmentation window.png", caption="Example of a segmentation mask drawn on the input image to add a window to the room")
st.write("Tip: ")
st.write("In the concepts dropdown, you can select 'keep background' (which is a white color). Everything drawn in this color will use "
"the original underlying segmentation mask. This can be useful to help with generating other objects, since you give the model a some "
"freedom to generate outside the object borders."
)
st.image("content/keep background 1.png", caption="Image with a poster drawn on the wall.")
st.image("content/keep background 2.png", caption="Image with a poster drawn on the wall surrounded by 'keep background'.")