Upload 42 files

.gitignore

@@ -0,0 +1,8 @@
+__pycache__
+/venv
+/tmp
+/outputs
+/log
+/.idea
+/.vscode
+/krita_config.yaml

CHANGELOG.md

@@ -0,0 +1,62 @@
+# UI Changelog
+
+## 2022-12-28
+
+- Added "Alt Dock Behaviour" under "SD Plugin Config".
+  - _Modifies default Krita dock behaviour!_
+    - Dragging title bar of docker now drags all stacked/tabbed dockers out instead of just one docker.
+    - Dragging the tab now drags the specific docker out instead of only re-arranging the tab.
+  - Enables floating stacked/tabbed dockers.
+  - Enables subdividing dock areas further.
+  - See: <https://doc.qt.io/qt-6/qmainwindow.html#DockOption-enum>
+- All generations are added to group layer per batch with generation info.
+  - For batches of generations, all but the last image generated is hidden by default.
+
+## 2022-12-20
+
+- **UI Overhaul**: A few miscellaneous changes with some big ones:
+  - All tabs are now their own dockers to allow more flexibility in arranging.
+    - "Restore Defaults" will make all dockers re-appear and arrange themselves.
+  - Progress & number of pending requests now shown.
+  - All dropdowns now support searching, useful if your model checkpoint list is really long.
+
+## 2022-12-04
+
+- Add Interrupt button.
+
+## 2022-11-15
+
+- Scripts/features that increase the image size (Simple upscaling, SD upscaling, Outpaint Mk 2, etc) will now expand the canvas when image generation is complete **only if** _there is no active selection_.
+  - If there is a selection, the image will be scaled to fit the selection region.
+  - Using Ctrl+A to select the entire image is considered an active selection!
+
+## 2022-11-08
+
+- Inpainting is finally 100% fixed! No more weird borders. Blur works properly.
+- Inpainting Full Resolution and Mask Blur were deemed obsolete and removed.
+  - See <https://github.com/Interpause/auto-sd-paint-ext/wiki/Usage-Guide#inpainting> on better ways to do so.
+
+## 2022-10-31
+
+- Moved base size/max size & some other quick config options based on user feedback.
+
+## 2022-10-25
+
+- Will now save previous tab user was on.
+- Fixed seed being truncated to 32-bit int.
+- Prevent sending image generation request when cannot connect to backend.
+
+## 2022-10-24
+
+- UI no longer freezes when generating images or network activity like getting backend config
+  - Pressing "start xxx" multiple times will queue generation requests on the backend
+  - Will not mess with the current selection region or layer when inserting images once done
+
+## 2022-10-21
+
+- No need to manually hide inpainting layer anymore; It will be auto-hidden.
+- Color correction can be toggled separately for img2img/inpainting.
+- Status bar:
+  - In middle of page to be more visible even when scrolling.
+  - Warning when using features with no document open.
+- Inpaint is now the default tab.

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2022 John-Henry Lim
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,138 @@
+# auto-sd-paint-ext
+
+Formerly known as `auto-sd-krita`.
+
+> Extension for AUTOMATIC1111's webUI with Krita Plugin (other drawing studios soon?)
+
+Outdated demo | New UI (TODO: demo image)
+--- | ---
+![demo image](https://user-images.githubusercontent.com/42513874/194701722-e7a3f7eb-be4a-4f43-93a5-480835c9260f.jpg) | ![demo image 2](https://user-images.githubusercontent.com/42513874/199507299-66729f9b-3581-43a3-b5f4-57eb90b8f981.png)
+
+Why use this?
+
+- Optimized workflow (txt2img, img2img, inpaint, outpaint, upscale) & UI design.
+- Only drawing studio plugin that exposes the Script API.
+- Easily create/save profiles (prompts, samplers, model, etc used).
+- Some of the above isn't actually implemented yet.
+
+## Quick Jump
+
+- Full Installation & Workflow Tutorial Video! (Coming Soon...)
+- [Installation Guide](https://github.com/Interpause/auto-sd-paint-ext/wiki/Install-Guide)
+- [Usage Guide](https://github.com/Interpause/auto-sd-paint-ext/wiki/Usage-Guide)
+  - [Step by Step Guide to Better Inpainting](https://github.com/Interpause/auto-sd-paint-ext/wiki/Usage-Guide#inpainting-step-by-step)
+- [Update Guide](https://github.com/Interpause/auto-sd-paint-ext/wiki/Update-Guide)
+- [Features](https://github.com/Interpause/auto-sd-paint-ext/wiki/Features)
+- [TODO](https://github.com/Interpause/auto-sd-paint-ext/wiki/TODO)
+- [Contribution Guide](https://github.com/Interpause/auto-sd-paint-ext/wiki/Contribution-Guide)
+
+(Outdated) Usage & Workflow Demo:
+
+[![Youtube Video](http://img.youtube.com/vi/nP8MuRwcDN8/0.jpg)](https://youtu.be/nP8MuRwcDN8 "Inpaint like a pro with Stable Diffusion! auto-sd-krita workflow guide")
+
+### Differences from Video
+
+- All webUI scripts have been tested to work!
+  - SD Upscale, Outpainting Mk 2, Img2Img Alt, etc
+- Inpainting experience is better
+  - Inpaint mask is auto-hidden
+  - Better mask blur & inpaint full resolution technique than webUI
+- UI no longer freezes during image update
+- UI has been improved, takes up less space
+- Error messages have been improved
+
+## Breaking Changes
+
+- The URL is different now, so reset "Backend URL" to default under the Config tab.
+- It is now an AUTOMATIC1111 extension.
+  - Do <https://github.com/Interpause/auto-sd-krita/wiki/Quick-Switch-Using-Existing-AUTOMATIC1111-Install> in reverse for a quick fix.
+- `krita_config.yaml` was renamed to `auto-sd-paint-ext-backend.yaml`.
+
+## FAQ
+
+Q: How does the base_size, max_size system work?
+
+A:
+
+It is an alternative to AUTO's highres fix that works for all modes, not just txt2img.
+
+The selection will be resized such that the shorter dimension is base_size. However, if the aforementioned resize causes the longer dimension to exceed max_size, the shorter dimension will be resized to less than base_size. Setting base_size and max_size higher can be used to generate higher resolution images (along with their issues), essentially **disabling the system**, _though it might make sense for img2img mode_.
+
+This is actually smarter than the builtin highres fix + firstphase width/height system. Thank the original plugin writer, @sddebz, for writing this.
+
+<hr/>
+
+Q: Outpainting tab?
+
+A:
+While the outpainting tab is still WIP, the outpainting scripts (under img2img tab) works perfectly fine! Alternatively, if you want more control over outpainting, you can:
+
+1. Expand the canvas
+2. Scribble in the newly added blank area
+3. img2img on the blank area + some of the image
+
+<hr/>
+
+Q: Is the model loaded into memory twice?
+
+A: No, it shares the same backend. Both the Krita plugin and webUI can be used concurrently.
+
+<hr/>
+
+Q: How can you commit to updating regularly?
+
+A: It is easy for me.
+
+<hr/>
+
+Q: Will it work with other Krita plugin backends?
+
+A: Unfortunately no, all plugins so far have different APIs. The official API is coming soon though...
+
+## UI Changelog
+
+See [CHANGELOG.md](./CHANGELOG.md) for the full changelog.
+
+### 2022-12-28
+
+- Added "Alt Dock Behaviour" under "SD Plugin Config".
+  - _Modifies default Krita dock behaviour!_
+    - Dragging title bar of docker now drags all stacked/tabbed dockers out instead of just one docker.
+    - Dragging the tab now drags the specific docker out instead of only re-arranging the tab.
+  - Enables floating stacked/tabbed dockers.
+  - Enables subdividing dock areas further.
+  - See: <https://doc.qt.io/qt-6/qmainwindow.html#DockOption-enum>
+- All generations are added to group layer per batch with generation info.
+  - For batches of generations, all but the last image generated is hidden by default.
+
+### 2022-12-20
+
+- **UI Overhaul**: A few miscellaneous changes with some big ones:
+  - All tabs are now their own dockers to allow more flexibility in arranging.
+    - "Restore Defaults" will make all dockers re-appear and arrange themselves.
+  - Progress & number of pending requests now shown.
+  - All dropdowns now support searching, useful if your model checkpoint list is really long.
+
+### 2022-12-04
+
+- Add Interrupt button.
+
+### 2022-11-15
+
+- Scripts/features that increase the image size (Simple upscaling, SD upscaling, Outpaint Mk 2, etc) will now expand the canvas when image generation is complete **only if** _there is no active selection_.
+  - If there is a selection, the image will be scaled to fit the selection region.
+  - Using Ctrl+A to select the entire image is considered an active selection!
+
+### 2022-11-08
+
+- Inpainting is finally 100% fixed! No more weird borders. Blur works properly.
+- Inpainting Full Resolution and Mask Blur were deemed obsolete and removed.
+  - See <https://github.com/Interpause/auto-sd-paint-ext/wiki/Usage-Guide#inpainting> on better ways to do so.
+
+## Credits
+
+- [@sddebz](https://github.com/sddebz) for writing the original backend API and Krita plugin while keeping the Gradio webUI functionality intact.
+
+## License
+
+MIT for the Krita Plugin backend server & frontend plugin. Code has been nearly completely rewritten compared to original plugin by now.

backend/__init__.py

@@ -0,0 +1,3 @@
+from .app import router
+
+__all__ = ["router"]

backend/app.py

@@ -0,0 +1,386 @@
+from __future__ import annotations
+
+import logging
+import os
+import time
+
+import modules
+from fastapi import APIRouter, Request
+from fastapi.responses import StreamingResponse
+from modules import shared
+from modules.call_queue import wrap_gradio_gpu_call
+from PIL import Image, ImageOps
+from starlette.concurrency import iterate_in_threadpool
+
+from .config import LOGGER_NAME, NAME_SCRIPT_LOOPBACK, NAME_SCRIPT_UPSCALE
+from .script_hack import get_script_info, get_scripts_metadata, process_script_args
+from .structs import (
+    ConfigResponse,
+    ImageResponse,
+    Img2ImgRequest,
+    Txt2ImgRequest,
+    UpscaleRequest,
+    UpscaleResponse,
+)
+from .utils import (
+    b64_to_img,
+    bytewise_xor,
+    get_encrypt_key,
+    get_sampler_index,
+    get_upscaler_index,
+    img_to_b64,
+    load_config,
+    merge_default_config,
+    parse_prompt,
+    prepare_backend,
+    prepare_mask,
+    save_img,
+    sddebz_highres_fix,
+)
+
+router = APIRouter()
+
+log = logging.getLogger(LOGGER_NAME)
+
+# NOTE: how to run a script
+# - get scripts_txt2img/scripts_img2img from modules.scripts
+# - construct array args, where 0th element is selected script
+# - refer to script.args_from & script.args_to to figure out which elements in
+#   array args to populate
+#
+# The way scripts are handled is they are loaded one by one, append to a list of
+# scripts, which each script taking up "slots" in the input args array.
+# So the more scripts, the longer array args would be for the last script.
+
+# NOTE: where to draw the line on what is done by the backend vs the frontend?
+# TODO: Create separate Outpainting route, add img2img structs to Upscale route
+# - yes I know its highly inconsistent what should be a route or not, but to prevent
+#   incredibly hacky workarounds on the frontend for script calling, it should be
+#   done by the backend, which has better access to the script information.
+# - Upscale tab UI:
+#    - Upscaler dropdown + 0.5x downscale checkbox + SD upscale checkbox
+#    - SD upscale checkbox hides 0.5x downscale checkbox, renames upscaler dropdown
+#    - to prescaler, and shows modified img2img UI (ofc uses its own cfg namespace)
+# - Outpaint tab UI:
+#    - modified img2img UI with own cfg namespace
+#    - try and hijack more control (Pixel to expand per direction instead of all directions)
+#    - self-sketch mode: basically sketch + inpaint but the inpaint mask is auto-calculated
+#    - option to select poor man, mk 2 or self-sketch
+# TODO: Consider using pipeline directly instead of Gradio API for less surprises & better control
+
+
+@router.get("/config", response_model=ConfigResponse)
+async def get_state():
+    """Get information about backend API.
+
+    Returns config from `krita_config.yaml`, other metadata,
+    the path to the rendered image and image mask, etc.
+
+    Returns:
+        Dict: information.
+    """
+    opt = load_config().plugin
+    prepare_backend(opt)
+
+    sample_path = os.path.abspath(opt.sample_path)
+    return {
+        **opt.dict(),
+        "sample_path": sample_path,
+        "upscalers": [upscaler.name for upscaler in shared.sd_upscalers],
+        "samplers": [sampler.name for sampler in modules.sd_samplers.samplers],
+        "samplers_img2img": [
+            sampler.name for sampler in modules.sd_samplers.samplers_for_img2img
+        ],
+        "scripts_txt2img": get_scripts_metadata(False),
+        "scripts_img2img": get_scripts_metadata(True),
+        "face_restorers": [model.name() for model in shared.face_restorers],
+        "sd_models": modules.sd_models.checkpoint_tiles(),  # yes internal API has spelling error
+    }
+
+
+@router.post("/txt2img", response_model=ImageResponse)
+def f_txt2img(req: Txt2ImgRequest):
+    """Post request for Txt2Img.
+
+    Args:
+        req (Txt2ImgRequest): Request.
+
+    Returns:
+        Dict: Outputs and info.
+    """
+    log.info(f"txt2img:\n{req}")
+
+    opt = load_config().txt2img
+    req = merge_default_config(req, opt)
+    prepare_backend(req)
+
+    script_ind, script, meta = get_script_info(req.script, False)
+    args = process_script_args(script_ind, script, meta, req.script_args)
+
+    width, height = sddebz_highres_fix(
+        req.base_size, req.max_size, req.orig_width, req.orig_height
+    )
+
+    output = wrap_gradio_gpu_call(modules.txt2img.txt2img)(
+        parse_prompt(req.prompt),  # prompt
+        parse_prompt(req.negative_prompt),  # negative_prompt
+        "None",  # prompt_style: saved prompt styles (unsupported)
+        "None",  # prompt_style2: saved prompt styles (unsupported)
+        req.steps,  # steps
+        get_sampler_index(req.sampler_name),  # sampler_index
+        req.restore_faces,  # restore_faces
+        req.tiling,  # tiling
+        req.batch_count,  # n_iter
+        req.batch_size,  # batch_size
+        req.cfg_scale,  # cfg_scale
+        req.seed,  # seed
+        req.subseed,  # subseed
+        req.subseed_strength,  # subseed_strength
+        req.seed_resize_from_h,  # seed_resize_from_h
+        req.seed_resize_from_w,  # seed_resize_from_w
+        req.seed_enable_extras,  # seed_enable_extras
+        height,  # height
+        width,  # width
+        req.highres_fix,  # enable_hr: high res fix
+        req.denoising_strength,  # denoising_strength: only applicable if high res fix in use
+        0,  # hr_scale (overrided by hr_resize_x/y)
+        req.upscaler_name,  # hr_upscaler: upscaler to use for highres fix
+        0,  # hr_second_pass_steps: 0 uses same num of steps as generation to refine details
+        req.orig_width,  # hr_resize_x
+        req.orig_height,  # hr_resize_y
+        *args,
+    )
+    images = output[0]
+    info = output[1]
+
+    if images is None or len(images) < 1:
+        log.warning("Interrupted!")
+        return {"outputs": [], "info": info}
+
+    if shared.opts.return_grid:
+        if not req.include_grid and len(images) > 1 and script_ind == 0:
+            images = images[1:]
+
+    if not script or (width == images[0].width and height == images[0].height):
+        log.info(
+            f"img size: {images[0].width}x{images[0].height}, target: {req.orig_width}x{req.orig_height}"
+        )
+        images = [
+            modules.images.resize_image(0, image, req.orig_width, req.orig_height)
+            for image in images
+        ]
+
+    # save images for debugging/logging purposes
+    if req.save_samples:
+        output_paths = [
+            save_img(image, opt.sample_path, filename=f"{int(time.time())}_{i}.png")
+            for i, image in enumerate(images)
+        ]
+        log.info(f"saved: {output_paths}")
+
+    images = [img_to_b64(image) for image in images]
+
+    log.info(f"output sizes: {[len(i) for i in images]}")
+    log.info(f"finished txt2img!")
+    return {"outputs": images, "info": info}
+
+
+@router.post("/img2img", response_model=ImageResponse)
+def f_img2img(req: Img2ImgRequest):
+    """Post request for Img2Img.
+
+    Args:
+        req (Img2ImgRequest): Request.
+
+    Returns:
+        Dict: Outputs and info.
+    """
+    log.info(f"img2img:\n{req.dict(exclude={'src_img', 'mask_img'})}")
+
+    opt = load_config().img2img
+    req = merge_default_config(req, opt)
+    prepare_backend(req)
+
+    script_ind, script, meta = get_script_info(req.script, True)
+    args = process_script_args(script_ind, script, meta, req.script_args)
+
+    image = b64_to_img(req.src_img)
+    mask = (
+        prepare_mask(b64_to_img(req.mask_img))
+        if req.mode == 1 and req.mask_img is not None
+        else None
+    )
+
+    orig_width, orig_height = image.size
+
+    if script and script.title() == NAME_SCRIPT_UPSCALE:
+        # in SD upscale mode, width & height determines tile size
+        width = height = req.base_size
+    else:
+        width, height = sddebz_highres_fix(
+            req.base_size, req.max_size, orig_width, orig_height
+        )
+
+    # NOTE:
+    # - image & mask repeated due to Gradio API have separate tabs for each mode...
+    # - mask is used only in inpaint mode
+    # - mask_mode determines whethere init_img_with_mask or init_img_inpaint is used,
+    #   I dont know why
+    # - new color sketch functionality in webUI is irrelevant so None is used for their options.
+    # - the internal code for img2img is confusing and duplicative...
+
+    output = wrap_gradio_gpu_call(modules.img2img.img2img)(
+        req.mode,  # mode
+        parse_prompt(req.prompt),  # prompt
+        parse_prompt(req.negative_prompt),  # negative_prompt
+        "None",  # prompt_style: saved prompt styles (unsupported)
+        "None",  # prompt_style2: saved prompt styles (unsupported)
+        image,  # init_img
+        {"image": image, "mask": mask},  # init_img_with_mask
+        None,  # init_img_with_mask_orig # only used by webUI color sketch if init_img_with_mask isn't dict
+        image,  # init_img_inpaint
+        mask,  # init_mask_inpaint
+        # using 1 for uploaded mask mode; processing done by prepare_mask to ensure its correct
+        1,  # mask_mode: internally checks if equal 0. 1 enables alpha mask (remove erased parts)
+        req.steps,  # steps
+        get_sampler_index(req.sampler_name),  # sampler_index
+        0,  # req.mask_blur,  # mask_blur
+        None,  # mask_alpha # only used by webUI color sketch if init_img_with_mask isn't dict
+        req.inpainting_fill,  # inpainting_fill
+        req.restore_faces,  # restore_faces
+        req.tiling,  # tiling
+        req.batch_count,  # n_iter
+        req.batch_size,  # batch_size
+        req.cfg_scale,  # cfg_scale
+        req.denoising_strength,  # denoising_strength
+        req.seed,  # seed
+        req.subseed,  # subseed
+        req.subseed_strength,  # subseed_strength
+        req.seed_resize_from_h,  # seed_resize_from_h
+        req.seed_resize_from_w,  # seed_resize_from_w
+        req.seed_enable_extras,  # seed_enable_extras
+        height,  # height
+        width,  # width
+        req.resize_mode,  # resize_mode
+        False,  # req.inpaint_full_res,  # inpaint_full_res
+        0,  # req.inpaint_full_res_padding,  # inpaint_full_res_padding
+        req.invert_mask,  # inpainting_mask_invert
+        "",  # img2img_batch_input_dir (unspported)
+        "",  # img2img_batch_output_dir (unspported)
+        *args,
+    )
+    images = output[0]
+    info = output[1]
+
+    if images is None or len(images) < 1:
+        log.warning("Interrupted!")
+        return {"outputs": [], "info": info}
+
+    if shared.opts.return_grid:
+        if not req.include_grid and len(images) > 1 and script_ind == 0:
+            images = images[1:]
+        # This is a workaround.
+        if script and script.title() == NAME_SCRIPT_LOOPBACK and len(images) > 1:
+            images = images[1:]
+
+    # NOTE: this is a dumb assumption:
+    # if size of image is different from size given to pipeline (after sbbedz fix)
+    # then it must be intentional (i.e. SD Upscale/outpaint) so dont scale back
+    if not script or (width == images[0].width and height == images[0].height):
+        log.info(
+            f"img Size: {images[0].width}x{images[0].height}, target: {orig_width}x{orig_height}"
+        )
+        images = [
+            modules.images.resize_image(0, image, orig_width, orig_height)
+            for image in images
+        ]
+
+    if req.mode == 1:
+
+        def apply_mask(img):
+            """Mask inpaint using original mask, including alpha."""
+            r, g, b = img.split()  # img2img/inpaint gives rgb image
+            a = ImageOps.invert(mask) if req.invert_mask else mask
+            return Image.merge("RGBA", (r, g, b, a))
+
+        images = [apply_mask(x) for x in images]
+
+    # save images for debugging/logging purposes
+    if req.save_samples:
+        output_paths = [
+            save_img(image, opt.sample_path, filename=f"{int(time.time())}_{i}.png")
+            for i, image in enumerate(images)
+        ]
+        log.info(f"saved: {output_paths}")
+
+    images = [img_to_b64(image) for image in images]
+
+    log.info(f"output sizes: {[len(i) for i in images]}")
+    log.info(f"finished img2img!")
+    return {"outputs": images, "info": info}
+
+
+@router.post("/upscale", response_model=UpscaleResponse)
+def f_upscale(req: UpscaleRequest):
+    """Post request for upscaling.
+
+    Args:
+        req (UpscaleRequest): Request.
+
+    Returns:
+        Dict: Output.
+    """
+    log.info(f"upscale:\n{req.dict(exclude={'src_img'})}")
+
+    opt = load_config().upscale
+    req = merge_default_config(req, opt)
+    prepare_backend(req)
+
+    image = b64_to_img(req.src_img).convert("RGB")
+    orig_width, orig_height = image.size
+
+    upscaler_index = get_upscaler_index(req.upscaler_name)
+    upscaler = shared.sd_upscalers[upscaler_index]
+
+    if upscaler.name == "None":
+        log.info(f"No upscaler selected, will do nothing")
+        return
+
+    if req.downscale_first:
+        image = modules.images.resize_image(0, image, orig_width // 2, orig_height // 2)
+
+    image = upscaler.scaler.upscale(image, upscaler.scale, upscaler.data_path)
+    if req.save_samples:
+        output_path = save_img(
+            image, opt.sample_path, filename=f"{int(time.time())}.png"
+        )
+        log.info(f"saved: {output_path}")
+
+    output = img_to_b64(image)
+    log.info(f"output size: {len(output)}")
+    log.info("finished upscale!")
+    return {"output": output}
+
+
+async def app_encryption_middleware(req: Request, call_next):
+    """Used to decrypt/encrypt HTTP request body."""
+    is_encrypted = "X-Encrypted-Body" in req.headers
+    # only supported method now is XOR
+    assert not is_encrypted or req.headers["X-Encrypted-Body"] == "XOR"
+    if is_encrypted:
+        key = get_encrypt_key()
+        assert key is not None, "Unable to decrypt request without key."
+        body = await req.body()
+        body = bytewise_xor(body, key)
+        # NOTE: FastAPI refuses to work with requests that have already been consumed idk why
+        async def receive():
+            return dict(type="http.request", body=body, more_body=False)
+
+        req = Request(req.scope, receive, req._send)
+
+    res: StreamingResponse = await call_next(req)
+    if is_encrypted:
+        res.headers["X-Encrypted-Body"] = req.headers["X-Encrypted-Body"]
+        body = [bytewise_xor(chunk, key) async for chunk in res.body_iterator]
+        res.body_iterator = iterate_in_threadpool(iter(body))
+    return res

backend/config.py

@@ -0,0 +1,154 @@
+from __future__ import annotations
+
+from typing import Any
+
+from pydantic import BaseModel, Field
+
+SCRIPT_NAME = "Interpause Backend API"
+SCRIPT_ID = "interpause_backend_api"
+ROUTE_PREFIX = "/sdapi/interpause"
+CONFIG_PATH = "auto-sd-paint-ext-backend.yaml"
+LOGGER_NAME = "auto-sd-paint-ext"
+ENCRYPT_FILE = "xor_pass.txt"
+
+# names of scripts to apply workarounds for
+NAME_SCRIPT_LOOPBACK = "Loopback"
+NAME_SCRIPT_UPSCALE = "SD upscale"
+
+
+class BaseOptions(BaseModel):
+    sample_path: str = "outputs/krita-out"
+    """Where to save generated images to."""
+    save_samples: bool = False
+    """Whether to save temporary images (useful for debugging)."""
+
+
+class GenerationOptions(BaseModel):
+    sd_model: str = "model.ckpt"
+    """Model to use for generation."""
+    script: str = "None"
+    """Which script to use."""
+    script_args: list = Field(default_factory=list)
+    """List of args for script."""
+
+    prompt: Any = "dog"
+    """Requested prompt."""
+    negative_prompt: Any = ""
+    """Requested negative prompt."""
+    seed: int = -1
+    """Seed used for noise generation. Incremented by 1 for each image rendered."""
+
+    seed_enable_extras: bool = False
+    """Enable subseed variation."""
+    subseed: int = -1
+    """Subseed to use for subseed variation. Incremented by 1 for each image rendered."""
+    subseed_strength: float = 0.0
+    """Strength of subseed compared to seed. 0.0 will be completely original seed, 1.0 will be completely subseed."""
+    seed_resize_from_h: int = 0
+    """Original resolution seed was used at. Used to resize latent noise to attempt to generate same image with a different resolution."""
+    seed_resize_from_w: int = 0
+    """Original resolution seed was used at. Used to resize latent noise to attempt to generate same image with a different resolution."""
+
+    sampler_name: str = "Euler a"
+    """Exact name of sampler to use. Name should follow exact spelling and capitalization as in the WebUI."""
+    steps: int = 30
+    """Number of steps for diffusion."""
+    cfg_scale: float = 7.5
+    """Guidance scale for diffusion."""
+    denoising_strength: float = 0.35
+    """Strength of denoising from 0.0 to 1.0."""
+
+    batch_count: int = 1
+    """Number of batches to render."""
+    batch_size: int = 1
+    """Number of images per batch to render."""
+
+    base_size: int = 512
+    """Native/base resolution of model used."""
+    max_size: int = 768
+    """Max input resolution allowed to prevent image artifacts."""
+    tiling: bool = False
+    """Whether to generate a tileable image."""
+    highres_fix: bool = False
+    """Whether to enable workaround for higher resolution at cost of time."""
+    firstphase_height: int = 512
+    """Max height for first phase of highres fix (before upscaling to requested resolution)."""
+    firstphase_width: int = 512
+    """Max width for first phase of highres fix (before upscaling to requested resolution)."""
+
+    # upscale_overlap: int = 64
+    # """Size of overlap in pixels for upscaling.""" Configure this in WebUI
+    upscaler_name: str = "None"
+    """Exact name of upscaler to use."""
+    filter_nsfw: bool = False
+    """filter NSFW content."""
+
+    include_grid: bool = False
+    """Whether to include the image grid in the results sent to Krita"""
+
+
+class SamplerParamOptions(BaseModel):
+    # TODO: More conveniently expose config options for samplers/explain them.
+    pass
+
+
+class FaceRestorationOptions(BaseModel):
+    restore_faces: bool = False
+    """Whether to use GFPGAN for face restoration."""
+    face_restorer: str = "CodeFormer"
+    """Exact name of face restorer to use."""
+    codeformer_weight: float = 0.5
+    """Strength of face restoration if using CodeFormer. 0.0 is the strongest and 1.0 is the weakest."""
+
+
+class InpaintingOptions(BaseModel):
+    inpainting_fill: int = 1
+    """What to fill inpainted region with. 0 is blur/fill, 1 is original, 2 is latent noise, and 3 is latent empty."""
+    inpaint_full_res: bool = False
+    """(DISABLED) Whether to use the full resolution for inpainting."""
+    inpaint_full_res_padding: int = 0
+    """(DISABLED) Padding when using full resolution for inpainting."""
+    mask_blur: int = 0
+    """(DISABLED) Size of blur at boundaries of mask."""
+    invert_mask: bool = False
+    """Whether to invert the mask."""
+    inpaint_mask_weight: float = 1.0
+    """Mask weight for specialized inpainting models."""
+
+
+class Txt2ImgOptions(BaseOptions, GenerationOptions, FaceRestorationOptions):
+    pass
+
+
+class Img2ImgOptions(
+    BaseOptions, GenerationOptions, InpaintingOptions, FaceRestorationOptions
+):
+    mode: int = 0
+    """Img2Img mode. 0 is normal img2img on the selected region, 1 is inpainting, and 2 (unsupported) is batch processing."""
+    resize_mode: int = 1
+    """Unused by Krita plugin since rescaling is done by us. 0 is stretch to fit, 1 is cover, 2 is contain."""
+
+    steps: int = 50
+
+    color_correct: bool = True
+    """Apply color correction after img2img/inpaint to match original & blend better."""
+    do_exact_steps: bool = True
+    """Do exactly the number of steps specified by the slider instead of less during img2img/inpaint."""
+
+
+class UpscaleOptions(BaseOptions):
+    upscaler_name: str = "None"
+    """Exact name of upscaler to use."""
+    downscale_first: bool = False
+    """Whether to downscale the image by x0.5 first."""
+
+
+class PluginOptions(BaseOptions):
+    sample_path: str = "outputs/krita-in"
+
+
+class MainConfig(BaseModel):
+    txt2img: Txt2ImgOptions = Txt2ImgOptions()
+    img2img: Img2ImgOptions = Img2ImgOptions()
+    upscale: UpscaleOptions = UpscaleOptions()
+    plugin: PluginOptions = PluginOptions()

backend/script_hack.py

@@ -0,0 +1,161 @@
+"""
+Attempting to map Gradio UI elements present in scripts to allow
+converting to pyQt elements on the plugin side.
+"""
+
+import logging
+from typing import List, Sequence, Tuple
+
+import gradio as gr
+import modules
+
+from .config import LOGGER_NAME
+
+log = logging.getLogger(LOGGER_NAME)
+
+
+def inspect_ui(script: modules.scripts.Script, is_img2img: bool):
+    """Get metadata about accepted arguments by inspecting GUI. Needs Gradio Blocks context."""
+    elems = script.ui(is_img2img)
+
+    metadata = []
+    if not isinstance(elems, Sequence):
+        return metadata
+
+    for elem in elems:
+        data = {
+            "type": "None",
+            "label": elem.label,
+            "val": elem.value,
+            "is_index": False,
+        }
+        if isinstance(elem, gr.HTML):
+            data.update(val="")
+        elif isinstance(elem, gr.Markdown):
+            data.update(val="")
+        elif isinstance(elem, gr.Slider):
+            data.update(
+                type="range",
+                min=elem.minimum,
+                max=elem.maximum,
+                step=elem.step,
+            )
+        elif isinstance(elem, gr.Radio):
+            data.update(
+                type="combo",
+                is_index=elem.type == "index",
+                opts=elem.choices,
+            )
+        elif isinstance(elem, gr.Dropdown):
+            data.update(
+                type="combo",
+                is_index=elem.type == "index",
+                opts=elem.choices,
+            )
+        elif isinstance(elem, gr.Textbox):
+            data.update(
+                type="text",
+            )
+        elif isinstance(elem, gr.Checkbox):
+            data.update(
+                type="checkbox",
+            )
+        elif isinstance(elem, gr.CheckboxGroup):
+            data.update(
+                type="multiselect",
+                is_index=elem.type == "index",
+                opts=elem.choices,
+            )
+        elif isinstance(elem, gr.File):
+            data.update(val="")  # unsupported
+        else:
+            data.update(val="")  # unsupported
+        metadata.append(data)
+
+    return metadata
+
+
+img2img_script_meta = None
+txt2img_script_meta = None
+
+
+def get_scripts_metadata(is_img2img: bool):
+    """Get metadata about accepted arguments for scripts."""
+    # NOTE: inspect_ui is quite slow, so cache this
+    global txt2img_script_meta, img2img_script_meta
+    if is_img2img:
+        runner = modules.scripts.scripts_img2img
+    else:
+        runner = modules.scripts.scripts_txt2img
+    metadata = {"None": []}
+    if (
+        is_img2img
+        and img2img_script_meta
+        and len(img2img_script_meta) - 1 == len(runner.titles)
+    ):
+        return img2img_script_meta
+    elif txt2img_script_meta and len(txt2img_script_meta) - 1 == len(runner.titles):
+        return txt2img_script_meta
+
+    with gr.Blocks(visible=False, analytics_enabled=False):
+        for name, script in zip(runner.titles, runner.selectable_scripts):
+            metadata[name] = inspect_ui(script, is_img2img)
+    if is_img2img:
+        img2img_script_meta = metadata
+    else:
+        txt2img_script_meta = metadata
+    return metadata
+
+
+def get_script_info(
+    script_name: str, is_img2img: bool
+) -> Tuple[int, modules.scripts.Script, List[dict]]:
+    """Get index of script, script instance and argument metadata by name.
+
+    Args:
+        script_name (str): Exact name of script.
+        is_img2img (bool): Whether the script is for img2img or txt2img.
+
+    Raises:
+        KeyError: Script cannot be found.
+
+    Returns:
+        Tuple[int, Script, List[dict]]: Index of script, script itself and arguments metadata.
+    """
+    if is_img2img:
+        runner = modules.scripts.scripts_img2img
+    else:
+        runner = modules.scripts.scripts_txt2img
+    # in API, index 0 means no script, scripts are indexed from 1 onwards
+    names = ["None"] + runner.titles
+    if script_name == "None":
+        return 0, None, []
+    for i, n in enumerate(names):
+        if n == script_name:
+            script = runner.selectable_scripts[i - 1]
+            return i, script, get_scripts_metadata(is_img2img)[n]
+    raise KeyError(f"script not found for type {type}: {script_name}")
+
+
+def process_script_args(
+    script_ind: int, script: modules.scripts.Script, meta: List[dict], args: list
+) -> list:
+    """Get the position arguments required."""
+    if script is None:
+        return [0]  # 0th element selects which script to use. 0 is None.
+
+    # convert strings back to indexes
+    for i, (o, arg) in enumerate(zip(meta, args)):
+        if o["is_index"]:
+            if isinstance(arg, list):
+                args[i] = [o["opts"].index(v) for v in arg]
+            else:
+                args[i] = o["opts"].index(arg)
+
+    log.info(
+        f"Script selected: {script.filename}, Args Range: [{script.args_from}:{script.args_to}]"
+    )
+    # pad the args like the internal API requires...
+    args = [script_ind] + [0] * (script.args_from - 1) + args
+    log.info(f"Script args:\n{args}")
+    return args

backend/structs.py

@@ -0,0 +1,85 @@
+from __future__ import annotations
+
+from typing import Dict, List, Optional
+
+from pydantic import BaseModel
+
+from .config import Img2ImgOptions, PluginOptions, Txt2ImgOptions, UpscaleOptions
+from .utils import optional
+
+
+@optional
+class DefaultTxt2ImgOptions(Txt2ImgOptions):
+    pass
+
+
+class Txt2ImgRequest(DefaultTxt2ImgOptions):
+    """Text2Img API request. If optional attributes aren't set, the defaults
+    from `krita_config.yaml` will be used.
+    """
+
+    orig_width: int
+    """Requested image width."""
+    orig_height: int
+    """Requested image height."""
+
+
+@optional
+class DefaultImg2ImgOptions(Img2ImgOptions):
+    pass
+
+
+class Img2ImgRequest(DefaultImg2ImgOptions):
+    """Img2Img API request. If optional attributes aren't set, the defaults from
+    `krita_config.yaml` will be used.
+    """
+
+    src_img: str
+    """Image being used."""
+    mask_img: Optional[str] = None
+    """Image mask being used."""
+
+
+@optional
+class DefaultUpscaleOptions(UpscaleOptions):
+    pass
+
+
+class UpscaleRequest(DefaultUpscaleOptions):
+    """Upscale API request. If optional attributes aren't set, the defaults from
+    `krita_config.yaml` will be used.
+    """
+
+    src_img: str
+    """Image being used."""
+
+
+class ConfigResponse(PluginOptions):
+    sample_path: str
+    """Where the Krita plugin should save the selected region and mask."""
+    upscalers: List[str]
+    """List of available upscalers."""
+    samplers: List[str]
+    """List of available samplers."""
+    samplers_img2img: List[str]
+    """List of available samplers specifically for img2img (upstream separated them for a reason)."""
+    scripts_txt2img: Dict[str, List[Dict]]
+    """List of available txt2img scripts."""
+    scripts_img2img: Dict[str, List[Dict]]
+    """List of available img2img scripts."""
+    face_restorers: List[str]
+    """List of available face restorers."""
+    sd_models: List[str]
+    """List of available models."""
+
+
+class ImageResponse(BaseModel):
+    outputs: List[str]
+    """List of generated images encoded in base64."""
+    info: str
+    """Generation info already jsonified."""
+
+
+class UpscaleResponse(BaseModel):
+    output: str
+    """Upscaled image in base64."""

backend/utils.py

@@ -0,0 +1,329 @@
+from __future__ import annotations
+
+import inspect
+import logging
+import os
+import secrets
+from base64 import b64decode, b64encode
+from io import BytesIO
+from itertools import cycle
+from math import ceil
+
+import modules
+import yaml
+from modules import shared
+from PIL import Image
+from pydantic import BaseModel
+
+from .config import CONFIG_PATH, ENCRYPT_FILE, LOGGER_NAME, MainConfig
+
+log = logging.getLogger(LOGGER_NAME)
+
+
+def load_config():
+    """Load default config (including those not exposed in the API yet) from
+    `CONFIG_PATH` in the current working directory.
+
+    Will create `CONFIG_PATH` if it has yet to exist using `MainConfig` from
+    `config.py`.
+
+    Returns:
+        MainConfig: config
+    """
+    if not os.path.isfile(CONFIG_PATH):
+        cfg = MainConfig()
+        with open(CONFIG_PATH, "w") as f:
+            yaml.safe_dump(cfg.dict(), f)
+
+    with open(CONFIG_PATH) as file:
+        obj = yaml.safe_load(file)
+        return MainConfig.parse_obj(obj)
+
+
+def merge_default_config(config: BaseModel, default: BaseModel):
+    """Replace unset and None fields in opt with values from default with the
+    same field name in place.
+
+    Unset fields does not include fields that are explicitly set to None but
+    includes fields with a default value due to being unset.
+
+    Args:
+        config (BaseModel): Config object.
+        default (BaseModel): Default to merge from.
+
+    Returns:
+        BaseModel: Modified config.
+    """
+
+    for field in config.__fields__:
+        if not field in config.__fields_set__ or field is None:
+            setattr(config, field, getattr(default, field, None))
+
+    return config
+
+
+def prepare_backend(opt: BaseModel):
+    """Misc configuration and preparation tasks before calling internal API.
+
+    Currently includes:
+    - Ensuring the output/input folders exist
+    - Set the global face restorer model to the selected one
+    - Set the global SD model to the selected one
+    - Set the global upscaler to the selected one
+    - Set other misc global webUI/backend settings
+
+    Args:
+        opt (BaseModel): Option/Request object
+    """
+    # the `shared` module handles app state for the underlying codebase
+
+    if hasattr(opt, "face_restorer"):
+        shared.opts.face_restoration_model = opt.face_restorer
+        shared.opts.code_former_weight = opt.codeformer_weight
+
+    if hasattr(opt, "sd_model"):
+        shared.opts.sd_model_checkpoint = opt.sd_model
+        modules.sd_models.reload_model_weights(shared.sd_model)
+
+    if hasattr(opt, "upscaler_name"):
+        shared.opts.upscaler_for_img2img = opt.upscaler_name
+
+    if hasattr(opt, "color_correct"):
+        shared.opts.img2img_color_correction = opt.color_correct
+        shared.opts.img2img_fix_steps = opt.do_exact_steps
+
+    if hasattr(opt, "filter_nsfw"):
+        shared.opts.filter_nsfw = opt.filter_nsfw
+
+    if hasattr(opt, "inpaint_mask_weight"):
+        shared.opts.inpainting_mask_weight = opt.inpaint_mask_weight
+
+    # Ensure the output/input folders exist
+    if hasattr(opt, "sample_path"):
+        os.makedirs(opt.sample_path, exist_ok=True)
+
+
+def optional(*fields):
+    """Decorator function used to modify a pydantic model's fields to all be optional.
+    Alternatively, you can  also pass the field names that should be made optional as arguments
+    to the decorator.
+    Taken from https://github.com/samuelcolvin/pydantic/issues/1223#issuecomment-775363074
+    """
+
+    def dec(_cls):
+        for field in fields:
+            _cls.__fields__[field].required = False
+        return _cls
+
+    if fields and inspect.isclass(fields[0]) and issubclass(fields[0], BaseModel):
+        cls = fields[0]
+        fields = cls.__fields__
+        return dec(cls)
+
+    return dec
+
+
+def save_img(image: Image.Image, sample_path: str, filename: str):
+    """Saves an image.
+
+    Args:
+        image (Image): Image to save.
+        sample_path (str): Folder to save the image in.
+        filename (str): Name to save the image as.
+
+    Returns:
+        str: Absolute path where the image was saved.
+    """
+    path = os.path.join(sample_path, filename)
+    image.save(path)
+    return os.path.abspath(path)
+
+
+def img_to_b64(image: Image.Image):
+    """Convert an image to base64-encoded string.
+
+    Args:
+        image (Image): Image to encode.
+
+    Returns:
+        str: Base64-encoded image.
+    """
+    buf = BytesIO()
+    image.save(buf, format="png")
+    return b64encode(buf.getvalue()).decode("utf-8")
+
+
+def b64_to_img(enc: str):
+    """Convert base64-encoded string to image.
+
+    Args:
+        enc (str): Base64-encoded image.
+
+    Returns:
+        Image: Image.
+    """
+    return Image.open(BytesIO(b64decode(enc)))
+
+
+def sddebz_highres_fix(
+    base_size: int, max_size: int, orig_width: int, orig_height: int
+):
+    """Calculate an appropiate image resolution given the base input size of the
+    model and max input size allowed.
+
+    The max input size is due to how Stable Diffusion currently handles resolutions
+    larger than its base/native input size of 512, which can cause weird issues
+    such as duplicated features in the image. Hence, it is typically better to
+    render at a smaller appropiate resolution before using other methods to upscale
+    to the original resolution. Setting max_size to 512, matching the base_size,
+    imitates how the highres fix works.
+
+    Stable Diffusion also messes up for resolutions smaller than 512. In which case,
+    it is better to render at the base resolution before downscaling to the original.
+
+    This method requires less user input than the builtin highres fix, which uses
+    firstphase_width and firstphase_height.
+
+    The original plugin writer, @sddebz, wrote this. I modified it to `ceil`
+    instead of `round` to make selected region resizing easier in the plugin, and
+    to avoid rounding to 0.
+
+    Args:
+        base_size (int): Native/base input size of the model.
+        max_size (int): Max input size to accept.
+        orig_width (int): Original width requested.
+        orig_height (int): Original height requested.
+
+    Returns:
+        Tuple[int, int]: Appropiate (width, height) to use for the model.
+    """
+
+    def rnd(r, x, z=64):
+        """Scale dimension x with stride z while attempting to preserve aspect ratio r."""
+        return z * ceil(r * x / z)
+
+    ratio = orig_width / orig_height
+
+    # height is smaller dimension
+    if orig_width > orig_height:
+        width, height = rnd(ratio, base_size), base_size
+        if width > max_size:
+            width, height = max_size, rnd(1 / ratio, max_size)
+    # width is smaller dimension
+    else:
+        width, height = base_size, rnd(1 / ratio, base_size)
+        if height > max_size:
+            width, height = rnd(ratio, max_size), max_size
+
+    new_ratio = width / height
+
+    log.info(
+        f"img size: {orig_width}x{orig_height} -> {width}x{height}, "
+        f"aspect ratio: {ratio:.2f} -> {new_ratio:.2f}, {100 * (new_ratio - ratio) / ratio :.2f}% change"
+    )
+    return width, height
+
+
+def parse_prompt(val):
+    """Parse different representations of prompt/negative prompt.
+
+    Args:
+        val (Any): Prompt to parse.
+
+    Raises:
+        SyntaxError: Value of the prompt key cannot be parsed.
+
+    Returns:
+        str: Correctly formatted prompt.
+    """
+    if val is None:
+        return ""
+    # Below cases are meant for prompts read from the yaml config
+    if isinstance(val, str):
+        return val
+    if isinstance(val, list):
+        return ", ".join(val)
+    if isinstance(val, dict):
+        prompt = ""
+        for item, weight in val.items():
+            if not prompt == "":
+                prompt += " "
+            if weight is None:
+                prompt += f"{item}"
+            else:
+                prompt += f"({item}:{weight})"
+        return prompt
+    raise SyntaxError(f"prompt field in {CONFIG_PATH} is invalid")
+
+
+def get_sampler_index(sampler_name: str):
+    """Get index of sampler by name.
+
+    Args:
+        sampler_name (str): Exact name of sampler.
+
+    Raises:
+        KeyError: Sampler cannot be found.
+
+    Returns:
+        int: Index of sampler.
+    """
+    for index, sampler in enumerate(modules.sd_samplers.samplers):
+        if sampler_name == sampler.name or sampler_name in sampler.aliases:
+            return index
+    raise KeyError(f"sampler not found: {sampler_name}")
+
+
+def get_upscaler_index(upscaler_name: str):
+    """Get index of upscaler by name.
+
+    Args:
+        upscaler_name (str): Exact name of upscaler.
+
+    Raises:
+        KeyError: Upscaler cannot be found.
+
+    Returns:
+        int: Index of sampler.
+    """
+    for index, upscaler in enumerate(shared.sd_upscalers):
+        if upscaler.name == upscaler_name:
+            return index
+    raise KeyError(f"upscaler not found: {upscaler_name}")
+
+
+def prepare_mask(mask: Image.Image):
+    """Prepare mask for usage.
+
+    Args:
+        mask (Image): mask.
+
+    Returns:
+        Image: The luminance mask.
+    """
+    return mask.getchannel("A")
+
+
+def bytewise_xor(msg: bytes, key: bytes):
+    """Used for decrypting/encrypting request/response bodies."""
+    return bytes(v ^ k for v, k in zip(msg, cycle(key)))
+
+
+def get_encrypt_key():
+    """Read encryption key from file."""
+    try:
+        with open(ENCRYPT_FILE) as f:
+            return f.read().strip().encode("utf-8")
+    except:
+        if not os.path.exists(ENCRYPT_FILE):
+            log.warning(
+                f"Encryption key file doesn't exist at {os.path.abspath(ENCRYPT_FILE)}."
+            )
+            log.warning(f"Creating random encryption key.")
+            with open(ENCRYPT_FILE, "w") as f:
+                f.write(secrets.token_hex(16))
+            log.warning(
+                f"Key in {ENCRYPT_FILE} is completely optional. It can be used to encrypt messages between backend & Krita and is editable."
+            )
+            return get_encrypt_key()
+    return None

frontends/krita/krita_diff.desktop

@@ -0,0 +1,8 @@
+[Desktop Entry]
+Type=Service
+ServiceTypes=Krita/PythonPlugin
+X-KDE-Library=krita_diff
+X-Python-2-Compatible=false
+X-Krita-Manual=manual.html
+Name=Stable Diffusion Plugin
+Comment=Expose the power of AUTOMATIC1111's Stable Diffusion fork for creating to your heart's desire.

frontends/krita/krita_diff/__init__.py

@@ -0,0 +1,73 @@
+from krita import DockWidgetFactory, DockWidgetFactoryBase, Krita
+
+from .defaults import (
+    TAB_CONFIG,
+    TAB_IMG2IMG,
+    TAB_INPAINT,
+    TAB_SDCOMMON,
+    TAB_TXT2IMG,
+    TAB_UPSCALE,
+)
+from .docker import create_docker
+from .extension import SDPluginExtension
+from .pages import (
+    ConfigPage,
+    Img2ImgPage,
+    InpaintPage,
+    SDCommonPage,
+    Txt2ImgPage,
+    UpscalePage,
+)
+from .script import script
+from .utils import reset_docker_layout
+
+instance = Krita.instance()
+instance.addExtension(SDPluginExtension(instance))
+instance.addDockWidgetFactory(
+    DockWidgetFactory(
+        TAB_SDCOMMON,
+        DockWidgetFactoryBase.DockLeft,
+        create_docker(SDCommonPage),
+    )
+)
+instance.addDockWidgetFactory(
+    DockWidgetFactory(
+        TAB_TXT2IMG,
+        DockWidgetFactoryBase.DockLeft,
+        create_docker(Txt2ImgPage),
+    )
+)
+instance.addDockWidgetFactory(
+    DockWidgetFactory(
+        TAB_IMG2IMG,
+        DockWidgetFactoryBase.DockLeft,
+        create_docker(Img2ImgPage),
+    )
+)
+instance.addDockWidgetFactory(
+    DockWidgetFactory(
+        TAB_INPAINT,
+        DockWidgetFactoryBase.DockLeft,
+        create_docker(InpaintPage),
+    )
+)
+instance.addDockWidgetFactory(
+    DockWidgetFactory(
+        TAB_UPSCALE,
+        DockWidgetFactoryBase.DockLeft,
+        create_docker(UpscalePage),
+    )
+)
+instance.addDockWidgetFactory(
+    DockWidgetFactory(
+        TAB_CONFIG,
+        DockWidgetFactoryBase.DockLeft,
+        create_docker(ConfigPage),
+    )
+)
+
+
+# dumb workaround to ensure its only created once
+if script.cfg("first_setup", bool):
+    instance.notifier().windowCreated.connect(reset_docker_layout)
+    script.cfg.set("first_setup", False)

frontends/krita/krita_diff/client.py

@@ -0,0 +1,402 @@
+import json
+import socket
+from typing import Any, Dict, List
+from urllib.error import URLError
+from urllib.parse import urljoin, urlparse
+from urllib.request import Request, urlopen
+
+from krita import QObject, QThread, pyqtSignal
+
+from .config import Config
+from .defaults import (
+    ERR_BAD_URL,
+    ERR_NO_CONNECTION,
+    LONG_TIMEOUT,
+    OFFICIAL_ROUTE_PREFIX,
+    ROUTE_PREFIX,
+    SHORT_TIMEOUT,
+    STATE_DONE,
+    STATE_READY,
+    STATE_URLERROR,
+    THREADED,
+)
+from .utils import bytewise_xor, fix_prompt, get_ext_args, get_ext_key, img_to_b64
+
+# NOTE: backend queues up responses, so no explicit need to block multiple requests
+# except to prevent user from spamming themselves
+
+# TODO: tab showing all queued up requests (local plugin instance only)
+
+
+def get_url(cfg: Config, route: str = ..., prefix: str = ROUTE_PREFIX):
+    base = cfg("base_url", str)
+    if not urlparse(base).scheme in {"http", "https"}:
+        return None
+    url = urljoin(base, prefix)
+    if route is not ...:
+        url = urljoin(url, route)
+    # print("url:", url)
+    return url
+
+
+# krita doesn't reexport QtNetwork
+class AsyncRequest(QObject):
+    timeout = None
+    finished = pyqtSignal()
+    result = pyqtSignal(object)
+    error = pyqtSignal(Exception)
+
+    def __init__(
+        self,
+        url: str,
+        data: Any = None,
+        timeout: int = ...,
+        method: str = ...,
+        headers: dict = ...,
+        key: str = None,
+    ):
+        """Create an AsyncRequest object.
+
+        By default, AsyncRequest has no timeout, will infer whether it is "POST"
+        or "GET" based on the presence of `data` and uses JSON to transmit. It
+        also assumes the response is JSON.
+
+        Args:
+            url (str): URL to request from.
+            data (Any, optional): Payload to send. Defaults to None.
+            timeout (int, optional): Timeout for request. Defaults to `...`.
+            method (str, optional): Which HTTP method to use. Defaults to `...`.
+            key (Union[str, None], Optional): Key to use for encryption/decryption. Defaults to None.
+        """
+        super(AsyncRequest, self).__init__()
+        self.url = url
+        self.data = None if data is None else json.dumps(data).encode("utf-8")
+        self.headers = {} if headers is ... else headers
+
+        self.key = None
+        if isinstance(key, str) and key.strip() != "":
+            self.key = key.strip().encode("utf-8")
+
+        if self.key is not None:
+            self.headers["X-Encrypted-Body"] = "XOR"
+        if timeout is not ...:
+            self.timeout = timeout
+        if method is ...:
+            self.method = "GET" if data is None else "POST"
+        else:
+            self.method = method
+        if self.data is not None:
+            if self.key is not None:
+                # print(f"Encrypting with ${self.key}:\n{self.data}")
+                self.data = bytewise_xor(self.data, self.key)
+                # print(f"Encrypt Result:\n{self.data}")
+            self.headers["Content-Type"] = "application/json"
+            self.headers["Content-Length"] = str(len(self.data))
+
+    def run(self):
+        req = Request(self.url, headers=self.headers, method=self.method)
+        try:
+            with urlopen(req, self.data, self.timeout) as res:
+                data = res.read()
+                enc_type = res.getheader("X-Encrypted-Body", None)
+                assert enc_type in {"XOR", None}, "Unknown server encryption!"
+                if enc_type == "XOR":
+                    assert self.key, f"Key needed to decrypt server response!"
+                    # print(f"Decrypting with ${self.key}:\n{data}")
+                    data = bytewise_xor(data, self.key)
+                    # print(f"Decrypt Result:\n{data}")
+                self.result.emit(json.loads(data))
+        except Exception as e:
+            self.error.emit(e)
+        finally:
+            self.finished.emit()
+
+    @classmethod
+    def request(cls, *args, **kwargs):
+        req = cls(*args, **kwargs)
+        if THREADED:
+            thread = QThread()
+            # NOTE: need to keep reference to thread or it gets destroyed
+            req.thread = thread
+            req.moveToThread(thread)
+            thread.started.connect(req.run)
+            req.finished.connect(thread.quit)
+            # NOTE: is this a memory leak?
+            # For some reason, deleteLater occurs while thread is still running, resulting in crash
+            # req.finished.connect(req.deleteLater)
+            # thread.finished.connect(thread.deleteLater)
+            return req, lambda: thread.start()
+        else:
+            return req, lambda: req.run()
+
+
+class Client(QObject):
+    status = pyqtSignal(str)
+    config_updated = pyqtSignal()
+
+    def __init__(self, cfg: Config, ext_cfg: Config):
+        """It is highly dependent on config's structure to the point it writes directly to it. :/"""
+        super(Client, self).__init__()
+        self.cfg = cfg
+        self.ext_cfg = ext_cfg
+        self.short_reqs = set()
+        self.long_reqs = set()
+        # NOTE: this is a hacky workaround for detecting if backend is reachable
+        self.is_connected = False
+
+    def handle_api_error(self, exc: Exception):
+        """Handle exceptions that can occur while interacting with the backend."""
+        self.is_connected = False
+        try:
+            # wtf python? socket raises an error that isnt an Exception??
+            if isinstance(exc, socket.timeout):
+                raise TimeoutError
+            else:
+                raise exc
+        except URLError as e:
+            self.status.emit(f"{STATE_URLERROR}: {e.reason}")
+        except TimeoutError as e:
+            self.status.emit(f"{STATE_URLERROR}: response timed out")
+        except json.JSONDecodeError as e:
+            self.status.emit(f"{STATE_URLERROR}: invalid JSON response")
+        except ValueError as e:
+            self.status.emit(f"{STATE_URLERROR}: Invalid backend URL")
+        except ConnectionError as e:
+            self.status.emit(f"{STATE_URLERROR}: connection error during request")
+        except Exception as e:
+            # self.status.emit(f"{STATE_URLERROR}: Unexpected Error")
+            # self.status.emit(str(e))
+            assert False, e
+
+    def post(
+        self, route, body, cb, base_url=..., is_long=True, ignore_no_connection=False
+    ):
+        if not ignore_no_connection and not self.is_connected:
+            self.status.emit(ERR_NO_CONNECTION)
+            return
+        url = get_url(self.cfg, route) if base_url is ... else urljoin(base_url, route)
+        if not url:
+            self.status.emit(ERR_BAD_URL)
+            return
+        # TODO: how to cancel this? destroy the thread after sending API interrupt request?
+        req, start = AsyncRequest.request(
+            url,
+            body,
+            LONG_TIMEOUT if is_long else SHORT_TIMEOUT,
+            key=self.cfg("encryption_key"),
+        )
+
+        if is_long:
+            self.long_reqs.add(req)
+        else:
+            self.short_reqs.add(req)
+
+        def handler():
+            self.long_reqs.discard(req)
+            self.short_reqs.discard(req)
+            if is_long and len(self.long_reqs) == 0:
+                self.status.emit(STATE_DONE)
+
+        req.result.connect(cb)
+        req.error.connect(lambda e: self.handle_api_error(e))
+        req.finished.connect(handler)
+        start()
+
+    def get(self, route, cb, base_url=..., is_long=False, ignore_no_connection=False):
+        self.post(
+            route,
+            None,
+            cb,
+            base_url=base_url,
+            is_long=is_long,
+            ignore_no_connection=ignore_no_connection,
+        )
+
+    def common_params(self, has_selection):
+        """Parameters nearly all the post routes share."""
+        tiling = self.cfg("sd_tiling", bool) and not (
+            self.cfg("only_full_img_tiling", bool) and has_selection
+        )
+
+        # its fine to stuff extra stuff here; pydantic will shave off irrelevant params
+        params = dict(
+            sd_model=self.cfg("sd_model", str),
+            batch_count=self.cfg("sd_batch_count", int),
+            batch_size=self.cfg("sd_batch_size", int),
+            base_size=self.cfg("sd_base_size", int),
+            max_size=self.cfg("sd_max_size", int),
+            tiling=tiling,
+            upscaler_name=self.cfg("upscaler_name", str),
+            restore_faces=self.cfg("face_restorer_model", str) != "None",
+            face_restorer=self.cfg("face_restorer_model", str),
+            codeformer_weight=self.cfg("codeformer_weight", float),
+            filter_nsfw=self.cfg("filter_nsfw", bool),
+            do_exact_steps=self.cfg("do_exact_steps", bool),
+            include_grid=self.cfg("include_grid", bool),
+            save_samples=self.cfg("save_temp_images", bool),
+        )
+        return params
+
+    def get_config(self):
+        def cb(obj):
+            try:
+                assert "sample_path" in obj
+                assert len(obj["upscalers"]) > 0
+                assert len(obj["samplers"]) > 0
+                assert len(obj["samplers_img2img"]) > 0
+                assert len(obj["face_restorers"]) > 0
+                assert len(obj["sd_models"]) > 0
+                assert len(obj["scripts_txt2img"]) > 0
+                assert len(obj["scripts_img2img"]) > 0
+            except:
+                self.status.emit(
+                    f"{STATE_URLERROR}: incompatible response, are you running the right API?"
+                )
+                print("Invalid Response:\n", obj)
+                return
+
+            # replace only after verifying
+            self.cfg.set("sample_path", obj["sample_path"])
+            # NOTE: sorting these lists is risky; ivent 100% verified that I removed all reliance on indexes
+            self.cfg.set("upscaler_list", obj["upscalers"])
+            self.cfg.set("txt2img_sampler_list", obj["samplers"])
+            self.cfg.set("img2img_sampler_list", obj["samplers_img2img"])
+            self.cfg.set("inpaint_sampler_list", obj["samplers_img2img"])
+            self.cfg.set("txt2img_script_list", list(obj["scripts_txt2img"].keys()))
+            self.cfg.set("img2img_script_list", list(obj["scripts_img2img"].keys()))
+            self.cfg.set("inpaint_script_list", list(obj["scripts_img2img"].keys()))
+            self.cfg.set("face_restorer_model_list", obj["face_restorers"])
+            self.cfg.set("sd_model_list", obj["sd_models"])
+
+            # extension script cfg
+            obj["scripts_inpaint"] = obj["scripts_img2img"]
+            for ext_type in {"scripts_txt2img", "scripts_img2img", "scripts_inpaint"}:
+                metadata: Dict[str, List[dict]] = obj[ext_type]
+                self.ext_cfg.set(f"{ext_type}_len", len(metadata))
+                for ext_name, ext_meta in metadata.items():
+                    old_val = self.ext_cfg(get_ext_key(ext_type, ext_name))
+                    new_val = json.dumps(ext_meta)
+                    if new_val != old_val:
+                        self.ext_cfg.set(get_ext_key(ext_type, ext_name), new_val)
+                        for i, opt in enumerate(ext_meta):
+                            key = get_ext_key(ext_type, ext_name, i)
+                            self.ext_cfg.set(key, opt["val"])
+
+            self.is_connected = True
+            self.status.emit(STATE_READY)
+            self.config_updated.emit()
+
+        self.get("config", cb, ignore_no_connection=True)
+
+    def post_txt2img(self, cb, width, height, has_selection):
+        params = dict(orig_width=width, orig_height=height)
+        if not self.cfg("just_use_yaml", bool):
+            seed = (
+                int(self.cfg("txt2img_seed", str))  # Qt casts int as 32-bit int
+                if not self.cfg("txt2img_seed", str).strip() == ""
+                else -1
+            )
+            ext_name = self.cfg("txt2img_script", str)
+            ext_args = get_ext_args(self.ext_cfg, "scripts_txt2img", ext_name)
+            params.update(self.common_params(has_selection))
+            params.update(
+                prompt=fix_prompt(self.cfg("txt2img_prompt", str)),
+                negative_prompt=fix_prompt(self.cfg("txt2img_negative_prompt", str)),
+                sampler_name=self.cfg("txt2img_sampler", str),
+                steps=self.cfg("txt2img_steps", int),
+                cfg_scale=self.cfg("txt2img_cfg_scale", float),
+                seed=seed,
+                highres_fix=self.cfg("txt2img_highres", bool),
+                denoising_strength=self.cfg("txt2img_denoising_strength", float),
+                script=ext_name,
+                script_args=ext_args,
+            )
+
+        self.post("txt2img", params, cb)
+
+    def post_img2img(self, cb, src_img, mask_img, has_selection):
+        params = dict(mode=0, src_img=img_to_b64(src_img))
+        if not self.cfg("just_use_yaml", bool):
+            seed = (
+                int(self.cfg("img2img_seed", str))  # Qt casts int as 32-bit int
+                if not self.cfg("img2img_seed", str).strip() == ""
+                else -1
+            )
+            ext_name = self.cfg("img2img_script", str)
+            ext_args = get_ext_args(self.ext_cfg, "scripts_img2img", ext_name)
+            params.update(self.common_params(has_selection))
+            params.update(
+                prompt=fix_prompt(self.cfg("img2img_prompt", str)),
+                negative_prompt=fix_prompt(self.cfg("img2img_negative_prompt", str)),
+                sampler_name=self.cfg("img2img_sampler", str),
+                steps=self.cfg("img2img_steps", int),
+                cfg_scale=self.cfg("img2img_cfg_scale", float),
+                denoising_strength=self.cfg("img2img_denoising_strength", float),
+                color_correct=self.cfg("img2img_color_correct", bool),
+                script=ext_name,
+                script_args=ext_args,
+                seed=seed,
+            )
+
+        self.post("img2img", params, cb)
+
+    def post_inpaint(self, cb, src_img, mask_img, has_selection):
+        assert mask_img, "Inpaint layer is needed for inpainting!"
+        params = dict(
+            mode=1, src_img=img_to_b64(src_img), mask_img=img_to_b64(mask_img)
+        )
+        if not self.cfg("just_use_yaml", bool):
+            seed = (
+                int(self.cfg("inpaint_seed", str))  # Qt casts int as 32-bit int
+                if not self.cfg("inpaint_seed", str).strip() == ""
+                else -1
+            )
+            fill = self.cfg("inpaint_fill_list", "QStringList").index(
+                self.cfg("inpaint_fill", str)
+            )
+            ext_name = self.cfg("inpaint_script", str)
+            ext_args = get_ext_args(self.ext_cfg, "scripts_inpaint", ext_name)
+            params.update(self.common_params(has_selection))
+            params.update(
+                prompt=fix_prompt(self.cfg("inpaint_prompt", str)),
+                negative_prompt=fix_prompt(self.cfg("inpaint_negative_prompt", str)),
+                sampler_name=self.cfg("inpaint_sampler", str),
+                steps=self.cfg("inpaint_steps", int),
+                cfg_scale=self.cfg("inpaint_cfg_scale", float),
+                denoising_strength=self.cfg("inpaint_denoising_strength", float),
+                color_correct=self.cfg("inpaint_color_correct", bool),
+                script=ext_name,
+                script_args=ext_args,
+                seed=seed,
+                invert_mask=self.cfg("inpaint_invert_mask", bool),
+                # mask_blur=self.cfg("inpaint_mask_blur", int),
+                inpainting_fill=fill,
+                # inpaint_full_res=self.cfg("inpaint_full_res", bool),
+                # inpaint_full_res_padding=self.cfg("inpaint_full_res_padding", int),
+                inpaint_mask_weight=self.cfg("inpaint_mask_weight", float),
+                include_grid=False,  # it is never useful for inpaint mode
+            )
+
+        self.post("img2img", params, cb)
+
+    def post_upscale(self, cb, src_img):
+        params = (
+            {
+                "src_img": img_to_b64(src_img),
+                "upscaler_name": self.cfg("upscale_upscaler_name", str),
+                "downscale_first": self.cfg("upscale_downscale_first", bool),
+            }
+            if not self.cfg("just_use_yaml", bool)
+            else {"src_img": img_to_b64(src_img)}
+        )
+        self.post("upscale", params, cb)
+
+    def post_interrupt(self, cb):
+        # get official API url
+        url = get_url(self.cfg, prefix=OFFICIAL_ROUTE_PREFIX)
+        self.post("interrupt", {}, cb, base_url=url)
+
+    def get_progress(self, cb):
+        # get official API url
+        url = get_url(self.cfg, prefix=OFFICIAL_ROUTE_PREFIX)
+        self.get("progress", cb, base_url=url)

frontends/krita/krita_diff/config.py

@@ -0,0 +1,87 @@
+from dataclasses import asdict
+from typing import Any
+
+from krita import QObject, QReadWriteLock, QSettings
+
+from .defaults import CFG_FOLDER, CFG_NAME, DEFAULTS, ERR_MISSING_CONFIG
+
+
+class Config(QObject):
+    def __init__(self, folder=CFG_FOLDER, name=CFG_NAME, model=DEFAULTS):
+        """Sorta like a controller for QSettings.
+
+        I'm going to treat this as a singleton global app state, but implemented
+        correctly such that it should be theoretically possible to have multiple
+        instances (maybe multiple dockers controlling multiple remotes?)
+
+        If model is None, Config will not check if keys exist.
+
+        Args:
+            folder (str, optional): Which folder to store settings in. Defaults to CFG_FOLDER.
+            name (str, optional): Name of settings file. Defaults to CFG_NAME.
+            model (Any, optional): Data model representing config & defaults. Defaults to DEFAULTS.
+        """
+        # See: https://doc.qt.io/qt-6/qsettings.html#accessing-settings-from-multiple-threads-or-processes-simultaneously
+        # but im too lazy to figure out creating separate QSettings per worker, so we will just lock
+        super(Config, self).__init__()
+        self.model = model  # is immutable
+        self.lock = QReadWriteLock()
+        self.config = QSettings(QSettings.IniFormat, QSettings.UserScope, folder, name)
+
+        # add in new config settings
+        self.restore_defaults(overwrite=False)
+
+    def __call__(self, key: str, type: type = str):
+        """Shorthand for Config.get()"""
+        return self.get(key, type)
+
+    def get(self, key: str, type: type = str):
+        """Get config value by key & cast to type.
+
+        Args:
+            key (str): Name of config option.
+            type (type, optional): Type to cast config value to. Defaults to str.
+
+        Returns:
+            Any: Config value.
+        """
+        self.lock.lockForRead()
+        try:
+            # notably QSettings assume strings too unless specified
+            if self.model is not None:
+                assert self.config.contains(key) and hasattr(
+                    self.model, key
+                ), ERR_MISSING_CONFIG
+            val = self.config.value(key, type=type)
+            return val
+        finally:
+            self.lock.unlock()
+
+    def set(self, key: str, val: Any, overwrite: bool = True):
+        """Set config value by key.
+
+        Args:
+            key (str): Name of config option.
+            val (Any): Config value.
+            overwrite (bool, optional): Whether to overwrite an existing value. Defaults to False.
+        """
+        self.lock.lockForWrite()
+        try:
+            if self.model is not None:
+                assert hasattr(self.model, key), ERR_MISSING_CONFIG
+            if overwrite or not self.config.contains(key):
+                self.config.setValue(key, val)
+        finally:
+            self.lock.unlock()
+
+    def restore_defaults(self, overwrite: bool = True):
+        """Reset settings to default.
+
+        Args:
+            overwrite (bool, optional): Whether to overwrite existing settings, else add only new ones. Defaults to True.
+        """
+        if self.model is None:
+            return
+        defaults = asdict(self.model)
+        for k, v in defaults.items():
+            self.set(k, v, overwrite)

frontends/krita/krita_diff/defaults.py

@@ -0,0 +1,127 @@
+from dataclasses import dataclass, field
+from typing import List
+
+# set combo box to error msg instead of blank when cannot retrieve options from backend
+ERROR_MSG = "Retrieval Failed"
+
+# Used for status bar
+STATE_READY = "Ready"
+STATE_INIT = "Errors will be shown here"
+STATE_URLERROR = "Network error"
+STATE_RESET_DEFAULT = "All settings reset"
+STATE_WAIT = "Please wait..."
+STATE_DONE = "Done!"
+STATE_INTERRUPT = "Interrupted!"
+
+# Other currently hardcoded stuff
+SHORT_TIMEOUT = 10
+LONG_TIMEOUT = None  # requests that might take "forever", i.e., image generation with high batch count
+REFRESH_INTERVAL = 3000  # 3 seconds between auto-config refresh
+ETA_REFRESH_INTERVAL = 1000  # 1 second between eta refresh
+CFG_FOLDER = "krita"  # which folder in ~/.config to store config
+CFG_NAME = "krita_diff_plugin"  # name of config file
+EXT_CFG_NAME = "krita_diff_plugin_scripts"  # name of config file
+# selection mask can only be added after image is added, so timeout is needed
+ADD_MASK_TIMEOUT = 200
+THREADED = True
+ROUTE_PREFIX = "/sdapi/interpause/"
+OFFICIAL_ROUTE_PREFIX = "/sdapi/v1/"
+
+# error messages
+ERR_MISSING_CONFIG = "Report this bug, developer missed out a config key somewhere."
+ERR_NO_DOCUMENT = "No document open yet!"
+ERR_NO_CONNECTION = "Cannot reach backend!"
+ERR_BAD_URL = "Invalid backend URL!"
+
+# tab IDs
+TAB_SDCOMMON = "krita_diff_sdcommon"
+TAB_CONFIG = "krita_diff_config"
+TAB_TXT2IMG = "krita_diff_txt2img"
+TAB_IMG2IMG = "krita_diff_img2img"
+TAB_INPAINT = "krita_diff_inpaint"
+TAB_UPSCALE = "krita_diff_upscale"
+
+
+@dataclass(frozen=True)
+class Defaults:
+    base_url: str = "http://127.0.0.1:7860"
+    encryption_key: str = ""
+    just_use_yaml: bool = False
+    create_mask_layer: bool = True
+    save_temp_images: bool = False
+    fix_aspect_ratio: bool = True
+    only_full_img_tiling: bool = True
+    filter_nsfw: bool = False
+    do_exact_steps: bool = True
+    sample_path: str = "."
+    minimize_ui: bool = False
+    first_setup: bool = True  # only used for the initial docker layout
+    alt_dock_behavior: bool = False
+    hide_layers: bool = True
+
+    sd_model_list: List[str] = field(default_factory=lambda: [ERROR_MSG])
+    sd_model: str = "model.ckpt"
+    sd_batch_size: int = 1
+    sd_batch_count: int = 1
+    sd_base_size: int = 512
+    sd_max_size: int = 768
+    sd_tiling: bool = False
+    upscaler_list: List[str] = field(default_factory=lambda: [ERROR_MSG])
+    upscaler_name: str = "None"
+    face_restorer_model_list: List[str] = field(default_factory=lambda: [ERROR_MSG])
+    face_restorer_model: str = "None"
+    codeformer_weight: float = 0.5
+    include_grid: bool = False
+
+    txt2img_prompt: str = ""
+    txt2img_negative_prompt: str = ""
+    txt2img_sampler_list: List[str] = field(default_factory=lambda: [ERROR_MSG])
+    txt2img_sampler: str = "Euler a"
+    txt2img_steps: int = 20
+    txt2img_cfg_scale: float = 7.0
+    txt2img_denoising_strength: float = 0.7
+    txt2img_seed: str = ""
+    txt2img_highres: bool = False
+    txt2img_script: str = "None"
+    txt2img_script_list: List[str] = field(default_factory=lambda: [ERROR_MSG])
+    # TODO: Seed variation
+
+    img2img_prompt: str = ""
+    img2img_negative_prompt: str = ""
+    img2img_sampler_list: List[str] = field(default_factory=lambda: [ERROR_MSG])
+    img2img_sampler: str = "Euler a"
+    img2img_steps: int = 40
+    img2img_cfg_scale: float = 12.0
+    img2img_denoising_strength: float = 0.8
+    img2img_seed: str = ""
+    img2img_color_correct: bool = False
+    img2img_script: str = "None"
+    img2img_script_list: List[str] = field(default_factory=lambda: [ERROR_MSG])
+
+    inpaint_prompt: str = ""
+    inpaint_negative_prompt: str = ""
+    inpaint_sampler_list: List[str] = field(default_factory=lambda: [ERROR_MSG])
+    inpaint_sampler: str = "LMS"
+    inpaint_steps: int = 100
+    inpaint_cfg_scale: float = 5.0
+    inpaint_denoising_strength: float = 0.40
+    inpaint_seed: str = ""
+    inpaint_invert_mask: bool = False
+    # inpaint_mask_blur: int = 4
+    inpaint_fill_list: List[str] = field(
+        # NOTE: list order corresponds to number to use in internal API!!!
+        default_factory=lambda: ["blur", "preserve", "latent noise", "latent empty"]
+    )
+    inpaint_fill: str = "preserve"
+    # inpaint_full_res: bool = False
+    # inpaint_full_res_padding: int = 32
+    inpaint_color_correct: bool = False
+    inpaint_script: str = "None"
+    inpaint_script_list: List[str] = field(default_factory=lambda: [ERROR_MSG])
+    inpaint_mask_weight: float = 1.0
+
+    upscale_upscaler_name: str = "None"
+    upscale_downscale_first: bool = False
+
+
+DEFAULTS = Defaults()

frontends/krita/krita_diff/docker.py

@@ -0,0 +1,34 @@
+from krita import DockWidget, QScrollArea
+
+from .script import script
+from .style import style
+
+
+def create_docker(page):
+    class Docker(DockWidget):
+        def __init__(self, *args, **kwargs):
+            super(Docker, self).__init__(*args, **kwargs)
+            self.setWindowTitle(page.name)
+            self.create_interface()
+            self.update_interface()
+            self.connect_interface()
+            self.setWidget(self.widget)
+
+        def create_interface(self):
+            self.page_widget = page()
+            self.widget = QScrollArea()
+            self.widget.setStyleSheet(style)
+            self.widget.setWidget(self.page_widget)
+            self.widget.setWidgetResizable(True)
+
+        def update_interface(self):
+            self.page_widget.cfg_init()
+
+        def connect_interface(self):
+            self.page_widget.cfg_connect()
+            script.config_updated.connect(lambda: self.update_interface())
+
+        def canvasChanged(self, canvas):
+            pass
+
+    return Docker

frontends/krita/krita_diff/extension.py

@@ -0,0 +1,63 @@
+from krita import Extension, QMainWindow, QTimer
+
+from .defaults import REFRESH_INTERVAL
+from .script import script
+
+
+class SDPluginExtension(Extension):
+    def __init__(self, instance):
+        super().__init__(instance)
+
+        self.instance = instance
+        # store original window docker config
+        self.dock_opts = None
+
+    def setup(self):
+        self.update_timer = QTimer()
+        self.update_timer.timeout.connect(lambda: script.action_update_config())
+        self.update_timer.start(REFRESH_INTERVAL)
+        script.config_updated.connect(lambda: self.update_global())
+        self.instance.notifier().windowCreated.connect(lambda: self.update_global())
+        script.action_update_config()
+
+    def update_global(self):
+        window = self.instance.activeWindow()
+        if not window:
+            return
+        qwin = window.qwindow()
+        if not self.dock_opts:
+            self.dock_opts = qwin.dockOptions()
+
+        # NOTE: This changes the default behaviour of Krita for all dockers!
+        if script.cfg("alt_dock_behavior", bool):
+            qwin.setDockOptions(
+                QMainWindow.AnimatedDocks
+                | QMainWindow.AllowTabbedDocks
+                | QMainWindow.GroupedDragging
+                | QMainWindow.AllowNestedDocks
+                # | QMainWindow.VerticalTabs
+            )
+        else:
+            qwin.setDockOptions(self.dock_opts)
+
+    def createActions(self, window):
+        txt2img_action = window.createAction(
+            "txt2img", "Apply txt2img", "tools/scripts"
+        )
+        txt2img_action.triggered.connect(lambda: script.action_txt2img())
+        img2img_action = window.createAction(
+            "img2img", "Apply img2img", "tools/scripts"
+        )
+        img2img_action.triggered.connect(lambda: script.action_img2img())
+        upscale_x_action = window.createAction(
+            "img2img_upscale", "Apply img2img upscale", "tools/scripts"
+        )
+        upscale_x_action.triggered.connect(lambda: script.action_sd_upscale())
+        upscale_x_action = window.createAction(
+            "img2img_inpaint", "Apply img2img inpaint", "tools/scripts"
+        )
+        upscale_x_action.triggered.connect(lambda: script.action_inpaint())
+        simple_upscale_action = window.createAction(
+            "simple_upscale", "Apply simple upscaler", "tools/scripts"
+        )
+        simple_upscale_action.triggered.connect(lambda: script.action_simple_upscale())

frontends/krita/krita_diff/krita_diff.action

@@ -0,0 +1,69 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<ActionCollection version="2" name="Scripts">
+    <Actions category="Scripts">
+        <text>Stable Diffusion Plugin</text>
+
+        <Action name="txt2img">
+            <icon/>
+            <text>Apply txt2img transform</text>
+            <whatsThis/>
+            <toolTip/>
+            <iconText/>
+            <activationFlags>10000</activationFlags>
+            <activationConditions>0</activationConditions>
+            <shortcut>ctrl+alt+q</shortcut>
+            <isCheckable>false</isCheckable>
+            <statusTip/>
+        </Action>
+        <Action name="img2img">
+            <icon/>
+            <text>Apply img2img transform</text>
+            <whatsThis/>
+            <toolTip/>
+            <iconText/>
+            <activationFlags>10000</activationFlags>
+            <activationConditions>0</activationConditions>
+            <shortcut>ctrl+alt+w</shortcut>
+            <isCheckable>false</isCheckable>
+            <statusTip/>
+        </Action>
+        <!--
+        <Action name="img2img_upscale">
+            <icon/>
+            <text>Apply SD upscale transform</text>
+            <whatsThis/>
+            <toolTip/>
+            <iconText/>
+            <activationFlags>10000</activationFlags>
+            <activationConditions>0</activationConditions>
+            <shortcut>ctrl+alt+e</shortcut>
+            <isCheckable>false</isCheckable>
+            <statusTip/>
+        </Action>
+        -->
+        <Action name="img2img_inpaint">
+            <icon/>
+            <text>Apply inpaint transform</text>
+            <whatsThis/>
+            <toolTip/>
+            <iconText/>
+            <activationFlags>10000</activationFlags>
+            <activationConditions>0</activationConditions>
+            <shortcut>ctrl+alt+r</shortcut>
+            <isCheckable>false</isCheckable>
+            <statusTip/>
+        </Action>
+        <Action name="simple_upscale">
+            <icon/>
+            <text>Apply ESRGAN upscaler</text>
+            <whatsThis/>
+            <toolTip/>
+            <iconText/>
+            <activationFlags>10000</activationFlags>
+            <activationConditions>0</activationConditions>
+            <shortcut>ctrl+alt+t</shortcut>
+            <isCheckable>false</isCheckable>
+            <statusTip/>
+        </Action>
+    </Actions>
+</ActionCollection>

frontends/krita/krita_diff/manual.html

@@ -0,0 +1,15 @@
+<h1>Stable Diffusion Plugin</h1>
+
+<p>Expose the power of AUTOMATIC1111's Stable Diffusion fork for creating to your heart's desire.</p>
+
+<p>Forked from <a href="https://github.com/sddebz/stable-diffusion-krita-plugin"
+        target="_blank">https://github.com/sddebz/stable-diffusion-krita-plugin</a>,
+    which is based on the feature-rich and performant <a href="https://github.com/AUTOMATIC1111/stable-diffusion-webui"
+        target="_blank">AUTOMATIC1111's Fork</a>.
+</p>
+
+<p>Usage guide at <a href="https://github.com/Interpause/auto-sd-paint-ext/wiki"
+        target="_blank">https://github.com/Interpause/auto-sd-paint-ext/wiki</a>.</p>
+<p>Report bugs & suggest features at <a href="https://github.com/Interpause/auto-sd-paint-ext/issues"
+        target="_blank">https://github.com/Interpause/auto-sd-paint-ext/issues</a>.
+</p>

frontends/krita/krita_diff/pages/__init__.py

@@ -0,0 +1,6 @@
+from .common import SDCommonPage
+from .config import ConfigPage
+from .img2img import Img2ImgPage
+from .inpaint import InpaintPage
+from .txt2img import Txt2ImgPage
+from .upscale import UpscalePage

frontends/krita/krita_diff/pages/common.py

@@ -0,0 +1,123 @@
+from krita import QHBoxLayout, QPushButton, QVBoxLayout, QWidget
+
+from ..script import script
+from ..widgets import QCheckBox, QComboBoxLayout, QLabel, QSpinBoxLayout
+
+# Notes:
+# - move tiling mode to config?
+# - move upscaler/face restorer to config?
+
+
+class SDCommonPage(QWidget):
+    name = "SD Common Options"
+
+    def __init__(self, *args, **kwargs):
+        super(SDCommonPage, self).__init__(*args, **kwargs)
+
+        self.title = QLabel("<em>Quick Config</em>")
+
+        # Model list
+        self.sd_model_layout = QComboBoxLayout(
+            script.cfg, "sd_model_list", "sd_model", label="SD model:"
+        )
+
+        # batch size & count
+        self.batch_count_layout = QSpinBoxLayout(
+            script.cfg, "sd_batch_count", label="Batch count:", min=1, max=9999, step=1
+        )
+        self.batch_size_layout = QSpinBoxLayout(
+            script.cfg, "sd_batch_size", label="Batch size:", min=1, max=9999, step=1
+        )
+        batch_layout = QHBoxLayout()
+        batch_layout.addLayout(self.batch_count_layout)
+        batch_layout.addLayout(self.batch_size_layout)
+
+        # base/max size adjustment
+        self.base_size_layout = QSpinBoxLayout(
+            script.cfg, "sd_base_size", label="Base size:", min=64, max=8192, step=64
+        )
+        self.max_size_layout = QSpinBoxLayout(
+            script.cfg, "sd_max_size", label="Max size:", min=64, max=8192, step=64
+        )
+        size_layout = QHBoxLayout()
+        size_layout.addLayout(self.base_size_layout)
+        size_layout.addLayout(self.max_size_layout)
+
+        # global upscaler
+        self.upscaler_layout = QComboBoxLayout(
+            script.cfg, "upscaler_list", "upscaler_name", label="Upscaler:"
+        )
+
+        # Restore faces
+        self.face_restorer_layout = QComboBoxLayout(
+            script.cfg,
+            "face_restorer_model_list",
+            "face_restorer_model",
+            label="Face restorer:",
+        )
+        self.codeformer_weight_layout = QSpinBoxLayout(
+            script.cfg,
+            "codeformer_weight",
+            label="CodeFormer weight (max 0, min 1):",
+            step=0.01,
+        )
+
+        # Tiling mode
+        self.tiling = QCheckBox(script.cfg, "sd_tiling", "Tiling mode")
+
+        # Interrupt button
+        self.interrupt_btn = QPushButton("Interrupt")
+
+        layout = QVBoxLayout()
+        layout.setContentsMargins(0, 0, 0, 0)
+
+        layout.addWidget(self.title)
+        layout.addLayout(self.upscaler_layout)
+        layout.addLayout(self.face_restorer_layout)
+        layout.addLayout(self.codeformer_weight_layout)
+        layout.addWidget(self.tiling)
+        layout.addLayout(self.sd_model_layout)
+        layout.addLayout(batch_layout)
+        layout.addLayout(size_layout)
+        layout.addWidget(self.interrupt_btn)
+        layout.addStretch()
+
+        self.setLayout(layout)
+
+    def cfg_init(self):
+        self.sd_model_layout.cfg_init()
+        self.batch_count_layout.cfg_init()
+        self.batch_size_layout.cfg_init()
+        self.base_size_layout.cfg_init()
+        self.max_size_layout.cfg_init()
+        self.upscaler_layout.cfg_init()
+        self.face_restorer_layout.cfg_init()
+        self.codeformer_weight_layout.cfg_init()
+        self.tiling.cfg_init()
+
+        self.title.setVisible(not script.cfg("minimize_ui", bool))
+
+    def cfg_connect(self):
+        self.sd_model_layout.cfg_connect()
+        self.batch_count_layout.cfg_connect()
+        self.batch_size_layout.cfg_connect()
+        self.base_size_layout.cfg_connect()
+        self.max_size_layout.cfg_connect()
+        self.upscaler_layout.cfg_connect()
+        self.face_restorer_layout.cfg_connect()
+        self.codeformer_weight_layout.cfg_connect()
+        self.tiling.cfg_connect()
+
+        # Hide codeformer_weight when model isnt codeformer
+        def toggle_codeformer_weights(visible):
+            self.codeformer_weight_layout.qspin.setVisible(visible)
+            self.codeformer_weight_layout.qlabel.setVisible(visible)
+
+        self.face_restorer_layout.qcombo.currentTextChanged.connect(
+            lambda t: toggle_codeformer_weights(t == "CodeFormer")
+        )
+        toggle_codeformer_weights(
+            self.face_restorer_layout.qcombo.currentText() == "CodeFormer"
+        )
+
+        self.interrupt_btn.released.connect(lambda: script.action_interrupt())

frontends/krita/krita_diff/pages/config.py

@@ -0,0 +1,182 @@
+from functools import partial
+
+from krita import QHBoxLayout, QLineEdit, QPushButton, QVBoxLayout, QWidget
+
+from ..defaults import DEFAULTS
+from ..script import script
+from ..utils import reset_docker_layout
+from ..widgets import QCheckBox, QLabel, QLineEditLayout, StatusBar
+
+
+class ConfigPage(QWidget):
+    name = "SD Plugin Config"
+
+    def __init__(self, *args, **kwargs):
+        super(ConfigPage, self).__init__(*args, **kwargs)
+
+        self.status_bar = StatusBar()
+
+        self.base_url = QLineEdit()
+        self.base_url_reset = QPushButton("Default")
+        inline1 = QHBoxLayout()
+        inline1.addWidget(self.base_url)
+        inline1.addWidget(self.base_url_reset)
+
+        self.enc_key = QLineEditLayout(
+            script.cfg, "encryption_key", "Optional Encryption Key"
+        )
+
+        # Plugin settings
+        self.just_use_yaml = QCheckBox(
+            script.cfg, "just_use_yaml", "(unrecommended) Ignore settings"
+        )
+        self.create_mask_layer = QCheckBox(
+            script.cfg, "create_mask_layer", "Use selection as mask"
+        )
+        self.save_temp_images = QCheckBox(
+            script.cfg, "save_temp_images", "Save images for debug"
+        )
+        self.fix_aspect_ratio = QCheckBox(
+            script.cfg, "fix_aspect_ratio", "Adjust selection aspect ratio"
+        )
+        self.only_full_img_tiling = QCheckBox(
+            script.cfg, "only_full_img_tiling", "Disallow tiling with selection"
+        )
+        self.include_grid = QCheckBox(
+            script.cfg, "include_grid", "Include txt2img/img2img grid"
+        )
+        self.minimize_ui = QCheckBox(script.cfg, "minimize_ui", "Squeeze the UI")
+        self.alt_docker = QCheckBox(
+            script.cfg, "alt_dock_behavior", "Alt Docker Behaviour"
+        )
+        self.hide_layers = QCheckBox(script.cfg, "hide_layers", "Auto hide layers")
+
+        # webUI/backend settings
+        self.filter_nsfw = QCheckBox(script.cfg, "filter_nsfw", "Filter NSFW")
+        self.img2img_color_correct = QCheckBox(
+            script.cfg, "img2img_color_correct", "Color correct img2img"
+        )
+        self.inpaint_color_correct = QCheckBox(
+            script.cfg, "inpaint_color_correct", "Color correct inpaint"
+        )
+        self.do_exact_steps = QCheckBox(
+            script.cfg,
+            "do_exact_steps",
+            "Exact number of steps for denoising",
+        )
+
+        self.refresh_btn = QPushButton("Auto-Refresh Options Now")
+        self.restore_defaults = QPushButton("Restore Defaults")
+
+        self.info_label = QLabel()
+        self.info_label.setOpenExternalLinks(True)
+        self.info_label.setWordWrap(True)
+
+        # scroll_area = QScrollArea()
+
+        layout = QVBoxLayout()
+        layout.setContentsMargins(0, 0, 0, 0)
+        layout_inner = QVBoxLayout()
+        layout_inner.setContentsMargins(0, 0, 0, 0)
+
+        layout_inner.addWidget(QLabel("<em>Plugin settings:</em>"))
+        layout_inner.addWidget(self.minimize_ui)
+        layout_inner.addWidget(self.alt_docker)
+        layout_inner.addWidget(self.hide_layers)
+        layout_inner.addWidget(self.create_mask_layer)
+        layout_inner.addWidget(self.fix_aspect_ratio)
+        layout_inner.addWidget(self.only_full_img_tiling)
+        layout_inner.addWidget(self.include_grid)
+        layout_inner.addWidget(self.save_temp_images)
+        # layout_inner.addWidget(self.just_use_yaml)
+
+        layout_inner.addWidget(QLabel("<em>Backend/webUI settings:</em>"))
+        layout_inner.addWidget(self.filter_nsfw)
+        layout_inner.addWidget(self.img2img_color_correct)
+        layout_inner.addWidget(self.inpaint_color_correct)
+        layout_inner.addWidget(self.do_exact_steps)
+
+        # TODO: figure out how to set height of scroll area when there are too many options
+        # or maybe an option search bar
+        # scroll_area.setLayout(layout_inner)
+        # scroll_area.setWidgetResizable(True)
+        # layout.addWidget(scroll_area)
+        layout.addWidget(self.status_bar)
+        layout.addWidget(QLabel("<em>Backend url:</em>"))
+        layout.addLayout(inline1)
+        layout.addLayout(self.enc_key)
+        layout.addLayout(layout_inner)
+        layout.addWidget(self.refresh_btn)
+        layout.addWidget(self.restore_defaults)
+        layout.addWidget(self.info_label)
+        layout.addStretch()
+
+        self.setLayout(layout)
+
+    def cfg_init(self):
+        # NOTE: update timer -> cfg_init, setText seems to reset cursor position so we prevent it
+        base_url = script.cfg("base_url", str)
+        if self.base_url.text() != base_url:
+            self.base_url.setText(base_url)
+
+        self.enc_key.cfg_init()
+        self.just_use_yaml.cfg_init()
+        self.create_mask_layer.cfg_init()
+        self.save_temp_images.cfg_init()
+        self.fix_aspect_ratio.cfg_init()
+        self.only_full_img_tiling.cfg_init()
+        self.include_grid.cfg_init()
+        self.filter_nsfw.cfg_init()
+        self.img2img_color_correct.cfg_init()
+        self.inpaint_color_correct.cfg_init()
+        self.do_exact_steps.cfg_init()
+        self.minimize_ui.cfg_init()
+        self.alt_docker.cfg_init()
+        self.hide_layers.cfg_init()
+
+        info_text = """
+            <em>Tip:</em> Only a selected few backend/webUI settings are exposed above.<br/>
+            <em>Tip:</em> You should look through & configure all the backend/webUI settings at least once.
+            <br/><br/>
+            <a href="http://127.0.0.1:7860/" target="_blank">Configure all settings in webUI</a><br/>
+            <a href="https://github.com/Interpause/auto-sd-paint-ext/wiki" target="_blank">Read the guide</a><br/>
+            <a href="https://github.com/Interpause/auto-sd-paint-ext/issues" target="_blank">Report bugs or suggest features</a>
+            """
+        if script.cfg("minimize_ui", bool):
+            info_text = "\n".join(info_text.split("\n")[-4:-1])
+        self.info_label.setText(info_text)
+
+    def cfg_connect(self):
+        self.base_url.textChanged.connect(partial(script.cfg.set, "base_url"))
+        # NOTE: this triggers on every keystroke; theres no focus lost signal...
+        self.base_url.textChanged.connect(lambda: script.action_update_config())
+        self.base_url_reset.released.connect(
+            lambda: self.base_url.setText(DEFAULTS.base_url)
+        )
+        self.enc_key.cfg_connect()
+        self.just_use_yaml.cfg_connect()
+        self.create_mask_layer.cfg_connect()
+        self.save_temp_images.cfg_connect()
+        self.fix_aspect_ratio.cfg_connect()
+        self.only_full_img_tiling.cfg_connect()
+        self.include_grid.cfg_connect()
+        self.filter_nsfw.cfg_connect()
+        self.img2img_color_correct.cfg_connect()
+        self.inpaint_color_correct.cfg_connect()
+        self.do_exact_steps.cfg_connect()
+        self.minimize_ui.cfg_connect()
+        self.alt_docker.cfg_connect()
+        self.hide_layers.cfg_connect()
+
+        def restore_defaults():
+            script.restore_defaults()
+            reset_docker_layout()
+            script.cfg.set("first_setup", False)
+            # retrieve list of available stuff again
+            script.action_update_config()
+
+        self.refresh_btn.released.connect(lambda: script.action_update_config())
+        self.restore_defaults.released.connect(restore_defaults)
+        self.minimize_ui.toggled.connect(lambda _: script.config_updated.emit())
+        self.alt_docker.toggled.connect(lambda _: script.config_updated.emit())
+        script.status_changed.connect(lambda s: self.status_bar.set_status(s))

frontends/krita/krita_diff/pages/extension.py

@@ -0,0 +1,135 @@
+import json
+from functools import partial
+from typing import List
+
+from krita import QVBoxLayout, QWidget
+
+from ..config import Config
+from ..script import script
+from ..utils import get_ext_key
+from ..widgets import (
+    QCheckBox,
+    QComboBoxLayout,
+    QLineEditLayout,
+    QMultiCheckBoxLayout,
+    QSpinBoxLayout,
+)
+
+
+# TODO: dynamically adjust script options available without needing to restart plugin
+class ExtWidget(QWidget):
+    def __init__(self, ext_cfg: Config, ext_type: str, ext_name: str, *args, **kwargs):
+        """Dynamically create form for script based on metadata.
+
+        Args:
+            ext_cfg (Config): Config object to get metadata from.
+            ext_type (str): Whether metadata is in "scripts_txt2img", "scripts_img2img" or "scripts_inpaint"
+            ext_name (str): Name of script.
+        """
+        super(ExtWidget, self).__init__(*args, **kwargs)
+
+        get_key = partial(get_ext_key, ext_type, ext_name)
+
+        try:
+            meta: List[dict] = json.loads(ext_cfg(get_key()))
+        except json.JSONDecodeError:
+            meta = []
+            print(f"Script metadata is invalid: {ext_cfg(get_key())}")
+
+        layout = QVBoxLayout()
+        self.widgets = []
+        for i, o in enumerate(meta):
+            w = None
+            k = get_key(i)
+            if o["type"] == "range":
+                w = QSpinBoxLayout(
+                    ext_cfg,
+                    k,
+                    label=o["label"],
+                    min=o["min"],
+                    max=o["max"],
+                    step=o["step"],
+                )
+            elif o["type"] == "combo":
+                w = QComboBoxLayout(ext_cfg, o["opts"], k, label=o["label"])
+            elif o["type"] == "text":
+                w = QLineEditLayout(ext_cfg, k, o["label"])
+            elif o["type"] == "checkbox":
+                w = QCheckBox(ext_cfg, k, o["label"])
+            elif o["type"] == "multiselect":
+                w = QMultiCheckBoxLayout(ext_cfg, o["opts"], k, o["label"])
+            else:
+                continue
+            self.widgets.append(w)
+            if isinstance(w, QWidget):
+                layout.addWidget(w)
+            else:
+                layout.addLayout(w)
+        self.setLayout(layout)
+
+    def cfg_init(self):
+        for w in self.widgets:
+            w.cfg_init()
+
+    def cfg_connect(self):
+        for w in self.widgets:
+            w.cfg_connect()
+
+
+class ExtSectionLayout(QVBoxLayout):
+    def __init__(self, cfg_prefix: str, *args, **kwargs):
+        super(ExtSectionLayout, self).__init__(*args, **kwargs)
+
+        # NOTE: backend will send empty scripts followed by the real one, have to
+        # detect for that
+        self.is_init = False
+
+        self.dropdown = QComboBoxLayout(
+            script.cfg,
+            f"{cfg_prefix}_script_list",
+            f"{cfg_prefix}_script",
+            label="Scripts:",
+        )
+        self.addLayout(self.dropdown)
+
+        self.ext_type = f"scripts_{cfg_prefix}"
+        self.ext_names = partial(script.cfg, f"{cfg_prefix}_script_list", "QStringList")
+        self.ext_widgets = {}
+
+    def init_ui_once_if_ready(self):
+        """Init UI only once, and only when its ready (aka metadata is present)."""
+        if self.is_init:
+            return
+        if len(self.ext_names()) != script.ext_cfg(f"{self.ext_type}_len", int):
+            return
+
+        self.is_init = True
+        for ext_name in self.ext_names():
+            ext_widget = ExtWidget(script.ext_cfg, self.ext_type, ext_name)
+            ext_widget.setVisible(False)
+            self.addWidget(ext_widget)
+            self.ext_widgets[ext_name] = ext_widget
+        self._cfg_connect()
+
+    def cfg_init(self):
+        self.dropdown.cfg_init()
+        self.init_ui_once_if_ready()
+        for widget in self.ext_widgets.values():
+            widget.cfg_init()
+
+    def cfg_connect(self):
+        self.dropdown.cfg_connect()
+        self.init_ui_once_if_ready()
+        self.dropdown.qcombo.currentTextChanged.connect(lambda s: self._update(s))
+
+    def _update(self, selected):
+        for w in self.ext_widgets.values():
+            w.setVisible(False)
+        widget = self.ext_widgets.get(selected, None)
+        if widget and selected != "None":
+            widget.setVisible(True)
+
+    def _cfg_connect(self):
+        for widget in self.ext_widgets.values():
+            widget.cfg_connect()
+        self._update(self.dropdown.qcombo.currentText())

frontends/krita/krita_diff/pages/img2img.py

@@ -0,0 +1,31 @@
+from krita import QPushButton
+
+from ..script import script
+from ..widgets import TipsLayout
+from .img_base import SDImgPageBase
+
+
+class Img2ImgPage(SDImgPageBase):
+    name = "Img2Img"
+
+    def __init__(self, *args, **kwargs):
+        super(Img2ImgPage, self).__init__(cfg_prefix="img2img", *args, **kwargs)
+
+        self.btn = QPushButton("Start img2img")
+        self.tips = TipsLayout(
+            ["Select what you want the model to perform img2img on."]
+        )
+
+        self.layout.addLayout(self.denoising_strength_layout)
+        self.layout.addWidget(self.btn)
+        self.layout.addLayout(self.tips)
+        self.layout.addStretch()
+
+    def cfg_init(self):
+        super(Img2ImgPage, self).cfg_init()
+
+        self.tips.setVisible(not script.cfg("minimize_ui", bool))
+
+    def cfg_connect(self):
+        super(Img2ImgPage, self).cfg_connect()
+        self.btn.released.connect(lambda: script.action_img2img())

frontends/krita/krita_diff/pages/img_base.py

@@ -0,0 +1,90 @@
+from krita import QHBoxLayout, QVBoxLayout, QWidget
+
+from ..script import script
+from ..widgets import (
+    QComboBoxLayout,
+    QLineEditLayout,
+    QPromptLayout,
+    QSpinBoxLayout,
+    StatusBar,
+)
+from .extension import ExtSectionLayout
+
+
+class SDImgPageBase(QWidget):
+    def __init__(self, cfg_prefix: str, *args, **kwargs):
+        super(SDImgPageBase, self).__init__(*args, **kwargs)
+
+        self.status_bar = StatusBar()
+
+        self.prompt_layout = QPromptLayout(
+            script.cfg, f"{cfg_prefix}_prompt", f"{cfg_prefix}_negative_prompt"
+        )
+
+        self.seed_layout = QLineEditLayout(
+            script.cfg, f"{cfg_prefix}_seed", label="Seed:", placeholder="Random"
+        )
+
+        self.sampler_layout = QComboBoxLayout(
+            script.cfg,
+            f"{cfg_prefix}_sampler_list",
+            f"{cfg_prefix}_sampler",
+            label="Sampler:",
+        )
+
+        self.steps_layout = QSpinBoxLayout(
+            script.cfg, f"{cfg_prefix}_steps", label="Steps:", min=1, max=9999, step=1
+        )
+        self.cfg_scale_layout = QSpinBoxLayout(
+            script.cfg,
+            f"{cfg_prefix}_cfg_scale",
+            label="CFG scale:",
+            min=1.0,
+            max=9999.0,
+        )
+
+        self.ext_layout = ExtSectionLayout(cfg_prefix)
+
+        inline_layout = QHBoxLayout()
+        inline_layout.addLayout(self.steps_layout)
+        inline_layout.addLayout(self.cfg_scale_layout)
+
+        self.layout = layout = QVBoxLayout()
+        layout.setContentsMargins(0, 0, 0, 0)
+
+        layout.addWidget(self.status_bar)
+        layout.addLayout(self.ext_layout)
+        layout.addLayout(self.prompt_layout)
+        layout.addLayout(self.seed_layout)
+        layout.addLayout(self.sampler_layout)
+        layout.addLayout(inline_layout)
+
+        self.setLayout(layout)
+
+        # not added so inheritants can place it wherever they want
+        self.denoising_strength_layout = QSpinBoxLayout(
+            script.cfg,
+            f"{cfg_prefix}_denoising_strength",
+            label="Denoising strength:",
+            step=0.01,
+        )
+
+    def cfg_init(self):
+        self.ext_layout.cfg_init()
+        self.prompt_layout.cfg_init()
+        self.seed_layout.cfg_init()
+        self.sampler_layout.cfg_init()
+        self.steps_layout.cfg_init()
+        self.cfg_scale_layout.cfg_init()
+        self.denoising_strength_layout.cfg_init()
+
+    def cfg_connect(self):
+        self.ext_layout.cfg_connect()
+        self.prompt_layout.cfg_connect()
+        self.seed_layout.cfg_connect()
+        self.sampler_layout.cfg_connect()
+        self.steps_layout.cfg_connect()
+        self.cfg_scale_layout.cfg_connect()
+        self.denoising_strength_layout.cfg_connect()
+
+        script.status_changed.connect(lambda s: self.status_bar.set_status(s))

frontends/krita/krita_diff/pages/inpaint.py

@@ -0,0 +1,97 @@
+from krita import QHBoxLayout, QPushButton
+
+from ..script import script
+from ..widgets import QCheckBox, QComboBoxLayout, QSpinBoxLayout, TipsLayout
+from .img_base import SDImgPageBase
+
+
+class InpaintPage(SDImgPageBase):
+    name = "Inpaint"
+
+    def __init__(self, *args, **kwargs):
+        super(InpaintPage, self).__init__(cfg_prefix="inpaint", *args, **kwargs)
+        self.layout.addLayout(self.denoising_strength_layout)
+
+        self.invert_mask = QCheckBox(script.cfg, "inpaint_invert_mask", "Invert mask")
+        # self.mask_blur_layout = QSpinBoxLayout(
+        #     script.cfg, "inpaint_mask_blur", "Mask blur (px):", min=0, max=9999, step=1
+        # )
+        self.inpaint_mask_weight = QSpinBoxLayout(
+            script.cfg, "inpaint_mask_weight", "Mask weight:", step=0.01
+        )
+
+        inline1 = QHBoxLayout()
+        inline1.addWidget(self.invert_mask)
+        inline1.addLayout(self.inpaint_mask_weight)
+        # inline1.addLayout(self.mask_blur_layout)
+
+        self.fill_layout = QComboBoxLayout(
+            script.cfg, "inpaint_fill_list", "inpaint_fill", label="Inpaint fill:"
+        )
+
+        # self.full_res = QCheckBox(script.cfg, "inpaint_full_res", "Inpaint full res")
+        # self.full_res_padding_layout = QSpinBoxLayout(
+        #     script.cfg,
+        #     "inpaint_full_res_padding",
+        #     "Padding (px):",
+        #     min=0,
+        #     max=9999,
+        #     step=1,
+        # )
+
+        inline2 = QHBoxLayout()
+        # inline2.addWidget(self.full_res)
+        # inline2.addLayout(self.full_res_padding_layout)
+
+        self.tips = TipsLayout(
+            [
+                "Ensure the inpaint layer is selected.",
+                "Select what the model will see when inpainting. <em>Inpaint full res</em> is unnecessary.",
+            ]
+        )
+        self.tips2 = TipsLayout(
+            [
+                '<a href="https://github.com/Interpause/auto-sd-paint-ext/wiki/Usage-Guide#inpainting" target="_blank">Inpaint Full Res & Mask Blur is obsolete; Click for new method.</a>'
+            ],
+            prefix="",
+        )
+        self.btn = QPushButton("Start inpaint")
+
+        self.layout.addLayout(self.fill_layout)
+        self.layout.addLayout(inline1)
+        self.layout.addLayout(inline2)
+        self.layout.addWidget(self.btn)
+        self.layout.addLayout(self.tips2)
+        self.layout.addLayout(self.tips)
+        self.layout.addStretch()
+
+    def cfg_init(self):
+        super(InpaintPage, self).cfg_init()
+        # self.mask_blur_layout.cfg_init()
+        self.fill_layout.cfg_init()
+        self.inpaint_mask_weight.cfg_init()
+        # self.full_res_padding_layout.cfg_init()
+        self.invert_mask.cfg_init()
+        # self.full_res.cfg_init()
+
+        self.tips.setVisible(not script.cfg("minimize_ui", bool))
+
+    def cfg_connect(self):
+        super(InpaintPage, self).cfg_connect()
+        # self.mask_blur_layout.cfg_connect()
+        self.fill_layout.cfg_connect()
+        self.inpaint_mask_weight.cfg_connect()
+        # self.full_res_padding_layout.cfg_connect()
+
+        self.invert_mask.cfg_connect()
+
+        # def toggle_fullres(enabled):
+        #     # hide/show fullres padding
+        #     self.full_res_padding_layout.qlabel.setVisible(enabled)
+        #     self.full_res_padding_layout.qspin.setVisible(enabled)
+
+        # self.full_res.cfg_connect()
+        # self.full_res.toggled.connect(toggle_fullres)
+        # toggle_fullres(self.full_res.isChecked())
+
+        self.btn.released.connect(lambda: script.action_inpaint())

frontends/krita/krita_diff/pages/txt2img.py

@@ -0,0 +1,49 @@
+from krita import QHBoxLayout, QPushButton
+
+from ..script import script
+from ..widgets import QCheckBox, TipsLayout
+from .img_base import SDImgPageBase
+
+
+class Txt2ImgPage(SDImgPageBase):
+    name = "Txt2Img"
+
+    def __init__(self, *args, **kwargs):
+        super(Txt2ImgPage, self).__init__(cfg_prefix="txt2img", *args, **kwargs)
+
+        self.highres = QCheckBox(script.cfg, "txt2img_highres", "Highres fix")
+
+        inline_layout = QHBoxLayout()
+        inline_layout.addWidget(self.highres)
+        inline_layout.addLayout(self.denoising_strength_layout)
+
+        self.tips = TipsLayout(
+            ["Set base_size & max_size higher for AUTO's txt2img highres fix to work."]
+        )
+
+        self.btn = QPushButton("Start txt2img")
+
+        self.layout.addLayout(inline_layout)
+        self.layout.addWidget(self.btn)
+        self.layout.addLayout(self.tips)
+        self.layout.addStretch()
+
+    def cfg_init(self):
+        super(Txt2ImgPage, self).cfg_init()
+        self.highres.cfg_init()
+
+        self.tips.setVisible(not script.cfg("minimize_ui", bool))
+
+    def cfg_connect(self):
+        super(Txt2ImgPage, self).cfg_connect()
+
+        def toggle_highres(enabled):
+            # hide/show denoising strength
+            self.denoising_strength_layout.qlabel.setVisible(enabled)
+            self.denoising_strength_layout.qspin.setVisible(enabled)
+
+        self.highres.cfg_connect()
+        self.highres.toggled.connect(toggle_highres)
+        toggle_highres(self.highres.isChecked())
+
+        self.btn.released.connect(lambda: script.action_txt2img())

frontends/krita/krita_diff/pages/upscale.py

@@ -0,0 +1,60 @@
+from krita import QPushButton, QVBoxLayout, QWidget
+
+from ..script import script
+from ..widgets import QCheckBox, QComboBoxLayout, QLabel, StatusBar
+
+
+# TODO: Become SD Upscale tab.
+class UpscalePage(QWidget):
+    name = "Upscale"
+
+    def __init__(self, *args, **kwargs):
+        super(UpscalePage, self).__init__(*args, **kwargs)
+
+        self.status_bar = StatusBar()
+
+        self.upscaler_layout = QComboBoxLayout(
+            script.cfg, "upscaler_list", "upscale_upscaler_name", label="Upscaler:"
+        )
+
+        self.downscale_first = QCheckBox(
+            script.cfg,
+            "upscale_downscale_first",
+            "Downscale image x0.5 before upscaling",
+        )
+
+        self.note = QLabel(
+            """
+NOTE:<br/>
+ - txt2img & img2img will use the <em>Quick Config</em> Upscaler when needing to scale up.<br/>
+ - Upscaling manually is only useful if the image was resized via Krita.<br/>
+ - In the future, SD Upscaling will replace this tab! For now, use the WebUI.
+            """
+        )
+        self.note.setWordWrap(True)
+
+        self.btn = QPushButton("Start upscaling")
+
+        layout = QVBoxLayout()
+        layout.setContentsMargins(0, 0, 0, 0)
+
+        layout.addWidget(self.status_bar)
+        layout.addWidget(self.note)
+        layout.addLayout(self.upscaler_layout)
+        layout.addWidget(self.downscale_first)
+        layout.addWidget(self.btn)
+        layout.addStretch()
+
+        self.setLayout(layout)
+
+    def cfg_init(self):
+        self.upscaler_layout.cfg_init()
+        self.downscale_first.cfg_init()
+
+        self.note.setVisible(not script.cfg("minimize_ui", bool))
+
+    def cfg_connect(self):
+        self.upscaler_layout.cfg_connect()
+        self.downscale_first.cfg_connect()
+        self.btn.released.connect(lambda: script.action_simple_upscale())
+        script.status_changed.connect(lambda s: self.status_bar.set_status(s))

frontends/krita/krita_diff/script.py

@@ -0,0 +1,425 @@
+import itertools
+import os
+import time
+from typing import Union
+
+from krita import (
+    Document,
+    Krita,
+    Node,
+    QImage,
+    QObject,
+    Qt,
+    QTimer,
+    Selection,
+    pyqtSignal,
+)
+
+from .client import Client
+from .config import Config
+from .defaults import (
+    ADD_MASK_TIMEOUT,
+    ERR_NO_DOCUMENT,
+    ETA_REFRESH_INTERVAL,
+    EXT_CFG_NAME,
+    STATE_INTERRUPT,
+    STATE_RESET_DEFAULT,
+    STATE_WAIT,
+)
+from .utils import (
+    b64_to_img,
+    find_optimal_selection_region,
+    get_desc_from_resp,
+    img_to_ba,
+    save_img,
+)
+
+
+# Does it actually have to be a QObject?
+# The only possible use I see is for event emitting
+class Script(QObject):
+    cfg: Config
+    """config singleton"""
+    client: Client
+    """API client singleton"""
+    status: str
+    """Current status (shown in status bar)"""
+    app: Krita
+    """Krita's Application instance (KDE Application)"""
+    doc: Document
+    """Currently opened document if any"""
+    node: Node
+    """Currently selected layer in Krita"""
+    selection: Selection
+    """Selection region in Krita"""
+    x: int
+    """Left position of selection"""
+    y: int
+    """Top position of selection"""
+    width: int
+    """Width of selection"""
+    height: int
+    """Height of selection"""
+    status_changed = pyqtSignal(str)
+    config_updated = pyqtSignal()
+
+    def __init__(self):
+        super(Script, self).__init__()
+        # Persistent settings (should reload between Krita sessions)
+        self.cfg = Config()
+        # used for webUI scripts aka extensions not to be confused with their extensions
+        self.ext_cfg = Config(name=EXT_CFG_NAME, model=None)
+        self.client = Client(self.cfg, self.ext_cfg)
+        self.client.status.connect(self.status_changed.emit)
+        self.client.config_updated.connect(self.config_updated.emit)
+        self.eta_timer = QTimer()
+        self.eta_timer.setInterval(ETA_REFRESH_INTERVAL)
+        self.eta_timer.timeout.connect(lambda: self.action_update_eta())
+
+    def restore_defaults(self, if_empty=False):
+        """Restore to default config."""
+        self.cfg.restore_defaults(not if_empty)
+        self.ext_cfg.config.remove("")
+
+        if not if_empty:
+            self.status_changed.emit(STATE_RESET_DEFAULT)
+
+    def update_selection(self):
+        """Update references to key Krita objects as well as selection information."""
+        self.app = Krita.instance()
+        self.doc = self.app.activeDocument()
+
+        # self.doc doesnt exist at app startup
+        if not self.doc:
+            self.status_changed.emit(ERR_NO_DOCUMENT)
+            return
+
+        self.node = self.doc.activeNode()
+        self.selection = self.doc.selection()
+
+        is_not_selected = (
+            self.selection is None
+            or self.selection.width() < 1
+            or self.selection.height() < 1
+        )
+        if is_not_selected:
+            self.x = 0
+            self.y = 0
+            self.width = self.doc.width()
+            self.height = self.doc.height()
+            self.selection = None  # for the two other cases of invalid selection
+        else:
+            self.x = self.selection.x()
+            self.y = self.selection.y()
+            self.width = self.selection.width()
+            self.height = self.selection.height()
+
+        assert (
+            self.doc.colorDepth() == "U8"
+        ), f'Only "8-bit integer/channel" supported, Document Color Depth: {self.doc.colorDepth()}'
+        assert (
+            self.doc.colorModel() == "RGBA"
+        ), f'Only "RGB/Alpha" supported, Document Color Model: {self.doc.colorModel()}'
+
+    def adjust_selection(self):
+        """Adjust selection region to account for scaling and striding to prevent image stretch."""
+        if self.selection is not None and self.cfg("fix_aspect_ratio", bool):
+            x, y, width, height = find_optimal_selection_region(
+                self.cfg("sd_base_size", int),
+                self.cfg("sd_max_size", int),
+                self.x,
+                self.y,
+                self.width,
+                self.height,
+                self.doc.width(),
+                self.doc.height(),
+            )
+
+            self.x = x
+            self.y = y
+            self.width = width
+            self.height = height
+
+    def get_selection_image(self) -> QImage:
+        """QImage of selection"""
+        return QImage(
+            self.doc.pixelData(self.x, self.y, self.width, self.height),
+            self.width,
+            self.height,
+            QImage.Format_RGBA8888,
+        ).rgbSwapped()
+
+    def get_mask_image(self) -> Union[QImage, None]:
+        """QImage of mask layer for inpainting"""
+        if self.node.type() not in {"paintlayer", "filelayer"}:
+            return None
+
+        return QImage(
+            self.node.pixelData(self.x, self.y, self.width, self.height),
+            self.width,
+            self.height,
+            QImage.Format_RGBA8888,
+        ).rgbSwapped()
+
+    def img_inserter(self, x, y, width, height, group: str = None):
+        """Return frozen image inserter to insert images as new layer."""
+        # Selection may change before callback, so freeze selection region
+        has_selection = self.selection is not None
+        glayer = self.doc.createGroupLayer(group) if group else None
+
+        def create_layer(name: str):
+            """Create new layer in document or group"""
+            layer = self.doc.createNode(name, "paintLayer")
+            parent = self.doc.rootNode()
+            if glayer:
+                glayer.addChildNode(layer, None)
+                parent.addChildNode(glayer, None)
+            else:
+                parent.addChildNode(layer, None)
+            return layer
+
+        # TODO: Insert images inside a group layer for better organization
+        # Group layer name can contain model name, prompt, etc
+        def insert(layer_name, enc):
+            nonlocal x, y, width, height, has_selection
+            print(f"inserting layer {layer_name}")
+            print(f"data size: {len(enc)}")
+
+            # QImage.Format_RGB32 (4) is default format after decoding image
+            # QImage.Format_RGBA8888 (17) is format used in Krita tutorial
+            # both are compatible, & converting from 4 to 17 required a RGB swap
+            # Likewise for 5 & 18 (their RGBA counterparts)
+            image = b64_to_img(enc)
+            print(
+                f"image created: {image}, {image.width()}x{image.height()}, depth: {image.depth()}, format: {image.format()}"
+            )
+
+            # NOTE: Scaling is usually done by backend (although I am reconsidering this)
+            # The scaling here is for SD Upscale or Upscale on a selection region rather than whole image
+            # Image won't be scaled down ONLY if there is no selection; i.e. selecting whole image will scale down,
+            # not selecting anything won't scale down, leading to the canvas being resized afterwards
+            if has_selection and (image.width() != width or image.height() != height):
+                print(f"Rescaling image to selection: {width}x{height}")
+                image = image.scaled(
+                    width, height, transformMode=Qt.SmoothTransformation
+                )
+
+            # Resize (not scale!) canvas if image is larger (i.e. outpainting or Upscale was used)
+            if image.width() > self.doc.width() or image.height() > self.doc.height():
+                # NOTE:
+                # - user's selection will be partially ignored if image is larger than canvas
+                # - it is complex to scale/resize the image such that image fits in the newly scaled selection
+                # - the canvas will still be resized even if the image fits after transparency masking
+                print("Image is larger than canvas! Resizing...")
+                new_width, new_height = self.doc.width(), self.doc.height()
+                if image.width() > self.doc.width():
+                    x, width, new_width = 0, image.width(), image.width()
+                if image.height() > self.doc.height():
+                    y, height, new_height = 0, image.height(), image.height()
+                self.doc.resizeImage(0, 0, new_width, new_height)
+
+            ba = img_to_ba(image)
+            layer = create_layer(layer_name)
+            # layer.setColorSpace() doesn't pernamently convert layer depth etc...
+
+            # Don't fail silently for setPixelData(); fails if bit depth or number of channels mismatch
+            size = ba.size()
+            expected = layer.pixelData(x, y, width, height).size()
+            assert expected == size, f"Raw data size: {size}, Expected size: {expected}"
+
+            print(f"inserting at x: {x}, y: {y}, w: {width}, h: {height}")
+            layer.setPixelData(ba, x, y, width, height)
+            return layer
+
+        if glayer:
+            return insert, glayer
+        return insert
+
+    def apply_txt2img(self):
+        # freeze selection region
+        insert, glayer = self.img_inserter(
+            self.x, self.y, self.width, self.height, group="a"
+        )
+        mask_trigger = self.transparency_mask_inserter()
+
+        def cb(response):
+            if len(self.client.long_reqs) == 1:  # last request
+                self.eta_timer.stop()
+            assert response is not None, "Backend Error, check terminal"
+            outputs = response["outputs"]
+            glayer_name, layer_names = get_desc_from_resp(response, "txt2img")
+            layers = [
+                insert(name if name else f"txt2img {i + 1}", output)
+                for output, name, i in zip(outputs, layer_names, itertools.count())
+            ]
+            if self.cfg("hide_layers", bool):
+                for layer in layers[:-1]:
+                    layer.setVisible(False)
+            glayer.setName(glayer_name)
+            self.doc.refreshProjection()
+            mask_trigger(layers)
+
+        self.eta_timer.start(ETA_REFRESH_INTERVAL)
+        self.client.post_txt2img(
+            cb, self.width, self.height, self.selection is not None
+        )
+
+    def apply_img2img(self, mode):
+        insert, glayer = self.img_inserter(
+            self.x, self.y, self.width, self.height, group="a"
+        )
+        mask_trigger = self.transparency_mask_inserter()
+        mask_image = self.get_mask_image()
+
+        path = os.path.join(self.cfg("sample_path", str), f"{int(time.time())}.png")
+        mask_path = os.path.join(
+            self.cfg("sample_path", str), f"{int(time.time())}_mask.png"
+        )
+        if mode == 1 and mask_image is not None:
+            if self.cfg("save_temp_images", bool):
+                save_img(mask_image, mask_path)
+            # auto-hide mask layer before getting selection image
+            self.node.setVisible(False)
+            self.doc.refreshProjection()
+
+        sel_image = self.get_selection_image()
+        if self.cfg("save_temp_images", bool):
+            save_img(sel_image, path)
+
+        def cb(response):
+            if len(self.client.long_reqs) == 1:  # last request
+                self.eta_timer.stop()
+            assert response is not None, "Backend Error, check terminal"
+
+            outputs = response["outputs"]
+            layer_name_prefix = (
+                "inpaint" if mode == 1 else "sd upscale" if mode == 2 else "img2img"
+            )
+            glayer_name, layer_names = get_desc_from_resp(response, layer_name_prefix)
+            layers = [
+                insert(name if name else f"{layer_name_prefix} {i + 1}", output)
+                for output, name, i in zip(outputs, layer_names, itertools.count())
+            ]
+            if self.cfg("hide_layers", bool):
+                for layer in layers[:-1]:
+                    layer.setVisible(False)
+            glayer.setName(glayer_name)
+            self.doc.refreshProjection()
+            # dont need transparency mask for inpaint mode
+            if mode == 0:
+                mask_trigger(layers)
+
+        method = self.client.post_inpaint if mode == 1 else self.client.post_img2img
+        self.eta_timer.start()
+        method(
+            cb,
+            sel_image,
+            mask_image,  # is unused by backend in img2img mode
+            self.selection is not None,
+        )
+
+    def apply_simple_upscale(self):
+        insert = self.img_inserter(self.x, self.y, self.width, self.height)
+        sel_image = self.get_selection_image()
+
+        path = os.path.join(self.cfg("sample_path", str), f"{int(time.time())}.png")
+        if self.cfg("save_temp_images", bool):
+            save_img(sel_image, path)
+
+        def cb(response):
+            assert response is not None, "Backend Error, check terminal"
+            output = response["output"]
+            insert(f"upscale", output)
+            self.doc.refreshProjection()
+
+        self.client.post_upscale(cb, sel_image)
+
+    def transparency_mask_inserter(self):
+        """Mask out extra regions due to adjust_selection()."""
+        orig_selection = self.selection.duplicate() if self.selection else None
+
+        def add_mask(layers):
+            self.doc.waitForDone()
+            cur_selection = self.selection
+            cur_layer = self.doc.activeNode()
+            for layer in layers:
+                self.doc.setActiveNode(layer)
+                self.doc.setSelection(orig_selection)
+                self.app.action("add_new_transparency_mask").trigger()
+            self.doc.setSelection(cur_selection)  # reset to current selection
+            self.doc.setActiveNode(cur_layer)  # reset to current layer
+
+        def trigger_mask_adding(layers):
+            if self.cfg("create_mask_layer", bool):
+                # need timeout to ensure layer exists first else crash
+                QTimer.singleShot(ADD_MASK_TIMEOUT, lambda: add_mask(layers))
+
+        return trigger_mask_adding
+
+    # Actions
+    def action_txt2img(self):
+        self.status_changed.emit(STATE_WAIT)
+        self.update_selection()
+        if not self.doc:
+            return
+        self.adjust_selection()
+        self.apply_txt2img()
+
+    def action_img2img(self):
+        self.status_changed.emit(STATE_WAIT)
+        self.update_selection()
+        if not self.doc:
+            return
+        self.adjust_selection()
+        self.apply_img2img(mode=0)
+
+    def action_sd_upscale(self):
+        assert False, "disabled"
+        self.status_changed.emit(STATE_WAIT)
+        self.update_selection()
+        self.apply_img2img(mode=2)
+
+    def action_inpaint(self):
+        self.status_changed.emit(STATE_WAIT)
+        self.update_selection()
+        if not self.doc:
+            return
+        self.adjust_selection()
+        self.apply_img2img(mode=1)
+
+    def action_simple_upscale(self):
+        self.status_changed.emit(STATE_WAIT)
+        self.update_selection()
+        if not self.doc:
+            return
+        self.apply_simple_upscale()
+
+    def action_update_config(self):
+        """Update certain config/state from the backend."""
+        self.client.get_config()
+
+    def action_interrupt(self):
+        def cb(resp=None):
+            self.status_changed.emit(STATE_INTERRUPT)
+
+        self.client.post_interrupt(cb)
+
+    def action_update_eta(self):
+        def cb(resp=None):
+            # print(resp)
+            # NOTE: progress & eta_relative is bugged upstream when there is multiple jobs
+            # so we use a substitute that seems to work
+            state = resp["state"]
+            cur_step = state["sampling_step"]
+            total_steps = state["sampling_steps"]
+            # doesnt take into account batch count
+            num_jobs = len(self.client.long_reqs) - 1
+
+            self.status_changed.emit(
+                f"Step {cur_step}/{total_steps} ({num_jobs} in queue)"
+            )
+
+        self.client.get_progress(cb)
+
+
+script = Script()

frontends/krita/krita_diff/style.py

@@ -0,0 +1,8 @@
+# NOTE: see https://bugreports.qt.io/browse/QTBUG-22862
+# We cant set layout padding via stylesheet
+
+style = """
+QTabBar::tab {
+    padding: 4px 2px 4px 2px;
+}
+"""

frontends/krita/krita_diff/utils.py

@@ -0,0 +1,237 @@
+import json
+import re
+from itertools import cycle
+from math import ceil
+
+from krita import Krita, QBuffer, QByteArray, QImage, QIODevice, Qt
+
+from .config import Config
+from .defaults import (
+    TAB_CONFIG,
+    TAB_IMG2IMG,
+    TAB_INPAINT,
+    TAB_SDCOMMON,
+    TAB_TXT2IMG,
+    TAB_UPSCALE,
+)
+
+
+def fix_prompt(prompt: str):
+    """Replace empty prompts with None."""
+    return prompt if prompt != "" else None
+
+
+def get_ext_key(ext_type: str, ext_name: str, index: int = None):
+    """Get name of config key where the ext values would be stored."""
+    return "_".join(
+        [
+            ext_type,
+            re.sub(r"\W+", "", ext_name.lower()),
+            "meta" if index is None else str(index),
+        ]
+    )
+
+
+def get_ext_args(ext_cfg: Config, ext_type: str, ext_name: str):
+    """Get args for script in positional list form."""
+    raw = ext_cfg(get_ext_key(ext_type, ext_name))
+    meta = []
+    try:
+        meta = json.loads(raw)
+    except json.JSONDecodeError:
+        print(f"Invalid metadata: {raw}")
+    args = []
+    for i, o in enumerate(meta):
+        typ = type(o["val"])
+        if issubclass(typ, list):
+            typ = "QStringList"
+        val = ext_cfg(get_ext_key(ext_type, ext_name, i), typ)
+        args.append(val)
+    return args
+
+
+def find_fixed_aspect_ratio(
+    base_size: int, max_size: int, orig_width: int, orig_height: int
+):
+    """Copy of `krita_server.utils.sddebz_highres_fix()`.
+
+    This is used by `find_optimal_selection_region()` below to adjust the selected region.
+    """
+
+    def rnd(r, x, z=64):
+        """Scale dimension x with stride z while attempting to preserve aspect ratio r."""
+        return z * ceil(r * x / z)
+
+    ratio = orig_width / orig_height
+
+    # height is smaller dimension
+    if orig_width > orig_height:
+        width, height = rnd(ratio, base_size), base_size
+        if width > max_size:
+            width, height = max_size, rnd(1 / ratio, max_size)
+    # width is smaller dimension
+    else:
+        width, height = base_size, rnd(1 / ratio, base_size)
+        if height > max_size:
+            width, height = rnd(ratio, max_size), max_size
+
+    return width / height
+
+
+def find_optimal_selection_region(
+    base_size: int,
+    max_size: int,
+    orig_x: int,
+    orig_y: int,
+    orig_width: int,
+    orig_height: int,
+    canvas_width: int,
+    canvas_height: int,
+):
+    """Adjusts the selected region in order to attempt to preserve the original
+    aspect ratio of the selection. This prevents the image from being stretched
+    after being scaled and strided.
+
+    After grasping what @sddebz intended to do, I fixed some logical errors &
+    made it clearer.
+
+    Iterating the padding is naive, but easier to understand & verify then figuring
+    out how to grow the rectangle using the fixed aspect ratio alone while accounting
+    for the canvas boundary. Also, it only grows the selection, not shrink, to
+    prevent clipping what the user selected.
+
+    Args:
+        base_size (int): Native/base input size of the model.
+        max_size (int): Max input size to accept.
+        orig_x (int): Original left position of selection.
+        orig_y (int): Original top position of selection.
+        orig_width (int): Original width of selection.
+        orig_height (int): Original height of selection.
+        canvas_width (int): Canvas width.
+        canvas_height (int): Canvas height.
+
+    Returns:
+        Tuple[int, int, int, int]: Best x, y, width, height to use.
+    """
+    orig_ratio = orig_width / orig_height
+    fix_ratio = find_fixed_aspect_ratio(base_size, max_size, orig_width, orig_height)
+
+    # h * (w/h - w/h) = w
+    xpad_limit = ceil(abs(fix_ratio - orig_ratio) * orig_height) * 2
+    # w * (h/w - h/w) = h
+    ypad_limit = ceil(abs(1 / fix_ratio - 1 / orig_ratio) * orig_width) * 2
+
+    best_x = orig_x
+    best_y = orig_y
+    best_width = orig_width
+    best_height = orig_height
+    best_delta = abs(fix_ratio - orig_ratio)
+    for x in range(1, xpad_limit + 1):
+        for y in range(1, ypad_limit + 1):
+            # account for boundary of canvas
+            # padding is on both sides i.e the selection grows while center anchored
+            x1 = max(0, orig_x - x // 2)
+            x2 = min(canvas_width, x1 + orig_width + x)
+            y1 = max(0, orig_y - y // 2)
+            y2 = min(canvas_height, y1 + orig_height + y)
+
+            new_width = x2 - x1
+            new_height = y2 - y1
+            new_ratio = new_width / new_height
+            new_delta = abs(fix_ratio - new_ratio)
+            if new_delta < best_delta:
+                best_delta = new_delta
+                best_x = x1
+                best_y = y1
+                best_width = new_width
+                best_height = new_height
+
+    return best_x, best_y, best_width, best_height
+
+
+def save_img(img: QImage, path: str):
+    """Expects QImage"""
+    # png is lossless; setting compression to max (0) won't affect quality
+    # NOTE: save_img WILL FAIL when using remote backend
+    try:
+        img.save(path, "PNG", 0)
+    except:
+        pass
+
+
+def img_to_ba(img: QImage):
+    """Converts QImage to QByteArray"""
+    ptr = img.bits()
+    ptr.setsize(img.byteCount())
+    return QByteArray(ptr.asstring())
+
+
+def img_to_b64(img: QImage):
+    """Converts QImage to base64-encoded string"""
+    ba = QByteArray()
+    buffer = QBuffer(ba)
+    buffer.open(QIODevice.WriteOnly)
+    img.save(buffer, "PNG", 0)
+    return ba.toBase64().data().decode("utf-8")
+
+
+def b64_to_img(enc: str):
+    """Converts base64-encoded string to QImage"""
+    ba = QByteArray.fromBase64(enc.encode("utf-8"))
+    return QImage.fromData(ba, "PNG")
+
+
+def bytewise_xor(msg: bytes, key: bytes):
+    """Used for decrypting/encrypting request/response bodies."""
+    return bytes(v ^ k for v, k in zip(msg, cycle(key)))
+
+
+def get_desc_from_resp(resp: dict, type: str = ""):
+    """Get description of image generation from backend response."""
+    try:
+        info = json.loads(resp["info"])
+        seeds = info["all_seeds"]
+        glayer_desc = f"""[{type}]
+Prompt: {info['prompt']},
+Negative Prompt: {info['negative_prompt']},
+Model: {info['sd_model_hash']},
+Sampler: {info['sampler_name']},
+Scale: {info['cfg_scale']},
+Steps: {info['steps']}"""
+        layers_desc = []
+        for (seed,) in zip(seeds):
+            layers_desc.append(f"Seed: {seed}")
+        return glayer_desc, layers_desc
+    except:
+        return f"[{type}]", cycle([None])
+
+
+def reset_docker_layout():
+    """NOTE: Default stacking of dockers hardcoded here."""
+    docker_ids = {
+        TAB_SDCOMMON,
+        TAB_CONFIG,
+        TAB_IMG2IMG,
+        TAB_TXT2IMG,
+        TAB_UPSCALE,
+        TAB_INPAINT,
+    }
+    instance = Krita.instance()
+    # Assumption that currently active window is the main window
+    window = instance.activeWindow()
+    dockers = {
+        d.objectName(): d for d in instance.dockers() if d.objectName() in docker_ids
+    }
+    qmainwindow = window.qwindow()
+    # Reset all dockers
+    for d in dockers.values():
+        d.setFloating(False)
+        d.setVisible(True)
+        qmainwindow.addDockWidget(Qt.LeftDockWidgetArea, d)
+
+    qmainwindow.tabifyDockWidget(dockers[TAB_SDCOMMON], dockers[TAB_CONFIG])
+    qmainwindow.tabifyDockWidget(dockers[TAB_TXT2IMG], dockers[TAB_IMG2IMG])
+    qmainwindow.tabifyDockWidget(dockers[TAB_TXT2IMG], dockers[TAB_INPAINT])
+    qmainwindow.tabifyDockWidget(dockers[TAB_TXT2IMG], dockers[TAB_UPSCALE])
+    dockers[TAB_SDCOMMON].raise_()
+    dockers[TAB_INPAINT].raise_()

frontends/krita/krita_diff/widgets/__init__.py

@@ -0,0 +1,8 @@
+from .checkbox import QCheckBox, QMultiCheckBoxLayout
+from .combo_box import QComboBoxLayout
+from .line_edit import QLineEditLayout
+from .misc import QLabel
+from .prompt import QPromptLayout
+from .spin_box import QSpinBoxLayout
+from .status_bar import StatusBar
+from .tips import TipsLayout

frontends/krita/krita_diff/widgets/checkbox.py

@@ -0,0 +1,81 @@
+from functools import partial
+
+from krita import QCheckBox as _QCheckBox
+from krita import QHBoxLayout, QVBoxLayout
+
+from ..config import Config
+from .misc import QLabel
+
+
+class QCheckBox(_QCheckBox):
+    def __init__(self, cfg: Config, field_cfg: str, label: str = None, *args, **kwargs):
+        """QCheckBox compatible with the config system.
+
+        Args:
+            cfg (Config): Config to connect to.
+            field_cfg (str): Config key to read/write value to.
+            label (str): Label, uses `field_cfg` if None. Defaults to None.
+        """
+        label = field_cfg if label is None else label
+        super(QCheckBox, self).__init__(label, *args, **kwargs)
+
+        self.cfg = cfg
+        self.field_cfg = field_cfg
+
+    def cfg_init(self):
+        self.setChecked(self.cfg(self.field_cfg, bool))
+
+    def cfg_connect(self):
+        self.toggled.connect(partial(self.cfg.set, self.field_cfg))
+
+
+# TODO: adjust number of checkboxes based on options without needing restart
+class QMultiCheckBoxLayout(QVBoxLayout):
+    def __init__(
+        self,
+        cfg: Config,
+        options_cfg: list,
+        selected_cfg: str,
+        label: str = None,
+        *args,
+        **kwargs
+    ):
+        """Layout for labelled multi-select CheckBox.
+
+        Args:
+            cfg (Config): Config to connect to.
+            options_cfg (list): List of options.
+            selected_cfg (str): Config key to read/write selected options to.
+            label (str, optional): Label, uses `selected_cfg` if None. Defaults to None.
+        """
+        super(QMultiCheckBoxLayout, self).__init__(*args, **kwargs)
+
+        self.cfg = cfg
+        self.options_cfg = options_cfg
+        self.selected_cfg = selected_cfg
+
+        self.qlabel = QLabel(self.selected_cfg if label is None else label)
+
+        # TODO: flexbox-like row breaking
+        self.row = QHBoxLayout()
+        self.qcheckboxes = []
+        for opt in self.options_cfg:
+            checkbox = _QCheckBox(opt)
+            self.qcheckboxes.append(checkbox)
+            self.row.addWidget(checkbox)
+
+        self.addWidget(self.qlabel)
+        self.addLayout(self.row)
+
+    def cfg_init(self):
+        val = set(self.cfg(self.selected_cfg, "QStringList"))
+        for box in self.qcheckboxes:
+            box.setChecked(box.text() in val)
+
+    def cfg_connect(self):
+        def update(_):
+            selected = [b.text() for b in self.qcheckboxes if b.isChecked()]
+            self.cfg.set(self.selected_cfg, selected)
+
+        for box in self.qcheckboxes:
+            box.toggled.connect(update)

frontends/krita/krita_diff/widgets/combo_box.py

@@ -0,0 +1,98 @@
+from functools import partial
+from typing import Union
+
+from krita import QComboBox, QHBoxLayout, Qt, QValidator
+
+from ..config import Config
+from .misc import QLabel
+
+
+class QOptionValidator(QValidator):
+    def __init__(self, opts: set, *args, **kwargs):
+        super(QOptionValidator, self).__init__(*args, **kwargs)
+        self.opts = opts
+
+    def validate(self, input, pos):
+        # Below validation rules make it impossible to type invalid options
+        if len(self.opts) < 2:
+            # List hasn't loaded yet
+            return QValidator.Intermediate, input, pos
+        elif input in self.opts:
+            return QValidator.Acceptable, input, pos
+        elif any(o.find(input) == 0 for o in self.opts):
+            return QValidator.Intermediate, input, pos
+        else:
+            return QValidator.Invalid, input, pos
+
+    def fixup(self, input):
+        return ""
+
+
+class QComboBoxLayout(QHBoxLayout):
+    def __init__(
+        self,
+        cfg: Config,
+        options_cfg: Union[str, list],
+        selected_cfg: str,
+        label: str = None,
+        *args,
+        **kwargs
+    ):
+        """Layout for labelled QComboBox.
+
+        Args:
+            cfg (Config): Config to connect to.
+            options_cfg (Union[str, list]): Config key to read available options from or list of options.
+            selected_cfg (str): Config key to read/write selected option to.
+            label (str, optional): Label, uses `selected_cfg` if None. Defaults to None.
+            num_chars (int, optional): Max length of qcombo in chars. Defaults to None.
+        """
+        super(QComboBoxLayout, self).__init__(*args, **kwargs)
+
+        # Used to connect to config stored in script
+        self.cfg = cfg
+        self.options_cfg = options_cfg
+        self.selected_cfg = selected_cfg
+        self._items = set()
+
+        self.qlabel = QLabel(self.selected_cfg if label is None else label)
+        self.qcombo = QComboBox()
+        self.qcombo.view().setTextElideMode(Qt.ElideLeft)
+        self.qcombo.setEditable(True)
+        self.qcombo.setInsertPolicy(QComboBox.NoInsert)
+        self.qcombo.setMinimumWidth(10)
+
+        self.addWidget(self.qlabel)
+        self.addWidget(self.qcombo)
+
+    def cfg_init(self):
+        opts = sorted(
+            set(
+                self.cfg(self.options_cfg, "QStringList")
+                if isinstance(self.options_cfg, str)
+                else self.options_cfg
+            ),
+            key=str.casefold,
+        )
+
+        # NOTE: assumes the None option will always be labelled as "None"
+        if "None" in opts:
+            opts.remove("None")
+            opts.insert(0, "None")
+
+        # prevent dropdown from closing when cfg_init is called by update
+        if set(opts) != self._items:
+            self._items = set(opts)
+            # as using editable mode, text isn't affected by clearing options
+            self.qcombo.clear()
+            self.qcombo.addItems(opts)
+            self.qcombo.setValidator(QOptionValidator(self._items))
+
+        # avoid resetting the auto-completer
+        if self.qcombo.currentText() != self.cfg(self.selected_cfg):
+            self.qcombo.setEditText(self.cfg(self.selected_cfg))
+
+    def cfg_connect(self):
+        # Possible to get invalid by backspacing after selecting option
+        # but no one would do that deliberately
+        self.qcombo.editTextChanged.connect(partial(self.cfg.set, self.selected_cfg))

frontends/krita/krita_diff/widgets/line_edit.py

@@ -0,0 +1,44 @@
+from functools import partial
+
+from krita import QHBoxLayout, QLineEdit
+
+from ..config import Config
+from .misc import QLabel
+
+
+class QLineEditLayout(QHBoxLayout):
+    def __init__(
+        self,
+        cfg: Config,
+        field_cfg: str,
+        label: str = None,
+        placeholder: str = "",
+        *args,
+        **kwargs
+    ):
+        """Layout for labelled QLineEdit.
+
+        Args:
+            cfg (Config): Config to connect to.
+            field_cfg (str): Config key to read/write value to.
+            label (str, optional): Label, uses `field_cfg` if None. Defaults to None.
+            placeholder (str, optional): Placeholder. Defaults to "".
+        """
+        super(QLineEditLayout, self).__init__(*args, **kwargs)
+
+        self.cfg = cfg
+        self.field_cfg = field_cfg
+
+        self.qedit = QLineEdit()
+        self.qedit.setPlaceholderText(placeholder)
+        self.addWidget(QLabel(field_cfg if label is None else label))
+        self.addWidget(self.qedit)
+
+    def cfg_init(self):
+        # NOTE: update timer -> cfg_init, setText seems to reset cursor position so we prevent it
+        val = self.cfg(self.field_cfg, str)
+        if self.qedit.text() != val:
+            self.qedit.setText(val)
+
+    def cfg_connect(self):
+        self.qedit.textChanged.connect(partial(self.cfg.set, self.field_cfg))

frontends/krita/krita_diff/widgets/misc.py

@@ -0,0 +1,13 @@
+from krita import QLabel as _QLabel
+from krita import Qt
+
+
+class QLabel(_QLabel):
+    """QLabel with overwritten default behaviours."""
+
+    def __init__(self, *args, **kwargs):
+        super(QLabel, self).__init__(*args, **kwargs)
+
+        self.setOpenExternalLinks(True)
+        self.setWordWrap(True)
+        self.setTextFormat(Qt.TextFormat.RichText)

frontends/krita/krita_diff/widgets/prompt.py

@@ -0,0 +1,58 @@
+from krita import QPlainTextEdit, QSizePolicy, QVBoxLayout
+
+from ..config import Config
+
+
+class QPromptEdit(QPlainTextEdit):
+    def __init__(self, placeholder="Enter prompt...", num_lines=5, *args, **kwargs):
+        super(QPromptEdit, self).__init__(*args, **kwargs)
+        self.setPlaceholderText(placeholder)
+        self.setFixedHeight(self.fontMetrics().lineSpacing() * num_lines)
+        self.setSizePolicy(QSizePolicy.MinimumExpanding, QSizePolicy.Maximum)
+
+
+class QPromptLayout(QVBoxLayout):
+    prompt_label: str = "Prompt:"
+    neg_prompt_label: str = "Negative prompt:"
+
+    def __init__(
+        self, cfg: Config, prompt_cfg: str, neg_prompt_cfg: str, *args, **kwargs
+    ):
+        """Layout for prompt and negative prompt.
+
+        Args:
+            cfg (Config): Config to connect to.
+            prompt_cfg (str): Config key to read/write prompt to.
+            neg_prompt_cfg (str): Config key to read/write negative prompt to.
+        """
+        super(QPromptLayout, self).__init__(*args, **kwargs)
+
+        # Used to connect to config stored in script
+        self.cfg = cfg
+        self.prompt_cfg = prompt_cfg
+        self.neg_prompt_cfg = neg_prompt_cfg
+
+        self.qedit_prompt = QPromptEdit(placeholder=self.prompt_label)
+        self.qedit_neg_prompt = QPromptEdit(placeholder=self.neg_prompt_label)
+
+        self.addWidget(self.qedit_prompt)
+        self.addWidget(self.qedit_neg_prompt)
+
+    def cfg_init(self):
+        # NOTE: update timer -> cfg_init, setText seems to reset cursor position so we prevent it
+        prompt = self.cfg(self.prompt_cfg, str)
+        neg_prompt = self.cfg(self.neg_prompt_cfg, str)
+        if self.qedit_prompt.toPlainText() != prompt:
+            self.qedit_prompt.setPlainText(prompt)
+        if self.qedit_neg_prompt.toPlainText() != neg_prompt:
+            self.qedit_neg_prompt.setPlainText(neg_prompt)
+
+    def cfg_connect(self):
+        self.qedit_prompt.textChanged.connect(
+            lambda: self.cfg.set(self.prompt_cfg, self.qedit_prompt.toPlainText())
+        )
+        self.qedit_neg_prompt.textChanged.connect(
+            lambda: self.cfg.set(
+                self.neg_prompt_cfg, self.qedit_neg_prompt.toPlainText()
+            )
+        )

frontends/krita/krita_diff/widgets/spin_box.py

@@ -0,0 +1,64 @@
+from functools import partial
+from math import isclose
+from typing import Union
+
+from krita import QDoubleSpinBox, QHBoxLayout, QSpinBox
+
+from ..config import Config
+from .misc import QLabel
+
+
+class QSpinBoxLayout(QHBoxLayout):
+    def __init__(
+        self,
+        cfg: Config,
+        field_cfg: str,
+        label: str = None,
+        min: Union[int, float] = 0.0,
+        max: Union[int, float] = 1.0,
+        step: Union[int, float] = 0.1,
+        *args,
+        **kwargs
+    ):
+        """Layout for labelled QSpinBox/QDoubleSpinBox.
+
+        Will infer which to use based on type of min, max and step.
+
+        Args:
+            cfg (Config): Config to connect to.
+            field_cfg (str): Config key to read/write value to.
+            label (str, optional): Label, uses `field_cfg` if None. Defaults to None.
+            min (Union[int, float], optional): Min value. Defaults to 0.0.
+            max (Union[int, float], optional): Max value. Defaults to 1.0.
+            step (Union[int, float], optional): Value step. Defaults to 0.1.
+        """
+        super(QSpinBoxLayout, self).__init__(*args, **kwargs)
+
+        self.cfg = cfg
+        self.field_cfg = field_cfg
+
+        self.qlabel = QLabel(field_cfg if label is None else label)
+
+        is_integer = (
+            float(step).is_integer()
+            and float(min).is_integer()
+            and float(max).is_integer()
+        )
+        self.cast = int if is_integer else float
+
+        self.qspin = QSpinBox() if is_integer else QDoubleSpinBox()
+        self.qspin.setMinimum(self.cast(min))
+        self.qspin.setMaximum(self.cast(max))
+        self.qspin.setSingleStep(self.cast(step))
+        self.addWidget(self.qlabel)
+        self.addWidget(self.qspin)
+
+    def cfg_init(self):
+        val = self.cfg(self.field_cfg, self.cast)
+        cur = self.qspin.value()
+        # prevent cursor from jumping when cfg_init is called by update
+        if not isclose(val, cur):
+            self.qspin.setValue(self.cfg(self.field_cfg, self.cast))
+
+    def cfg_connect(self):
+        self.qspin.valueChanged.connect(partial(self.cfg.set, self.field_cfg))

frontends/krita/krita_diff/widgets/status_bar.py

@@ -0,0 +1,22 @@
+from ..defaults import (
+    STATE_DONE,
+    STATE_INIT,
+    STATE_INTERRUPT,
+    STATE_READY,
+    STATE_URLERROR,
+)
+from .misc import QLabel
+
+
+class StatusBar(QLabel):
+    def __init__(self, *args, **kwargs):
+        super(StatusBar, self).__init__(*args, **kwargs)
+        self.set_status(STATE_INIT)
+
+    def set_status(self, s):
+        if s == STATE_READY and STATE_URLERROR not in self.text():
+            return
+        if s == STATE_DONE and STATE_INTERRUPT == self.text():
+            return
+
+        self.setText(f"<b>Status:</b> {s}")

frontends/krita/krita_diff/widgets/tips.py

@@ -0,0 +1,18 @@
+from typing import List
+
+from krita import QVBoxLayout
+
+from .misc import QLabel
+
+
+class TipsLayout(QVBoxLayout):
+    def __init__(self, tips: List[str], prefix="<em>Tip:</em> ", *args, **kwargs):
+        super(TipsLayout, self).__init__(*args, **kwargs)
+
+        self.tips = [QLabel(prefix + t) for t in tips]
+        for t in self.tips:
+            self.addWidget(t)
+
+    def setVisible(self, visible: bool):
+        for t in self.tips:
+            t.setVisible(visible)

install.py

@@ -0,0 +1,26 @@
+import os
+from pathlib import Path
+
+from launch import git, run
+
+REPO_LOCATION = Path(__file__).parent
+auto_update = os.environ.get("AUTO_UPDATE", "True").lower() in {"true", "yes"}
+
+if auto_update:
+    print("[auto-sd-paint-ext] Attempting auto-update...")
+
+    try:
+        # current_hash = run(
+        #     f'"{git}" -C {REPO_LOCATION} rev-parse HEAD',
+        #     "[auto-sd-paint-ext] Get commit hash.",
+        # ).strip()
+
+        run(f'"{git}" -C "{REPO_LOCATION}" fetch', "[auto-sd-paint-ext] Fetch upstream.")
+
+        run(f'"{git}" -C "{REPO_LOCATION}" pull', "[auto-sd-paint-ext] Pull upstream.")
+    except Exception as e:
+        print("[auto-sd-paint-ext] Auto-update failed:")
+        print(e)
+        print("[auto-sd-paint-ext] Ensure git was used to install extension.")
+
+# NOTE: if we ever get dependencies, we can install them here.

scripts/main.py

@@ -0,0 +1,146 @@
+import logging
+from pathlib import Path
+
+import backend
+import gradio as gr
+from backend.app import app_encryption_middleware
+from backend.config import LOGGER_NAME, ROUTE_PREFIX, SCRIPT_ID, SCRIPT_NAME
+from backend.utils import get_encrypt_key
+from fastapi import FastAPI
+from modules import script_callbacks, scripts, shared
+
+PLUGIN_LOCATION = Path(scripts.basedir()) / "frontends" / "krita"
+
+
+class BackendScript(scripts.Script):
+    def title(self):
+        return SCRIPT_NAME
+
+    def show(self, is_img2img):
+        return scripts.AlwaysVisible
+
+    def ui(self, is_img2img):
+        return []
+
+    def process(self, _):
+        pass
+
+
+started = False
+
+
+def on_app_started(demo: gr.Blocks, app: FastAPI):
+    # NOTE: There is currently a glitch where the on_app_started() callback is called twice
+    # Surprisingly, it only breaks the encryption middleware and causes duplicate logs
+    # Below is a workaround to fix said issues
+    # Downside is that for now, restarting Gradio via the webUI will just break the extension
+    global started
+    if started:
+        return
+    started = True
+
+    logger = logging.getLogger(LOGGER_NAME)
+    logger.setLevel(logging.INFO)
+    handler = logging.StreamHandler()
+    handler.setFormatter(
+        logging.Formatter(
+            fmt="%(name)s:%(levelname)s: %(message)s",
+            datefmt="%Y-%m-%d %H:%M:%S",
+        )
+    )
+    logger.addHandler(handler)
+
+    if shared.cmd_opts.api:
+        app.include_router(backend.router, prefix=ROUTE_PREFIX, tags=[SCRIPT_NAME])
+        app.middleware("http")(app_encryption_middleware)
+        # on first run, this creates a key file
+        get_encrypt_key()
+        if not shared.cmd_opts.listen:
+            logger.info(
+                "Add --listen to COMMANDLINE_ARGS to enable usage as a remote backend."
+            )
+    else:
+        logger.warning("COMMANDLINE_ARGS does not contain --api, API won't be mounted.")
+    # if you wanted to do anything massive to the UI, you could modify demo, but why?
+
+
+def on_ui_settings():
+    # hook to add our own settings to the settings tab
+    pass
+
+
+def krita_help(folder):
+    folder = "<path_to_pykrita>" if not folder else folder
+    return f"""
+        Search for "Command Prompt" in the Start Menu, right-click and click "Run as Administrator...", paste the follow commands and hit Enter:
+        ```bat
+        mklink /j "{folder}\\krita_diff" "{(PLUGIN_LOCATION / 'krita_diff').resolve()}"
+        mklink "{folder}\\krita_diff.desktop" "{(PLUGIN_LOCATION / 'krita_diff.desktop').resolve()}"
+        ```
+
+        Linux command:
+        ```sh
+        ln -s "{(PLUGIN_LOCATION / 'krita_diff').resolve()}" "{folder}/krita_diff"
+        ln -s "{(PLUGIN_LOCATION / 'krita_diff.desktop').resolve()}" "{folder}/krita_diff.desktop"
+        ```
+        """
+
+
+def on_ui_tabs():
+    # hook to create our own UI tab
+    with gr.Blocks(analytics_enabled=False) as interface:
+        gr.Markdown(
+            """
+        ### Generate Krita Plugin Symlink Command
+
+        1. Launch Krita.
+        2. On the menubar, go to `Settings > Manage Resources...`.
+        3. In the window that appears, click `Open Resource Folder`.
+        4. In the file explorer that appears, look for a folder called `pykrita` or create it.
+        5. Enter the `pykrita` folder and copy the folder location from the address bar.
+        6. Paste the folder location below.
+        """
+        )
+        folder = gr.Textbox(
+            placeholder="C:\\\\...\\pykrita", label="Pykrita Folder Location", lines=1
+        )
+        out = gr.Markdown(krita_help(""))
+        folder.change(krita_help, folder, out)
+        gr.Markdown(
+            """
+        **NOTE**: Symlinks will break if you move or rename the repository or any 
+        of its parent folders or otherwise change the path such that the symlink 
+        becomes invalid. In which case, repeat the above steps with the new `pykrita` 
+        folder location and (auto-detected) repository location.
+
+        **NOTE**: Ensure `webui-user.bat`/`webui-user.sh` contains `--api` in `COMMANDLINE_ARGS`!
+        """
+        )
+        gr.Markdown(
+            """
+            ### Enabling the Krita Plugin
+
+            1. Restart Krita.
+            2. On the menubar, go to `Settings > Configure Krita...`
+            3. On the left sidebar, go to `Python Plugin Manager`.
+            4. Look for `Stable Diffusion Plugin` and tick the checkbox.
+            5. Restart Krita again for changes to take effect.
+
+            The `SD Plugin` docked window should appear on the left of the Krita window. If it does not, look on the menubar under `Settings > Dockers` for `SD Plugin`.
+
+            ### Next Steps
+
+            - [Troubleshooting](https://github.com/Interpause/auto-sd-paint-ext/wiki/Troubleshooting)
+            - [Update Guide](https://github.com/Interpause/auto-sd-paint-ext/wiki/Update-Guide)
+            - [Usage Guide](https://github.com/Interpause/auto-sd-paint-ext/wiki/Usage-Guide)
+            """
+        )
+        gr.Markdown("TODO: Control/status panel")
+
+    return [(interface, "auto-sd-paint-ext Guide/Panel", SCRIPT_ID)]
+
+
+# NOTE: see modules/script_callbacks.py for all callbacks
+script_callbacks.on_app_started(on_app_started)
+script_callbacks.on_ui_tabs(on_ui_tabs)
+script_callbacks.on_ui_settings(on_ui_settings)