Package: hordelib
hordelibhordelibis a wrapper aroundComfyUIprimarily to enable theAI Hordeto run inference pipelines designed visually in the ComfyUI GUI.The developers ofhordelibcan be found in the AI Horde Discord server:https://discord.gg/3DxrhksKznhordelibhas been the default inference backend library of theAI Hordesincehordelibv1.0.0.PurposeThe goal here is to be able to design inference pipelines in the excellent ComfyUI, and then call those inference pipelines programmatically. Whilst providing features that maintain compatibility with the existing horde implementation.InstallationIf being installed from pypi, use a requirements file of the form:--extra-index-url https://download.pytorch.org/whl/cu121
hordelib

...your other dependencies...Linux InstallationOn Linux you will need to install the Nvidia CUDA Toolkit. Linux installers are provided by Nvidia athttps://developer.nvidia.com/cuda-downloadsNote if you only have 16GB of RAM and a default /tmp on tmpfs, you will likely need to increase the size of your temporary space to install the CUDA Toolkit or it may fail to extract the archive. One way to do that is just before installing the CUDA Toolkit:sudomount-oremount,size=16G/tmpIf you only have 16GB of RAM you will also need swap space. So if you typically run without swap, add some. You won't be able to run this library without it.UsageHorde payloads can be processed simply with (for example):importosimporthordelibhordelib.initialise()fromhordelib.hordeimportHordeLibfromhordelib.shared_model_managerimportSharedModelManager# Wherever your models areos.environ["AIWORKER_CACHE_HOME"]="f:/ai/models"generate=HordeLib()SharedModelManager.loadModelManagers(compvis=True)SharedModelManager.manager.load("Deliberate")data={"sampler_name":"k_dpmpp_2m","cfg_scale":7.5,"denoising_strength":1.0,"seed":123456789,"height":512,"width":512,"karras":False,"tiling":False,"hires_fix":False,"clip_skip":1,"control_type":None,"image_is_control":False,"return_control_map":False,"prompt":"an ancient llamia monster","ddim_steps":25,"n_iter":1,"model":"Deliberate",}pil_image=generate.basic_inference_single_image(data).imagepil_image.save("test.png")Note thathordelib.initialise()will erase all command line arguments from argv. So make sure you parse them before you call that.Seetests/run_*.pyfor more standalone examples.LoggingIf you don't wanthordelibto setup and control the logging configuration initialise with:importhordelibhordelib.initialise(setup_logging=False)Acknowledgmentshordelibdepends on a large number of open source projects, and most of these dependencies are automatically downloaded and installed when you installhordelib. Due to the nature and purpose ofhordelibsome dependencies are bundled directlyinsidehordelibitself.ComfyUIA powerful and modular stable diffusion GUI with a graph/nodes interface. Licensed under the terms of the GNU General Public License v3.0.The entire purpose ofhordelibis to access the power of ComfyUI.Controlnet Preprocessors for ComfyUICustom nodes for ComfyUI providing Controlnet preprocessing capability. Licened under the terms of the Apache License 2.0.ComfyUI Face Restore NodeCustom nodes for ComfyUI providing face restoration.DMCA AbuseOn 26th May 2023 anindividualissued aDMCA takedown noticeto Github againsthordelibwhich claimed their name had been removed from the copyright header in the AGPL license in the 7 files listed in the takedown notice. This claim was true, and this attribution had been removed by ahordelibcontributor prior to being committed into thehordelibrepository.Unfortunately, it appears the individual making the DMCA claim was acting in bad faith, and even when their name was restored to the copyright attribution in the files, they persisted to press the DMCA takedown claim, which, due to the nature of the Github process, resulted in hordelib being subject to a DMCA takedown on Github.This version ofhordelibhas the 7 files mentioned in the DMCA takedown removed and replaced with alternatives and the functionality required to run the AI Horde Worker was restored on 7th June 2023.DevelopmentRequirements:git(install git)tox(pip install tox)Set the environmental variableAIWORKER_CACHE_HOMEto point to a model directory.Note the model directory must currently be in the original AI Horde directory structure:<AIWORKER_CACHE_HOME>\
    clip\
    codeformer\
    compvis\
        Deliberate.ckpt
        ...etc...
    controlnet\
    embeds\
    esrgan\
    gfpgan\
    safety_checker\Running the TestsSimply execute:tox(ortox -qfor less noisy output)This will take a while the first time as it installs all the dependencies.If the tests run successfully images will be produced in theimages/folder.Running a specific test filetox -- -k <filename>for exampletox -- -k test_initialisationRunning a specific predefined test suitetox listThis will list all groups of tests which are involved in either the development, build or CI proccess. Tests which have the word 'fix' in them will automatically apply changes when run, such as to linting or formatting. You can do this by running:tox -e [test_suite_name_here]Directory Structurehordelib/pipeline_designs/Contains ComfyUI pipelines in a format that can be opened by the ComfyUI web app. These saved directly from the web app.hordelib/pipelines/Contains the above pipeline JSON files converted to the format required by the backend pipeline processor. These are converted from the web app, seeConverting ComfyUI pipelinesbelow.hordelib/nodes/These are the custom ComfyUI nodes we use forhordelibspecific processing.Running ComfyUI Web ApplicationIn this example we install the dependencies in the OS default environment. When using the git version ofhordelib, from the project root:pip install -r requirements.txt --extra-index-url https://download.pytorch.org/whl/cu118 --upgradeEnsure ComfyUI is installed and patched, one way is running the tests:toxFrom then on to run ComfyUI:cd ComfyUIpython main.pyThen open a browser at:http://127.0.0.1:8188/Designing ComfyUI PipelinesUse the standard ComfyUI web app. Use the "title" attribute to name the nodes, these names become parameter names in thehordelib. For example, a KSampler with the "title" of "sampler2" would become a parametersampler2.seed,sampler2.cfg, etc. Load the pipelinehordelib/pipeline_designs/pipeline_stable_diffusion.jsonin the ComfyUI web app for an example.Save any new pipeline inhordelib/pipeline_designsusing the naming convention "pipeline_<name>.json".Convert the JSON for the model (seeConverting ComfyUI pipelinesbelow) and save the resulting JSON inhordelib/pipelinesusing the same filename as the previous JSON file.That is all. This can then be called fromhordelibusing therun_image_pipeline()method inhordelib.comfy.Comfy()Converting ComfyUI pipelinesIn addition to the design file saved from the UI, we need to save the pipeline file in the backend format. This file is created in thehordelibproject root namedcomfy-prompt.jsonautomatically if you run a pipeline through the ComfyUI version embedded inhordelib. Running ComfyUI withtox -e comfyuiautomatically patches ComfyUI so this JSON file is saved.Build ConfigurationThe main config files for the project are:pyproject.toml,tox.iniandrequirements.txtPyPi PublishingPypi publishing is automatic all from the GitHub website.Create a PR frommaintoreleasesLabel the PR with "release:patch" (0.0.1) or "release:minor" (0.1.0)Merge the PR with a standard merge commit (not squash)Standalone "clean" environment test from PypiHere's an example:Start in a new empty directory. Create requirements.txt:--extra-index-url https://download.pytorch.org/whl/cu118
hordelibCreate the directoryimages/and copy thetest_db0.jpginto it.Copyrun_controlnet.pyfrom thehordelib/tests/directory.Build a venv:python -m venv venv
.\venv\Scripts\activate
pip install -r requirements.txtRun the test we copied:python run_controlnet.py

The `images/` directory should have our test images.Updating the embedded version of ComfyUIWe use a ComfyUI version pinned to a specific commit, seehordelib/consts.py:COMFYUI_VERSIONTo test if the latest version works and upgrade to it, from the project root simply:cd ComfyUIChange CWD to the embedded comfygit checkout masterSwitch to master branchgit pullGet the latest comfyui codegit rev-parse HEADUpdate the hash inhordelib.consts:COMFYUI_VERSIONcd ..Get back to the hordelib project roottoxSee if everything still worksNow ComfyUI is pinned to a new version.ComfyUI PatchingWe patch the ComfyUI source code to:Modify the model manager to allow us to dynamically move models between VRAM, RAM and disk cache.Allow make ComfyUI output some handy JSON we need for development purposes.To create a patch file:Make the required changes to a clean install of ComfyUI and then rungit diff > yourfile.patchthen move the patch file to wherever you want to save it.Note that the patch filereallyneeds to be in UTF-8 format and some common terminals, e.g. Powershell, won't do this by default. In Powershell to create a patch file use:git diff | Set-Content -Encoding utf8 -Path yourfile.patchPatches can be applied with thehordelib.install_comfyui.Installerclassesapply_patch()method.