Voyages in the Domain of Artificial Ingenuity: Unveiling the Enigma of Stable Diffusion and the Odyssey of Comfy UI
Tags: stablediffusion, ai, docker, jupyter
Published: 2023-08-10 12:41:55

This marks the [second phase of my expedition](https://medium.com/@betmig.dev/unveiling-the-magical-fusion-8f311376c75e) into the realm of Artificial Intelligence, the realm of [Stable Diffusion](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0), and the intricate domain of [Ren’py](https://renpy.org/).


These preceding fortnights have ushered in a mélange of challenges, predominantly stemming from the necessity to attend to mundane chores and obligations of life’s terrestrial hue — procuring provisions, disbursing dues, harmonizing with the rhythms of existence in Sweden, navigating the labyrinthine corridors of academia, all while endeavoring to maintain an equilibrium in the sphere of my physical well-being. Nevertheless, I have not been deterred; I’ve persevered, striving to effect marginal ameliorations within my ambit. My aspiration remains unwavering, a steadfast beacon amid these exigencies.


![Depiction of my life at this given time.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rq2ggmjw9u0q50rh6qtr.gif)

The crux of my investigations has gravitated chiefly around the concept of Stable Diffusion, with [AUTOMATIC1111’s SD Web UI](https://github.com/AUTOMATIC1111/stable-diffusion-webui) as my conduit for experimentation. My endeavors have transpired within the ethereal confines of cloud computing, leveraging the manifold offerings of [Vast AI](https://vast.ai/) services.

This odyssey has been nothing short of captivating. It amuses me to no end to observe these myriad machines dispersed across the globe, each occupying peculiar niches in remote locales. Their diversity is beguiling — some are reliant upon antiquated **DVDs** to store their ephemeral data, while others subsist on _meager HDD_ resources.

![Hackers Hacking](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p33xtblr3310f74t9qvm.gif)

The pecuniary valuation of this heterogeneous spectrum fluctuates in consonance with the unique settings and geographic coordinates. A **gamble**, one might contend, but a gamble worth its salt when it comes to these cerebral pursuits. 😽 However, it would be fallacious to deem this service a _paragon of reliability_ for those who seek to cultivate it into a professional enterprise or a robust commercial venture. For modest explorations and preliminary trials, it indeed suffices — **a mantra** I recite to myself in moments of introspection.

Alas, the _fiscal toll_ mounts with celerity, particularly when one orchestrates a pause in the operations of their virtual assemblage, motivated by the desire to safeguard the troves of your _safetensors_, the invaluable _loRas_, and the intricate _embeddings_ painstakingly cultivated. One **gingerly** presses the pause button, persuaded that the financial consequences would be inconsequential.

![Depiction of Vast AI reacting to my naivety](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s0zem7tv7i3i3oyx5uwh.gif)

Ah, the folly of such assumptions! The realm of monetary tolls has ensnared me, as my 20 units of currency, my “cushion,” were **promptly** depleted. 😿

Sorrowful reverberations resounded as I conceded to the inevitability of **terminating my instance**, all while bemoaning the inexorable act of reinstating the intricate webwork. 🤌🏽

In my moments of **disenchantment**, I directed my acerbic reflections toward the constructs of capitalism, questioning the notion of why this intricate edifice of technology couldn’t be proffered in a manner that transcended the 💸 shackles of cost. Amidst the tapestry of exasperation, interwoven with tearful frustration, I resolved to plunge deeper into the realms of comprehension. I yearned to seize greater reins of authority over my endeavors. While the allure of the **web-based user interface** was undeniable, its constraints spurred me to seek an alternative, a more congenial solution.

**Can one unearth a graphical interface that bestows the latitude to sculpt one’s workflow with meticulous artistry? **Indeed, the path to such enlightenment exists!

![Comfy-UI showing all it’s gorgeousness](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lcqbmtoa7q33r0xyj4c0.png)

I present to you the [Comfy-UI for Stable Diffusion](https://github.com/comfyanonymous/ComfyUI) — a robust and modular interface befitting the contours of stable diffusion, replete with an interface that embraces the lexicon of graphs and nodes. **Precisely the mechanism I craved!** The expedition mandates a **rerun**, an exercise in uploading my constructions afresh (accompanied by the obligatory _growls_) and situating them within the embrace of the virtual cloud (with accompanying _double-growls_) once again.

Alas, verity struck, punctuating the euphoria of discovery. Vast AI **failed** to yield a ready-made Comfy UI template tailored for the indolent. The conundrum of crafting my **docker image** and navigating its ascent awaited resolution. Fortuity graced my quest, [unveiling a docker image](https://hub.docker.com/layers/runpod/stable-diffusion/comfy-1.0.0/images/sha256-26670d72a5fb2c21bcbb886ef5b15c186d8a3ed3b4517f6863b253b5b213b66d?context=explore) that aligned with my needs. Yet, the groundwork of configuring the rudiments of my template remained a challenge to surmount.

Curiosity might be kindled, prompting contemplation regarding the trajectory of progress. The trajectory is a trifle elementary.

Embark upon a sojourn to the realms of [Vast.AI](https://cloud.vast.ai/), enlisting thyself within its precincts. Secure an endowment of no less than **20 monetary units** in your digital coffer.

Go into the [Console](https://cloud.vast.ai/), whence a window of opportunity beckons by the name of “**Instance Configuration.**” A lateral panel dubbed “**Edit Image and Config**” awaits your prompt.

![Click on “Console”](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5beni56j1ipom0b107pk.png)

![Click on “**Edit Image and Configuration**”](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/y1goosru6mh51iggnvdj.png)

Select the “Template Slot” option, ushering forth the genesis of your unique template.

![On the “Template Slot” pane, click on “Select.”](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/norf4yj9wkvyg4x0w4cq.png)

Within the confines of “**Enter full docker image/tag name for docker pull,**” meticulously replicate the designated nomenclature of [this docker image](https://hub.docker.com/layers/runpod/stable-diffusion/comfy-1.0.0/images/sha256-26670d72a5fb2c21bcbb886ef5b15c186d8a3ed3b4517f6863b253b5b213b66d?context=explore).

In this instance, the apellation to replicate reads as:

`runpod/stable-diffusion:comfy-1.0.0.`

![Type the docker image tag name on the first text field. Type your docker create/run options on the second text field.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kx2fhen13on23m8wdy2l.png)

Kindle your creative flame within the precincts of “**Create/Run options,**” etching forth the following:

`-e JUPYTER_DIR=/ -p 3000:3000`

The elucidations herein elucidate the nature of a bespoke container, analogous to a vessel containing a tapestry of software routines that unfurl autonomously. Consider it an emulation of a _microcosmic realm_ within the vast expanse of your computational macrocosm.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sen4t7avv4cx6s7i3o7s.gif)

The inaugural parameter, “_-e JUPYTER_DIR=/,_” functions akin to a parchment inscribed with guiding coordinates, ushering the container to a designated alcove within its virtual haven. A dictum to navigate, a rudder to chart the course.

The secondary parameter, “_-p 3000:3000,_” mirrors the act of fashioning a portal betwixt the extramural veracity and the sanctum of the container’s realm. An invitation to discourse, as if to say, “_Should any wayfarer from the exterior yearn to engage in dialogue, let them approach the portal marked ‘3000,’ and I shall orchestrate the conveyance of their communiqué to the inner sanctum._”

![Opening Moria Gates](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ust0of8kyg9icvtazjqs.gif)

In facile parlance, these parameters serve as custodians of the container’s creative abode, orchestrating its endeavors while allowing external entreaties to traverse through the portal labeled ‘3000.’

With the segue to “**Launch Mode**” a decree for the inauguration of a jupyter-python notebook is beckoned.

![Select the first option “Run a Jupyter-python notebook”](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ix2icwwziw23nkuux72w.png)

Into the parchment of “**On-start script,**” the following dictates shall be inscribed:

`env | grep _ >> /etc/environment; touch ~/.no_auto_tmux; sleep 5;

sed -i '/rsync -au --remove-source-files \/venv\/ \/workspace\/venv\//a source \/workspace\/venv\/bin\/activate\n pip install jupyter_core' /start.sh;

/start.sh`

The uninitiated may ponder over the _esoteric prose_ enunciated thus. Envision the establishment of a _mystical kitchen_ within a vast culinary carriage, analogous to a **container**. The kitchen’s machinations must be meticulously calibrated to dispense culinary delights in harmonious synchrony. Consider the following interpretation:

- _env | grep _ >> /etc/environment;_ : Analogous to an inventory, wherein the distinctive ingredients and implements, marked with enigmatic symbols (like ingredients distinguished by the presence of an underscore in their designations), are meticulously documented on a conspicuous slate within the kitchen, visible to all participants.


- _touch ~/.no_auto_tmux;_ : This gesture parallels affixing a note upon the kitchen wall, decreeing, “**Hark, the culinary corps, let us abstain from the automatic invocation of this multifaceted contraption for the present.**” It is a deliberation upon the orchestration of the kitchen’s mechanics.


- _sleep 5;_ : Imagine an interlude, a brief intermission lasting five moments, before the grand inauguration of the kitchen’s operations. A pause to allow the culinary artisans to don their aprons and gather their bearings.


- _sed -i ‘/rsync -au — remove-source-files \/venv\/ \/workspace\/venv\//a source \/workspace\/venv\/bin\/activate\n pip install jupyter_core’ /start.sh;_ : Analogous to a culinary formula, the script (**“start.sh”**) guides the chefs through the preparatory stages. The script is adapted to accommodate new directives — first, don the specialized apron (activate the specific environment), followed by a sprinkle of the condiment ‘jupyter_core’ upon the culinary endeavors (installation of ‘jupyter_core’).


- _/start.sh_ : A clarion call, akin to issuing the command, “_Culinary crew, let the symphony of preparation commence!_” It marks the initiation of the actual culinary process, governed by the instructions encoded within the script.


![Cheffs kiss](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2cwfoiay4qcutx5jtmot.gif)

Concluding the ensemble, a moniker and explication for the template shall be appended for future reference. A final act — the button marked “**Select and Save**” — seals the covenant.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ggdz5vbfqpqdxtytmv4g.png)

In possession of your bespoke template, commence by affirming the ‘1X’ designation for the GPUs atop the central pinnacle.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ta5f627thyoqluiut9v2.png)

Manipulate the sliders judiciously to unearth a mechanism that aligns with your objectives. Personally, my predilections incline toward parsimony, tethered to dependable storage. Hence, I shift the slider — a shift toward “Disk Space To Allocate,” 300GB at the very least (a personal proclivity). The filter sliders should be engaged to align with a minimum of **20GB in GPU RAM**. Prudence would recommend a selection of “_Secure Cloud (Only Trusted Datacenters)_” for the sake of safety and reliability.

Typically, a solitary instance of the “**1x RTX 3090**” breed suffices for most exigencies. Should an attractive offer grace your purview, the “**Rent**” button stands ready to be pressed.

With the die cast, the instance **shall awaken**, its presence evident within the pantheon of instances delineated on the left. Herein lies the gateway to the user interface, a portal offering _Comfy UI’s vista_, alongside direct ingress to the expanse of _Jupyter notebook_. This sanctum allows for manual transference of files or even the beckoning of models from esteemed repositories such as [civitai](https://civitai.com/) or [huggingface](https://huggingface.co/) through the medium of command lines.

![Awaken Instance](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eivtmlnmo4vggkgqxl0o.png)

Assuming the guided journey remains faithful, **Comfy UI** shall be unfurled in the cloud’s ephemeral canvas, with the power to usher forth the invocation of safetensors and myriad embeddings within the realm of Stable Diffusion.

In our **forthcoming chapter**, the nuances of configuring the **Comfy UI** workflow shall be illuminated, each facet cast under the gleam of meticulous detail. Until then, let your curiosity be unshackled. Immerse yourself in the exploration, for the prerogative remains to cease and recommence, a cardinal tenet should adversity come knocking.

![Character from Brain Vs Heart](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mpbncvudsza0snhbhr4d.png)

With fervent wishes for your exploratory odyssey,

Betmig