Package: discoart
Create compelling Disco Diffusion artworks in one lineDiscoArt is an elegant way of creating compelling Disco Diffusion[*]artworks for generative artists, AI enthusiasts and hard-core developers. DiscoArt has a modern & professional API with a beautiful codebase, ensuring high usability and maintainability. It introduces handy features such as result recovery and persistence, gRPC/HTTP serving w/o TLS, post-analysis, easing the integration to larger cross-modal or multi-modal applications.[*]Disco Diffusion is a Google Colab Notebook that leverages CLIP-Guided Diffusion to allow one to create compelling and beautiful images from text prompts.💯Best-in-class: industry-level engineering, top-notch code quality, lean dependencies, small RAM/VRAM footprint; important bug fixes, feature improvementsvs. the original DD5.6.👼Available to all: smooth install forself-hosting, Google Colabfree tier, non-GUI (IPython) environment, and CLI! No brainfuck, no dependency hell, no stackoverflow.🎨Focus on create not code: one-linercreate()with a Pythonic interface, autocompletion in IDE, and powerful features. Fetch real-time results anywhere anytime, no more worry on session outrage on Google Colab. Set initial state easily for more efficient parameter exploration.🏭Ready for integration & production: built on top ofDocArraydata structure, enjoy smooth integration withJina,CLIP-as-serviceand other cross-/multi-modal applications.☁️As-a-service: simplypython -m discoart serve, DiscoArt is now a high-performance low-latency service supports gRPC/HTTP/websockets and TLS. Scaling up/down is one-line; Cloud-native features e.g. Kubernetes, Prometheus and Grafana is one-line.Unbelievable simple thanks to Jina.Gallery with promptsDo you see thediscoart-idin each tweet? To get the config & prompts, simply:fromdiscoartimportshow_configshow_config('discoart-id')InstallPython 3.7+ and CUDA-enabled PyTorch is required.pipinstalldiscoartThis applies to bothself-hosting,Google Colab, system integration, non-GUI environments.Self-hosted Jupyter: to run a Jupyter Notebook on your own GPU machine, the easiest way is touse our prebuilt Docker image.Use it from CLI:python -m discoart createandpython -m discoart configare CLI commands.Use it as a service:python -m discoart serveallows one to run it as gRPC/HTTP/websockets service.GUIDiscoArt is theinfrastructurefor creating Disco Diffusion artworks. The built-in Jupyter Notebook support gives you basic yet limited user experience, e.g. it does not offer any intuitive GUI forprompt scheduling. Note that DiscoArt is developer-centric and API-first, hence improving consumer-facing experience is out of the scope. There are services, platforms and products (not Jina AI affiliated) that already integrate DiscoArt as a service and provide nice GUI on top of it, e.g. Fever Dreams, Replicate, RunPod and Renderflux.Click to see third-party GUIFever Dreams: a free community-powered service with nice GUI and gallery, where people generate and share their DiscoArt artworks, prompts and configs.Replicate: a free form-based GUI of DiscoArt with sandbox user experience and the visualizations.RunPod: a paid GPU cloud provider that runs DiscoArt container with a simple and clean GUI to visualize the configs and creations.Renderflux: a paid creative art platform that wraps DiscoArt and provides end-to-end GUI for creation management.Please be aware that these platforms, products or companies are not affiliated with Jina AI.They define their own terms of services, paywall and data and privacy policies, which are not in the scope of DiscoArt MIT License.Get StartedCreate artworksfromdiscoartimportcreateda=create()That's it! It will create with thedefault text prompts and parameters.Set prompts and parametersSupported parameters arelisted here. You can specify them increate():fromdiscoartimportcreateda=create(text_prompts='A painting of sea cliffs in a tumultuous storm, Trending on ArtStation.',init_image='https://d2vyhzeko0lke5.cloudfront.net/2f4f6dfa5a05e078469ebe57e77b72f0.png',skip_steps=100,)In case you forgot a parameter, just lookup the cheatsheet at anytime:fromdiscoartimportcheatsheetcheatsheet()The difference on the parameters between DiscoArt and DD5.6is explained here.Visualize resultsFinal results and intermediate results are created under the current working directory, i.e../{name-docarray}/{i}-done.png
./{name-docarray}/{i}-step-{j}.png
./{name-docarray}/{i}-progress.png
./{name-docarray}/{i}-progress.gif
./{name-docarray}/da.protobuf.lz4where:name-docarrayis the name of the run, you can specify it otherwise it is a random name.i-*is up to the value ofn_batches.*-done-*is the final image on done.*-step-*is the intermediate image at certain step, updated in real-time.*-progress.pngis the sprite image of all intermediate results so far, updated in real-time.*-progress.gifis the animated gif of all intermediate results so far, updated in real-time.da.protobuf.lz4is the compressed protobuf of all intermediate results so far, updated in real-time.The save frequency is controlled bysave_rate.Moreover,create()returnsda, aDocumentArray-type object. It contains the following information:All arguments passed tocreate()function, including seed, text prompts and model parameters.4 generated image and its intermediate steps' images, where4is determined byn_batchesand is the default value.This allows you to further post-process, analyze, export the results with powerful DocArray API.Images are stored as Data URI in.uri, to save the first image as a local file:da[0].save_uri_to_file('discoart-result.png')To save all final images:foridx,dinenumerate(da):d.save_uri_to_file(f'discoart-result-{idx}.png')You can also display all four final images in a grid:da.plot_image_sprites(skip_empty=True,show_index=True,keep_aspect_ratio=True)Or display them one by one:fordinda:d.display()Or take one particular run:da[0].display()Visualize intermediate stepsYou can also zoom into a run (say the first run) and check out intermediate steps:da[0].chunks.plot_image_sprites(skip_empty=True,show_index=True,keep_aspect_ratio=True)You can.display()the chunks one by one, or save one via.save_uri_to_file(), or save all intermediate steps as a GIF:da[0].chunks.save_gif('lighthouse.gif',show_index=True,inline_display=True,size_ratio=0.5)Note that >=0.7.14, a 20FPS gif is generated which includes all intermedidate steps.Show/save/load configsTo show the config of a Document/DocumentArray,fromdiscoartimportshow_configshow_config(da)# show the config of the first runshow_config(da[3])# show the config of the fourth runshow_config('discoart-06030a0198843332edc554ffebfbf288')# show the config of the run with a known DocArray IDTo save the config of a Document/DocumentArray,fromdiscoartimportsave_configsave_config(da,'my.yml')# save the config of the first runsave_config(da[3],'my.yml')# save the config of the fourth runTo runcreatefrom a YAML config of Document/DocumentArray,fromdiscoartimportcreate,load_configconfig=load_config('my.yml')create(**config)You can also export the config as an SVG image:fromdiscoart.configimportsave_config_svgsave_config_svg(da)One can also generate runnable Python code directly from the config:fromdiscoart.configimportexport_pythonexport_python(da)Pull results anywhere anytimeIf you are a free-tier Google Colab user, one annoy thing is the lost of sessions from time to time. Or sometimes you just early stop the run as the first image is not good enough, and a keyboard interrupt will prevent.create()to return any result. Either case, you can easily recover the results by pulling the last session ID.Find the session ID. It appears on top of the image.Pull the result via that IDon any machine at any time, not necessarily on Google Colab:fromdocarrayimportDocumentArrayda=DocumentArray.pull('discoart-3205998582')Reuse a Document as initial stateConsider a Document as a self-contained data with config and image, one can use it as the initial state for the future run. Its.tagswill be used as the initial parameters;.uriif presented will be used as the initial image.fromdiscoartimportcreatefromdocarrayimportDocumentArrayda=DocumentArray.pull('discoart-3205998582')create(init_document=da[0],cut_ic_pow=0.5,tv_scale=600,cut_overview='[12]*1000',cut_innercut='[12]*1000',use_secondary_model=False,)If you just want to initialize from a known DocArray ID, then simply:fromdiscoartimportcreatecreate(init_document='discoart-3205998582')Environment variablesYou can set environment variables to control the meta-behavior of DiscoArt. The environment variables must be set before importing DiscoArt, either in Bash or in Python viaos.environ.DISCOART_LOG_LEVEL='DEBUG'# more verbose logsDISCOART_OPTOUT_CLOUD_BACKUP='1'# opt-out from cloud backupDISCOART_DISABLE_IPYTHON='1'# disable ipython dependencyDISCOART_DISABLE_RESULT_SUMMARY='1'# disable result summary after the run endsDISCOART_DEFAULT_PARAMETERS_YAML='path/to/your-default.yml'# use a custom default parameters fileDISCOART_CUT_SCHEDULES_YAML='path/to/your-schedules.yml'# use a custom cut schedules fileDISCOART_MODELS_YAML='path/to/your-models.yml'# use a custom list of models fileDISCOART_OUTPUT_DIR='path/to/your-output-dir'# use a custom output directory for all images and resultsDISCOART_CACHE_DIR='path/to/your-cache-dir'# use a custom cache directory for models and downloadsDISCOART_DISABLE_REMOTE_MODELS='1'# disable the listing of diffusion models on Github, remote diffusion models allows user to use latest models without updating the codebase.DISCOART_REMOTE_MODELS_URL='https://yourdomain/models.yml'# use a custom remote URL for fetching models listDISCOART_DISABLE_CHECK_MODEL_SHA='1'# disable checking local model SHA matches the remote model SHADISCOART_DISABLE_TQDM='1'# disable tqdm progress bar on diffusionCLIDiscoArt provides two commandscreateandconfigthat allows you to run DiscoArt from CLI.python-mdiscoartcreatemy.ymlwhich creates artworks from the YAML config filemy.yml. You can also do:catconfig.yml|python-mdiscoartcreateSo how can I have my ownmy.ymland what does it look like? That's the second command:python-mdiscoartconfigmy.ymlwhich forks thedefault YAML configand export them tomy.yml. Now you can modify it and run it withpython -m discoart createcommand.If no output path is specified, thenpython -m discoart configwill print the default config to stdout.To get help on a command, add--helpat the end, e.g.:python-mdiscoartcreate--helpusage: python -m discoart create [-h] [YAML_CONFIG_FILE]

positional arguments:
  YAML_CONFIG_FILE  The YAML config file to use, default is stdin.

optional arguments:
  -h, --help        show this help message and exitServingServing DiscoArt is super easy. Simply run the following command:python-mdiscoartserveYou shall see:Now send request to the server via curl/Javascript, e.g.curl\-XPOSThttp://0.0.0.0:51001/post\-H'Content-Type: application/json'\-d'{"execEndpoint":"/create", "parameters": {"text_prompts": ["A beautiful painting of a singular lighthouse", "yellow color scheme"]}}'That's it.You can of course pass all parameters that accepted bycreate()function in the JSON.Polling intermediate resultsWe already know thatcreatefunction is slow even on GPU it could take 10 minutes to finish an artwork. This means the after sending the above request, the client will have to wait 10 minutes for the response. There is nothing wrong with this behavior given that everything runs synchronously. However, in practice, client may expect a progress or intermediate results in the middle instead of waiting for the end./resultendpoint is designed for this purpose. It will return the intermediate results as soon as they are available. All you need is to specifyname_docarrayin the request parameters as you specified in/createendpoint. Here is an example:Let's createmydisco-123by sending the following request to/createendpoint:curl\-XPOSThttp://0.0.0.0:51001/post\-H'Content-Type: application/json'\-d'{"execEndpoint":"/create", "parameters": {"name_docarray": "mydisco-123", "text_prompts": ["A beautiful painting of a singular lighthouse", "yellow color scheme"]}}'Now that the above request is being processed on the server, you can periodically checkmydisco-123progress by sending the following request to/resultendpoint:curl\-XPOSThttp://0.0.0.0:51001/post\-H'Content-Type: application/json'\-d'{"execEndpoint":"/result", "parameters": {"name_docarray": "mydisco-123"}}'A JSON will be returned with up-to-date progress, with image as DataURI, loss, steps etc.The JSON Schema of Document/DocumentArray is described here.Note,/resultwon't be blocked by/createthanks to the smart routing of Jina Gateway. To learn/play more about those endpoints, you can check ReDoc or the Swagger UI embedded in the server.Skip & CancelSend to/skip, to skip the current run and move to the next run as defined inn_batches:curl\-XPOSThttp://0.0.0.0:51001/post\-H'Content-Type: application/json'\-d'{"execEndpoint":"/skip"}'Send to/stop, to stop the current run cancel all runsn_batches:curl\-XPOSThttp://0.0.0.0:51001/post\-H'Content-Type: application/json'\-d'{"execEndpoint":"/stop"}'Unblocking/createrequestIt is possible to have an unblocked/createendpoint: the client request to/createwill beimmediatelyreturned, without waiting for the results to be finished. You now have to fully rely on/resultto poll the result.To enable this feature:Copy-paste thedefaultflow.ymlfiletomyflow.yml;Changefloating: falsetofloating: trueunderdiscoartexecutor section;Run the following command:python-mdiscoartservemyflow.ymlBeware that the request velocity is now underyour control. That is, if the client sends 10/createrequests in a second, then the server will start 10create()in parallel! This can easily lead to OOM. Hence, the suggestion is only enabling this feature if you are sure that the client is not sending too many requests, e.g. you control the client request rate; or you are using DiscoArt behind a BFF (backend for frontend).Scaling outIf you have multiple GPUs and you want to run multiple DiscoArt instances in parallel by leveraging GPUs in a time-multiplexed fashion, you can copy-paste thedefaultflow.ymlfileand modify it as follows:jtype:Flowwith:protocol:httpmonitoring:trueport:51001port_monitoring:51002# prometheus monitoring portenv:JINA_LOG_LEVEL:debugDISCOART_DISABLE_IPYTHON:1DISCOART_DISABLE_RESULT_SUMMARY:1executors:-name:discoartuses:DiscoArtExecutorenv:CUDA_VISIBLE_DEVICES:RR0:3# change this if you have multiple GPUreplicas:3# change this if you have larger VRAM-name:polleruses:ResultPollerHerereplicas: 3says spawning three DiscoArt instances,CUDA_VISIBLE_DEVICES: RR0:3makes sure they use the first three GPUs in a round-robin fashion.Name it asmyflow.ymland then runpython-mdiscoartservemyflow.ymlCustomizationThanks to Jina, there are tons of things you can customize! You can change the port number; change protocol to gRPC/Websockets; add TLS encryption; enable/disable Prometheus monitoring; you can also export it to Kubernetes deployment bundle simply via:jinaexportkubernetesmyflow.ymlFor more features and YAML configs,please check out Jina docs.Use gRPC gatewayTo switch from HTTP to gRPC gateway is simple:jtype:Flowwith:protocol:grpc...and then restart the server.There are multiple advantages of using gRPC gateway:Much faster and smaller network overhead.Feature-rich, like compression, status monitoring, etc.In general, if you are using the DiscoArt server behind a BFF (backend for frontend), or your DiscoArt server doesnotdirectly serve HTTP traffic from end-users, then you should use gRPC protocol.To communicate with a gRPC DiscoArt server, one can use a Jina Client:# !pip install jinafromjinaimportClientc=Client(host='grpc://0.0.0.0:51001')da=c.post('/create',parameters={'name_docarray':'mydisco-123','text_prompts':['A beautiful painting of a singular lighthouse','yellow color scheme',],},)# check intermediate resultsda=c.post('/result',parameters={'name_docarray':'mydisco-123'})To use an existing Document/DocumentArray as init Document forcreate:fromjinaimportClientc=Client(host='grpc://0.0.0.0:51001')old_da=create(...)da=c.post('/create',old_da,# this can be a DocumentArray or a single Documentparameters={'width_height':[1024,768],},)This equals to runcreate(init_document=old_da, width_height=[1024, 768])on the server. Note:follow-up parameters have higher priorities than the parameters ininit_document.ifinit_documentis a DocumentArray, then the first Document in the array will be used as the init Document.there is no need to do any serialization before sending, Jina automatically handles it.Hosting on Google ColabThough not recommended, it is also possible to use Google Colab to host DiscoArt server.
Please check out the following tutorials:https://docs.jina.ai/how-to/google-colab/https://clip-as-service.jina.ai/hosting/colab/Run in DockerWe provide a prebuilt Docker image for running DiscoArt out of the box. To update Docker image to latest version:dockerpulljinaai/discoart:latestUse Jupyter notebookThe default entrypoint is starting a Jupyter notebook# docker build . -t jinaai/discoart  # if you want to build yourselfdockerrun-p51000:8888-v$(pwd):/home/jovyan/-v$HOME/.cache:/root/.cache--gpusalljinaai/discoartNow you can visithttp://127.0.0.1:51000to access the notebookEnable GPU in Docker on WindowsYou can use it on Windows Subsystem for Linux (WSL), Check the official guidehere.# Make sure you install Windows 11 or Windows 10, version 21H2dockerrun-p8888:8888-v$HOME/.cache:/root/.cache--gpusalljinaai/discoartUse as a service# docker build . -t jinaai/discoart  # if you want to build yourselfdockerrun--entrypoint"python"-p51001:51001-v$(pwd):/home/jovyan/-v$HOME/.cache:/root/.cache--gpusalljinaai/discoart-mdiscoartserveYour DiscoArt server is now running athttp://127.0.0.1:51001.Release cycleDocker images are built on every release, so one can lock it to a specific version, say0.5.1:dockerrun-p51000:8888-v$(pwd):/home/jovyan/-v$HOME/.cache:/root/.cache--gpusalljinaai/discoart:0.5.1What's next?Next is create.😎If you are already a DD user: you are ready to go! There is no extra learning, DiscoArt respects the same parameter semantics as DD5.6. So just unleash your creativity!Read more about their differences here.You can always dofrom discoart import cheatsheet; cheatsheet()to check all new/modified parameters.👶If you are aDALL·E Flowor new user: you may want to take step by step, as Disco Diffusion works in a very different way than DALL·E. It is much more advanced and powerful: e.g. Disco Diffusion can take weighted & structured text prompts; it can initialize from a image with controlled noise; and there are way more parameters one can tweak. Impatient prompt like"armchair avocado"will give you nothing but confusion and frustration. I highly recommend you to check out the following resources before trying your own prompt:Zippy's Disco Diffusion Cheatsheet v0.3EZ Charts - Diffusion Parameter StudiesDisco Diffusion 70+ Artist StudiesA Traveler’s Guide to the Latent SpaceDisco Diffusion Illustrated SettingsCoar’s Disco Diffusion GuideSupportJoin ourSlack communityand chat with other community members about ideas.Join ourEngineering All Handsmeet-up to discuss your use case and learn Jina's new features.When?The second Tuesday of every monthWhere?Zoom (see our public events calendar/.ical)
andlive stream on YouTubeSubscribe to the latest video tutorials on ourYouTube channelJoin UsDiscoArt is backed byJina AIand licensed underMIT License.We are actively hiringAI engineers, solution engineers to build the next neural search ecosystem in open-source.