-------------------------------------------------------------------------------- -------------------------------------------------------------------------------- Spaghetti AI for Windows using NVIDIA GPU I wanted to come up with a unique name for the app, so I chose something silly. -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- Important Information -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- Storage Requirements You should make sure that you have at least 10 gigabytes of space, perhaps more, before installing this initially. However, you should make sure you have at least 50 gigabytes more of space before downloading model data. For perspective, on my computer the ".venv" folder has just over 50,000 files, and nearly 6,000 folders, and is nearly 6 gigabytes. The model data folder ("model_data") has the following folders and these are their sizes: 5.10 GB models--circulus--canvers-realistic-v3.6 5.10 GB models--circulus--canvers-real-v3.7.5 3.97 GB models--circulus--canvers-real-v3.8 3.97 GB models--circulus--canvers-real-v3.8.1 1.98 GB models--runwayml--stable-diffusion-v1-5 2.15 GB models--stabilityai--sd-x2-latent-upscaler 12.50 GB models--stabilityai--sdxl-turbo 6.62 GB models--stabilityai--stable-diffusion-xl-base-1.0 5.81 GB models--stabilityai--stable-diffusion-xl-refiner-1.0 General Requirements You must have a NVIDIA graphics card with Cuda installed. I already had Cuda installed. If you need Cuda, it's in a step below. This script is written to be used on my Windows computer using the GPU. This script will not work using a CPU. While there is code in the script for CPU usage, it doesn't work on a Windows computer. It might be related to the Triton module not being available in Windows. There seem to be something that might work, but it would have meant installing something from elsewhere that I was not willing to try. The version of CUDA might matter. If you have one that doesn't work, you might have to install a different version of CUDA. If you have installed Automatic1111, you don't need to do step 1 and 2 as that program required Python and Git. You don't need Automatic1111 installed to use this app. About Automatic1111: https://github.com/AUTOMATIC1111/stable-diffusion-webui#installation-and-running Antivirus/Firewall Issues Your Antivirus/Firewall software will need to be set to allow the command prompt to download and install Python components, as well as run the app. What you allow or don't allow is up to you. You may need to reinstall some of it if you don't allow it through the first time and it doesn't work for you. I don't know what the minimum you need to allow would be, or if some or all of it can operate in a sandboxed environment. It is important to remember that security settings by your firewall and antivirus program may prevent the app from installing and/or functioning properly. I use Comodo and I had to allow everything, and install it more than once, to get it to install properly. In Advanced View, I had to set files to trusted in Contained Apps and unblock files in Blocked Applications. When popup alerts came up, I had to allow the files, have it remember my answer and have it install as an installer or updater. I also had to go to "Tasks > Containment Tasks > Reset the Container" in order to have the files not install in a sandboxed environment. Then I had to install it again. It is also important to note that you should not have any other programs opened while installing this app. Until I had fully allowed the app through my firewall and antivirus program, at times my computer completely froze up. You don't want to lose any work you are working on. -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- Instructions -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- Step 1 (if you haven't done already): Install Python 3.10.6: https://www.python.org/ftp/python/3.10.6/python-3.10.6-amd64.exe Make sure to check "Add Python to PATH" -------------------- Step 2 (if you haven't done already): I'm not certain if you need this. You might not. I already had it installed on my system. Install git https://git-scm.com/download/win -------------------- Step 3: Create directory manually: C:/Spaghetti_AI This can be anywhere, and the directory can be named anything, but if you change it here you need to change it in all the other places in these instructions and in some of the files you download. -------------------- Step 4: You need to open a regular Command Prompt. You can press Win key + R to get "Run" window. In that, enter: cmd In Command Prompt, copy this line and press Enter: cd C:/Spaghetti_AI && py -m venv .venv && .venv\Scripts\activate.bat This will change the directory of the command prompt to the directory you created. Next, it creates a virtual environment for Python as described here: https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/ https://huggingface.co/docs/diffusers/installation Finally, it activates that virtual environment as also described there. The command prompt will now look like this: (.venv) C:/Spaghetti_AI> You don't need to reopen the Command Prompt each time. Wait until a task finishes and then you will be able to enter the next command. It's best you do not close it. If you do close it, you will need to go to the directory again each time and activate the virtual environment like this, by copying this and pressing Enter: cd C:/Spaghetti_AI && .venv\Scripts\activate.bat You do not want to install Python packages outside of the virtual environment as it would replace the packages in your main installation. That could cause conflicts in other programs. Always make sure you have: (.venv) C:/Spaghetti_AI> In future steps. (though I have added the code to activate it every time as a precaution) -------------------- Step 5 (optional): pip is the package installer for Python. If you want to make sure pip is up-to-date, in Command Prompt (that is doing things in your virtual environment), copy this line and press Enter: cd C:/Spaghetti_AI && .venv\Scripts\activate.bat && py -m pip install --upgrade pip Since you are using a virtual environment, packages that are installed will be installed into this folder rather than into Python's main program: C:/Spaghetti_AI/.venv/Lib/site-packages That way you don't cause issues updating things you may not want to have updated. -------------------- Step 6 (if needed; unsure of this step, I didn't do it): If you don't have Cuda and need to get it, it's here: https://developer.nvidia.com/cuda-toolkit-archive I have 12.2 installed. Some things mention needing 12.1. In Step 8, as of writing this, it specifically mentions Cuda 12.1. You may want to check the PyTorch website first in Step 8 if you need to install Cuda to see what version it is compatible with or you will not get past that step. That link above has a list of prior versions. As packages update, you might need something even newer. To see what version of Cuda you have, in Command Prompt, copy this line and press Enter: nvidia-smi In the top right corner of the text output that is displayed, your Cuda version will be displayed. An older page on this command is here: https://developer.nvidia.com/nvidia-system-management-interface The text output now looks different. -------------------- Step 7: You need to install PyTorch. You should visit their website and see what version they recommend based on the version of Cuda you have. https://pytorch.org/ As of writing this, in the "INSTALL PYTORCH" section I selected the Stable PyTorch build, Windows, Pip, Python and Cuda 12.1. That generated in the "Run this Command" section the command you see below. (with the added code to do this in the virtual environment) To install PyTorch in Command Prompt (that is doing things in your virtual environment), copy this line and press Enter: cd C:/Spaghetti_AI && .venv\Scripts\activate.bat && pip3 install -U xformers --index-url https://download.pytorch.org/whl/cu121 A lot of things will download and install, including any packages needed to run this. When complete, move on to the next step. (when the command prompt returns to where you can paste into it again) The torch file alone is over 2 gigabytes. -------------------- Step 8: This step installs the rest of what you need. You can read about them if you want: https://huggingface.co/docs/diffusers/index https://huggingface.co/docs/transformers/index https://huggingface.co/docs/accelerate/index https://pypi.org/project/ftfy/ https://github.com/modin-project/modin https://pypi.org/project/invisible-watermark/ https://pypi.org/project/torchsde/ https://github.com/gradio-app/gradio To install these, in Command Prompt (that is doing things in your virtual environment), all you need to do is copy this line and press Enter: cd C:/Spaghetti_AI && .venv\Scripts\activate.bat && pip install diffusers transformers accelerate ftfy modin[all] invisible_watermark torchsde A whole bunch of things, what might be 100 packages or more, will download and install, including any packages needed to run these. As of writing this, the script needs version 4.11.0 of Gradio. In some versions before that there was a bug that made dropdown menus not work properly. Another bug was that an image couldn't be downloaded using the download button when viewing a gallery. Even older versions simply won't work at all with this app. This worked for me: cd C:/Spaghetti_AI && .venv\Scripts\activate.bat && pip install https://gradio-builds.s3.amazonaws.com/d4068557953746662235d595ec435c42ceb24414/gradio-4.11.0-py3-none-any.whl That was found here: https://www.gradio.app/docs/blocks After selecting "main" from the version number dropdown in the left column. The link changes when there is a new version. A version history of Gradio is here: https://www.gradio.app/changelog When complete, move on to the next step. -------------------- Step 9: Add these files: spaghetti_ai_install.bat spaghetti_ai_launcher.bat spaghetti_ai_script.py To this folder: C:/Spaghetti_AI/.venv/app_files/ So that a file is here for example: C:/Spaghetti_AI/.venv/app_files/spaghetti_ai_script.py That's really the most important one that needs to be there as those two .bat files call it. There are some things you will need to change in that file, most notably the directory specified in "main_dir". -------------------- Step 10 (Important): I feel this is a very important step. When you try to create model data for each base model, as well as use the refiner and upscaler, model data will try to be downloaded. There is also an option to download all, or just the default data, at one time. That we will cover in a later step. But before you can download model data, you need to make sure two things are set to "0". At least one will not be when you get this script so that you read this section. All the model data could be 40 gigabytes or more of data. I strongly recommend that you then disable the script from downloading updates to the model data once you have downloaded what you want. HuggingFace doesn't just download a model once and never try again. If something is updated, it will download it again. It will not delete the older version. This could eventually allow it to use all the space on your hard drive as that could potentially add 5 to 15 gigabytes of data each time a model updates. There are two variables that you need to make sure are "0". If you do not use "spaghetti_ai_launcher.bat", then there is only one. In "spaghetti_ai_launcher.bat", if you use it, set this to 0: HF_HUB_OFFLINE In "spaghetti_ai_script.py", set this to 0: only_use_local_files To have model data download, both variables must be set to 0 or model data will not download. Eventually I may only use one of those. You can't set, that I can see, "HF_HUB_OFFLINE" in the python script. It should now block all requests online, including to download model data, whereas the other only prevents model data from being downloaded. Some people might not use "spaghetti_ai_launcher.bat", so it's hard to just pick one to use. If you use "spaghetti_ai_launcher.bat" to launch the script, you can set "HF_HUB_OFFLINE" in that file to 1 to not download data again and not set "only_use_local_files". That variable will automatically be changed to "1" if "HF_HUB_OFFLINE" is set to "1" "HF_HUB_OFFLINE" can be set permanently in Windows, until you change it, but this is the easiest way that doesn't impact other installations. If you wanted to permanently store that environment variable in Windows, which I don't recommend, it would impact other installations that use Hugging Face. This only has it apply when using the script. You can read about environment variables here at Hugging Face: https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables#hfhuboffline In regard to Hugging Face caching things, you can learn more on this page: https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations The first time you use the script you will need to set HF_HUB_OFFLINE to 0 like this there: set HF_HUB_OFFLINE=0 Do not include any spaces before or after the equals sign. Or access the app from the command line when you want to download data if "only_use_local_files" is set to "0". The script will not work otherwise. Because of the importance of this variable, I prefer to not have the script work with the .bat file until someone changes that value themselves. -------------------- Step 11: You're ready to run the script that runs the web interface. The first time the models download it will take time as it's gigabytes of data that you need to download. By default, an individual model file isn't downloaded until you actually try creating an image for that model. You can change that however which we will get to in a moment. In Command Prompt (that is doing things in your virtual environment), copy this line and press Enter: cd C:/Spaghetti_AI && .venv\Scripts\activate.bat && py .venv\app_files\spaghetti_ai_script.py That will launch the web interface. When done, you will get a message like this: Running on local URL: http://127.0.0.1:7860 If your browser did not open a web browser, visit the link that was in the command prompt. If it was not the one above, note it. However, if you have something like Automatic1111 open already, it will use a different port, like "7861". It might be this: http://127.0.0.1:7860 To download a lot of data at once rather than waiting until you use each model, you can add this at the end of the URL that opens after your browser opens: ?download_data=1 It will download dozens of gigabytes of data, so you may not want to do that. A "1" will download only the default model configuration for each main model, as well as refiner and upscaler data. A "2" will download all model data needed in "model_configuration_links_object", including online configurations you may not want. Or you can just download it as you use each model. Downloading all the model data will mean downloading dozens of gigabytes of data. Model files are downloaded to: C:/Spaghetti_AI/model_data You should know that this folder can grow considerably. You need to manually clean it out when you need to get rid of old models. Occasionally, new versions will be downloaded when you create an image. O(unless you turned off the ability to download model data) When that Ohappens, you may want to remove the old version if the new version Oworks. Otherwise, you could have 5 to 15 gigabytes of space added each Otime you use a model that updates. You should also know that temporary images created are likely stored Ohere: %USERPROFILE%/AppData/Local/Temp/gradio Those however are likely cleaned out automatically by your computer. A package called Triton apparently can't be installed on Windows so you will see an error in the Command Prompt go by about that every time you launch the web interface. It doesn't seem to impact anything when using a GPU on Windows. https://github.com/openai/triton/issues/1057 https://github.com/openai/triton/issues/1640 Someone offers something there that might work, but I didn't want to install something from someone randomly. Again, it seems to work without it when using a GPU. If trying to use a CPU using Windows, that might be what prevents the app from being able work. -------------------- Step 12: When you open the link, you'll see the display to create the imagery. You must leave the command prompt open. When you process an image, the command prompt will tell you the progress of what you are creating and approximately how long it will take for the step it is on. It will take longer as there will be things that happen for which there is no progress indicator, either in the command prompt or the progress shown in the browser. -------------------- Step 13: To create a shortcut to the command that needs to be launched every time, use: spaghetti_ai_launcher.bat You can then create a shortcut to that, which you could add to the desktop. You can follow the instructions below or use the shortcut that comes with this app: Spaghetti AI.lnk Right click on your desktop and then click on "New > Shortcut". For "Type the location of the item", enter: cmd Then click "Next". You can name the shortcut whatever you want, like "Spaghetti AI". Then click "Finish". Then right click on the shortcut you created and click "Properties". In "Target", replace what is there with the following: C:\Windows\System32\cmd.exe /k C:/Spaghetti_AI/.venv/app_files/spaghetti_ai_launcher.bat That will execute that file each time you click on it. If for some reason that doesn't work sometimes, and rather than get the program to launch it simply opens the command prompt, then you must enter this each time in the command prompt: cd C:/Spaghetti_AI && .venv\Scripts\activate.bat && py .venv\app_files\spaghetti_ai_script.py You might also not be able to use a shortcut. If so, just copy "spaghetti_ai_launcher.bat" and put it on your desktop and name it what you want. The downside, if you do that, would be reconfiguring your firewall to use it from that location too. It's best to create a shortcut. -------------------- File List You can add all of these files to: C:\Spaghetti_AI\.venv\app_files Although some of these files are not needed. - Instructions.txt This file. Instructions on how to install the app. - README.md - requirements.txt These files are for use on HuggingFace. They are not otherwise needed. - spaghetti_ai_install.txt You should avoid using this. I wanted to create an easier way to install it, but it's best to read the instructions. "C:/Spaghetti_AI" needs to be set in "main_dir" in this file if you change the path and use this file. - spaghetti_ai_launcher.bat This is the file that launches "spaghetti_ai_script.py". This sets some variables that need to be set there, like disabling the automatic downloading of model data and turning off telemetry. The shortcuts in this section should point to this. "C:/Spaghetti_AI" needs to be set in "main_dir" in this file if you change the path. - spaghetti_ai_script.py The main script. "C:/Spaghetti_AI" needs to be set in "main_dir" in this file if you change the path. If "only_use_local_files" is set to "1", you will need to change it to download model data. - Spaghetti AI (no icon).lnk - Spaghetti AI.lnk - spaghetti_ai_icon_for_shortcut.ico These files are simply to provide a shortcut to the app. I created a .ico file to have a nice icon that you can use with the shortcut. Using a custom icon in a shortcut might not work, unless you do it yourself, so I also included one that does not have a custom icon. In Windows, you can right click on the shortcut and then click "Properties". On the "Shortcut" tab you can then click "Change Icon". You can then browse to the location of the icon provided or another. If you change the path from "C:/Spaghetti_AI", the shortcut will have to be edited in the "Target" field of the "Shortcut" tab. -------------------- Various Information You get this error: "CUDA out of memory. Tried to allocate 82.00 MiB. GPU 0 has a total capacty of 12.00 GiB of which 0 bytes is free. Of the allocated memory 16.97 GiB is allocated by PyTorch, and 439.95 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF" Try setting these to 1 in spaghetti_ai_script.py: use_sequential_cpu_offload_for_base_model = 1 use_sequential_cpu_offload_for_refiner = 1 use_sequential_cpu_offload_for_upscaler = 1