please help me run this project. i cloned the github using command prompt. now, what command do i type to start it? :""Skip to content
oobabooga
/
text-generation-webui

Type / to search

Code
Issues
243
Pull requests
30
Discussions
Actions
Projects
Wiki
Security
Insights
Owner avatar
text-generation-webui
Public
oobabooga/text-generation-webui
Go to file
t
Add file
Folders and files
Name		
Latest commit
oobabooga
oobabooga
Merge pull request #5680 from oobabooga/dev
1934cb6
 · 
last week
History
.github
Update stalebot message
last month
characters
Improve the default character
6 months ago
css
Big picture fixes (#5565)
3 weeks ago
docker
Installer: add back INSTALL_EXTENSIONS environment variable (for docker)
last week
docs
Document StreamingLLM
last week
extensions
API: don't use settings.yaml for default values
last week
grammars
Add roleplay.gbnf grammar (#5368)
2 months ago
instruction-templates
Removed extra spaces from Mistral instruction template that were caus…
last month
js
Big picture fixes (#5565)
3 weeks ago
loras
Add dummy file
last year
models
Synthia instruction templates (#5041)
3 months ago
modules
Document StreamingLLM
last week
presets
Reduce the number of built-in presets (#5217)
2 months ago
prompts
Remove duplicate code
10 months ago
training
Training: Update llama2-chat-format.json (#5593)
2 weeks ago
.gitignore
Add .vs to .gitignore
3 months ago
CMD_FLAGS.txt
Update CMD_FLAGS.txt
6 months ago
Colab-TextGen-GPU.ipynb
Use the correct PyTorch in the Colab notebook
2 weeks ago
LICENSE
Initial commit
2 years ago
README.md
Document StreamingLLM
last week
cmd_linux.sh
Add Conda env deactivation to installer scripts
6 months ago
cmd_macos.sh
Add Conda env deactivation to installer scripts
6 months ago
cmd_windows.bat
Use call for conda deactivate in Windows installer (#4042)
6 months ago
cmd_wsl.bat
Move one-click-installers into the repository
6 months ago
convert-to-safetensors.py
Make the code more like PEP8 for readability (#862)
last year
download-model.py
Revert "Replace hashlib.sha256 with hashlib.file_digest so we don't n…
3 weeks ago
one_click.py
Small fix for cuda 11.8 in the one-click installer
last week
requirements.txt
Add numba to requirements.txt
last week
requirements_amd.txt
Add numba to requirements.txt
last week
requirements_amd_noavx2.txt
Add numba to requirements.txt
last week
requirements_apple_intel.txt
Add numba to requirements.txt
last week
requirements_apple_silicon.txt
Add numba to requirements.txt
last week
requirements_cpu_only.txt
Add numba to requirements.txt
last week
requirements_cpu_only_noavx2.txt
Add numba to requirements.txt
last week
requirements_noavx2.txt
Add numba to requirements.txt
last week
requirements_nowheels.txt
Add numba to requirements.txt
last week
server.py
Minor logging improvements
last month
settings-template.yaml
Add prompt_lookup_num_tokens parameter (#5296)
2 months ago
setup.cfg
Various one-click installer improvements (#4994)
2 months ago
start_linux.sh
One-click installer: delete the Miniconda installer after completion
2 weeks ago
start_macos.sh
One-click installer: delete the Miniconda installer after completion
2 weeks ago
start_windows.bat
Installer: validate the checksum for the miniconda installer on Windows
last week
start_wsl.bat
Fixes by @jllllll
6 months ago
update_wizard_linux.sh
Create an update wizard (#5623)
2 weeks ago
update_wizard_macos.sh
Create an update wizard (#5623)
2 weeks ago
update_wizard_windows.bat
Move update_wizard_windows.sh to update_wizard_windows.bat (oops)
2 weeks ago
update_wizard_wsl.bat
Move update_wizard_wsl.sh to update_wizard_wsl.bat
2 weeks ago
wsl.sh
Create an update wizard (#5623)
2 weeks ago
Repository files navigation
README
AGPL-3.0 license
Text generation web UI
A Gradio web UI for Large Language Models.

Its goal is to become the AUTOMATIC1111/stable-diffusion-webui of text generation.

Image1	Image2
Image1	Image2
Features
3 interface modes: default (two columns), notebook, and chat.
Multiple model backends: Transformers, llama.cpp (through llama-cpp-python), ExLlamaV2, AutoGPTQ, AutoAWQ, GPTQ-for-LLaMa, CTransformers, QuIP#.
Dropdown menu for quickly switching between different models.
Large number of extensions (built-in and user-contributed), including Coqui TTS for realistic voice outputs, Whisper STT for voice inputs, translation, multimodal pipelines, vector databases, Stable Diffusion integration, and a lot more. See the wiki and the extensions directory for details.
Chat with custom characters.
Precise chat templates for instruction-following models, including Llama-2-chat, Alpaca, Vicuna, Mistral.
LoRA: train new LoRAs with your own data, load/unload LoRAs on the fly for generation.
Transformers library integration: load models in 4-bit or 8-bit precision through bitsandbytes, use llama.cpp with transformers samplers (llamacpp_HF loader), CPU inference in 32-bit precision using PyTorch.
OpenAI-compatible API server with Chat and Completions endpoints -- see the examples.
How to install
Clone or download the repository.
Run the start_linux.sh, start_windows.bat, start_macos.sh, or start_wsl.bat script depending on your OS.
Select your GPU vendor when asked.
Once the installation ends, browse to http://localhost:7860/?__theme=dark.
Have fun!
To restart the web UI in the future, just run the start_ script again. This script creates an installer_files folder where it sets up the project's requirements. In case you need to reinstall the requirements, you can simply delete that folder and start the web UI again.

The script accepts command-line flags. Alternatively, you can edit the CMD_FLAGS.txt file with a text editor and add your flags there.

To get updates in the future, run update_wizard_linux.sh, update_wizard_windows.bat, update_wizard_macos.sh, or update_wizard_wsl.bat.

Setup details and information about installing manually
List of command-line flags
Documentation
https://github.com/oobabooga/text-generation-webui/wiki

Downloading models
Models should be placed in the folder text-generation-webui/models. They are usually downloaded from Hugging Face.

GGUF models are a single file and should be placed directly into models. Example:
text-generation-webui
└── models
    └── llama-2-13b-chat.Q4_K_M.gguf
The remaining model types (like 16-bit transformers models and GPTQ models) are made of several files and must be placed in a subfolder. Example:
text-generation-webui
├── models
│   ├── lmsys_vicuna-33b-v1.3
│   │   ├── config.json
│   │   ├── generation_config.json
│   │   ├── pytorch_model-00001-of-00007.bin
│   │   ├── pytorch_model-00002-of-00007.bin
│   │   ├── pytorch_model-00003-of-00007.bin
│   │   ├── pytorch_model-00004-of-00007.bin
│   │   ├── pytorch_model-00005-of-00007.bin
│   │   ├── pytorch_model-00006-of-00007.bin
│   │   ├── pytorch_model-00007-of-00007.bin
│   │   ├── pytorch_model.bin.index.json
│   │   ├── special_tokens_map.json
│   │   ├── tokenizer_config.json
│   │   └── tokenizer.model
In both cases, you can use the "Model" tab of the UI to download the model from Hugging Face automatically. It is also possible to download it via the command-line with

python download-model.py organization/model
Run python download-model.py --help to see all the options.

Google Colab notebook
https://colab.research.google.com/github/oobabooga/text-generation-webui/blob/main/Colab-TextGen-GPU.ipynb

Contributing
If you would like to contribute to the project, check out the Contributing guidelines.

Community
Subreddit: https://www.reddit.com/r/oobabooga/
Discord: https://discord.gg/jwZCF2dPQN
Acknowledgment
In August 2023, Andreessen Horowitz (a16z) provided a generous grant to encourage and support my independent work on this project. I am extremely grateful for their trust and recognition.

About
A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

Resources
 Readme
License
 AGPL-3.0 license
 Activity
Stars
 34.3k stars
Watchers
 298 watching
Forks
 4.6k forks
Report repository
Releases 34
snapshot-2024-03-10
Latest
last week
+ 33 releases
Sponsor this project
ko_fi
ko-fi.com/oobabooga
Packages
No packages published
Contributors
302
@oobabooga
@jllllll
@dependabot[bot]
@mcmonkey4eva
@matatonic
@missionfloyd
@FartyPants
@Ph0rk0z
@TheLounger
@mayaeary
@xanthousm
@Brawlence
@EliasVincent
@nikita-skakun
+ 288 contributors
Deployments
2
 github-pages last year
Languages
Python
90.9%
 
CSS
3.5%
 
JavaScript
2.4%
 
Shell
1.4%
 
Batchfile
0.7%
 
Jupyter Notebook
0.6%
 
Dockerfile
0.5%
Footer
© 2024 GitHub, Inc.
Footer navigation
Terms
Privacy
Security
Status
Docs
Contact
Manage cookies
Do not share my personal information
""