Spaces:
Sleeping
Sleeping
File size: 2,957 Bytes
24d11d4 a388980 24d11d4 4c4fd67 48ea851 f177e4b 24d11d4 a388980 11dbfc8 a265324 0fe4fa3 11dbfc8 a388980 11dbfc8 a388980 11dbfc8 d16d319 3ebd805 5741be4 a3810f8 5741be4 7764421 5741be4 7764421 5741be4 a3810f8 7764421 31ef570 7764421 31ef570 7764421 11dbfc8 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 |
---
title: chat-ui
emoji: 🔥
colorFrom: purple
colorTo: purple
sdk: docker
pinned: false
license: apache-2.0
base_path: /chat
app_port: 3000
---
# Chat UI
A chat interface using open source models, eg OpenAssistant. It is a SvelteKit app and it powers the [HuggingChat app on hf.co/chat](https://huggingface.co/chat).
## Launch
```bash
npm install
npm run dev
```
## Environment
Default configuration is in `.env`. Put custom config and secrets in `.env.local`, it will override the values in `.env`.
Check out [.env](./.env) to see what needs to be set.
Basically you need to create a `.env.local` with the following contents:
```
MONGODB_URL=<url to mongo, for example a free MongoDB Atlas sandbox instance>
HF_ACCESS_TOKEN=<your HF access token from https://huggingface.co/settings/tokens>
```
## Duplicating to a Space
Create a `DOTENV_LOCAL` secret to your space with the following contents:
```
MONGODB_URL=<url to mongo, for example a free MongoDB Atlas sandbox instance>
HF_ACCESS_TOKEN=<your HF access token from https://huggingface.co/settings/tokens>
```
Where the contents in `<...>` are replaced by the MongoDB URL and your [HF Access Token](https://huggingface.co/settings/tokens).
## Running Local Inference
Both the example above use the HF Inference API or HF Endpoints API.
If you want to run the model locally, you need to run this inference server locally: https://github.com/huggingface/text-generation-inference
And add this to your `.env.local`, feel free to adjust/remove the parameters and the preprompt:
```
MODELS=`[{
"name": "...",
"endpoints": [{"url": "http://127.0.0.1:8080/generate_stream"}],
"userMessageToken": "<|prompter|>",
"assistantMessageToken": "<|assistant|>",
"messageEndToken": "</s>",
"preprompt": "Below are a series of dialogues between various people and an AI assistant. The AI tries to be helpful, polite, honest, sophisticated, emotionally aware, and humble-but-knowledgeable. The assistant is happy to help with almost anything, and will do its best to understand exactly what is needed. It also tries to avoid giving false or misleading information, and it caveats when it isn't entirely sure about the right answer. That said, the assistant is practical and really does its best, and doesn't let caution get too much in the way of being useful.\n-----\n",
"parameters": {
"temperature": 0.9,
"top_p": 0.95,
"repetition_penalty": 1.2,
"top_k": 50,
"truncate": 1000,
"max_new_tokens": 1000
}
}]`
```
## Building
To create a production version of your app:
```bash
npm run build
```
You can preview the production build with `npm run preview`.
> To deploy your app, you may need to install an [adapter](https://kit.svelte.dev/docs/adapters) for your target environment.
|