nsarrazin HF staff commited on
Commit
fae93d9
1 Parent(s): 002a2a0

Added a few sections to the readme and reorganized it (#264)

Browse files
Files changed (1) hide show
  1. README.md +86 -37
README.md CHANGED
@@ -16,64 +16,113 @@ app_port: 3000
16
 
17
  A chat interface using open source models, eg OpenAssistant. It is a SvelteKit app and it powers the [HuggingChat app on hf.co/chat](https://huggingface.co/chat).
18
 
19
- ## Launch
 
 
 
 
 
 
 
 
 
 
20
 
21
  ```bash
22
- npm install
23
- npm run dev
24
  ```
25
 
26
- ## Environment
27
-
28
- Default configuration is in `.env`. Put custom config and secrets in `.env.local`, it will override the values in `.env`.
29
 
30
- Check out [.env](./.env) to see what needs to be set.
31
 
32
- Basically you need to create a `.env.local` with the following contents:
33
 
34
- ```
35
- MONGODB_URL=<url to mongo, for example a free MongoDB Atlas sandbox instance>
36
- HF_ACCESS_TOKEN=<your HF access token from https://huggingface.co/settings/tokens>
37
  ```
38
 
39
- ## Duplicating to a Space
40
 
41
- Create a `DOTENV_LOCAL` secret to your space with the following contents:
42
 
43
- ```
44
- MONGODB_URL=<url to mongo, for example a free MongoDB Atlas sandbox instance>
45
- HF_ACCESS_TOKEN=<your HF access token from https://huggingface.co/settings/tokens>
46
- ```
47
 
48
- Where the contents in `<...>` are replaced by the MongoDB URL and your [HF Access Token](https://huggingface.co/settings/tokens).
49
 
50
- ## Running Local Inference
51
 
52
- Both the example above use the HF Inference API or HF Endpoints API.
53
 
54
- If you want to run the model locally, you need to run this inference server locally: https://github.com/huggingface/text-generation-inference
 
 
 
55
 
56
- And add this to your `.env.local`, feel free to adjust/remove the parameters and the preprompt:
57
 
 
 
 
 
 
 
 
 
58
  ```
59
- MODELS=`[{
60
- "name": "...",
61
- "endpoints": [{"url": "http://127.0.0.1:8080/generate_stream"}],
62
- "userMessageToken": "<|prompter|>",
63
- "assistantMessageToken": "<|assistant|>",
64
- "messageEndToken": "</s>",
65
- "preprompt": "Below are a series of dialogues between various people and an AI assistant. The AI tries to be helpful, polite, honest, sophisticated, emotionally aware, and humble-but-knowledgeable. The assistant is happy to help with almost anything, and will do its best to understand exactly what is needed. It also tries to avoid giving false or misleading information, and it caveats when it isn't entirely sure about the right answer. That said, the assistant is practical and really does its best, and doesn't let caution get too much in the way of being useful.\n-----\n",
66
- "parameters": {
67
- "temperature": 0.9,
68
- "top_p": 0.95,
69
- "repetition_penalty": 1.2,
70
- "top_k": 50,
71
- "truncate": 1000,
72
- "max_new_tokens": 1000
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
73
  }
74
- }]`
75
  ```
76
 
 
 
 
 
 
 
 
 
 
 
77
  ## Building
78
 
79
  To create a production version of your app:
 
16
 
17
  A chat interface using open source models, eg OpenAssistant. It is a SvelteKit app and it powers the [HuggingChat app on hf.co/chat](https://huggingface.co/chat).
18
 
19
+ 1. [Setup](#setup)
20
+ 2. [Launch](#launch)
21
+ 3. [Extra parameters](#extra-parameters)
22
+ 4. [Deploying to a HF Space](#deploying-to-a-hf-space)
23
+ 5. [Building](#building)
24
+
25
+ ## Setup
26
+
27
+ The default config for Chat UI is stored in the `.env` file. You will need to override some values to get Chat UI to run locally. This is done in `.env.local`.
28
+
29
+ Start by creating a `.env.local` file in the root of the repository. The bare minimum config you need to get Chat UI to run locally is the following:
30
 
31
  ```bash
32
+ MONGODB_URL=<the URL to your mongoDB instance>
33
+ HF_ACCESS_TOKEN=<your access token>
34
  ```
35
 
36
+ ### Database
 
 
37
 
38
+ The chat history is stored in a MongoDB instance, and having a DB instance available is needed for Chat UI to work.
39
 
40
+ You can use a local MongoDB instance. The easiest way is to spin one up is using docker:
41
 
42
+ ```bash
43
+ docker run -d -p 27017:27017 --name mongo-chatui mongo:latest
 
44
  ```
45
 
46
+ In which case the url of your DB will be `MONGODB_URL=mongodb://localhost:27017`.
47
 
48
+ Alternatively, you can use a [free MongoDB Atlas](https://www.mongodb.com/pricing) instance for this, Chat UI should fit comfortably within the free tier. After which you can set the `MONGODB_URL` variable in `.env.local` to match your instance.
49
 
50
+ ### Hugging Face Access Token
 
 
 
51
 
52
+ You will need a Hugging Face access token to run Chat UI locally, using the remote inference endpoints. You can get one from [your Hugging Face profile](https://huggingface.co/settings/tokens).
53
 
54
+ ## Launch
55
 
56
+ After you're done with the `.env.local` file you can run Chat UI locally with:
57
 
58
+ ```bash
59
+ npm install
60
+ npm run dev
61
+ ```
62
 
63
+ ## Extra parameters
64
 
65
+ ### OpenID connect
66
+
67
+ The login feature is disabled by default and users are attributed a unique ID based on their browser. But if you want to use OpenID to authenticate your users, you can add the following to your `.env.local` file:
68
+
69
+ ```bash
70
+ OPENID_PROVIDER_URL=<your OIDC issuer>
71
+ OPENID_CLIENT_ID=<your OIDC client ID>
72
+ OPENID_CLIENT_SECRET=<your OIDC client secret>
73
  ```
74
+
75
+ These variables will enable the openID sign-in modal for users.
76
+
77
+ ### Custom models
78
+
79
+ You can customize the parameters passed to the model or even use a new model by updating the `MODELS` variable in your `.env.local`. The default one can be found in `.env` and looks like this :
80
+
81
+ ```json
82
+ MODELS=`[
83
+ {
84
+ "name": "OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5",
85
+ "datasetName": "OpenAssistant/oasst1",
86
+ "description": "A good alternative to ChatGPT",
87
+ "websiteUrl": "https://open-assistant.io",
88
+ "userMessageToken": "<|prompter|>",
89
+ "assistantMessageToken": "<|assistant|>",
90
+ "messageEndToken": "</s>",
91
+ "preprompt": "Below are a series of dialogues between various people and an AI assistant. The AI tries to be helpful, polite, honest, sophisticated, emotionally aware, and humble-but-knowledgeable. The assistant is happy to help with almost anything, and will do its best to understand exactly what is needed. It also tries to avoid giving false or misleading information, and it caveats when it isn't entirely sure about the right answer. That said, the assistant is practical and really does its best, and doesn't let caution get too much in the way of being useful.\n-----\n",
92
+ "promptExamples": [
93
+ {
94
+ "title": "Write an email from bullet list",
95
+ "prompt": "As a restaurant owner, write a professional email to the supplier to get these products every week: \n\n- Wine (x10)\n- Eggs (x24)\n- Bread (x12)"
96
+ }, {
97
+ "title": "Code a snake game",
98
+ "prompt": "Code a basic snake game in python, give explanations for each step."
99
+ }, {
100
+ "title": "Assist in a task",
101
+ "prompt": "How do I make a delicious lemon cheesecake?"
102
+ }
103
+ ],
104
+ "parameters": {
105
+ "temperature": 0.9,
106
+ "top_p": 0.95,
107
+ "repetition_penalty": 1.2,
108
+ "top_k": 50,
109
+ "truncate": 1000,
110
+ "max_new_tokens": 1024
111
+ }
112
  }
113
+ ]`
114
  ```
115
 
116
+ You can change things like the parameters, or customize the preprompt to better suit your needs. You can also add more models by adding more objects to the array, with different preprompts for example.
117
+
118
+ ### Running your own models
119
+
120
+ If you want to, you can even run your own models, by having a look at our endpoint project, [text-generation-inference](https://github.com/huggingface/text-generation-inference). You can then add your own endpoint to the `MODELS` variable in `.env.local` and it will be picked up as well.
121
+
122
+ ## Deploying to a HF Space
123
+
124
+ Create a `DOTENV_LOCAL` secret to your HF space with the content of your .env.local, and they will be picked up automatically when you run.
125
+
126
  ## Building
127
 
128
  To create a production version of your app: