nsarrazin HF staff commited on
Commit
e34af36
1 Parent(s): f567f41

Fix README linting & add details about auth

Browse files
Files changed (1) hide show
  1. README.md +18 -11
README.md CHANGED
@@ -117,28 +117,35 @@ You can change things like the parameters, or customize the preprompt to better
117
 
118
  ### Running your own models using a custom endpoint
119
 
120
- If you want to, you can even run your own models, by having a look at our endpoint project, [text-generation-inference](https://github.com/huggingface/text-generation-inference). You can then add your own endpoint to the `MODELS` variable in `.env.local`. Using the default `.env` information provided above as an example, the endpoint information is added after `websiteUrl` and before `userMessageToken` parameters.
121
 
122
  ```
123
- "websiteUrl": "https://open-assistant.io",
124
- "endpoints": [{"url": "https://HOST:PORT/generate_stream"}],
125
- "userMessageToken": "<|prompter|>",
 
126
  ```
127
 
 
 
128
  ### Custom endpoint authorization
129
 
130
- Custom endpoints may require authorization. In those situations, we will need to generate a base64 encoding of the username and password.
 
 
131
 
132
  `echo -n "USER:PASS" | base64`
133
 
134
  > VVNFUjpQQVNT
135
 
 
 
136
  You can then add the generated information and the `authorization` parameter to your `.env.local`.
137
 
138
  ```
139
- "endpoints": [
140
  {
141
- "url": "https://HOST:PORT/generate_stream",
142
  "authorization": "Basic VVNFUjpQQVNT",
143
  }
144
  ]
@@ -146,16 +153,16 @@ You can then add the generated information and the `authorization` parameter to
146
 
147
  ### Models hosted on multiple custom endpoints
148
 
149
- If the model being hosted will be available on multiple servers/instances add the `weight` parameter to your `.env.local`.
150
 
151
  ```
152
- "endpoints": [
153
  {
154
- "url": "https://HOST:PORT/generate_stream",
155
  "weight": 1
156
  }
157
  {
158
- "url": "https://HOST:PORT/generate_stream",
159
  "weight": 2
160
  }
161
  ...
 
117
 
118
  ### Running your own models using a custom endpoint
119
 
120
+ If you want to, you can even run your own models locally, by having a look at our endpoint project, [text-generation-inference](https://github.com/huggingface/text-generation-inference). You can then add your own endpoints to the `MODELS` variable in `.env.local`, by adding an `"endpoints"` key for each model in `MODELS`.
121
 
122
  ```
123
+ {
124
+ // rest of the model config here
125
+ "endpoints": [{"url": "https://HOST:PORT/generate_stream"}]
126
+ }
127
  ```
128
 
129
+ If `endpoints` is left unspecified, ChatUI will look for the model on the hosted Hugging Face inference API using the model name.
130
+
131
  ### Custom endpoint authorization
132
 
133
+ Custom endpoints may require authorization, depending on how you configure them. Authentication will usually be set either with `Basic` or `Bearer`.
134
+
135
+ For `Basic` we will need to generate a base64 encoding of the username and password.
136
 
137
  `echo -n "USER:PASS" | base64`
138
 
139
  > VVNFUjpQQVNT
140
 
141
+ For `Bearer` you can use a token, which can be grabbed from [here](https://huggingface.co/settings/tokens).
142
+
143
  You can then add the generated information and the `authorization` parameter to your `.env.local`.
144
 
145
  ```
146
+ "endpoints": [
147
  {
148
+ "url": "https://HOST:PORT/generate_stream",
149
  "authorization": "Basic VVNFUjpQQVNT",
150
  }
151
  ]
 
153
 
154
  ### Models hosted on multiple custom endpoints
155
 
156
+ If the model being hosted will be available on multiple servers/instances add the `weight` parameter to your `.env.local`. The `weight` will be used to determine the probability of requesting a particular endpoint.
157
 
158
  ```
159
+ "endpoints": [
160
  {
161
+ "url": "https://HOST:PORT/generate_stream",
162
  "weight": 1
163
  }
164
  {
165
+ "url": "https://HOST:PORT/generate_stream",
166
  "weight": 2
167
  }
168
  ...