Spaces:
Runtime error
Runtime error
andreasmadsen
commited on
Commit
•
fc7333a
1
Parent(s):
2e6d1bb
fix docs regarding TGI endpoint url (#408)
Browse filesThe README specify to use a `/generate_stream` url, when using a custom
TGI endpoint. This is incorrect, as it breaks APIs, such as the
conversation summary that depends on the `/` TGI interface.
This turns out to be a documentation-only bug, as `/` will point to
`/generate_stream` when `stream: true` is set in the request JSON.
Simply using the `/` url as the TGI endpoint therefore fixes the
summarization bug.
Fixes: #278
README.md
CHANGED
@@ -164,7 +164,7 @@ To do this, you can add your own endpoints to the `MODELS` variable in `.env.loc
|
|
164 |
|
165 |
{
|
166 |
// rest of the model config here
|
167 |
-
"endpoints": [{"url": "https://HOST:PORT
|
168 |
}
|
169 |
|
170 |
```
|
@@ -191,7 +191,7 @@ You can then add the generated information and the `authorization` parameter to
|
|
191 |
|
192 |
"endpoints": [
|
193 |
{
|
194 |
-
"url": "https://HOST:PORT
|
195 |
"authorization": "Basic VVNFUjpQQVNT",
|
196 |
}
|
197 |
]
|
@@ -232,11 +232,11 @@ If the model being hosted will be available on multiple servers/instances add th
|
|
232 |
|
233 |
"endpoints": [
|
234 |
{
|
235 |
-
"url": "https://HOST:PORT
|
236 |
"weight": 1
|
237 |
}
|
238 |
{
|
239 |
-
"url": "https://HOST:PORT
|
240 |
"weight": 2
|
241 |
}
|
242 |
...
|
|
|
164 |
|
165 |
{
|
166 |
// rest of the model config here
|
167 |
+
"endpoints": [{"url": "https://HOST:PORT"}]
|
168 |
}
|
169 |
|
170 |
```
|
|
|
191 |
|
192 |
"endpoints": [
|
193 |
{
|
194 |
+
"url": "https://HOST:PORT",
|
195 |
"authorization": "Basic VVNFUjpQQVNT",
|
196 |
}
|
197 |
]
|
|
|
232 |
|
233 |
"endpoints": [
|
234 |
{
|
235 |
+
"url": "https://HOST:PORT",
|
236 |
"weight": 1
|
237 |
}
|
238 |
{
|
239 |
+
"url": "https://HOST:PORT",
|
240 |
"weight": 2
|
241 |
}
|
242 |
...
|