Spaces:
Sleeping
Sleeping
Felix Marty
commited on
Commit
•
17ca086
1
Parent(s):
590064e
better text
Browse files- app.py +4 -2
- defaults.py +1 -1
app.py
CHANGED
@@ -15,11 +15,13 @@ with gr.Blocks() as demo:
|
|
15 |
"""
|
16 |
Let's try out TorchServe + BetterTransformer!
|
17 |
|
18 |
-
BetterTransformer is a feature made available with PyTorch 1.13. allowing to use a fastpath execution for encoder attention blocks.
|
19 |
|
20 |
-
As a one-liner, you can use BetterTransformer
|
21 |
|
22 |
```
|
|
|
|
|
23 |
better_model = BetterTransformer.transform(model)
|
24 |
```
|
25 |
|
|
|
15 |
"""
|
16 |
Let's try out TorchServe + BetterTransformer!
|
17 |
|
18 |
+
BetterTransformer is a stable feature made available with [PyTorch 1.13](https://pytorch.org/blog/PyTorch-1.13-release/) allowing to use a fastpath execution for encoder attention blocks.
|
19 |
|
20 |
+
As a one-liner, you can convert your 🤗 Transformers models to use BetterTransformer thanks to the [🤗 Optimum](https://huggingface.co/docs/optimum/main/en/index) library:
|
21 |
|
22 |
```
|
23 |
+
from optimum.bettertransformer import BetterTransformer
|
24 |
+
|
25 |
better_model = BetterTransformer.transform(model)
|
26 |
```
|
27 |
|
defaults.py
CHANGED
@@ -35,4 +35,4 @@ BATCH_SIZE = 8 # fixed!
|
|
35 |
|
36 |
HEADERS = {"Content-Type": "text/plain"}
|
37 |
ADDRESS_VANILLA = "http://3.83.142.46:8080/predictions/my_tc"
|
38 |
-
ADDRESS_BETTERTRANSFORMER = "http://3.95.
|
|
|
35 |
|
36 |
HEADERS = {"Content-Type": "text/plain"}
|
37 |
ADDRESS_VANILLA = "http://3.83.142.46:8080/predictions/my_tc"
|
38 |
+
ADDRESS_BETTERTRANSFORMER = "http://3.95.136.2:8080/predictions/my_tc"
|