Update README.md
Browse files
README.md
CHANGED
|
@@ -138,6 +138,8 @@ cargo run --example quantized-t5 --release -- \
|
|
| 138 |
--temperature 0
|
| 139 |
```
|
| 140 |
|
|
|
|
|
|
|
| 141 |
Available weight files:
|
| 142 |
- aya-101.Q2_K.gguf
|
| 143 |
- aya-101.Q3_K.gguf
|
|
|
|
| 138 |
--temperature 0
|
| 139 |
```
|
| 140 |
|
| 141 |
+
Note: this runs on CPU. To use GPU, replace `Device::Cpu` with `Device::new_cuda(0)?` in `candle/candle-examples/examples/quantized-t5/main.rs`
|
| 142 |
+
|
| 143 |
Available weight files:
|
| 144 |
- aya-101.Q2_K.gguf
|
| 145 |
- aya-101.Q3_K.gguf
|