Text2Text Generation
Transformers
GGUF
101 languages
t5
text-generation-inference
kcoopermiller commited on
Commit
77d827e
1 Parent(s): 954769e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -138,6 +138,8 @@ cargo run --example quantized-t5 --release -- \
138
  --temperature 0
139
  ```
140
 
 
 
141
  Available weight files:
142
  - aya-101.Q2_K.gguf
143
  - aya-101.Q3_K.gguf
 
138
  --temperature 0
139
  ```
140
 
141
+ Note: this runs on CPU. To use GPU, replace `Device::Cpu` with `Device::new_cuda(0)?` in `candle/candle-examples/examples/quantized-t5/main.rs`
142
+
143
  Available weight files:
144
  - aya-101.Q2_K.gguf
145
  - aya-101.Q3_K.gguf