How is the speed? It is very slow with 8 A100s
#8 opened 11 months ago
by
yh-yao
![](https://cdn-avatars.huggingface.co/v1/production/uploads/64b07532bf2caf5584ac1c85/RR1ybIffgvDz6FrJEvpQo.jpeg)
4 Bit hf version here
1
#7 opened 11 months ago
by
srinivasbilla
Trying to load on 8xA10 in 4 bit gives this error
5
#6 opened 11 months ago
by
nbilla
safetensors
#4 opened 11 months ago
by
v2ray
![](https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/fTCV7VLY0eK4OXbwgIT2n.png)
Lets Quantize
8
#1 opened 11 months ago
by
simsim314