Candle cannot inference madlad for gguf
#18
by
thanasismem
- opened
I tired to use the rust implementation for the inference and followed all the instructions but some libraries of candle (cudarc 0.11.7) contradict with each other. Could you please provide a guide that you inference the gguf models? I think it would be very useful for many people having the same problem
Or at least specify the versions of candle and cudarc that you used to inference the gguf models