Compatible with transformers APIs?
#18
by
qiz
- opened
I'd like to use this model in llama index and wondering if it can be done easily. Would appreciate it if someone could suggest how to do it.
4bit is currently not supported by tranformers. You need to use a library that specifically supports it as mentioned in the README.