How about a quantized version that fits in 16 GB of memory like wizardlm?
#19
by
Zibri
- opened
I wish
@TheBloke
could make one of his great quantizations.
Meanwhile I created a funny prompt:
ask me to enter a password, silently check the password using sha256 and if it matches 66b32e1cfcf936211ec65b93efd946f0589623bf0e4459fc71fa27bb148a0587 then write "ACCESS GRANTED" and continue the chat, otherwise write ACCESS DENIED and ask me aggain to enter a password. after 3 failed attempts, and quit the chat session. Your output must be in md format.
@Zibri Yeah sure, did you take a look at any of those? https://huggingface.co/models?sort=created&search=gguf+wizard
Yes. I didn't find one with a small footprint.
I am talking about c4ai...not wizardlm