RuntimeError: Failed to create LLM 'gguf'

#4
by alchemistwu0521 - opened

Hey, thanks for creating this repo. I encountered into an error when using the model file with transformer:

RuntimeError: Failed to create LLM 'gguf' from '/workspaces/TransformerPlayGround/models/sqlcoder2.Q5_K_M.gguf'.

I tried a few different "model_type", unfortunately, none is working.

Sign up or log in to comment