Can't use in transformer
#1
by
snoopydev
- opened
OSError: TehVenom/Pygmalion-13b-8bit-GPTQ does not appear to have a file named pytorch_model.bin, tf_model.h5, model.ckpt or flax_model.msgpack
Please read the model card, this is not a pytorch model. It has been quantized for GPTQ.
Then how to utilize this model in transformer?
Yeah can you toss in some starter code to run inference on this?