THUDM

/

chatglm2-6b-int4

like 230

I get the same error when I run

from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm2-6b-32k-int4", trust_remote_code=True)
model = AutoModel.from_pretrained("THUDM/chatglm2-6b-32k-int4", trust_remote_code=True).half().cuda()
model = model.eval()

error: AttributeError: 'NoneType' object has no attribute 'int4WeightExtractionHalf'