最新的commit好像有bug,但是不确定该怎么修复
#26
by
qingsonglv
- opened
https://huggingface.co/THUDM/chatglm-6b/commit/fdb7a601d8f8279806124542e11549bdd76f62f6
这段代码:
tokens = tokenizer(text, return_tensors='pt')['input_ids'].cuda()
gen_kwargs = {"max_length": 512, "num_beams": 1, "do_sample": True, "top_p": 0.7,
"temperature": 0.95}
outputs = model.generate(input_ids=tokens, **gen_kwargs)
preds = tokenizer.batch_decode(outputs, skip_special_tokens=True)
用最新的版本decode会报这个错误:
Traceback (most recent call last):
File "inference_adgen.py", line 28, in <module>
preds = tokenizer.batch_decode(outputs, skip_special_tokens=True)
File "/usr/local/lib/python3.8/dist-packages/transformers/tokenization_utils_base.py", line 3437, in batch_d
ecode
return [
File "/usr/local/lib/python3.8/dist-packages/transformers/tokenization_utils_base.py", line 3438, in <listco
mp>
self.decode(
File "/root/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b/fdb7a601d8f8279806124542e11549b
dd76f62f6/tokenization_chatglm.py", line 276, in decode
if self.pad_token_id in token_ids: # remove pad
RuntimeError: Boolean value of Tensor with more than one value is ambiguous
断点打印对应的变量如下:
> /root/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b/fdb7a601d8f8279806124542e11549bdd76f6
2f6/tokenization_chatglm.py(277)decode()
-> if self.pad_token_id in token_ids: # remove pad
(Pdb) p self.pad_token_id
20003
(Pdb) p token_ids
[tensor([ 20005, 85421, 20061, 95898, 20032, 88554, 20061, 97257, 84555,
20032, 85107, 20061, 86268, 20032, 85347, 20061, 91689, 20032,
89768, 20061, 105428, 20032, 85173, 93942, 20061, 90984, 20032,
85173, 90936, 20061, 84703, 85509, 150001, 150004, 20005, 84703,
85509, 94681, 83829, 85388, 85023, 92871, 106418, 83825, 94707,
20006, 83827, 94681, 83839, 84690, 105428, 89768, 20006, 85109,
94705, 107834, 20006, 83887, 98530, 85425, 116447, 20006, 97284,
83842, 101233, 109473, 109473, 83825, 139235, 83823, 92194, 91231,
20006, 87122, 91689, 20006, 86955, 86800, 83823, 150005],
device='cuda:0')]
(Pdb)
Fixed
zxdu20
changed discussion status to
closed