模型推理脚本报错

#29
by Jay0324 - opened

企业微信截图_ac735786-1d41-4ea8-b3a3-5a8c7e08369f.png

最后一步 outputs = model.generate(**inputs, **gen_kwargs)的时候报错

kv_cache 居然是一个字符串'past_key_values' ,而且device_map的问题也没有解决,还要自己去找各个设备不统一的点。。

Knowledge Engineering Group (KEG) & Data Mining at Tsinghua University org

按照github的要求装依赖,transformers是4.40

transformers改为4.40 ok了

zRzRzRzRzRzRzR changed discussion status to closed

Sign up or log in to comment