fix batch infer

#10

by kuaizhirui - opened 21 days ago

←

21 days ago

解决左pad之后 batch infer总是输出unk的问题或者和单条样本推理结果不一致的问题，本质上精度不一致的问题，由expanded_attn_mask和combined_attention_mask相加导致的，因此先换成torch.finfo(dtype).min的一半

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Ready to merge

This branch is ready to get merged automatically.

· Sign up or log in to comment