how to stop generating when eos token appears?
#2
by
shanekong
- opened
i hope to stop generating when the eos token appears; i add two lines code in inference.py call() method,
self.generator.eos = self.eos_index
self.generator.real_eos = self.eos_index
but it generates unfluent sentence when it is called in batch.
text = inference.request([inp] * 5, append_right_eos=False)
ok, I will test your case in my environment and give you a response later.