Open Ended Generation
#7
by
nik8482
- opened
Hi,
Is there anyway to stop the model from asking itself questions after generating an initial response? This is an example of responses I'm currently getting
Ideally I want the model to stop generating after the first response, I could reduce the max tokens length however I get responses which seem to get cut off midway
Thanks for the help!
Add "<human>:" as a stop word.
Example: (here I have ["<human>:"] inside the variable self.model_conf['stop_words'])
self.stop_words_ids = [
self.tokenizer(stop_word, return_tensors="pt")["input_ids"].squeeze()
for stop_word in self.model_conf["stop_words"]
]
self.stopping_criteria = StoppingCriteriaList(
[StoppingCriteriaSub(stops=self.stop_words_ids)]
)
and then add this as parameter to your generate() call:
stopping_criteria=self.stopping_criteria,
Sven i think we work on the same project. Let's catch up. :-)