starcoder model.generate params
#57
by
vplechuc
- opened
Hi, where I can find which params are accepted for model.generate(...) function if model is defined as
model = AutoModelForCausalLM.from_pretrained(base_model).to(device) (here device is "cuda")
Is there any docs that I could check?
Basically I need to know if I can tell to the model the "temperature" I want to use and how many "results" I want on the output
Thanks
you can check these docs: https://huggingface.co/docs/transformers/v4.34.0/en/main_classes/text_generation#transformers.GenerationMixin.generate It does accept temperature and num_return_sequences.
loubnabnl
changed discussion status to
closed