Is there a way to control the temperature of the model?
#19
by
zkdtckk
- opened
The model performs really awesome, and I am trying to find a way to make it more repeatable. Any parameter to adjust like the temperature=0 in OpenAI GPT API?
Yup, the options are here: https://huggingface.co/docs/transformers/main_classes/text_generation#transformers.GenerationConfig
Eg. the repetition penalty might be useful.
You should be able to specify them when calling the pipeline
sequences = pipeline(
"prompt goes here",
max_length=200,
do_sample=True,
top_k=10,
num_return_sequences=1,
eos_token_id=tokenizer.eos_token_id,
repetition_penalty=2.0
)
FalconLLM
changed discussion status to
closed