Unable to reproduce the score of gemma_2b at pass@1 in humaneval.
I want to reproduce the humaneval pass@1 score of gemma_2b. When I tested with the parameters below, I got a score of 0.11.
completion = model.generate(input_ids=inputs["input_ids"],
max_length=512,
num_return_sequences=1,
eos_token_id=tokenizer.eos_token_id)
After modifying the parameters as shown below, I got 0.14.
completion = model.generate(input_ids=inputs["input_ids"],
max_length=512,
num_return_sequences=1,
eos_token_id=tokenizer.eos_token_id,
do_sample=True,
temperature=0.7,
top_k=50,
top_p=0.95) # 增加 max_length 的值
I would like to know how to modify the parameters to achieve 0.22.