bigscience

/

bloom

like 4.56k

Previous scores did not strip end of sequence tokens. The updated scores ignore the end of sequence tokens (</s>).
This is equivalent to evaluating on code generations decoded with tokenizer.decode(code_tokens, skip_special_tokens=True)

cc @TimeRobber @loubnabnl