google

/

gemma-2-9b-it

like 279

I came across the following error once I increase the number of "num_return_sequences" to any value greater then 1. Any idea?
"RuntimeError: shape mismatch: value tensor of shape [32, 8, 612, 256] cannot be broadcast to indexing result of shape [16, 8, 612, 256]"