Runtime Error

#28
by prudhvi95 - opened

I am getting the following error when running jais on a large input text. Below is the code. What is the maximum context length for Jais-13b?

Error: The size of tensor a (2048) must match the size of tensor b (2049) at non-singleton dimension 3/

generate_ids = model.generate(
    inputs,
    top_p=0.9,
    temperature=0.3,
    max_new_tokens=1024,
    min_length=input_len + 4,
    repetition_penalty=1.2,
    do_sample=True,
)

for JAIS-13B and JAIS-30B-v1 it is 2048. for JAIS-30B-v3 it is 8192. it says so in the whitepaper

Sign up or log in to comment