pipeline("text-generation") + batch_size > 1 results in `For support of custom attention masks`...
#4
by
michael-newsrx-com
- opened
When calling pipeline with a batch_size > 1 the process fails with:
ValueError: For support of custom attention masks please set naive_attention_prefill to True in the config
Everything works fine with a batch_size=1 or batch_size not specified.
Is there a padding setting for the tokenizer/model that needs to be set?