doesn't work with facebook/bart-base
hi,
great work!
Tried it out and works for t5-small but throws an error for facebook/bart-base
do you happen to know if this will be fixed by huggingface (and on what timeline?)
NotImplementedError: in user code:
File "/usr/local/lib/python3.7/dist-packages/transformers/generation_tf_utils.py", line 605, in generate *
seed=model_kwargs.pop("seed", None),
File "/usr/local/lib/python3.7/dist-packages/transformers/generation_tf_utils.py", line 1687, in _generate *
input_ids,
File "/usr/local/lib/python3.7/dist-packages/transformers/generation_tf_utils.py", line 2854, in beam_search_body_fn *
log_probs = logits_processor(flatten_beam_dim(running_sequences), flatten_beam_dim(log_probs), cur_len)
File "/usr/local/lib/python3.7/dist-packages/transformers/generation_tf_logits_process.py", line 94, in __call__ *
scores = processor(input_ids, scores, cur_len)
File "/usr/local/lib/python3.7/dist-packages/transformers/generation_tf_logits_process.py", line 427, in __call__ *
raise NotImplementedError("TFNoRepeatNGramLogitsProcessor is only implemented for eager execution.")
NotImplementedError: TFNoRepeatNGramLogitsProcessor is only implemented for eager execution.
Hi @OttoZastrow !
Sadly, I couldn't make an XLA-compatible version of the logit processor that is triggered with the no_repeat_ngram_size generate option, so you can't use that flag with XLA. You can still use it without XLA, though.
The processor, as it is coded now, relies on dynamic slicing of data in a way that is not supported by XLA at the moment. That implies that we either wait for XLA support, or find a way of working around this bottleneck. I can't provide an estimate of a timeline to fix it, but I'm confident that our development team won't have the resources to tackle the issue for the next 2-3 months :)
