Generate: sequence bias can handle same terminations
Author: gante
Pull request: yes

# What does this PR do?

Fixes the issue raised by @stas00 [here](https://github.com/huggingface/transformers/pull/24334#issuecomment-1631324670).

In a nutshell, when I designed `SequenceBiasLogitsProcessor`, I've committed the fallacy of early optimization -- the resulting solution was not compatible with biasing sequences that had the same termination. This PR opts for a simpler solution that is more inclusive (but probably slower).

Existing tests and docstring example passing 👍 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hi @gante, 

Thank you for trying to fix it. The failure is still there with this PR's branch.

```
  File "/mnt/nvme0/code/huggingface/m4-master/m4/evaluation/launch.py", line 143, in <module>
    main(args)
  File "/mnt/nvme0/code/huggingface/m4-master/m4/evaluation/launch.py", line 97, in main
    score = evaluator(task, accelerator, model, args)
  File "/mnt/nvme0/code/huggingface/m4-master/m4/evaluation/evaluators/in_contexter.py", line 263, in in_contexter
    metric = task.add_batch_metric(metric, **kwargs)
  File "/mnt/nvme0/code/huggingface/m4-master/m4/models/vgpt2/evaluation_open_ended_vqa_in_context_vgpt2.py", line 336, in add_batch_metric
    generated_tokens = self.generate_tokens(**kwargs)
  File "/mnt/nvme0/code/huggingface/m4-master/m4/models/vgpt2/evaluation_open_ended_vqa_in_context_vgpt2.py", line 312, in generate_tokens
    generated_tokens = unwrapped_model.generate(
  File "/home/stas/anaconda3/envs/py38-pt20/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/mnt/nvme0/code/huggingface/transformers-master-2/src/transformers/generation/utils.py", line 1613, in generate
    return self.beam_search(
  File "/mnt/nvme0/code/huggingface/transformers-master-2/src/transformers/generation/utils.py", line 2930, in beam_search
    next_token_scores_processed = logits_processor(input_ids, next_token_scores)
  File "/mnt/nvme0/code/huggingface/transformers-master-2/src/transformers/generation/logits_process.py", line 92, in __call__
    scores = processor(input_ids, scores)
  File "/mnt/nvme0/code/huggingface/transformers-master-2/src/transformers/generation/logits_process.py", line 618, in __call__
    self._prepare_bias_variables(scores)
  File "/mnt/nvme0/code/huggingface/transformers-master-2/src/transformers/generation/logits_process.py", line 674, in _prepare_bias_variables
    raise ValueError(
ValueError: Setting a bias on sequences that share a common token termination is not yet supported. Please open an issue if you see this error message (after checking that it doesn't already exist).
```

@stas00 that is odd -- the exception was entirely removed 👀 

On my end, I can't reach the exception as before. Can I kindly request to try again? 🤗 (and, if it still fails, may I have some way to reproduce it?)

ah, my bad. somehow I got the wrong branch I think. I have retested with the latest of this PR and the problem is no more. Thank you for the fix, Joao

oh, I know. I was on the branch of the original PR #24334 - hence the confusion