Constraint decoding using long constraints, model keeps resetting the constraint
Author: gansem
Pull request: no
Labels: name: bug

### System Info

transformers: 4.21.0
torch: 1.11.0+cpu
python: 3.8.10
Linux-5.13.0-1033-gcp

### Who can help?

@patrickvonplaten,
@Narsil
@patil-suraj

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Hi, I am currently working with the constrained decoding methods of the transformers. I have found that when the constraints get very long the model will at some point reset the constraint. Once the constraint is reset the model is no longer able to generate a sensible sequence and will thus keep resetting the constraint every couple tokens. I have posted a small example below, in this example I feed the entire text as the constraint and simply want the model to output the input:

```ruby
from transformers import BartTokenizer, BartForConditionalGeneration, PhrasalConstraint

tokenizer = BartTokenizer.from_pretrained("facebook/bart-base", extra_ids=0)
model = BartForConditionalGeneration.from_pretrained("facebook/bart-base")

long_text = """Amsterdam is the capital and most populous city of the Netherlands; with a population of 907,976 within the city proper, 1,558,755 in the urban area and 2,480,394 in the metropolitan area. Found within the Dutch province of North Holland, Amsterdam is colloquially referred to as the "Venice of the North", due to the large number of canals which form a UNESCO World Heritage Site. Amsterdam was founded at the Amstel, that was dammed to control flooding; the city's name derives from the Amstel dam. Originating as a small fishing village in the late 12th century, Amsterdam became one of the most important ports in the world during the Dutch Golden Age of the 17th century, and became the leading centre for the finance and trade sectors. In the 19th and 20th centuries, the city expanded and many new neighborhoods and suburbs were planned and built. The 17th-century canals of Amsterdam and the 19–20th century Defence Line of Amsterdam are on the UNESCO World Heritage List. Sloten, annexed in 1921 by the municipality of Amsterdam, is the oldest part of the city, dating to the 9th century."""

input_ids = tokenizer(long_text, 
                        return_tensors="pt", 
                        padding=True, 
                        truncation=True, 
                        max_length=1024).input_ids

print(input_ids.shape)
force_article_ids = input_ids[0].tolist()

constraints = [PhrasalConstraint(force_article_ids)]

outputs = model.generate(input_ids[0].unsqueeze(0), 
                            max_length=512,
                            num_beams=2,
                            constraints=constraints)

tokenizer.decode(outputs[0], skip_special_tokens=True)
```

### Expected behavior

The expected output here should be the same as the input, however, the output I get is:

` ` `
Amsterdam is the capital and most populous city of the Netherlands; with a population of 907,976 within the city proper, 1,558,755 in the urban area and 2,480,394 in the metropolitan area. Found within the Dutch province of North Holland, Amsterdam is colloquially referred to as the "Venice of the North", due to the large number of canals which form a UNESCO World Heritage Site. Amsterdam was founded at the Amstel, that was dammed to control flooding; the city\'s name derives from the AmsterelAmstAmsterAmstadAmestAmstrAm stAm StAmStAmstedAm AmsterdamAmstadtAmstaAmstenAmsteAmstraAmtAmetAmthAmothAmstonAmlandAmastAmselAmandAmantAmatAmartAmAmAmottAmostAmotAmachtAmersonAmottenAmoorAmoldAmasterAm.Am ·AmnAmenAmethAmiensAmensAmienAmäAmeterAmogenAmetersAm SterAmmsAmmarkAm OstAmertAmendAmentAmoterAmtsAmisAmemAmusAmokAméAmoisAmierAmmAmereAmamAmæAmēAmosAmischAmorAmmetAmersAmesAmnesAm AmAmseeAmseAmaterAmmarAmmaAm 18AmmatAm18AmstatAmareAm 16AmambAm16AmstyAmameAmalAmasAmperAmplAmlemAmollAmolAmoleAmolandAmolenAmolloAmopAmopenAmelandAmoyAmophAmateurAmomAmodAmoudAmoAm oAm�AmmanAmhofAmofAmafAmoffAmåAmoustAmachAmoutAmairAmbornAmandaAmáAmasteAmfoAm
` ` ` 

Is there a way I can prevent the model from resetting the constraint or stop the generation on reset?
Thanks in advance!

Comments:
@cwkeam - any ideas on this one by any chance?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

maybe you can set `max_new_tokens=512` in generate