Bad word list for text generation
Author: Peter-Devine
Pull request: no
Labels: name: High-Level feature

# 🚀 Feature request
Add a word of lists that you do not want the model to generate for whatever reason.

## Motivation
When creating a text generation model, especially if you will serve that model publicly, it is desirable to have assurances that a model is physically incapable of outputting certain tokens.

Such tokens would include profanity of all kinds, prejudicial language, or even just out-of-domain vocabulary.

## Your contribution

I am unsure as to your coding style guide, but I will detail how I would implement this below.

Firstly, you cannot simply set the offending word's values to `-Inf` as done here https://github.com/huggingface/transformers/blob/908fa43b543cf52a3238129624f502240725a6a6/src/transformers/modeling_utils.py#L1114 and here https://github.com/huggingface/transformers/blob/908fa43b543cf52a3238129624f502240725a6a6/src/transformers/modeling_utils.py#L1131
when calculating `top_k_top_p_filtering` as it would not capture multi-token words.

As an example, I will use the common Irish exclamation "feck" (see [here ](https://en.wikipedia.org/wiki/Feck) for an off-topic discussion of the word).

'feck' is tokenized into `['fe', 'ck']` in the `gpt2-xl` tokenizer. It would be unreasonable to simply set the logit values of any mention of the token `'fe'` or `'ck'` to -Inf as that would stop the creation of purely harmless words such as 'feasible' (`['fe', 'as', 'ible']`) and 'peck' (`['pe', 'ck']`). Therefore, I suggest a list of tokens that are not allowed at any one instance, and are updated, depending on the previous tokens.
This functionality would mean that a model would be allowed to generate the `'fe'` token, but would not be able to follow it with a `'ck'` token straight after.

I would operationalize this by having:
* a `bad_words_list` which the user has passed into the generate function
* a `possible_bad_words` dict which describes all the possible bad words that you can make with the current token history and how far along they are
* a `prohibited_tokens_list` which would prevent the model from choosing those tokens at a given time.

E.g.
Let's say that we pass the following list to the .generate function:
`bad_words_list = ["feck", "arse", "booze", "up yours"]`
which would be then tokenized and made into ids in the .generator function before the generation starts:
`bad_words_list = [tokenizer.encode(x, add_special_tokens=False) for x in bad_words_list]`

Then, the following function would be run just before the model outputs are obtained from both beam and no beam generators

```
def update_possible_bad_words(previous_token, bad_words_list, possible_bad_words):
    # Start with an empty list of prohibited bad words
    prohibited_tokens_list = []
    unmatched_bad_words= []

    # We cycle through the provided list of bad words, and if a bad word has only one token, then we add it to our prohibited list
    # Else, if a bad word starts with the token of our previous token, then we add it to our list of possible_bad_words
    for bad_word in bad_words_list:
        if len(bad_word) == 1:
            unmatched_bad_words.append(bad_word[0])
        elif previous_token == bad_word[0] and bad_word not in possible_bad_words.keys():
            possible_bad_words[bad_word] = 0

    # We cycle through all our possible bad words
    for bad_word in possible_bad_words.keys():
        bad_word_index = possible_bad_words[bad_word]

        # if the previous token matches the token currently indicated by the stored bad word index, then we increase this index by one.
        if previous_token == bad_word[bad_word_index]:
            new_bad_word_index = bad_word_index +1


            # If the length of the bad word is one greater than the currently stored bad word index, then that means that we need to stop the next token from being generated. Thus, we add it to the prohibited list. We also add this word to the unmatched_bad_words, as we can now consider deleting it from possible bad words as it has been potentially mitigated.
            if len(bad_word) == new_bad_word_index+1:
                prohibited_tokens_list.append(bad_word[-1])
                unmatched_bad_words.append(bad_word)

            # We set the dict value to be this new incremented index
            possible_bad_words[bad_word] = new_bad_word_index 
        else:
           # Else if there is no match (I.e. the pattern of tokens created is different to the bad word) then we can mark this word as unmatched
            unmatched_bad_words.append(bad_word)

    # We cycle through all words that were not matched and we delete them from the possible_bad_words dict, as long as their starting token was not the previous token, in which case it could still be possible bad word.
    for unmatched_bad_word in unmatched_bad_words:
        if previous_token == unmatched_bad_word[0]:
            possible_bad_words[unmatched_bad_word] = 0
        else:
            del possible_bad_words[unmatched_bad_word]

    return prohibited_tokens_list
```
and I would call this function like so:
`prohibited_tokens_list = update_possible_bad_words(input_ids[-1], bad_words_list, possible_bad_words)`
here
https://github.com/huggingface/transformers/blob/908fa43b543cf52a3238129624f502240725a6a6/src/transformers/modeling_utils.py#L827
and here
https://github.com/huggingface/transformers/blob/908fa43b543cf52a3238129624f502240725a6a6/src/transformers/modeling_utils.py#L934
where `possible_bad_words` has been initialized as an empty dict directly before the generation loop.

Finally, we would pass `prohibited_tokens_list` to `top_k_top_p_filtering`
https://github.com/huggingface/transformers/blob/908fa43b543cf52a3238129624f502240725a6a6/src/transformers/modeling_utils.py#L1100
and would simply perform `logits[prohibited_tokens_list] = filter_value` before or after the top p and k filtering in that function.

Comments:
I like this idea. Perhaps it can be added as an example, or even as an argument to the generation script.