There is not space after generating an 'special token' and the next word using gpt2.
Author: fabrahman
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

Hi,
I have used ```run_lm_finetuning.py``` to finetune gpt2 and then tried to do some generation. I have added a couple of special token to dictionary and the finetuned gpt2 without any problem.
Then when I am doing the generation using ```run_generation.py```, I realized whenever the model generates a special_token , it clubs it with the next generated token. For example, consider [SEP] is a special token and this is an output:

[SEP]and it has been a very , ....

And this happens with all of my special_tokens. That is if the next token of an special token isn't an special token, then there is no space between them.

Does anyone know the reason?
Best

Comments:
Can you show exactly how you ran ```run_generation.py```?

@enzoampil thanks, this is my command:
```python run_generation.py --model_type=gpt2 --model_name_or_path=gpt2_finetuned/ --top_k 10 --temperature 0.8 --top_p 0.0 --stop_token "
" ```

Can you try running ```python run_generation.py --model_type=gpt2 --model_name_or_path=gpt2_finetuned/ --top_k 10 --temperature 0.8 --top_p 0.0``` and post the output here. 

If the output looks fine at this point, just exclude the ```stop_token``` argument when running ```run_generation.py```

I see the problem. At the decoding step using ```tokenizer.decode```, we set ```skip_special_tokens=True```.

In other words, special tokens are removed at the decoding step, so you the ```stop_token``` argument should not be a special token.

@enzoampil 
1) I tried without --stop_token and the output is the same.
2) I don't think ``` skip_special_tokens=True ``` is the problem, since I had already set that to **False**.
Did you mean --stop_token is a problem? Am I doing something wrong?

Can you try to set ```clean_up_tokenization_spaces=False```

Yeah I actually tried that one too and still the output doesn't change. @enzoampil 

Would you mind sending over your model files? Can't seem to replicate the issue.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Is it solved?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


I know this is old, but I'm having the same issue (and I've never had it before). It was introduced after I added a Tokenizers `BPEDecoder` to my tokenizer. I'm using special tokens as a way of logically parsing my (non-language) input, and I need the special tokens in the output so that another stage of processing can use them for understanding the structure of the prediction. But there are no spaces between my special tokens and the next word. It's not a huge deal, I suppose, because I could fix it in post-processing, but I'd like to know what's up.

[EDIT] Just to note; in my case it has nothing to do with prediction. I'm just testing the encode/decode of my tokenizer and noticing these spaces missing in the decoded output.