refactor: replace magic strings with constant `SPIECE_UNDERLINE`
Author: scruel
Pull request: yes

# What does this PR do?
Add `constants` subfolder and replace magic strings with constant `SPIECE_UNDERLINE`

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@ArthurZucker


Comments:
How to fix the "duplicate" error in CI ckecks?

> There are a lot of changes here which are really out-of-scope for using a single `"SPIECE_UNDERLINE"` constant. Moving constants around has the potential to create many breaking changes and something which should be done with great care.

Agree, we should be careful, what I have done in this PR is just replacement, and shouldn't break anything, and I still keep some test cases (e.g., [this](https://github.com/scruel/transformers/blob/patch-ms/tests/models/xlm_prophetnet/test_modeling_xlm_prophetnet.py#L142-L143)) unchanged.
I will double-check everything again to make sure it won't break anything, and might revert some code in `tests` folder for convenience reason -- when we write test cases, we often just copy the output directly, rather than think which constants we shall put to replace the strings in the `assert` statement, what would you think?

> I don't think creating a whole new module constants isn't necessary at the moment. ... tokenization_utils_base.py

It's up to your team to decide, if you do confirm, I will update this PR, before this, I'd like to say that IMO it would be nicer to have a `constants` module:
- consider in most cases the only thing we need to import is just `SPIECE_UNDERLINE`, rather than let importer load the entire `tokenization_utils_base` file before importing constants.
- We may start to replace other magic strings and put them into this module, e.g., `<unk>`, `
`, `
`. Though it is not such a high priority task, but we do often replace those repeated strings to constants to make code more maintainable.

BTW, can you explain what is the `_import_structure` for? And why I can't have duplicate items in it? thanks!



> shouldn't break anything, and I still keep some test cases (e.g., [this](https://github.com/scruel/transformers/blob/patch-ms/tests/models/xlm_prophetnet/test_modeling_xlm_prophetnet.py#L142-L143)) unchanged.

Not breaking things is unfortunately more complicated that just having tests pass. Users expect to be able to import certain objects from specific paths, and moving them can cause breakages, even if our repo is fine

> It's up to your team to decide, if you do confirm

As a core maintainer, I'm making the decision that we don't need a constants module at the moment. It's something that can be revisited in the future and should be decoupled from the more general use of `SPIECE_UNDERLINE` which this PR is concerned with

> consider in most cases the only thing we need to import is just SPIECE_UNDERLINE, rather than let importer load the entire tokenization_utils_base file before importing constants.

Are there cases when we're using `SPIECE_UNDERLINE` and `tokenization_utils_base` isn't used? 

> BTW, can you explain what is the _import_structure for? And why I can't have duplicate items in it? thanks!

[Here's a comment](https://github.com/huggingface/transformers/blob/bf3dfd11605d8aa9e87bf872f076ef517399c7e4/src/transformers/__init__.py#L15) explaining _import_structure. You can't have duplicates as this is equivalent to saying you want import something twice


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.