Fix stop string matching for byte-fragment tokens
Author: Incheonkirin
Pull request: yes

# What does this PR do?

Fixes #46519.

This fixes a case where `StopStringCriteria` can miss stop strings with byte-level tokenizers. The full generated sequence decodes correctly, but some CJK characters are split into byte-fragment tokens, and decoding those tokens one by one can produce replacement characters instead. That means the text may visibly end with the stop string while the stopping criterion still does not match it.

This PR handles this during preprocessing by building the matching table in byte space for ByteLevel and SentencePiece ByteFallback tokenizers. Non-byte tokenizers keep the existing string path.

`StopStringCriteria.__call__` is unchanged, so the runtime matching path stays tensor-only.

I ran:
~~~bash
PYTHONPATH=src python -m pytest tests/generation/test_stopping_criteria.py -q -p no:cacheprovider
ruff check src/transformers/generation/stopping_criteria.py tests/generation/test_stopping_criteria.py
ruff format --check src/transformers/generation/stopping_criteria.py tests/generation/test_stopping_criteria.py
~~~

Local result: `19 passed`.

## Code Agent Policy

The Transformers repo is currently being overwhelmed by a large number of PRs and issue comments written by
code agents. We are currently bottlenecked by our ability to review and respond to them. As a result, 
**we ask that new users do not submit pure code agent PRs** at this time. 
You may use code agents in drafting or to help you diagnose issues. We'd also ask autonomous "OpenClaw"-like agents
not to open any PRs or issues for the moment.

PRs that appear to be fully agent-written will probably be closed without review, and we may block users who do this
repeatedly or maliciously. 

This is a rapidly-evolving situation that's causing significant shockwaves in the open-source community. As a result, 
this policy is likely to be updated regularly in the near future. For more information, please read [`CONTRIBUTING.md`](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md).

- [x] I confirm that this is not a pure code agent PR.

## Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? See #46519.
- [x] Did you make sure to update the documentation with your changes? No documentation update is needed for this bug fix.
- [x] Did you write any new necessary tests?

## Who can review?

cc @Rocketknight1 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46530). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46530)

Thanks! Glad this landed — the byte-fragment case was subtle, and it’s good to have it covered now.