Processor saving does not work when multiple tokenizers
Author: AmitMY
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.57.1
- Platform: macOS-26.0.1-arm64-arm-64bit
- Python version: 3.12.2
- Huggingface_hub version: 0.34.3
- Safetensors version: 0.5.3
- Accelerate version: 1.10.1
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.8.0 (NA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@Cyrilvallez 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

Currently, processors are saved with fixed names:
https://github.com/huggingface/transformers/blob/ca01fe4d133a498a81bdfd335024d14801e106a2/src/transformers/utils/__init__.py#L272-L278

That means, that if you create a processor that uses two of the same kind of subprocessors (for example, a byte level tokenizer and a BPE tokenizer, or two image processors, etc), they override eachother, because they use the same file name.

```py
import tempfile

from transformers import ProcessorMixin, AutoTokenizer, PreTrainedTokenizer


class OtherProcessor(ProcessorMixin):
    name = "other-processor"

    attributes = [
        "tokenizer1",
        "tokenizer2",
    ]
    tokenizer1_class = "AutoTokenizer"
    tokenizer2_class = "AutoTokenizer"

    def __init__(self,
                 tokenizer1: PreTrainedTokenizer,
                 tokenizer2: PreTrainedTokenizer
                 ):
        super().__init__(tokenizer1=tokenizer1,
                         tokenizer2=tokenizer2)


tokenizer1 = AutoTokenizer.from_pretrained("google/gemma-3-270m")
tokenizer2 = AutoTokenizer.from_pretrained("HuggingFaceTB/SmolLM2-1.7B")

processor = OtherProcessor(tokenizer1=tokenizer1,
                           tokenizer2=tokenizer2)

with tempfile.TemporaryDirectory() as temp_dir:
    # Save processor
    processor.save_pretrained(save_directory=temp_dir, push_to_hub=False)
    # Load processor
    new_processor = OtherProcessor.from_pretrained(temp_dir)

assert processor.tokenizer1.__class__ != processor.tokenizer2.__class__ # passes
assert new_processor.tokenizer1.__class__ != new_processor.tokenizer2.__class__ # fails
```

### Expected behavior

You should be able to use multiple processors within a processor. 


Comments:
I don't think `Processor` classes are designed to support having two of the same kind of preprocessor! This is because, as you noticed, the subclasses tend to overwrite each other's save files; there can only be one `tokenizer_config.json` in the repo.

We might update this in future, but probably not until there's a major model that needs it cc @zucchini-nlp 

Thanks for your response - 

This is a bit circular logic - 
we are building a new, large big model, with a dependency on multiple image processors, and multiple tokenizers https://github.com/sign/WeLT
We have 14k H100 hours to spend on training it, and we want to make it HF compatible from day 1 -
If HF does not support it, we will be forced to hack an ugly solution around it - and only then HF will want to support it?

I think it just makes sense to be as flexible as possible, without all this hard-coding.
We might need to support a fork of HF, but that's also not an optimal solution, because our implementation might not be your preferred one.

@Rocketknight1 The solution in #41879 is minimal, elegant, and backwards compatible.
What do you think?

This is a substantial rewrite to how Processor classes operate, but I don't really own that part of the codebase so I'm going to wait for @zucchini-nlp's opinion! 😅 

also cc @yonigozlan @molbap since I think they're working more on processors now!

Taking a look!

Hello @AmitMY! We already have a few models which require multiple tokenizers (see evolla for example). We have hardcoded logic to handle these specific models (see how the `save_pretrained` and `from_pretrained` methods are overriden in [processing_evolla](https://github.com/huggingface/transformers/blob/a0bf5a82eebf88ee9f52145be427f6f1541329f6/src/transformers/models/evolla/processing_evolla.py)). However this is clearly not a good solution long-term as we get more and more models needing multiple tokenizers (multiple image processors is a first I think, but we might as well handle it).
We have a big refactor of processors in the works right now, adding this to it might not be ideal, but happy to add a way to handle this once the refactor is merged.
In the meantime, you can use the trick in evolla to save and load multiple tokenizers, however it might be more complicated for image processors if you need that too.

Thanks @yonigozlan !
I think the way #41879 implemented it is very minimal - just using subdirectories.
So if this would be the solution down the line, I don't see a reason why not merge it right now.

Anyway, yes, I am having the same issue with image processors as well, and this solution does solve it.
I can hard code a solution, but i think this should go into the core repo. 

The existing solution is just bad, and you don't know it is bad until you try to load your processor from pretrained.

Hey @AmitMY , I totally agree that we need a better solution that overriding saving/loading method in the long-term. I looked quickly over the draft PR and I don't think it's justified to create a new folder to save a single config file per image processor. We'll need a clean solution with backwards compatibility for loading from old models

Since @yonigozlan seems to be working on a huge refactor, it's better for us to wait a bit to not cause conflicts with the ongoing work

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

still an issue in the last release. @yonigozlan should it not be?

Hello! Yes indeed, [this PR](https://github.com/huggingface/transformers/pull/42630) should fix it. It only worked before if the tokenizers name were in a hardcoded set of names that are used in the library, which is not ideal. Now as long as the tokenizers contain "tokenizer" in their name, it should work.

You can try it with `pip install git+https://github.com/huggingface/transformers.git@c17ef247cfb2591af7e33cc1c591f994e97942a0`.

Also, with the new changes to processors, your example can be simplified:
```python
import tempfile

from transformers import ProcessorMixin, AutoTokenizer, PreTrainedTokenizer


class OtherProcessor(ProcessorMixin):
    def __init__(self,
                 tokenizer1: PreTrainedTokenizer,
                 tokenizer2: PreTrainedTokenizer
                 ):
        super().__init__(tokenizer1=tokenizer1,
                         tokenizer2=tokenizer2)


tokenizer1 = AutoTokenizer.from_pretrained("google/gemma-3-270m")
tokenizer2 = AutoTokenizer.from_pretrained("HuggingFaceTB/SmolLM2-1.7B")

processor = OtherProcessor(tokenizer1=tokenizer1,
                           tokenizer2=tokenizer2)

with tempfile.TemporaryDirectory() as temp_dir:
    # Save processor
    processor.save_pretrained(save_directory=temp_dir, push_to_hub=False)
    # Load processor
    new_processor = OtherProcessor.from_pretrained(temp_dir)

assert processor.tokenizer1.__class__ != processor.tokenizer2.__class__
assert new_processor.tokenizer1.__class__ != new_processor.tokenizer2.__class__
```

Thanks @yonigozlan ! I don't love it, because I also have multiple image processors.
How should I deal with those? 

This is a different process than for tokenizers that we haven't addressed yet. We'll try to open a PR before the v5 rc cycle is over. I'll keep you updated in this issue.

Hey @AmitMY ! Closing this PR as having multiple subprocessors of any kind should work now. Feel free to notify us if you see any issues!