Return scores from semantic segmentation post-process
Author: guarin
Pull request: yes

# What does this PR do?

Adds a new `return_segmentation_scores: bool = False` argument to all `post_process_semantic_segmentation` methods. If `True`, the method will return segmentation maps and scores. If `False`, only the segmentation maps are returned.

This changes the return type from all `post_process_semantic_segmentation` methods from:
```
-> list[Tensor]
```
to 
```
-> list[Tensor] | list[SemanticSegmentationPostProcessorOutput]
```

`SemanticSegmentationPostProcessorOutput` is a new addition based on `BatchFeature` and is intended to be used as return type from `post_process` methods instead of the untyped dicts that we currently return. The class is fully backwards compatible with `dict`. 

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Code Agent Policy

The Transformers repo is currently being overwhelmed by a large number of PRs and issue comments written by
code agents. We are currently bottlenecked by our ability to review and respond to them. As a result, 
**we ask that new users do not submit pure code agent PRs** at this time. 
You may use code agents in drafting or to help you diagnose issues. We'd also ask autonomous "OpenClaw"-like agents
not to open any PRs or issues for the moment.

PRs that appear to be fully agent-written will probably be closed without review, and we may block users who do this
repeatedly or maliciously. 

This is a rapidly-evolving situation that's causing significant shockwaves in the open-source community. As a result, 
this policy is likely to be updated regularly in the near future. For more information, please read [`CONTRIBUTING.md`](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md).

- [ ] I confirm that this is not a pure code agent PR.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc
- kernels: @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46163). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @molbap @yonigozlan @nielsrogge, not sure if anyone's working on this already

Oop, didn't realize @guarin was working here! No insult intended :sweat_smile:

Following the discussion above, I agree that `return_logits` may be too narrow here, and that `return_scores` / `segmentation_scores` better describes the tensor this PR is trying to expose.

A related design point from #46139: I think this PR also exposes a broader gap in semantic segmentation APIs.

Today, each model/image processor owns its own post-processing path. That makes sense for model-specific restoration, but for users working across models within the same task, it creates extra learning cost: every model family may expose a slightly different final tensor, naming convention, and hard-decision rule.

So beyond the immediate naming issue, I wonder if a longer-term direction could be a task-oriented post-processing layer for semantic segmentation. In that design, each processor would still handle model-specific restoration, but would expose a shared pre-final score interface before the hard segmentation map is produced.

This kind of unified layer would also give external decoding methods a clearer place to fit. For example, RankSEG-style decoding does not need to change the model or preprocessing; it only replaces the final hard-decision step with a metric-aware decoding method. So it could naturally sit on top of this shared post-processing interface: reuse the official Transformers restoration path, then apply an alternative task-level decoder.

Not suggesting that this PR needs to solve the whole abstraction now, but I think the current naming/API discussion is a useful first step toward that direction.

run-slow: beit, chmv2, conditional_detr, deformable_detr, detr, dpt, eomt, grounding_dino, mask2former, maskformer, mobilenet_v2, mobilevit, oneformer, sam3, sapiens2, segformer

[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/27131196759)

This comment contains `run-slow`, running the specified jobs: 

models: ["models/beit", "models/chmv2", "models/conditional_detr", "models/deformable_detr", "models/detr", "models/dpt", "models/eomt", "models/grounding_dino", "models/mask2former", "models/maskformer", "models/mobilenet_v2", "models/mobilevit", "models/oneformer", "models/sam3", "models/sapiens2", "models/segformer"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/27131196759)

### Commit Info
| Context | Commit | Description |
|---------|--------|-------------|
| RUN | [2ad1d8b6](https://github.com/huggingface/transformers/commit/2ad1d8b6163493c2ecd171a4668c62cf729c900e) | workflow commit (merge commit) |
| PR | [c93e2ad1](https://github.com/guarin/transformers/commit/c93e2ad104b57282269679a1f19cad3f79168e03) | branch commit (from PR) |
| main | [5d82aacc](https://github.com/huggingface/transformers/commit/5d82aacc6a27ff47c3f5e55ae760d7cd52a6ce83) | base commit (on `main`) |

### Model CI Report

❌ **[3 new failed tests from this PR](https://huggingface.co/datasets/hf-internal-testing/transformers_pr_ci/raw/9a9077685a06bc56210ac42c0524b5c6f5ebd254/2026-06-08/runs/35891-27131196759/ci_results_run_models_gpu/new_failures_with_bad_commit.json)** 😭

- [eomt](https://github.com/huggingface/transformers/actions/runs/27131196759/job/80072999039):
    tests/models/eomt/test_image_processing_eomt.py::EomtImageProcessingTest::test_call_numpy (✅ ⟹ ❌)
    tests/models/eomt/test_image_processing_eomt.py::EomtImageProcessingTest::test_call_pil (✅ ⟹ ❌)
    tests/models/eomt/test_image_processing_eomt.py::EomtImageProcessingTest::test_call_pytorch (✅ ⟹ ❌)



run-slow: beit, chmv2, conditional_detr, deformable_detr, detr, dpt, eomt, grounding_dino, mask2former, maskformer, mobilenet_v2, mobilevit, oneformer, sam3, sapiens2, segformer

[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/27133295695)

This comment contains `run-slow`, running the specified jobs: 

models: ["models/beit", "models/chmv2", "models/conditional_detr", "models/deformable_detr", "models/detr", "models/dpt", "models/eomt", "models/grounding_dino", "models/mask2former", "models/maskformer", "models/mobilenet_v2", "models/mobilevit", "models/oneformer", "models/sam3", "models/sapiens2", "models/segformer"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/27133295695)

### Commit Info
| Context | Commit | Description |
|---------|--------|-------------|
| RUN | [3f111379](https://github.com/huggingface/transformers/commit/3f111379f9e1689df6841058960c91a1edeeae4d) | workflow commit (merge commit) |
| PR | [e11512da](https://github.com/guarin/transformers/commit/e11512da609b2841e69741ac607d812899effd39) | branch commit (from PR) |
| main | [5d82aacc](https://github.com/huggingface/transformers/commit/5d82aacc6a27ff47c3f5e55ae760d7cd52a6ce83) | base commit (on `main`) |

✅ No failing test specific to this PR 🎉 👏 !


Quick follow-up while this is still in review.

`return_segmentation_scores=True` is great, but in every `image_processing_*.py` the `argmax` is computed unconditionally — the `segmentation` field is just dropped at the end if the user didn't ask for it. Read this as: the processor is paying for a hard map a downstream decoder would throw away.

Could we add a paired `return_segmentation: bool = True` flag (default `True` for full BC) so that `return_segmentation_scores=True, return_segmentation=False` returns `SemanticSegmentationPostProcessorOutput(segmentation=None, segmentation_scores=...)`? Skips the argmax, skips the dataclass fill, and gives us a real "scores only" entry point.

Why a new flag rather than overloading `return_segmentation_scores`:

- one says *what to expose*, the other says *what to compute*. Splitting the two matches where this PR is already heading.
- the pipeline in `image_segmentation.py` is already calling the function with no kwargs and then doing `outputs.numpy()` on the result. Keeping `return_segmentation` as a separate, default-True knob means the pipeline keeps doing what it does today, and the score-only path is a deliberate opt-in that downstream decoders pick.

Long-term split I'm hoping for:

```
processor: raw outputs -> restored semantic scores (+ optional default hard map)
decoder:   scores      -> hard map
pipeline:  hard map    -> user-facing masks / labels
```

I'm not asking for a full decoder layer in this PR — just for the score-only contract to be reachable. Happy to send a small PR against this branch for the flag + per-processor `None` short-circuit + docstring update, it's a mostly mechanical change.


@Leev1s that's a valid concern! Definitely too much for this PR but something to keep in mind for the future. We'll have to see how much user interest there is for this feature. Making all possible outputs configurable will increase the number of arguments and code branching quite a bit.

@guarin Thanks for the response.🤗

The reason I'm still pushing a bit on this is that the current implementation makes `return_segmentation_scores=True` look like a score-output path, but internally it still always runs the final hard-decision step first. For downstream decoders, that means the official restoration path is available, but not quite as a clean scores-only entry point yet.

I think a paired opt-in flag keeps the current PR's default behavior unchanged while making the separation explicit:

```python
return_segmentation_scores=True
return_segmentation=False
```

would return restored scores with `segmentation=None`, and skip the final `argmax` / thresholding step. So the default path stays exactly as-is, but downstream methods that only need scores don't pay for or depend on the hard map.

@guarin 

Sent a small follow-up PR to make the `scores-only` decoder contract concrete.

@Leev1s the current pattern with `return_segmentation_scores` is consistent with other similar flags in transformers like `return_loss` that "add" an extra output to the default outputs. To make individual outputs fully configurable, and avoid any extra computation, we'll have to look beyond semantic segmentation and take other models and tasks into account to keep the functionalities consistent across the whole codebase. This will require more consideration and we'll have to see how much user interest there is for such a feature.

Do you have a particular use-case that benefits from excluding the segmentation computation completely that goes beyond the minor speed-up due to skipping the extra `argmax` computation? 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: beit, chmv2, conditional_detr, deformable_detr, detr, dpt, eomt, grounding_dino, mask2former, maskformer, mobilenet_v2, mobilevit, oneformer, sam3, sapiens2, segformer