Repository: ultralytics/ultralytics

Issue:
tflite inference in segmentation wrongly scales the first 4 upper pixels of the image
### Search before asking

- [X] I have searched the YOLOv8 [issues](https://github.com/ultralytics/ultralytics/issues) and found no similar bug report.


### YOLOv8 Component

Predict

### Bug

The faulty code is in forward of`autobackend.py` for tflite as it updates **each** output by denormalizing. Instead it should not affect the mask output:

```python
for output in self.output_details:
    x = self.interpreter.get_tensor(output["index"])
    if integer:
        scale, zero_point = output["quantization"]
        x = (x.astype(np.float32) - zero_point) * scale  # re-scale
    if x.ndim > 2:  # if task is not classification
        # Denormalize xywh by image size. See https://github.com/ultralytics/ultralytics/pull/1695
        # xywh are normalized in TFLite/EdgeTPU to mitigate quantization error of integer models
        x[:, [0, 2]] *= w
        x[:, [1, 3]] *= h
    y.append(x)
```

You can see the problem in this picture, it only affects the very top of the masks, so it is often a silent problem. In the following I visualize just one of the 32 masks:

![Screenshot from 2024-03-28 17-54-53](https://github.com/ultralytics/ultralytics/assets/10595147/bf86c0ed-0d8d-492b-828f-6de2082e2ce6)


### Environment

only for tflite in segmentation

### Minimal Reproducible Example

self explanatory

### Additional

_No response_

### Are you willing to submit a PR?

- [X] Yes I'd like to help by submitting a PR!


Discussion:
👋 Hello @charlesollion, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the [Docs](https://docs.ultralytics.com) for new users where you can find many [Python](https://docs.ultralytics.com/usage/python/) and [CLI](https://docs.ultralytics.com/usage/cli/) usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a [minimum reproducible example](https://docs.ultralytics.com/help/minimum_reproducible_example/) to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our [Tips for Best Training Results](https://docs.ultralytics.com/yolov5/tutorials/tips_for_best_training_results/).

Join the vibrant [Ultralytics Discord](https://ultralytics.com/discord) 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

## Install

Pip install the `ultralytics` package including all [requirements](https://github.com/ultralytics/ultralytics/blob/main/pyproject.toml) in a [**Python>=3.8**](https://www.python.org/) environment with [**PyTorch>=1.8**](https://pytorch.org/get-started/locally/).

```bash
pip install ultralytics
```

## Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including [CUDA](https://developer.nvidia.com/cuda)/[CUDNN](https://developer.nvidia.com/cudnn), [Python](https://www.python.org/) and [PyTorch](https://pytorch.org/) preinstalled):

- **Notebooks** with free GPU: <a href="https://console.paperspace.com/github/ultralytics/ultralytics"><img src="https://assets.paperspace.io/img/gradient-badge.svg" alt="Run on Gradient"/></a> <a href="https://colab.research.google.com/github/ultralytics/ultralytics/blob/main/examples/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> <a href="https://www.kaggle.com/ultralytics/yolov8"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a>
- **Google Cloud** Deep Learning VM. See [GCP Quickstart Guide](https://docs.ultralytics.com/yolov5/environments/google_cloud_quickstart_tutorial/)
- **Amazon** Deep Learning AMI. See [AWS Quickstart Guide](https://docs.ultralytics.com/yolov5/environments/aws_quickstart_tutorial/)
- **Docker Image**. See [Docker Quickstart Guide](https://docs.ultralytics.com/yolov5/environments/docker_image_quickstart_tutorial/) <a href="https://hub.docker.com/r/ultralytics/ultralytics"><img src="https://img.shields.io/docker/pulls/ultralytics/ultralytics?logo=docker" alt="Docker Pulls"></a>

## Status

<a href="https://github.com/ultralytics/ultralytics/actions/workflows/ci.yaml?query=event%3Aschedule"><img src="https://github.com/ultralytics/ultralytics/actions/workflows/ci.yaml/badge.svg" alt="Ultralytics CI"></a>

If this badge is green, all [Ultralytics CI](https://github.com/ultralytics/ultralytics/actions/workflows/ci.yaml?query=event%3Aschedule) tests are currently passing. CI tests verify correct operation of all YOLOv8 [Modes](https://docs.ultralytics.com/modes/) and [Tasks](https://docs.ultralytics.com/tasks/) on macOS, Windows, and Ubuntu every 24 hours and on every commit.
@charlesollion hi there! Thanks for bringing this to our attention 🙏. It seems like the issue stems from the denormalization process affecting segmentation mask outputs inappropriately. A quick workaround in `autobackend.py` would be to add a condition to skip this process for mask outputs. Something along these lines:

```python
if not output["name"].startswith("mask"):  # Assuming mask outputs have a 'mask' prefix
    if x.ndim > 2:  # if task is not classification
        x[:, [0, 2]] *= w
        x[:, [1, 3]] *= h
```

This is a temporary fix, and I'll be working on a more integrated solution for this issue. Your findings are incredibly helpful, and we appreciate your willingness to contribute with a PR! If you have any further insights or implement a solution, feel free to share it or discuss it further here before the PR. Happy coding!
@charlesollion TFLite Segmentation models are showing identical Val metrics/mAP50-95(M) vs every other format, so everything is likely working correctly:

https://github.com/ultralytics/ultralytics/actions/runs/8475291549/job/23223143167
<img width="1655" alt="Screenshot 2024-03-29 at 02 07 18" src="https://github.com/ultralytics/ultralytics/assets/26833433/6afef260-a2ea-469f-bc38-34d1b93c344d">

@charlesollion can you submit a PR with a proposed fix? The same CI will run on it and we can check the accuracy of the proposed changes in the PR.

Resolving patch:
diff --git a/ultralytics/nn/autobackend.py b/ultralytics/nn/autobackend.py
--- a/ultralytics/nn/autobackend.py
+++ b/ultralytics/nn/autobackend.py
@@ -543,7 +543,7 @@ def callback(request, userdata):
                     if integer:
                         scale, zero_point = output["quantization"]
                         x = (x.astype(np.float32) - zero_point) * scale  # re-scale
-                    if x.ndim > 2:  # if task is not classification
+                    if x.ndim == 3:  # if task is not classification, excluding masks (ndim=4) as well
                         # Denormalize xywh by image size. See https://github.com/ultralytics/ultralytics/pull/1695
                         # xywh are normalized in TFLite/EdgeTPU to mitigate quantization error of integer models
                         x[:, [0, 2]] *= w


Test patch: