TrOCR (image-to-text) produces incorrect output (':') on 12th Gen Intel CPU (i7-1260P) even with simple input
Author: eximius313
Pull request: no
Labels: name: bug

### System Info

```
- `transformers` version: 4.51.3
- Platform: Windows-10-10.0.26100-SP0
- Python version: 3.11.9
- Huggingface_hub version: 0.30.2
- Safetensors version: 0.5.3
- Accelerate version: 1.6.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (GPU?): 2.6.0+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
```

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

TrOCR (both pipeline and manual processing with VisionEncoderDecoderModel) consistently outputs ':' on CPU, even for simple English text images and regardless of num_threads.
Basic PyTorch ops, fill-mask (BERT), image-classification (ViT) work correctly on the same CPU and environment.

Code:
```python
import os
import torch

num_threads = 1
print(f"Setting OMP_NUM_THREADS and MKL_NUM_THREADS to: {num_threads}")
os.environ['OMP_NUM_THREADS'] = str(num_threads)
os.environ['MKL_NUM_THREADS'] = str(num_threads)
torch.set_num_threads(num_threads)


from transformers import TrOCRProcessor, VisionEncoderDecoderModel

from PIL import Image
import torch

image_path = "C:/ocr.png"
model_id = 'microsoft/trocr-large-printed'

try:
    processor = TrOCRProcessor.from_pretrained(model_id)
except Exception as e:
    exit()

try:
    model = VisionEncoderDecoderModel.from_pretrained(model_id)
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    model.to(device)

except Exception as e:
    exit()

try:
    img = Image.open(image_path).convert("RGB")
except Exception as e:
    exit()

try:
    pixel_values = processor(images=img, return_tensors="pt").pixel_values
    pixel_values = pixel_values.to(device)
except Exception as e:
    exit()

try:
    generated_ids = model.generate(pixel_values, max_length=50)
except Exception as e:
    exit()

try:
    generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
    print("\n--- OUTPUT ---")
    print(generated_text)
    print("------------------------------------\n")
except Exception as e:
    exit()
```

Image:
![Image](https://github.com/user-attachments/assets/f88e60a6-2eca-4dcd-99ba-5900a5179a21)

Steps to reproduce:
`python example.py`

### Expected behavior

It displays "This is some English text"

Comments:
Hi,

The TrOCR model is meant to be used on text-line images. Hence it relies on another model which first detects the text, which you can then crop from the image. Passing the cropped image to TrOCR should give a good result.

We're closing on getting FAST integrated which can do that: text detection (#35476). Next we can add a pipeline to Transformers which runs FAST and TrOCR in a sequence.

For now I'd recommend [CRAFT](https://github.com/clovaai/CRAFT-pytorch) as text detector - but you'll see that FAST is a lot better.

Thanks @NielsRogge,

even if I crop the image to:

![Image](https://github.com/user-attachments/assets/3afb7f19-2153-4ee4-aa8e-1d0c122b4b0c)
it still does not work.

I'm eagerly waiting for FAST.

Thanks

You would need to crop the "This is some" on one image, and "English text" on the other (each image = single line of text). TrOCR-printed has been trained that way.

@NielsRogge,
One line indeed worked, thanks.
Perhaps adding a warning would help in the future?