[Model] Add PaddleOCR-VL Model Support
Author: zhang-prog
Pull request: yes

# What does this PR do?
This PR adds PaddleOCR-VL model to Hugging Face Transformers from PaddleOCR.

Relevant Links:

[PaddleOCR](https://www.paddleocr.ai/latest/index.html)
https://huggingface.co/PaddlePaddle/PaddleOCR-VL

## Usage

### Use a pipeline

```python
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="PaddlePaddle/PaddleOCR-VL", dtype="bfloat16")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/ocr_demo2.jpg"},
            {"type": "text", "text": "OCR:"},
        ]
    }
]
result = pipe(text=messages)
print(result)
```

### Load model directly

```python
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("PaddlePaddle/PaddleOCR-VL")
model = AutoModelForImageTextToText.from_pretrained("PaddlePaddle/PaddleOCR-VL", dtype="bfloat16")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/ocr_demo2.jpg"},
            {"type": "text", "text": "OCR:"},
        ]
    }
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=100)
result = processor.decode(outputs[0][inputs["input_ids"].shape[-1]:-1])
print(result)

```

Comments:
@zucchini-nlp 
We have refactored the code to address the issues you mentioned in your comments. 
Please review the code again when you have time. 
Thank you for your efforts!!!

> @zhang-prog thanks for iterating!
> 
> There are a couple major comments which were not addressed and can be a blocker for merging.
> 
> 1. The model seems to not support batched inference in current state. We need to enable batching before merging if possible. Should not be hard I think given that the image tower is quite similar to existing models
> 2. We also need tests to make sure everything actually works and a documentation page. These files are usually auto-prefilled with empty files when you run `transformers add-new-model-like`
> 3. Let the modular copy automatically when possible. I think there are a few more modules which can be copied from similar models. If you struggle with finding a similar model, you can try out a [modular detector](https://github.com/huggingface/transformers/blob/main/utils/modular_model_detector.py)

@zucchini-nlp Thank you for your valuable insights! We’ve carefully addressed all comments and responded to your overall recommendations.

1. We support bs > 1, like this:

```python
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("PaddlePaddle/PaddleOCR-VL")
model = AutoModelForImageTextToText.from_pretrained("PaddlePaddle/PaddleOCR-VL", dtype="bfloat16")
messages1 = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/ocr_demo.jpg"},
            {"type": "text", "text": "OCR:"},
        ]
    }
]
messages2 = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/ocr_demo2.jpg"},
            {"type": "text", "text": "OCR:"},
        ]
    }
]
batch_messages = [messages1, messages2]
inputs = processor.apply_chat_template(
	batch_messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
    padding=True,
    padding_side='left',
).to(model.device)

generated_ids = model.generate(**inputs, max_new_tokens=1024)
generated_ids_trimmed = [out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)]
result = processor.batch_decode(generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False)
print(result)
```

2. We still have some issues to discuss. I replied to your comment and will generate the final version of the document once it’s completed.

3. We also added the `PaddleOCRVisionConfig` and `PaddleOCRTextConfig` into modular. 

Thank you for your efforts. ❤️
PTAL. 


@zucchini-nlp How do I properly add documentation pages and unit tests? I tried to use `transformers add-new-model-like`, which generates the new `modular_xxx.py` files, but this process might not be the right approach.

@zucchini-nlp PTAL. Thanks❤️

Sorry, taking a look.Got lost in my notifications 

@zucchini-nlp 

PTAL.❤️

`_checkpoint_conversion_mapping` and `ignore_keys_at_rope_validation` needs to be discussed.

I am working on the docs and tests.

Great, looking good already. We can keep the conversion mapping as is, no issue for us! There are also a few unresolved comments from the past iterations, if you can take a look 

Ping me when the docs/tests are added and the CI shows ✅ 

Don't merge. Working.....

@zucchini-nlp PTAL.❤️

we add tests and model_doc, and all the tests passed:

```
pytest tests/models/paddleocr_vl/test_modeling_paddleocr_vl.py
========== 100 passed, 115 skipped, 5 warnings in 1375.17s (0:22:55) ===========

RUN_SLOW=1 pytest -sv tests/models/paddleocr_vl/test_modeling_paddleocr_vl.py::PaddleOCRVLIntegrationTest
========== 4 passed, 2 warnings in 59.76s==========
```

both `ci/circleci: tests_generate` and `ci/circleci: tests_torch` are showing the same error. Could you help analyze the cause?

<img width="970" height="512" alt="image" src="https://github.com/user-attachments/assets/dda28c3b-f331-4b66-bae8-e071688805bf" />


@zucchini-nlp 
Why did it crash here?

<img width="1180" height="236" alt="image" src="https://github.com/user-attachments/assets/02ffd229-a83f-40a0-a960-1dee9f39bd9f" />

in my environment, the test passed:

<img width="1200" height="336" alt="image" src="https://github.com/user-attachments/assets/f33fc371-4d8b-4799-8c3e-fe984f2b8e0b" />


@zhang-prog worker crashed means that the tests might be using too much RAM. I see that the image sizes in tests are quite high, 600x400 images. Let's make dummy inputs and model as tiny as possible

Reviewing now 

@zucchini-nlp 

Now, CI shows ✅
PTAL. ❤️

run-slow: paddleocr_vl

This comment contains `run-slow`, running the specified jobs: 

models: ["models/paddleocr_vl"]
quantizations: []

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42178). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20130177344)

### Model CI Report

#### ❌ Failed tests

- [paddleocr_vl](https://github.com/huggingface/transformers/actions/runs/20130177344/job/57769680364):
    tests/models/paddleocr_vl/test_modeling_paddleocr_vl.py::PaddleOCRVLModelTest::test_flex_attention_with_grads
    tests/models/paddleocr_vl/test_modeling_paddleocr_vl.py::PaddleOCRVLIntegrationTest::test_small_model_integration_test
    tests/models/paddleocr_vl/test_modeling_paddleocr_vl.py::PaddleOCRVLIntegrationTest::test_small_model_integration_test_batch



@zucchini-nlp fixed, please try slow tests again. btw, some conversation can be marked as resolved?

run-slow: paddleocr_vl

This comment contains `run-slow`, running the specified jobs: 

models: ["models/paddleocr_vl"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20132511404)

### Model CI Report

#### ❌ Failed tests

- [paddleocr_vl](https://github.com/huggingface/transformers/actions/runs/20132511404/job/57777422708):
    tests/models/paddleocr_vl/test_modeling_paddleocr_vl.py::PaddleOCRVLModelTest::test_flex_attention_with_grads



@zucchini-nlp  uh, OOM on `test_flex_attention_with_grads` again, how can I pass this test by changing the parameters?

@zhang-prog the test changes hidden sizes of the model to be multiple of 16 iirc, prob that is the reason it is OOM'ing. If the model causes OOM and can't be made smaller, we can skip the test imo

@zucchini-nlp ok, I've reduced the value of  `hidden_size`, `intermediate_size` and `num_attention_heads`. Maybe this attempt can pass the test. let's try again!

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, paddleocr_vl

run-slow: paddleocr_vl

This comment contains `run-slow`, running the specified jobs: 

models: ["models/paddleocr_vl"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20135357040)

✅ No failing test specific to this PR 🎉 !


Great, ci green now and we can merge 🚀