processor tests: use tiny Hub repos to reduce CI memory by ~43%
Author: ydshieh
Pull request: yes

## Summary

Processor tests currently load full pretrained tokenizers (up to 256k vocab, ~300 MB each) at `setUpClass` time, causing excessive RSS growth per test run. This PR introduces tiny Hub repos (`hf-internal-testing/tiny-processor-*`) with trimmed tokenizers (~500 tokens) for 38 processor test files, plus a tiny `DacModel` for the Dia test.

**Changes:**
- `tests/test_processing_common.py`: added `tiny_model_id` class attribute. When set, `tmpdirname` holds the tiny processor (used by all tests); `full_tmpdirname` holds the full processor only when `model_id` is also set (used by the few tests that need hardcoded token IDs). If `model_id = None`, `full_tmpdirname` is skipped entirely.
- 38 processor test files: set `tiny_model_id = "hf-internal-testing/tiny-processor-<model>"`.
- `tests/models/dia/test_processing_dia.py`: use `hf-internal-testing/tiny-dac-44khz` (0.54 MB vs ~95 MB).
- `tests/models/seamless_m4t/test_processing_seamless_m4t.py`: use `hf-internal-testing/tiny-processor-seamless_m4t`.

## Memory impact (measured on Linux CI, 39 test files)

| | Total delta MB | Duration | Entries ≥ 100 MB |
|---|---|---|---|
| **Before** | 18,291 MB | 2,793s (46.5 min) | 36 |
| **After** | 7,482 MB | 326s (5.4 min) | 24 |
| **Reduction** | **−59%** | **−88% (8.6×)** | **−33%** |

**Notable individual fixes:**
- `llava`: 428 → 33 MB
- `fuyu`: 347 → 50 MB
- `altclip`: 254 → 6 MB
- `trocr`: 251 → 11 MB
- `colqwen2`: 111 → 5 MB
- `idefics3`: 102 → 4 MB
- `gemma3n`: 91 → 4 MB
- `lighton_ocr`: 37 → 4 MB

### Before — 38 models, 18,291 MB total delta, 46.5 min

```
   Max dMB   Total dMB   >=10MB   Wall Time  Model
----------  ----------  -------  ----------  --------------------
  +   428.8       490.8        3       27.2s  llava
  +   424.4       948.9       11      162.5s  minicpmv4_6
  +   402.3       988.8       12       71.2s  qwen2_5_omni
  +   347.7       860.2        4      103.3s  fuyu
  +   254.3       638.8        7      111.1s  altclip
  +   251.5       637.4        6      112.1s  trocr
  +   213.7       521.5        6      130.2s  glm4v
  +   210.3       545.6        7       63.5s  qwen3_vl
  +   172.9       461.3        7      132.8s  glm46v
  +   166.9       816.2       14       64.1s  qwen2_vl
  +   156.5       500.8        7       29.5s  llava_next_video
  +   155.2       442.3        9       50.2s  kosmos2_5
  +   128.2       655.8       12       63.0s  qwen2_5_vl
  +   126.6       264.8        2       19.3s  idefics2
  +   126.3       695.5       12       67.2s  internvl
  +   124.6       432.8        6       95.6s  ernie4_5_vl_moe
  +   124.1       428.2       10       66.6s  video_llama_3
  +   111.2       347.7       11       44.3s  colqwen2
  +   107.9       382.0        6       74.1s  qwen3_omni_moe
  +   102.3       600.8       13      106.1s  idefics3
  +    98.4       764.4       15       38.4s  smolvlm
  +    92.9       603.3       12       92.3s  mllama
  +    91.2       586.6       12      170.0s  gemma3n
  +    89.6       563.8       21       14.1s  owlv2
  +    84.8       298.4        6       60.2s  csm
  +    70.2       514.7       12       48.6s  qianfan_ocr
  +    69.1       327.5        8       43.0s  qwen2_audio
  +    61.1       547.8       13       95.0s  pixtral
  +    60.2       265.9        7       46.6s  janus
  +    56.2       246.1        5       25.4s  donut
  +    50.6       388.5       12      100.1s  mistral3
  +    43.2       402.1       15       69.4s  aria
  +    37.8       308.5       11      148.8s  lighton_ocr
  +    28.8       165.5        3      124.7s  glm_image
  +    26.2       431.1       25       14.1s  clip
  +    24.7       165.9        3      104.8s  aya_vision
  +    12.6        19.5        1        1.9s  parakeet
  +    10.0        30.8        1        1.6s  grounding_dino
```

### After — 38 models, 7,482 MB total delta, 5.4 min

```
   Max dMB   Total dMB   >=10MB   Wall Time  Model
----------  ----------  -------  ----------  --------------------
  +   376.8       548.1        2        6.1s  qwen3_vl
  +   327.6       513.2        3       16.9s  qwen2_5_omni
  +   323.6       650.2        4        8.4s  minicpmv4_6
  +   279.7       455.0        3        6.6s  glm4v
  +   245.6       379.4        2        8.2s  qwen2_vl
  +   237.4       394.9        2       18.8s  ernie4_5_vl_moe
  +   235.2       389.4        2        8.2s  internvl
  +   198.5       812.4       13       15.7s  smolvlm
  +   195.3       501.4        7        9.0s  llava_next_video
  +   170.9       304.0        2        8.2s  qwen2_5_vl
  +   147.5       235.9        3       17.4s  kosmos2_5
  +   145.3       360.3        4        3.8s  idefics2
  +   126.2       256.8        2        9.4s  glm46v
  +   119.4       154.2        2       10.4s  video_llama_3
  +    89.2       288.4        5       15.0s  qwen3_omni_moe
  +    61.2       171.8        3       13.9s  qwen2_audio
  +    50.7        95.5        2       15.6s  mistral3
  +    50.6        85.7        2       14.0s  fuyu
  +    46.9       239.9       10        2.2s  owlv2
  +    42.3        67.8        2        5.8s  csm
  +    41.7       122.6        4        5.3s  qianfan_ocr
  +    33.5        76.6        2        5.0s  llava
  +    30.3        67.1        2       11.9s  pixtral
  +    25.1        43.1        1        8.1s  mllama
  +    21.9        53.4        3        5.8s  aria
  +    15.1        47.6        2        2.2s  clip
  +    12.5        14.7        1        2.2s  parakeet
  +    11.4        25.7        1        1.8s  trocr
  +     7.3        11.1        0        9.8s  donut
  +     7.2        10.0        0        9.3s  glm_image
  +     6.9        27.0        0        2.1s  grounding_dino
  +     6.1         9.5        0        5.1s  altclip
  +     5.4         9.5        0        2.8s  colqwen2
  +     5.1        16.4        0        5.3s  aya_vision
  +     4.6        12.1        0        5.6s  janus
  +     4.6        12.8        0        4.7s  lighton_ocr
  +     4.0         9.2        0       13.7s  gemma3n
  +     4.0         8.9        0       11.2s  idefics3
```

Remaining high-memory entries (24) are all video/audio frame loading (`test_apply_chat_template_video_frame_sampling`) — a separate issue unrelated to tokenizer size.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Comments:
## Asset URL replacements (follow-up from this PR)

The commits below (on branch `debug_mem`) replaced all high-res / external test assets with low-res copies hosted in `hf-internal-testing/test-videos`.

### Videos

| Old asset | Resolution | Test files affected |
|-----------|-----------|---------------------|
| `raushan-testing-hf/videos-test/tiny_video.mp4` | 3840×2160 4K, 11 frames (~261 MB decoded) | `test_processing_common.py`, `ernie4_5_vl_moe`, `glm46v`, `glm4v`, `llava_next_video`, `llava_onevision`, `qwen2_5_omni`, `qwen2_5_vl`, `qwen2_vl`, `qwen3_omni_moe`, `qwen3_vl`, `smolvlm`, `video_llama_3` |
| `raushan-testing-hf/videos-test/Big_Buck_Bunny_720_10s_10MB.mp4` | 1280×720 | `test_processing_common.py` |
| `raushan-testing-hf/videos-test/sample_demo_1.mp4` | unknown | `test_processing_common.py`, `qwen2_5_omni`, `qwen3_omni_moe` |
| `hf-internal-testing/fixtures_videos/tennis.mp4` | 1280×720, 182 frames | `internvl` |

**Replaced with** (all in `hf-internal-testing/test-videos`):

| New asset | Resolution |
|-----------|-----------|
| `tiny_video_320x240.mp4` | 320×240, 11 frames |
| `big_buck_bunny_320x240_10s.mp4` | 320×240 |
| `sample_demo_1_320x240.mp4` | 320×240 |
| `tennis_320x240.mp4` | 320×240, 16 frames |

### Images (idefics2 — replaced external CDN URLs)

| Old asset | New asset (hf-internal-testing/test-videos) |
|-----------|---------------------------------------------|
| `cdn.britannica.com/.../Statue-of-Liberty-Island-New-York-Bay.jpg` | `statue_of_liberty_64x64.jpg` |
| `cdn.britannica.com/.../Skyline-Chicago.jpg` | `chicago_64x64.jpg` |
| `thumbs.dreamstime.com/.../golden-gate-bridge...jpg` | `golden_gate_64x64.jpg` |
| (image used in internvl interleaved test) | `view_64x64.jpg` |

### Image processor size overrides (new `_setup_image_processor` / `_setup_video_processor` hooks)

Beyond URL replacements, the following processor test classes now override spatial resolution in `setUpClass` so all tests use tiny tensors instead of the model's real-world defaults:

| Model | Default | Test override | Savings |
|-------|---------|--------------|---------|
| `donut` | 2560×1920 (document scanning) | 64×64 | ~59 MB per image → ~0.05 MB |
| `owlv2` | 960×960 | 64×64 | ~11 MB per image |
| `qianfan_ocr` | 448×448, max_patches=12 | 64×64, max_patches=1 | ~27 MB per image |
| `minicpmv4_6` (image) | scale_resolution=448, max_slice_nums=9 | scale_resolution=64, max_slice_nums=1 | ~21 MB per image |
| `minicpmv4_6` (video) | scale_resolution=448, max_slice_nums=9 | scale_resolution=64, max_slice_nums=1 | reduces per-frame features from 14 KB to 0.1 KB |
| `smolvlm` (video) | max_image_size=364 | max_image_size=64 | reduces video frame tensors ~32× |
| `idefics2` | shortest_edge=378 | shortest_edge=64, longest_edge=64 | 309 MB total → 17 MB total |
| `internvl` | 448×448 | 20×20 | reduces frame/image tensors ~500× |

## Memory benchmark update: before vs after (commits 5a6a7481 → 369048e883)

Ran on the 24 processor test files touched in this branch.

### Before (5a6a7481)
- **Peak RSS**: 1934.9 MB | **Net growth**: 858.2 MB | **Time**: 166s (2:46) | Tests: 849 passed, 474 skipped

### After (369048e883)
- **Peak RSS**: 1172.5 MB | **Net growth**: 287.2 MB | **Time**: 112s (1:51) | Tests: 849 passed, 474 skipped

**Peak RSS −762 MB (−39%) | Net growth −571 MB (−67%) | Time −54s (−33%)**

---

### Top improvements (Delta MB before → after)

| Test | Before | After |
|------|-------:|------:|
| `minicpmv4_6::test_apply_chat_template_video_frame_sampling` | +498 MB | gone from top 30 |
| `ernie4_5_vl_moe::test_apply_chat_template_video_frame_sampling` | +309 MB | +17 MB |
| `qwen2_5_omni::test_apply_chat_template_video_frame_sampling` | +288 MB | +52 MB |
| `qwen3_omni_moe::test_apply_chat_template_video_frame_sampling` | +191 MB | gone from top 30 |
| `qwen2_5_omni::test_apply_chat_template_audio_2` | +183 MB | gone from top 30 |
| `glm46v::test_apply_chat_template_video_frame_sampling` | +152 MB | gone from top 30 |
| `ernie4_5_vl_moe::test_apply_chat_template_video_0` | +119 MB | +48 MB |
| `internvl::test_apply_chat_template_video_frame_sampling` | +100 MB | gone from top 30 |
| `idefics2::test_add_special_tokens_processor` | +94 MB | gone from top 30 |
| `llava_next_video::test_get_num_multimodal_tokens_matches_processor_call` | +89 MB | +22 MB |
| `idefics2::test_process_interleaved_images_prompts_image_splitting` | +82 MB | gone from top 30 |
| `idefics2::test_non_nested_images_with_batched_text` | +80 MB | gone from top 30 |
| `glm4v::test_apply_chat_template_video_frame_sampling` | +71 MB | gone from top 30 |
| `donut::test_image_processor_defaults` | +66 MB | gone from top 30 |
| `qianfan_ocr::test_apply_chat_template_image_0` | +60 MB | gone from top 30 |
| `idefics3::test_image_processor_defaults` | +45 MB | gone from top 30 |