Add MiniViT: lightweight Vision Transformer for CIFAR-scale image classification
Author: justynigam
Pull request: no
Labels: name: New model

### Model description

**Propose adding a new vision model, `MiniViT`, a compact Vision Transformer variant optimized for small image datasets (e.g., CIFAR-10/100).**

Include model implementation, configuration, feature extractor, tests, example script, and model card for Hub integration.

**Motivation:**
- There is demand for small, efficient ViT variants for edge and educational use.
- `MiniViT` provides a minimal ViT architecture (patch embedding, few transformer blocks, lightweight MLP head) that is useful for teaching, quick experiments, and resource-constrained training.

**Model details (high-level):**
- Architecture: patch embedding -> positional embeddings -> N transformer encoder blocks (multi-head self-attention + MLP) -> classification token or global pooling -> classification head.
- Inputs: images (batch, channels, height, width). Preprocessing via `AutoFeatureExtractor` / `ViTFeatureExtractor` style.
- Outputs: logits for classification; supports `MiniViTForImageClassification` with `MiniViTConfig`.
- Design choices: small embed dimension (e.g., 128), few heads (e.g., 4), shallow depth (e.g., 6), dropout and LayerNorm similar to existing ViT implementations.

**Implementation plan:**
- Add model package: `src/transformers/models/minivit/`
  - `configuration_minivit.py` — `MiniViTConfig` with typical config fields.
  - `modeling_minivit.py` — core `MiniViTModel` and `MiniViTForImageClassification`.
  - `feature_extraction_minivit.py` — feature extractor (resize / center crop / normalize) or reuse `AutoFeatureExtractor`.
  - `tokenization_minivit.py` — not required for vision-only model.
  - `__init__.py` — exports and integration mappings.
- Update registries/mappings: Add to AutoConfig/AutoModel/AutoFeatureExtractor mappings.
- Tests/examples: Add forward/config tests; example training script; fast CI config.
- Documentation/Hub: Model card, docs page, push weights example.

Minimal inference example:
```python
from transformers import MiniViTForImageClassification, MiniViTConfig, AutoFeatureExtractor
import torch
from PIL import Image

config = MiniViTConfig(image_size=32, patch_size=4, hidden_size=128, num_hidden_layers=6, num_attention_heads=4, num_labels=10)
model = MiniViTForImageClassification(config)
feature_extractor = AutoFeatureExtractor.from_pretrained("minivit-config-or-local")

img = Image.open("example.png").convert("RGB")
inputs = feature_extractor(img, return_tensors="pt")
outputs = model(**inputs)
logits = outputs.logits
preds = torch.argmax(logits, dim=-1)
```

### Open source status

- [x] The model implementation is available
- [ ] The model weights are available

### Provide useful links for the implementation

Implementation plan and modeling details outlined above. Authors: @justynigam (proposer).

No existing weights, but architecture is minimal and educationally valuable. Links: 
- ViT paper: https://arxiv.org/abs/2010.11929
- CIFAR: https://www.cs.toronto.edu/~kriz/cifar.html

Comments:
@justynigam looks interesting, but how much compute do we need to train this model based on the architecture for full precision?

@mahimairaja MiniViT is very small — training CIFAR-10 in full precision (FP32) fits on a single modern GPU and should take minutes for 100 epochs.


We generally do not add architectures until there are significant checkpoints available, unfortunately! Transformers is not intended to be a repository for all possible layers and architectures - we add architectures as needed to support checkpoints.

> [@mahimairaja](https://github.com/mahimairaja) MiniViT is very small — training CIFAR-10 in full precision (FP32) fits on a single modern GPU and should take minutes for 100 epochs.

That is interesting

> We generally do not add architectures until there are significant checkpoints available, unfortunately! Transformers is not intended to be a repository for all possible layers and architectures - we add architectures as needed to support checkpoints.

I have made documentation for these model with CIFAR-10 Dataset