Aquiles-ai/MoonViT-3D

The model weights and their respective implementation were taken from moonshotai/Kimi-K2.6, so all credit and thanks go to the Moonshot AI team.

This checkpoint is the vision tower of Kimi-K2.6 stored in the native Kimi_K25VisionModel format, so it works directly with transformers — no trust_remote_code needed.

Example usage:

from transformers import AutoImageProcessor, AutoModel
import torch
from PIL import Image

img = Image.open("a_red_car.png")
processor = AutoImageProcessor.from_pretrained("Aquiles-ai/MoonViT-3D")
model = AutoModel.from_pretrained("Aquiles-ai/MoonViT-3D", dtype=torch.bfloat16).to("cuda")

inputs = processor(images=[img], return_tensors="pt")
with torch.no_grad():
    out = model(
        pixel_values=inputs["pixel_values"].to(torch.bfloat16).to("cuda"),
        grid_thw=inputs["image_grid_thw"].to("cuda"),
    )
print(out.last_hidden_state.shape)  # (num_patches, 1152)
print(out.pooler_output.shape)      # (num_merged_blocks, 4, 1152)

Batch processing:

from transformers import AutoImageProcessor, AutoModel
import torch
from PIL import Image

img = Image.open("a_red_car.png")
img2 = Image.open("a_gray_cat.png")
processor = AutoImageProcessor.from_pretrained("Aquiles-ai/MoonViT-3D")
model = AutoModel.from_pretrained("Aquiles-ai/MoonViT-3D", dtype=torch.bfloat16).to("cuda")

inputs = processor(images=[img, img2], return_tensors="pt")
with torch.no_grad():
    out = model(
        pixel_values=inputs["pixel_values"].to(torch.bfloat16).to("cuda"),
        grid_thw=inputs["image_grid_thw"].to("cuda"),
    )
print(out.last_hidden_state.shape)  # (total_patches, 1152) - flattened over the batch
print(out.pooler_output.shape)      # (total_merged_blocks, 4, 1152)

Actual use cases for this model:

• Image embedding extraction for similarity search (image retrieval), dataset deduplication, or image clustering.
• Frozen backbone for downstream fine-tuning: attach a classification, detection, or regression head on top of the features and train only that part.
• Building block for your own VLM: connect it to an LLM (via a projection layer such as a Q-Former or MLP) if you want to replicate something similar to Kimi-VL but with your own text decoder.
• Multimodal RAG: generate image embeddings to index alongside text in a vector database and perform cross-modal text-to-image retrieval.
• Frame-by-frame video processing: since it supports grid_thw with a temporal dimension, in theory you can pass sequences of frames and get spatio-temporal features, although the modelcard doesn't include an example of this — you'd need to build it yourself.