SmolVLM: Redefining small and efficient multimodal models
Authors: Andrés Marafioti
Orr Zohar
Miquel Farré
Merve noyan
Elie Bakouch
Pedro Manuel Cuenca Jiménez
Cyril Zakka
Loubna Ben allal
Anton Lozhkov
Nouamane Tazi
Vaibhav Srivastav
Joshua Lochner
Hugo Larcher
Mathieu Morlon
Lewis Tunstall
Leandro Von Werra
Thomas Wolf
Venue: COLM 2025 (2025)
Keywords: Vision Language Models
Large Multimodal Models
Vision Understanding
Video Understanding
TL;DR: We explore extremely efficient VLMs starting at 256M parameters, which run with less than 1GB

Abstract: Large Vision-Language Models (VLMs) deliver exceptional performance but require significant computational resources, limiting their deployment on mobile and edge devices. Smaller VLMs typically mirror design choices of larger models, such as extensive image tokenization, leading to inefficient GPU memory usage and constrained practicality for on-device applications.

We introduce SmolVLM, a series of compact multimodal models specifically engineered for resource-efficient inference. We systematically explore architectural configurations, tokenization strategies, and data curation optimized for low computational overhead. Through this, we identify key design choices that yield substantial performance gains on both image and video tasks within minimal memory footprints.

Our smallest model, SmolVLM-256M, uses less than 1GB GPU memory during inference and outperforms the 300-times larger Idefics-80B model, despite an 18-month development gap. Our largest model, at 2.2B parameters, rivals state-of-the-art VLMs consuming twice the GPU memory. SmolVLM models extend beyond static images, demonstrating robust video comprehension capabilities.

Our results emphasize that strategic architectural optimizations, aggressive yet efficient tokenization, and carefully curated training data significantly enhance multimodal performance, facilitating practical, energy-efficient deployments at significantly smaller scales.

SmolVLM: Redefining small and efficient multimodal models
Andrés Marafioti Orr Zohar Miquel Farré*;;
Merve Noyan", Elie Bakouch", Pedro Cuenca", Cyril Zakka", Loubna Ben Allal Anton Lozhkov", Nouamane Tazi", Vaibhav Srivastav", Joshua Lochner', Hugo Larcher', Mathieu Morlon", Lewis Tunstall Leandro von Werra", Thomas Wolf
"Hugging Face *Stanford University "Equal contribution
Code:
smolm
Abstract
Large Vision-Language Models (VLMs) deliver exceptional performance but require significant computational resources, limiting their deployment on mobile and edge devices. Smaller VLMs typically mirror design choices of larger models, such as extensive image tokenization, leading to inefficient
GPU memory usage and constrained practicality for on-device applications. We introduce SmolVLM, a series of compact multimodal models specifi- cally engineered for resource-efficient inference. We systematically explore architectural configurations, tokenization strategies, and data curation op- timized for low computational overhead. Through this, we identify key design choices that yield substantial performance gains on both image and video tasks within minimal memory footprints.
Our smallest model, SmolVLM-256MV uses less than 1GB GPU memory during inference and outperforms the 300-times larger Idefics-80B model, despite an 18-month development gap. Our largest model, at 2.2B pa- rameters, rivals state-of-the VLMs consuming twice the GPU memory. SmolVLM models extend beyond static images, demonstrating robust video comprehension capabilities.
Our results emphasize that strategic architectural optimizations, aggressive yet efficient tokenization, and carefully curated training data significantly enhance multimodal performance, facilitating practical, energy-efficiest deployments at significantly smaller scales.
50h
5095
800
30%
10
Usage
Figure 1: Smol yet Mighty: comparison of SmolVLM with other state-of-the-art small VLM models. Image results are sourced from the OpenCompass Open (eaan et al., 2024).
1 Introduction
Vision-Language Models (VLMs) have rapidly advanced in capability and adoption (Achiam et al., 2023; Bai et al., 2023; Beyer et al., 2024; Chen et al., 2024c; McKinzie et al., 2024), driving breakthroughs in cross-modal reasoning (Liu et al., 2024a; 2023) and document understanding (Appalaraju et al., 2021; Faysse et al., 2024; Livathinos et al., 2025; Nassar et al., 2025). However, these improvements typically entail large parameter counts and high computational demands.
1

---

11.20
Image
Input
11.00
Image
Linear Layer
Splitting
Encoder
Shuffe
Pace
10,00
enegi
What is this?
Tokenbzer
Figure 2: SmolVLM Architecture. Images are split into subimages, and frames are sampled from videos, and then encoded into visual features. These features are first rearranged via pixel-shuffle operation, then mapped into the LLM input space as visual tokens using an MLP projection. Visual tokens are then concatenated/ iiteeeddvveeeeeeeed with text embeddings (orange/red). This combined sequence is passed to the LLM for text output.
Since early large-scale VLMs like Flamingo (Alayrac et al., 2022a) and Idefics (Laurençon et al., 2023) demonstrated capabilities with 80B parameters, new models have slowly appeared at smaller sizes. However, these models often retain high memory demands due to architectural decisions made for their larger counterparts. For instance, Qwen2-VL (Wang et al., 2024) and InternVL 2.5 (Chen et al., 2024b) offer smaller variants (1B-2B), but retain significant computational overhead. Conversely, models from Meta (Llama 3.2 (Dubey et al., 2024)) and Google (Gemma 3) reserve vision capabilities for large-scale models. Even PaliGemma (Beyer et al., 2024), initially efficiency-focused, scaled up significantly in its second release (Steiner et al., 2024). In contrast, Moondream(Korrapat 2024) keeps focusing on improving performance while maintaining efficiency, and H2OVI-Missisippi((alll et al., 2024) explicitly targeted on-device deployment. Efficient processing particularly critical for video understanding tasks, exemplified by Apollo (Zohar et al., 2024b), where memory management is essential. Furthermore, reasoning LLMs generate more tokens during inference, compounding computational costs (DeepSeek-Aī, 2025; OpenAI et al., 2024). Therefore, efficiency per token becomes vital to ensure models remain practical for real-world use. Our contributions are:
Compact yet Powerful Models: We introduce SmoIVLM, family of powerful small-scale multimodal models, demonstrating that careful architectural design can substantially reduce resource requirements without sacrificing capability,
Efficient GPU Memory Usage: Our smallest model runs inference using less than
1GB GPU RAM, significantly lowering the barrier to on-device deployment. Systematic Architectural Exploration: We comprehensively investigate the impact of architectural choices, including encoder-LM parameter balance, tokenization methods, positional encoding, and training data composition, identifying critical factors that maximize performance in compact VLMs.
Fully Open-source Resources: To promote reproducibility and facilitate further research, we release all model weights, training datasets, and code, including mobile application showcasing inference on a smartphone.
2 Smoller Model Architecture
2.1 How to assign compute between vision and language towers?
VLMs utilize vision encoders (see Figure 2) to generate 'vision tokens' that are then fed into LM. We investigate optimal capacity allocation between vision encoders and language models (LMs) in compact VLMs. Specifically, we pair three SmolLM2 variants (135M, 360M, and 1.7B parameters) with two SigLIP encoders: a compact 93M SigLIP-B/ BBPPBB16666-- and larger 428M SigLIP-SO-400M. Typically, larger VLMs disproportionately allocate parameters to the LM; however, the LM is scaled down, this is longer the case.
2

---

0.55
0.66
0.55
0.51
9.50
0.50
3.60
0.43
9.40
0.40
3.40
0.47
Encoder Paroms
13M
3.16
418K
0.80
1010 1600
0.90
0.30
20:04
500M
0.45
lstal Parametiers (ND
Contest Langth
Model Size
Frame Asstrping Factor
Figure 3: Performance analysis of SmolVLM configurations. (Left) Impact of vision encoder and language model sizes. Smaller language models (135M) benefit less from larger vision encoders (SigLIP-SO-400M, 428M) compared to SELIP-BB/16. (93M), while larger language models gain more from powerful encoders. (Middle-left) Performance significantly improves with increased context lengths (2k to 16k tokens). (Middle-right) Optimal pixel shuffle factor (PS=2 vs. PS=4) varies by model size. (Right) Frame averaging reduces video performance, with rapid decline as more frames are averaged. Metrics average CIDEr (captioning) and accuracy (visual question answering).
Figure 3 (left) confirms that performance declines significantly when using a large encoder with the smallest LM (135M), highlighting inefficient encoder-LM balance. At interme- diate LM scale (360M), the larger encoder improves performance by 11.6%, yet this comes with substantial 66% increase parameters, making the compact encoder preferable. Only at the largest LM scale (1.7B), the larger encoder represent just a 10% parameter increase.
Finding 1. Compact multimodal models benefit from a balanced encoder-LM parameter allocation, making smaller vision encoders preferable for efficiency.
2.2 How can we efficiently pass the images the Language Model?
Following Laurençon et al. (2024), we adopt a self-attention architecture in which visual tokens from the vision encoder are concatenated with textual tokens and jointly processed by a language model (e.g., FROMAGe (Koh al., 2023), BLIP-2 (Li et al., 2023a)). This design requires significantly more context than the 2k-token limit used in SmoILM2, as single 512 x 512 image encoded with SigLIP-B/ extended the context capacity by increasing the RoPE base from 10k to 273k, following Liu et al. (2024c), and fine-tuned the model on a mix of long-context data (Dolma books (Soldaini et al., 2024), The Stack (Kocetkov et al., 2022)) and short-context sources (FineWeb-Edu (Penedo et al., 2024), DCLM (Li et al., 2024a), and math from SmolLM2).
requires 1024 tokens. To address this, we
stable at 16k tokens for the 1.7B LM, smaller models (135M, 360M) struggled beyond 8k. Experiments with our 2.2B SmolVLM confirmed consistent perfor- mance gains up to 16k tokens (Figure 3, middle). Accordingly, for SmolVLM and an 8k-token limit for smaller variants.
While fine-tuning
adopt 16k-token context
Finding 2. Compact VLMs significantly benefit from extended context lengths.
Extending the context window alone is not sufficient. Recent VLMs (e.g., MM1 (McKinzie et al., 2024), MiniCPM-V (Yao et al., 2024), InternVL (Chen al., 2024c)) combine the self- attention architecture with token compression techniques (Zohar et al., 2024b; Laurençon et al., 2024) fit longer sequences efficiently and reduce computational overhead.
One particularly effective compression method is pixel shuffle (space-to-depth), originally proposed for super-resolution tasks (Shi et al., 2016) and recently adopted by Idefics3. Pixel shuffle rearranges spatial features into additional channels, reducing spatial resolution but increasing representational density (Fig. 4). This reduces the total number of visual tokens by a factor of r2, where r is the shuffle ratio. However, higher ratios collapse larger spatial regions into single tokens, impairing tasks requiring precise localization, such as OCR. Models like Intern and Idefics3 use r = to balance compression and spatial fidelity. In contrast, our experiments (Fig. 3, right) show that smaller VLMs benefit from
3

---

64x61x1
32x32x1
16x16x16
8x8x64
Chamaale
Waoth
Wodts
Widti
Figure 4: Pixel shuffle. Rearranges encoded images, trading spatial resolution for increased channel depth. This reduces visual token count while preserving information density.
more aggressive compression (r (r= 4) as the reduced token count eases attention overhead and improves long-contex! modeling.
Finding Small VLMs benefit from more aggressive visual token compression.
2.3 How can we efficiently encode images and videos?
Balancing token allocation between images and videos crucial for efficient multimodal modeling: images benefit from higher resolution and more tokens to retain fidelity, whereas videos typically require fewer tokens per frame to handle longer sequences efficiently.
To achieve this, we successfully adopted image-splitting strategy inspired by UReader (Ye et al., 2023) and SPHINX (Lin et al., 2023b), where high-resolution images are divided into multiple sub-images along with downsized version of the original. This approach proved effective in maintaining image quality without excessive computational overhead. For videos, however, we found that strategies such as frame averaging, inspired by Liu et al. (2024f), negatively impacted performance. As shown in Figure 3 (right), combining multiple frames significantly degraded OpenCompass- Vodesss-iieoo results, particularly higher averaging factors (2, 4, 8). Consequently, frame averaging was excluded from SmolVLM's final design, and videos frames were instead rescaled to the resolution of the image encoder.
Finding 4. For small models, image splitting enhances performance for vision tasks, whereas video frame averaging does not.
3 Smol Instruction Tuning
3.1 Learned Tokens vs. String
A major design consideration in SmolVLM involves encoding split subimage positions effectively. Initially, we tried to use simple string tokens (e.g., <row col 2>), which caused early training plateaus called the "OCR loss plague" characterized by sudden loss drops without corresponding improvements in OCR performance (Figure 5, left, middle).
To address instability during training, we introduced positional tokens, significantly im- proving training convergence and reducing stalls. Although larger models were relatively robust using raw string positions, smaller models benefited substantially from positional tokens, achieving notably higher OCR accuracy and improved generalization across tasks. Figure 5 (center) shows that learned positional tokens consistently outperform naive string positions on multiple image and text benchmarks. Additionally, Figure 5 (right) illustrates that models leveraging learned tokens consistently score higher in both OpenCompass- Image and OpenCompass- Video evaluations, underscoring the effectiveness of structured positional tokenization in compact multimodal models.
Finding Learned positional tokens outperform raw tokens for compact VLMs.
4

---

256M string lostinss leamed
Mean and VQA Anc.
15
90
Paapppss-ieeeeeee Video
2.0
15
0.5
Learned Tokeas
Learaed Towens
-
without
without
with
with
0.
Optimization Steps
(M)
OpenCompaest-nnon
Figure 5: Tokenization Strategy Comparisons. (Left) Training loss curves illustrating the
"OCR loss plague" when using string-based tokens in smaller models. (Center) Aggre- gated evaluation metrics showing consistently higher scores with learned tokens (orange). (Right) Scatter plot of OpenCompass-Iman vs. OpenCompass-Vide Videe:: learned tokens domi- nate the higher-scoring region, especially in image-intensive tasks.
3.2 Structured Text Prompts and Media Segmentation
We evaluated how system prompts and explicit media intro/ /outro prefixes incrementally improve SmolVLM's performance on image (left) and video (right) benchmarks, as shown in Figure 6. Each violin plot represents three checkpoints for given configuration.
System Prompts. We prepend concise instructions to clarify task objectives and reduce ambiguity during zero-shot inference. For example, conversational datasets utilize prompts like "You are a useful conversational assistant," " whereas vision-focused tasks employ "You are a visual agent and should provide concise answers." The second violin plot in each subplot (Fig. 6) illustrates clear performance improvements from incorporating these system prompts, particularly evident in image-centric tasks.
Media Intro/Outro Tokens. To clearly demarcate visual content, we introduce textual markers around image and video segments (e.g,, "Here is an image..." " and "Here are N frames sampled from video..."). The outro tokens then transition back to textual instructions (e.g,, "Given this image/video..." "). The third violin indicates that this strategy substantially boosts performance video tasks waske where confusion between multiple frames is more likely-and and still yields measurable improvements on image tasks.
Masking User Prompts Drawing techniques from Allal et al. (2025), we explore user- prompt masking during supervised fine-tuning as a way to reduce overfitting. The right violin plot in Figure 6 shows that masking user queries (orange) yields improved perfor- mance in both image and video tasks, compared to the unmasked baseline (blue). This effect especially pronounced in multimodal QA, where questions often repetitive and can be trivially memorized by the model. Masking thus forces SmolVLM to rely on task-related content rather than superficial repetition, promoting better generalization.
Finding 6. System prompts and media intro/ outroo tokens significantly improve compact VLM performance, particularly for video tasks. During SFT, only train on completions.
3.3 Impact of Text Data Reuse from LLM-SFT
seemingly intuitive practice is to reuse text data from the final supervised fine-tuning stages of large language models, anticipating in-distribution prompts and higher-quality linguistic inputs. However, Figure 7 (left) shows that incorporating LLM-SFT text data (SmolTalk) can degrade performance in smaller multimodal architectures by as much 3.7% in video tasks and 6.5% in image tasks. We attribute this negative transfer to reduced data diversity, which outweighs any benefits of reusing text. In keeping with Zohar et al. (2024b), we therefore maintain strict 14% text proportion in our training mix. These findings highlight the importance of a carefully balanced data pipeline, rather than direct adoption of large-scale SFT text for small-scale multimodal models.
Finding 7. Adding text from SFT blend proved worse than new text SFT data.
5

---

No System Prompt
With System Frompt
With Intro/Cutre
With User Procpt Masked
0.55
0.55
Average Score
0.50
0.50
0.40
0.35
Image
Video
Figure 6: Cumulative Effect of Training Strategies on SmolVLM Performance. The visualization shows the progression of performance improvements different tokenization and prompt engineering strategies are applied sequentially to the SmolVLM base model. (Left) Image benchmark results show consistent improvements with each added strategy. (Right) Video benchmark results reveal similar patterns with more pronounced gains.
3.4 Optimizing Chain-of-Though Integration for Compact Models
Mhain-of-Thoughh (CoT) prompting, which exposes models to explicit reasoning steps dur- ing training, generally enhances reasoning capabilities in large models. However, its effect smaller multimodal architectures remains unclear. To investigate this, varied the proportion of CoT data integrated into the Mammoth dataset (Yue et al., 2024b), covering text, image, and video tasks. Figure 7 (middle) shows that incorporating minimal frac- tion (0.02-0.05%) of CoT examples slightly improved performance, but higher proportions markedly degraded results- especially in image tasks. These observations suggest that excessive crasoning-orienter textual data can overwhelm the limited capacity of smaller VLMs, thereby compromising their visual representation capabilities. Consequently, com- pact models benefit most from very sparse inclusion of CoT data rather than the extensive use typically beneficial in larger-scale architectures.
Finding Excessive CoT data harms compact model performance.
3.5 Impact of Video Sequence Length on Model Performance
Increasing video duration during training offers richer temporal context but comes at greater computational cost. To identify optimal duration, we trained SmolVLM average video lengths ranging from 1.5 to 3.5 minutes. Figure 7 (right) demonstrates clear performance improvements for both video and image benchmarks as video durations approached approx- imately 3.5 minutes, likely due to more effective cross-modal feature learning, Extending video duration beyond 3.5 minutes yielded minimal further gains, indicating diminishing returns relative to the added computational expense. Thus, moderately extending video sequences enhances performance significantly in smaller models, whereas overly long sequences do not proportionally justify their computational cost.
Finding 9. Moderately increasing video duration during training improves both video and image task performance in compact VLMs.
Experimental Results
We construct three variants SmolVLM, tailored to different computational environments:
SmoIVLM-256M: Our smallest model, combining the 93M-parameter SigLIP- B/16 (Zhai et al., 2023) and the SmolLM2-135M (Allal et al., 2025). It operates on < 1GB GRAM, making it ideal for resource-constrainee edge application.
SmolVLM-500M: A mid-range model featuring the same 93M-parameter SigLIP- B/16 paired with the larger SmoLLM2-360M balances memory efficiency and performance and is suitable for moderate-resource edge devices.

---

0.55
0.55
Withou: SmolTalks
With SmolTblk
OpenConpas. Average
.7%%%
0.45
5%
0.45
Vudco
Video
Image
Image
0.35
0.35
Video
Image
0.0
0.2
0.4
0.6
1.5 min
2,5 min
3.5 min
CoT Data
Average Video Duration
Figure 7: Impact of Training Strategies Smol-Scale Multimodal Models. (Left) Reusing text data from LLM-SFT (SmolTalk) reduces both image and video scores in smaller models. (Middle) A minimal fraction (005%)) of Chain-of-Thought (CoT) data yields optimal results, while heavier CoT usage degrades performance. (Right) Increasing average video duration beyond 3.5 min leads to diminished returns for both image and video tasks.
SmolVLM-2.2B: The largest variant, with a 400M-parameter SigLIP-SO400N and a 1.7B-parameter SmolLM2 backbone. This model maximizes performance while remaining deployable on higher-end edge systems.
4.1 Training Data
Model training proceeds in two stages: (1) a vision stage, and (2) a video stage. The vision training stage uses new mixture of the datasets used in Laurençon et al. (2024), to which we added MathWriting (Gervais al., 2024). The mixture was balanced to emphasize visual and structured data interpretation while maintaining the focus on reasoning and problem- solving capabilities. The visual components comprise document understanding, captioning and visual question answering (including 2% dedicated to multi-image reasoning), chart understanding, table understanding, and visual reasoning tasks. To preserve the model's performance in text-based tasks, retained modest of general knowledge Q&A and text- based reasoning & logic problems, which incorporate mathematics and coding challenges.
The video fine-tuning stage maintains 14% of text data and 33% of video to achieve optimal performance, following the learnings of Zohar et al. (2024b). For video, we sample visual description and captioning from LLaVA-video-178k (Zhang al., 2024), Video-STAR (Zohar et al., 2024a), Vript (Yang et al., 2024), and ShareGPT4 Video (Chen et al., 2023), tempo- ral understanding from Vista-400k (Ren al., 2024), and narrative comprehension from MovieChat (Song et al., 2024) and Video (Farré et al., 2024). Multi-image data was sampled from M4-Instruct (Liu al., 2024a) and Mammoth (Guo et al., 2024). The text samples were sourced from (Xu et al., 2024).
For a more granular description, Figure provides detailed overview of the training data distribution used in both our vision and video fine-tuning stages.
Evaluation details
We evaluated SmolVLM using VLMEEal (Duan et al., 2024) to ensure reproducibility. The full results are available online'. Currently, Open VLM Leaderboard covers 239 different VLMs and 31 different multi-modal benchmarks. Further, we plot the performance against the RAM required to run the evaluations. We argue that model size is usually used as proxy for the computational cost required to run model. This is misleading for VLMs because the architecture strongly influences how expensive is to run the model, in our opinion RAM usage is better proxy. For the experiments, input image resizing. For SmolVLM, this resizes the longest edge of images to 1536 the 256M and 500M models and 1920 in the 2.2B one.
use each model's 's default
'Open Leaderboard

---

1204
Text
Figure 8: Data Details. Training dataset details for Vision (Left) and video (Right), broken down by modality and sub-categories.
4.3 Strong Performance at a Tiny Scale
We evaluate SmoIVLM's performance relative to model size, comparing three variants (256M, 500M, and 2.2B) against efficient state-of-the-art (SOTA) open-source (OS) model. Table summarizes results across nine demanding vision-language benchmarks and five video benchmarks. We highlight the table MolmoE 7B with 1B activate parameters (Deitke et al., 2004))Molmmo--111BB7 for vision tasks and VL.2-2VL.2-2F Chen et al. (2024c) for video tasks. broader array competing models are showcased in Fig. 1.
Efficiency and Memory Footprint. SmolVLM demonstrates remarkable computational ef- ficiency compared to significantly larger models. Single-image inference requires only 0.8GB of VRAM for the 256M variant, 1.2GB for the 500M, and 4.9GB for the 2.2B dramaticalll Even compared to models of similar pa-
lower than 27.7GB required by MolmoE-A1B-77 rameter scales, SmolVLM is notably more efficient: Qwen2VL-2B requires 13.7GB VRAM and Intern VL2-2B requires 10.5GB VRAM, highlighting that parameter count alone does not dictate compute requirements. At batch size 64, memory usage for SmolVLM remains prac- tical: 15.0GB (256M), 16.0GB (500M), and 49. 99.9GB (2.2B). These results highlight SmolVL.M's substantial advantages for deployment in GPU-constrained environments.
Overall Gains from Scaling. Increasing SmolVLM's parameter count consistently yields substantial performance improvements across all evaluated benchmarks. The largest model (2.28) achieves the highest overall score at 59.8%, followed by the intermediate 500M variant (51.0%) and the smallest 256M variant (44.0%). Notably, even the smallest SmolVLM- 256M significantly surpasses the much larger Idefics 8ÓB model (see Fig. 1) on nearly all benchmarks, emphasizing effective vision capabilities at modest scales. The few ex- MMMU (29.0% vs. 42.3%) and A12D (46.4% vs. 56.3%) benchmarks where strong linguistic reasoning from large language backbone remains crucial. Intriguingly, visually oriented tasks such as OCRBench also benefit markedly from scaling language model capacity, with nearly 10-point improvement when moving from 256M (52.6%) to 500M (61.0%). These results underscore that larger language models pro- vide enhanced context management and improved multimodal reasoning, benefiting both iegguage-intensing and vision-centric tasks.
Comparison with Other Compact VLMs. Figure situates SmolVL.M-2.2B among recent small-scale VLMs by comparing OpenCompass benchmark performance against GPU memory consumption per image. SmolVLM--2225 achieves notably strong performance on MathVista (51.5) and ScienceQA (90.0), while maintaining exceptionally low GPU usage of 4.99t VRAM. contrast, models requiring significant more compute offer such as Qwen2VL-2B and InternVL2-2B aren't clearly better performers. Specifically, Qwen2VL-2B slightly surpasses SmolVLM-2.22 on AI2D (74.7 vs. 70.0) and ChartQA (73.5 vs. 68.8), yet falls short on Vista (48.0 vs. 51.5) and ScienceQA (78.7 90.0). Similarly, VnternVL.-22B
8

---

Capability
Benchmark
SmolVLM 256M
SmolVLM 500M
SmolVLM 2.28
Efficient OS
OCRBench (Iin 2024e)
52.6%
61.0%
72.9%
54.7%
Character Recugnition
MolmoE-AIIB 1877
AI2D (Kembhavi et al,, 2016)
46.4%
59.2%
70.0%
71.0%
Single-Image
Diagrams
MolmolEAA1B-7
ChartQA (Masry et 2022)
55.6%
62.8%
68.7%
48.0%
Chart Lnderstanding
Molmol-AAI A18-77
TextVQA (Singh et al, 2019)
50.2%
60.2%
73.0%
61.5%
Text Understanding
MolmoFF-A1B-77
DocVQA (Mathew al., 2021)
58.3%
70.3%
80.0%
77.7%
Document Understanding
Molmoe--AJ -1B---
ScienceQA 2022)
73.8%
80.0%
High Science
MolmoE-A1B-
MMMU (Yue et al., 2024a)
29.0%
33.7%
42.0%
33,9%
Multi-task
College-Ievel Multndiscipiine
MolmoE-A1B--7
Visa (Lu et al., 2024h)
35.9%
40.1%
51.5%
37.6%
General Math Underslanding
MolmolB A1B-7B
(Chen al., 2024a)
38.3%
Multidisiplinnry Reasoning
MolmoEEAA1B-7
Mideo-MME (Fu et al., 2024)
33.7%
42.2%
52.1%
45.0%
General Video Understanding
IntemVVI
Video
MLVU (Zhou et al., 2024)
40.6%
47.3%
55.2%
48.2%
MovieQA - MSRVTT-Cap
InternVLL2228
MVBench (Li et al., 2024b)
32.7%
39.7%
46.3%
60.2%
Multiviow Reasoring
VL2-22
WorldSense (Hong al., 2025)
29.7%
Temporal Physies
Qwenn2V1 17B
TempCompass (Liu 2024d)
43.1%
49.0%
33.7%
53.4%
Temporal Understanding
InternV/12228
Overall Avg.
Across Benchmarks
44.0%
51.0%
59.8%
RAM Usage
Batch size
=1
0.8 GB
1.2 GB
4.9 GB
27.7 GB
MolmoE-A11B-77
batch size 64
15.0
16.0 CB
49.9 GB
Table 1:
Performance comparison between SmolVLM models (256M-2.2B) and efficient open source models on vision- and video-language benchmarks. RAM is reported in GB.
achieves higher scores ScienceQA (94.1 vs. 90.0) and MMStar (49.8 vs. 46.0), but at more than double the VRAM cost.
Further comparisons highlight distinct trade-offs among size, memory footprint, and task- specific performance. MiniCPM-V2 (2.8B parameters) underperforms SmolVLM-2222B on most benchmarks. Other models such Moondream2 and PaliGemma (both around 2-3B parameters) exhibit significant variance across tasks: Moondream2, for instance, scores well CharlQA (72.2) with just 3.9GB VRAM but substantially underperforms MMMU (29.3). Conversely, PaliGemma excels at ScienceQA (94.3) yet struggles on ChartQA (33.7). This variability underscores how specialized training impacts per-task.
Video Benchmarks. Table provides comprehensive results across five diverse video benchmarks: Video-MME, MLVU, MVBench, TempCompass, and WorldSense. SmolVLM- 2.2B notably excels at Video-MME (52.1) and WorldSense (36.2), outperforming significantly larger models such as Qwen2 VL-7B (32.4 on WorldSense), showcasing strong capabilities in complex multimodal video comprehension tasks. The SmolVLM-500M variant also demonstrates robust performance, achieving competitive scores on TempCompass (49.0) and WorldSense (30.6), highlighting sophisticated temporal reasoning and real-world visual understanding at scale ideal for edge-device deployment. Despite their compact parameter counts, SmolVLM variants consistently balance efficient resource use with impressive accuracy, reinforcing their suitability for resource-constrained multimodal scenarios.
4.4 On-Device Performance
To comprehensively assess the deployment practicality of SmolVLM, we benchmarked its throughput across varying batch sizes two representative hardware platforms: NVIDIA A100 and NVIDIA L4 GPUs (see Figure 9). Our evaluations highlight SmolVLM' suitability for on-device and edge deployment scenarios.

---

1000
2003
2.56
relige
50054
coflee
000
operatiion The appliance
roloned devicon with
bowe and
City an tao The top The sppllionc &
Muzt ristteed by gff pullon lift which
Goafi
200
10
32
61
Visual Intelligence Hugging Face
Dabch Stze
MI Kozc Giica
501 Mas
Fou tru s208
Piotes iibers
80-
70-
900
Tolacs 8:0
whore can Snd
Figure 9: SmolVLM edge device. (Left) Examples of the HuggingSnap app, where SmolVLM LM can run locally, on the device, on consumer phones. For example, interactions can be done using mobile interface to detect objects and answer questions. (Right) Throughput in tokens per second on NVIDIA A100 GPUs (top) and different consumer personal comput- ers (bottom) across different batch sizes and model variants.
On the A100 GPU, the smallest SmolVLM-256M, variant achieves impressive throughput, scaling from 0.8 examples per second at batch size to 16.3 examples per second at batch size 64. The 500M variant similarly scales from 0.7 to 9.9 examples per second, while the largest 2.2B variant demonstrates more modest scaling (0.6 to 1.7 examples per second), indicative of its higher computational demands.
Evaluations on the L4 GPU further emphasize SmolVLM's edge compatibility. Here, the 256M variant reaches peak throughput at 2.7 examples per second with batch size 8, subse- quently diminishing due memory constraints. The 500M and 2.2B variants peak at lower batch sizes (1.4 and 0.25 examples per second, respectively), underscoring their efficiency even under more restrictive hardware conditions.
Finally, we accompany the release with several optimized ONNX (Open Neural Network Exchange) exports, facilitating pross-llaffom compatibility and broadening deployment opportunities across consumer-grade hardware targets. Notably, we demonstrate the ability to efficiently run these models locally within browser environment via WebGPU, with the 256M variant achieving up to 80 decode tokens per second MacBook Pro (M4 Max).
5 Conclusion
We introduced SmolVLM, a family of memory-efficiett Vision- Lsiion-Language Models ranging from 256M to 2.2B parameters. Remarkably, even our smallest variant requires less than 1GB of GPU memory yet surpasses state-of-theear 80B-parameter models from just 18 months ago (Laurençon et al., 2023). Our findings emphasize critical insight: scaling down large VLM architectures optimized under resource-rich conditions results in disproportionately high memory demands during inference with little advantage over specialized architectures. By contrast, SmolVLM's design philosophy explicitly prioritizes compact architectural innovations, aggressive but careful tokenization methods, and efficient training strategies, enabling powerful multimodal capabilities at fraction of the computational cost.
All model weights, training datasets, and training code are publicly released to encourage reproducibility, transparency, and continued innovation. We hope SmolVLM will inspire the next generation of lightweight, efficient VLMs, unlocking new possibilities for real-time multimodal inference with minimal power consumption.
10

---

References
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Tlge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. Xiv preprint ar arXfv:2303.08774 2023.
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Has- son, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob L. Menick, Sebastian Borgeaud, Andy Brock, Aida Nematzadeh, Sa- hand Sharifzadeh, Mikot aj Binkkwski,, Ricardo Barreira, Oriol Vinyals, Andrew Zis- serman, and Karén Simonyan. Flamingo: a visual language model for few-shot learn- ing. In S. Koyejo, Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 23716-23736. Curran As- sociates, Inc., 2022a. URL https:: https::/prooceeednggg neurips httppps.ccpppppppp cc/paper.pppppooccppcco lalpppppplleeeeeeeepppaaapppappe walles/paper/2// file/96ea1 Fil.66567777cffffo777ccccccc1110
pdf.
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: visual language model for few-shot learning. Advances in neural information processing systems, 35:23716-23773555 2022b.
Loubna Ben Allal, Anton Lozhkov Elie Bakouch, Gabriel Martín Blázquez, Guilherme
Penedo, Lewis Tunstall, Andrés Marafioti, Hynek Kydlfček, Agustín Piqueres Lajarín, Vaibhav Srivastav, Joshua Lochner, Caleb Fahlgren, Xuan-Son Nguyen, Clémentine Four- rier, Ben Burtenshaw, Hugo Larcher, Haojun Zhao, Cyril Zakka, Mathieu Morlon, Colin Raffel, Leandro von Werra, and Thomas Wolf. Smollm2: When smol goes big data-centric training of small language model, 2025. URL https: org absr 02737.
Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. Vqa: Visual question answering. Proceedings of the IEEE international conference computer vision, pp. 2433-2433, 2015.
Srikar Appalaraju, Bhavan Jasani, Bhargava Urala Kota, Yusheng Xie, and R Manmatha. Docformer: End-to-end transformer for document understanding. In Proceedings of the IEEE/CVF international conference computer vision, pp. 11199993-000 1003, 2021.
Jinze Bai, Shuai Bai, Shusheng Yang. Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. 2023.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025. URL https: http::///xxxvvoooee htrrgaabs/2502... 13923.
Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, et al. Paligemma: A versatile 3b vlm for transfer. arXiv preprint ar WxXi=:247.07722 2024.
Ali Furkan Biten, Rubēn Tito, Andrés Mafla, Lluis Gomez, Marçal Rusiñol, C.V. Jawahar, Ernest Valveny, and Dimosthenis Karatzas. Scene text visual question answering. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4290-4300, 2019. doi: 10.1109/ICC /Tt////CCC..10///333
Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem, and Juan Carlos Niebles. Activi- tynet: A large-scale video benchmark for human activity understanding. In Proceedings of the ieee conference computer vision and pattern recognition, pp. 961-970, 2015.
11

---

David Chen and Bill Dolan. Collecting highly parallel data for para- phrase evaluation.
URL Association for Computational Linguistics, January
2011.
Lin Chen, Jisong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. arXiv preprint ar arXi::2311.1279: 2023.
Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision-language models? arXiv preprint arXit:2403. 2024a.
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint d/Xiv:2412.05271, 2024b.
Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. ar Xiv preprint ar arXiv:2404.7682, 2024c.
DeepSeek-AL Deepseek-rl: Incentiizinng reasoning capability in llms via reinforcement learning, 2025. URL https://axxi..org/bbb/// 12948.
Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, Jiasen Lu, Taira Anderson, Erin Bransom, Kiana Ehsani, Huong Ngo, YenSung Chen, Ajay Patel, Mark Yatskar, Chris Callison-Burch, Andrew Head, Rose Hendrix, Favyen Bastani, Eli Van- derBilt, Nathan Lambert, Yvonne Chou, Arnavi Chheda, Jenna Sparks, Sam Skjonsberg, Michael Schmitz, Aaron Sarnat, Byron Bischoff, Pete Walsh, Chris Newell, Piper Wolters, Tanmay Gupta, Kuo-Hao Zeng, Jon Borchardt, Dirk Groeneveld, Crystal Nam, Sophie Lebrecht, Caitlin Wittlif, Carissa Schoenick, Oscar Michel, Ranjay Krishna, Luca Weihs, Noah A. Smith, Hannaneh Hajishirzi, Ross Girshick, Ali Farhadi, and Aniruddha Kemb- havi. Molmo and pixmo: Open weights and open data for state-of-the-an vision-language models, 2024. URL https: /tt////axxivv wtgg/bs//44al 17146.
Haodong Duan, Junming Yang, Yuxuan Qiao, Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong, Yuhang Zang, Pan Zhang, Jiaqi Wang, et al. Vimevalkit: An open-source toolkit for evaluating large multi-modality models. arviv preprint ar Xio:2407.111991 2024.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama herd of models. arXiv preprint or Xiv:2440 21783, 2024.
Miquel Farré, Andi Marafioti, Lewis Tunstall, Leandro Von Werra, and Thomas Wolf. co/
Finevideo.
2024.
Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Omrani, Gautier Viaud, Céline Hudelot, and Pierre Colombo. Colpali: Efficient document retrieval with vision language models. aiv preprint Xiv:2407.000 003000001440 0194 2024.
Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever com- prehensive evaluation benchmark of multi-modal llms in video analysis. ar preprint arXiv::405.21075, 2024.
Shaikat Galib, Shanshan Wang, Guanshuo Xu, Pascal Pfeiffer, Ryan Chesler, Mark Landry, and Sri Satish Ambati. H2ovl-mississippi vision language models technical report, 2024. URL https://arkiv.org//b/////// 13611.
12

---

Philippe Gervais, Asya Fadeeva, and Andrii Maksai. Mathwriting: A dataset for handwrit- ten mathematical expression recognition, 2024. URL http:p////axxw.org//////o 2404. 10690.
Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the v vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and paltern recognittin,, pp. 6904-6913, 2017.
Jarvis Guo, Tuney Zheng, Yuelin Bai, Bo Li, Yubo Wang, King Zhu, Yizhi Li, Graham Neubig, Wenhu Chen, and Xiang Yue. Mammoth-vl: Eliciting multimodal reasoning with instruction tuning at scale. 2024. URL //tps:///aav
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. Xiv preprint ar axXiv:2203.15556, 2022.
Jack Hong, Shilin Yan, Jiayin Cai, Xiaolong Jiang, Yao Hu, and Weidi Xie. Worldsense: Evaluating real-world omnimodal understanding for multimodal 1lms, 2025. URL https: //arxiv. http:/..og/bbb///////// 04326.
Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, Xinrong Zhang, Zheng Leng Thai, Kaihuo Zhang, Chongyi Wang, Yuan Yao, Chenyang Zhao, Jie Zhou, Jie Cai, Zhongwu Zhai, Ning Ding, Chao Jia, Guoyang Zeng, Dahai Li, Zhiyuan Liu, and Maosong Sun. Minicpm: Unveiling the potential of small language models with scalable training strategies, 2024. URL https://arxiv. wwwpiiioorr//bbbbe wrg//or////////////o/ 06395.
Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Joao Carreira. Perceiver: General perception with iterative attention. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 4651-4664. PMLR, 18-24 Jul wtt.gssml...rrss//eeeeeeeee wwwpppresssvvve 3139/ jajgle22 a.tml.
2021. URL https: / http://pooceeeenn...e...
Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (eds.), Computer Vision ECCV 2016, pp. 235-251, Cham, 2016. Springer International Publishing. ISBN 978-3-319-46493-.
Denis Kocetkov, Raymond Li, Loubna Ben Allal, Jia Li, Chenghao Mou, Carlos Muñoz Ferrandis, Yacine Jernite, Margaret Mitchell, Sean Hughes, Thomas Wolf, et al. The stack: 3 tb of permissively licensed source code. arXiv preprint arXio:2211.15533, 2022.
Jing Yu Koh, Ruslan Salakhutdinov, and Daniel Fried. Grounding language models to images for multimodal inputs and outputs, 2023.
Vik Korrapati. Moondream. Online, 2024. URL httph::://o/ooderee ai/. Accessed: 2025-03-
27.
Hugo Laurençon, Lucile Saulnier, Leo Tronchon, Stas Bekman, Amanpreet Singh, An- ton Lozhkov, Thomas Wang. Siddharth Karamcheti, Alexander M Rush, Douwe Kiela, Matthieu Cord, and Victor Sanh. OBELICS: An open web-scale filtered dataset of inter- leaved image-text documents. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL https: http::://ppmmes http:///eee///eeee////ee//. wwtp./wwww.efoormiiiii.e.e SKN2hf1BIZ.
Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh. What matters when building vision-language models? arviv preprint ar XrX::2245.002246 2024.
Hugo Laurençon, Andrés Marafioti, Victor Sanh, and Léo Tronchon. Building and better understanding vision-language models: insights and future directions, 2024. URL https: wwwrxxi.org/abs///e/ 12637.
13

---

Jeffrey Li, Alex Fang, Georgios Smyrnis, Maor Ivgi, Matt Jordan, Samir Gadre, Hritik Bansal, Etash Guha, Sedrick Keh, Kushal Arora, et al. Datacomp-LM: In search of the next generation of training sets for language models. arXiv preprint ar arXio:2406.11794, 2024a
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: bootstrapping language- image pre-training with frozen image encoders and large language models. In Proceddings of the 40th International Conference on Machine Learning, ICML'23, JMLR.org, 2023a.
Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, Limin Wang, and Yu Qiao. Mvbench: A comprehensive multi-modal video understanding benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 22195-22206, June 2024b.
Rui Li, Xiaohan Wang, Yuhui Zhang, Zeyu Wang, and Serena Yeung-Levy Temporal prefer- ence optimization for long-for form video understanding. arXiv preprint XrXi::2501.1391, 2025a.
Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. Textbooks are all you need ii: phi-1.5 technical report. ar preprint arXtt:2309.05463, 2023b.
Yuncheng Li, Yale Song, Liangliang Cao, Joel Tetreault, Larry Goldberg, Alejandro Jaimes, and Jiebo Luo. Tgif: A new dataset and benchmark on animated gif description. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4641-4650, 2016.
Zhiqi Li, Guo Chen, Shilong Liu, Shihao Wang, Vibashan VS, Yishen Ji, Shiyi Lan, Hao Zhang, Yilin Zhao, Subhashree Radhakrishnan, et al. Eagle 2: Building post-training data strategies from scratch for frontier vision-language models. ar Xiv preprint arXiv:2501.74818, 2025b.
Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. arXiv preprint a/Xie:2311.00122, 2023a.
Ziyi Lin, Chris Liu, Renrui Zhang, Peng Gao, Longtian Qiu, Han Xiao, Han Qiu, Chen Lin, Wenqi Shao, Keqin Chen, et al. Sphinx: The joint mixing of weights, tasks, and visual embeddings for multi-modal large language models. arXio preprint 2rXi::2311.07555, 2023b.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https: ieteenrriiee.eetfooooodddddoooo
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024a. URL https://llava-vl gi thub. hotplo/////////////////// 2024-01 30- 1a11-----aa-aameee llava-next/:
Jiajun Liu, Yibing Wang, Hanghang Ma, Xiaoping Wu, Xiaoqi Ma, Xiaoming Wei, Jianbin Jiao, Enhua Wu, and lie Hu. Kangaroo: A powerful video-language model supporting long-context video input. arXiv preprint ar 4rXiv:2408.15542, 2024b.
Xiaoran Liu, Hang Yan, Shuo Zhang, Chenxin An, Xipeng Qiu, and Dahua Lin. Scaling laws of rope-based extrapolation, 2024c. URL https://arxxv.org// abgp//////////////// 05209.
Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. Tempcompass: Do video Ilms really understand videos?, 2024d. URL
00476.
Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xu- Xu-Cheng Yin, Cheng-Lin Liu, Lianwen Jin, and Xiang Bai. Ocrbench: on the hidden mystery of ocr in large multimodal models. Science China Information Sciences, 6211)222001,, 2024e.
14

---

Zhijian Liu, Ligeng Zhu, Baifeng Shi, Zhuoyang Zhang, Yuming Lou, Shang Yang, Haocheng Xi, Shiyi Cao, Yuxian Gu, Dacheng Li, et al. Nvila: Efficient frontier visual language models. arviv preprint ar arXio:2412.04468, 2024f.
Zuyan Liu, Yuhao Dong, Ziwei Liu, Winston Hu, Jiwen Lu, and Yongming Rao. Oryx milm: On-demand spatial-temporal understanding at arbitrary resolution. arXiv preprint ar arXir:2409.1298 2024g.
Nikolaos Livathinos, Christoph Auer, Maksym Lysak, Ahmed Nassar, Michele Dolfi, Panos Vagenas, Cesar Berrospi Ramis, Matteo Omenetti, Kasper Dinkla, Yusik Kim, et al. Docling: An efficient open-source toolkít for ai-driven document conversion. In AAAI 25: Workshop on Open-Source AI for Mainstream Use, 2025.
Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Yaofeng Sun, et al. Deepseek-vl: towards real-world vision-language understanding. arXiv preprint 2rXio:2403.05525, 2024a.
Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. In S. Koyejo, S. Mo- hamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neu- ral Information Processing Systems, volume 35, pp. 2507-2521, Curran Associates, wnwpprrrreeiiinn..uuir.......... neurips. cc/paper Tiles//aper/2022//ille pdff.
Inc., 2022. URL Cona
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathe- matical reasoning of foundation models in visual contexts. In International Conference Learning Representations (ICLR), 2024b.
Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video- chatgpt: Towards detailed video understanding via large vision and language models. arxiv preprint ar arXiv:2306.0542 2023.
Ahmed Masry, Do Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. ChartQA: A benchmark for question answering about charts with visual and logical reasoning. In Findings of the Association for Computational Linguistics: ACL 2022, pp. 2263-2279, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653 2..18553/v/2022 findings-acl.177 URL https: http::////aaaatoooooooooroo. orgp://22.... findings- aaciinnsssal...... 177.
Minesh Mathew, Dimosthenis Karatzas, and C. V. Jawahar. Docvqa: A dataset for vqa on document images. In 2021 IEEE Winter Conference Applications of Computer Vision doi: 10.1109 (11109/WAAVA /t////WCC36600000 2021
(WACV), pp. 2199-22055 2021.
Brandon McKinzie, Zhe Gan, Jean-Philipppe Fauconnier Biard, Sam Dodge, Philipp Dufter, Bowen Zhang, Dhruti Shah, Xianzhi Du, Futang Peng, Haotian Zhang, Floris Weers, Anton Belyi, Karanjeet Singh, Doug Kang, Ankur Jain, Hongyu He, Max Schwarzer, Tom Gunter, Xiang Kong, Aonan Zhang, Jianyu Wang, Chong Wang, Nan Du, Tao Lei, Sam Wiseman, Mark Lee, Zirui Wang, Ruoming Pang, Peter Grasch, Alexander Toshev, and Yinfei Yang. Mml: Methods, analysis insights from multimodal I1m pre- pre-training, 2024. URL https://arxiv... org/ abs/2403. 24011096111...... 09611.
Ahmed Nassar, Andres Marafioti, Matteo Omenetti, Maksym Lysak, Nikolaos Livathinos, Christoph Auer, Lucas Morin, Rafael Teixeira de Lima, Yusik Kim, A. Said Gurbuz, Michele Dolfi, Miquel Farré, and Peter W. J. Staar. Smoldocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion, 2025. URL 11576.
https://arxiv.org/bbb//////
OpenAl, :, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, Ananya Kumar, Andre Saraiva, Andrea Vallone, Andrew
15

---

Duberstein, Andrew Kondrich, Andrey Mishchenko, Andy Applebaum, Angela Jiang, Ashvin Nair, Barret Zoph, Behrooz Ghorbani, Ben Rossen, Benjamin Sokolowsky, Boaz Barak, Bob McGrew, Borys Minaiev, Botao Hao, Bowen Baker, Brandon Houghton, Bran- don McKinzie, Brydon Eastman, Camillo Lugaresi, Cary Bassin, Cary Hudson, Chak Ming Li, Charles de Bourcy, Chelsea Voss, Chen Shen, Chong Zhang, Chris Koch, Chris Orsinger, Christopher Hesse, Claudia Fischer, Clive Chan, Dan Roberts, Daniel Kappler, Daniel Levy, Daniel Selsam, David Dohan, David Farhi, David Mely, David Robinson, Dimitris Tsipras, Doug Li, Dragos Oprica, Eben Freeman, Eddie Zhang, Edmund Wong, Eliz- abeth Proehl, Enoch Cheung, Eric Mitchell, Eric Wallace, Erik Ritter, Evan Mays, Fan Wang, Felipe Petroski Such, Filippo Raso, Florencia Leoni, Foivos Tsimpourlas, Fran- cis Song, Fred von Lohmann, Freddie Sulit, Geoff Salmon, Giambattista Parascandolo, Gildas Chabot, Grace Zhao, Greg Brockman, Guillaume Leclerc, Hadi Salman, Haiming Bao, Hao Sheng, Hart Andrin, Hessam Bagherinezhad, Hongyu Ren, Hunter Lightman, Hyung Won Chung, Ian Kivlichan, lan O'Connell, Ian Osband, Ignasi Clavera Gilaberte, Ilge Akkaya, Ilya Kostrikov, Ilya Sutskever, Irina Kofman, Jakub Pachocki, James Lennon, Jason Wei, Jean Harb, Jerry Twore, Jiacheng Feng, Jiahui Yu, Jiayi Weng, Jie Tang, Jieqi Yu, Joaquin Quiftonero Candela, Joe Palermo, Joel Pariih, Johannes Heidecke, John Hallman, John Rizzo, Jonathan Gordon, Jonathan Uesato, Jonathan Ward, Joost Huizinga, Julie Wang, Kai Chen, Kai Xiao, Karan Singhal, Karina Nguyen, Karl Cobbe, Katy Shi, Kayla Wood, Kendra Rimbach, Keren Gu-Lemberg, Kevin Liu, Kevin Lu, Kevin Stone, Kevin Yu, Lama Ahmad, Lauren Yang, Leo Liu, Leon Maksin, Leyton Ho, Liam Fedus, Lilian Weng, Linden Li, Lindsay McCallum, Lindsey Held, Lorenz Kuhn, Lukas Kondraciuk, Lukasz Kaiser, Luke Metz, Madelaine Boyd, Maja Trebacz, Manas Joglekar, Mark Chen, Marko Tintor, Mason Meyer, Matt Jones, Matt Kaufer, Max Schwarzer, Meghan Shah, Mehmet Yat- baz, Melody Y. Guan, Mengyuan Xu, Mengyuan Yan, Mia Glaese Mianna Chen, Michael Lampe, Michael Malek, Michele Wang, Michelle Fradin, Mike McClay, Mikhail Pavlov, Miles Wang, Mingxuan Wang, Mira Murati, Mo Bavarian, Mostafa Rohaninejad, Nat McAleese, Neil Chowdhury, Neil Chowdhury, Nick Ryder, Nikolas Tezak, Noam Brown, Ofir Nachum, Oleg Boiko, Oleg Murk, Olivia Watkins, Patrick Chao, Paul Ashbourn, Pavel Izmailov, Peter Zhokhov, Rachel Dias, Rahul Arora, Randall Lin, Rapha Gontijo Lopes, Raz Gaon, Reah Miyara, Reimar Leike, Renny Hwang, Rhythm Garg, Robin Brown, Roshan James, Rui Shu, Ryan Cheu, Ryan Greene, Saachi Jain, Sam Altman, Sam Toizer, Sam Toyer, Samuel Miserendino, Sandhini Agarwal, Santiago Hernandez, Sasha Baker, Scott McKinney, Scottie Yan, Shengjia Zhao, Shengli Hu, Shibani Santurkar, Shraman Ray Chaudhuri, Shuyuan Zhang, Siyuan Fu, Spencer Papay, Steph Lin, Suchir Balaji, Suvansh Sanjeev, Szymon Sidor, Tal Broda, Aidan Clark, Tao Wang, Taylor Gordon, Ted Sanders, Tejal Patwardhan, Thibault Sottiaux, Thomas Degry, Thomas Dimson, Tianhao Zheng, Timur Garipov, Tom Stasi, Trapit Bansal, Trevor Creech, Troy Peterson, Tyna Eloundou, Valerie Qi, Vineet Kosaraju, Vinnie Monaco, Vitchyr Pong, Vlad Fomenko, Weiyi Zheng, Wenda Zhou, Wes McCabe, Wojciech Zaremba, Yann Dubois, Yinghai Lu, Yining Chen, Young Cha, Yu Bai, Yuchen He, Yuchen Zhang, Yunyun Wang, Zheng Shao, and Zhuohan www./rrg/////////e/////eeee//e// ab///bbs/////// 22/20.2241.1111 16720.
Li. Openai ol system card, 2024. URL https: .rg.
Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Hamza Alobei- dli, Alessandro Cappelli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. The RefinedWeb dataset for Falcon LLM: Outperforming curated corpora with web data only. In Advances in Neural Information Processing Systems, 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, 2021.
Weiming Ren, Huan Yang, Jie Min, Cong Wei, and Wenhu Chen. Vista: Enhancing long- duration and high-resolution video understanding by video spatiotemporal augmenta- tion, 2024. URL httts:://arxit www.:/o/g///////// 2.t.s.//212....... 00927..
Min Shi, Fuxiao Liu, Shihao Wang, Shijia Liao, Subhashree Radhakrishnan, De-An Huang, Hongxu Yin, Karan Sapra, Yaser Yacoob, Humphrey Shi, et al. Eagle: Exploring the design space for multimodal Ilms with mixture of encoders. arv preprint arxi::2408.1 Stfi::208.1598,, 2024.
16

---

Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P. Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. Real-time single image and video super-resolution using efficient sub-pixel convolutional neural network, 2016. URL https://arxiv. org/abs/160s 051558588900088....
Yan Shu, Zheng Liu, Peitian Zhang, Minghao Qin, Junjie Zhou, Zhengyang Liang, Tiejun Huang, and Bo Zhao. Video-xl: Extra-long vision language model for hour-scale video understanding. ar Xiv preprint ar arXiv:2409. 14485, 2024.
Amanpreet Singh, Vivek Natarjan, Meet Shah, Yu Jiang, Xinlei Chen, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8317-8326, 2019.
Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, lan Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Pe- ters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Pete Walsh, Luke Zettlemoyer, Noah Smith, Hannaneh Hajishirzi, Iz. Beltagy, Dirk Groeneveld, Jesse Dodge, and Kyle Lo. Dolma: an open corpus of three trillion tokens for language model pretraining research, 2024.
Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, Yan Lu, Jenq-Neng Hwang, and Gaoang Wang. Moviechat: From dense token to sparse memory for long video understanding, 2024. URL https: / /ttxxvaaavvvvv/a////v//////aaavl wttpr/wwww.ogga/////////// 16449.
Andreas Steiner, André Susano Pinto, Michael Tschannen, Daniel Keysers, Xiao Wang, Yonatan Bitton, Alexey Gritsenko, Matthias Minderer, Anthony Sherbondy, Shangbang Long, Siyang Qin, Reeve Ingle, Emanuele Bugliarello, Sahar Kazemzadeh, Thomas Mes- nard, Ibrahim Alabdulmohsin, Lucas Beyer, and Xiaohua Zhai. Paligemma 2: A family of versatile vlms for transfer, 2024. httpht/ss///rxit http://www.rg//////////// 03555.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baappiin Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: a family of highly capable multimodal models. ar preprint ar arXio:2312.1180: 2023.
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models practical size, 2024. URL https://arxio. org/atb/2248.0001111 001108 1(3), 2024.
Peter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula, Shusheng Yang, Jihan Yang, Manoj Middepogu, Ziteng Wang, al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. Advances in Neural Information Processing Systems, 37:87310-87356, 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models, 2023.
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint ar 1rXi::2409.12191, 2024.
Yi Wang, Xinhao Li, Ziang Yan, Yinan He, Jiashuo Yu, Xiangyu Zeng, Chenting Wang, Changlian Ma, Haian Huang, Jianfei Gao, et al. Internvide02. Empowering video milms with long and rich context modeling. arXiv preprint ar 2ioo:2501.12386 2025.
Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. Longvideobench: A benchmark for long-context interleaved lideo-language understanding. Advances Neural Information Processing Systemss, 37:28828 28887,, 2024a.
17

---

Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, Zhenda Xie, Yu Wu, Kai Hu, Jiawei Wang, Yaofeng Sun, Yukun Li, Yishi Piao, Kang Guan, Aixin Liu, Xin Xie, Yuxiang You, Kai Dong, Xingkai Yu, Haowei Zhang, Liang Zhao, Yisong Wang, and Chong Ruan. Deepseek-v12 Mfxture-of-experts vision-language models for advanced multimodal understanding, 10302.
2024b. URL httpps://arxiin http://///aaxii.oo//b/////////// org/
Jun Xu, Tao Mei, Ting Yao, and Yong Rui. Msr-vtt: A large video description dataset for bridging video and language. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5288-52999 5296, 2016.
Zhangchen Xu, Fengqing Jiang, Luyao Niu, Yuntian Deng, Radha Poovendran, Yejin Choi, and Bill Yuchen Lin. Magpie: Alignment data synthesis from scratch by prompting aligned Iims with nothing. arXiv preprint ar ar::r:2406 2024.
Dongji Yang, Suyuan Huang, Chengqiang Lu, Xiaodong Han, Haoxin Zhang, Yan Gao, Yao Hu, and Hai Zhao. Vript: A video is worth thousands of words, 2024. URL https: ftaxiv.org/// abs 020.. 060400.
Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang,Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, Qianyu Chen, Huarong Zhou, Zhensheng Zou, Haoye Zhang, Shengding Hu, Zhi Zheng, Jie Zhou, Jie Cai, Xu Han, Guoyang Zeng, Dahai Li, Zhiyuan Liu, and Maosong Sun. Minicpm-v: A gpt-4v level mllm on your phone, 2024. URL
https://arxs wtwg/bs/////e/eeeooooeeee 200b./2408.00 01800.
Jiabo Ye, Anwen Hu, Haiyang Xu, Qinghao Ye, Ming Yan, Guohai Xu, Chenliang Li, Junfeng Tian, Qi Qian, Ji Zhang, et al. Ureader: Universal ocr-free visually-situated language understanding with multimodal large language model. ariio preprint ar 2511:22310.0516,, 2023.
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of CVPR, 2024a.
Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mammoth: Building math generalist models through hybrid instruction tuning, 2023. URL https://arxio. vrglabs/2309.055
2024b.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11975-11986, 2023.
Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, et al. Videollama 3: Frontier multimodal foundation models for image and video understanding. arviv preprint ar 2rX1:22501.13106, 2025.
Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data, 2024. URL https:://arke org/ 0tbs/2441..0777
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging alm-as-a-uudge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36, 2024.
Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. Mlvu: A comprehensive benchmark for multi-task long video understanding. ar Xiv preprint ar Xiw::40266000 X7X::2246.00464, 2024.
Orr Zohar, Xiaohan Wang, Yonatan Bitton, Idan Szpektor, and Serena Yeung-Levy. Video- star: Self-training enables video instruction tuning with any supervision, 2024a. URL
https://arxiv https://anxiv.orgg/bb/4499 06189.
18

---

Our Zohar, Xiaohan Wang, Yann Dubois, Nikhil Mehta, Tong Xiao, Philippe Hansen-Estruch, Licheng Yu, Xiaofang Wang, Felix Juefei-Xu, Ning Zhang, et al. Apollo: An exploration of video understanding in large multimodal models. arviv preprint ar Xio:2412. 2024b.
19


Reviews:
[{"review_id": "rms3V0Nff9", "reviewer": "Reviewer_ofar", "summary": "This paper introduces SmolVLM, which confronts high memory and compute costs of vision‑language models by pairing compact SigLIP encoders with lightweight SmolLMs, pixel‑shuffle compression, image‑splitting, and extended 8–16k contexts. Three variants (256 M–2.2 B) beat baselines on 14 image and 5 video benchmarks using under 1 GB GPU memory for real‑time edge deployment scenarios.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The work squarely targets the growing gap between state‑of‑the‑art VLM accuracy and the memory/power budgets of edge devices, which is very important for the VLM community.\n\n2. The authors vary encoder/LM size, context length, visual‑token compression, and video sampling, turning the paper into a design‑space exploration rather than a one‑off model proposal. The nine “Findings” distilled from Figure 3 and Section 2 provide reusable insights (e.g., balanced encoder‑LM sizing, pixel‑shuffle ratio, minimal CoT) rather than ad‑hoc tweaks.\n\n3. Three SmolVLM variants (256 M‑2.2 B) beat or match larger open‑source baselines on nine image and five video benchmarks while using 0.8‑4.9 GB VRAM (batch 1), a >5× savings over MolmoE‑7B and InternVL‑2B.\n\n4. Comprehensive benchmark coverage OCR, charts, science QA, math, temporal reasoning, and movie QA； this breadth supports the generality claim.\n\n5. This paper releases weights, training data mixtures, and code, plus a phone demo app, which is commendable for reproducibility and impact.", "reasons_to_reject": "1.  VRAM figures are reported, but there are no wall‑clock throughput or energy measurements to substantiate “real‑time” claims, which are critical for edge deployment.\n\n2. The paper notes spatial‑detail loss for OCR at high shuffle ratios yet adopts r = 4 for small models without a thorough error analysis on localisation‑heavy tasks.\n\n3. Ablations missing for alternative compression. No comparison against patch‑merging, adaptive pooling, or transformer down‑sampling leaves open whether pixel shuffle is uniquely effective.", "questions_to_authors": "After reading the rebuttal and comments from other reviewers, I adjust the score to 7 due to the feedback on fairness. But still think this work will be a good experimental practical～", "extra_scores": {}}, {"review_id": "uHMM5mKTGC", "reviewer": "Reviewer_RaaY", "summary": "This paper introduces **SmolVLM**, a family of compact multimodal vision-language models optimized for efficient inference on resource-constrained devices. Through systematic exploration of architecture, tokenization, and training data, the authors show that smaller vision encoders, extended context windows, aggressive visual token compression, and structured positional tokenization significantly enhance performance. The smallest model, SmolVLM-256M, achieves better results than the much larger Idefics-80B model using less than 1GB GPU memory. The largest variant (2.2B parameters) rivals state-of-the-art multimodal models while requiring half the GPU memory. Additionally, SmolVLM demonstrates robust performance on both image and video benchmarks, underscoring its practical applicability. The study emphasizes that careful architectural design and efficient training strategies are crucial for developing small-scale yet powerful multimodal models. All resources, including model weights, training data, and code, are publicly released to promote reproducibility and further innovation.", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. **Innovative and Efficient Model Design**:\n   The paper introduces SmolVLM, a compact yet powerful multimodal architecture that outperforms significantly larger models using dramatically less GPU memory, highlighting important advancements in efficient multimodal modeling.\n\n2. **Comprehensive Experimental Validation**:\n   Extensive experiments systematically explore architectural choices, tokenization methods, and data strategies, providing practical guidelines and insights valuable for researchers and practitioners interested in deploying vision-language models on resource-constrained edge devices.\n\n3. **Strong Open-Source Contribution**:\n   All model weights, training datasets, and source code are publicly available, fostering reproducibility and supporting future research. This openness significantly benefits the research community by providing ready-to-use resources for efficient multimodal model development.", "reasons_to_reject": "1. **Questionable comparison target in the abstract**  \n   The abstract highlights a performance comparison against Idefics-80B, which feels out of place because many recent *small-scale* multimodal models would be more relevant baselines.\n\n2. **Primarily engineering-oriented, with limited novelty**  \n   Most techniques and findings—such as extending context length by adjusting the RoPE base and compressing visual tokens via pixel shuffle—have already been explored in prior work, making the overall contribution incremental rather than groundbreaking.\n\n3. **Incomplete and potentially biased evaluation**  \n   The experimental section omits direct, side-by-side comparisons with several SOTA *small* VLMs (e.g., MiniCPM-V, NVILA). In addition, the study focuses on leaderboard averages without reporting real-world latency, energy, or on-device throughput—metrics that are critical for a paper whose primary claim is efficiency. This limited evaluation makes it hard to judge whether SmolVLM truly advances the state of the art beyond existing compact models.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "H3zrBaMBaL", "reviewer": "Reviewer_jgcB", "summary": "The paper describes a series of compact multimodal models specifically engineered for resource-efficient inference. First the authors identify key design criteria to be optimized for low computational overhead—architectural configurations, efficient tokenization strategies, and training data curation. Then they validate the design choices by showing how the resulting vision-language models (VLM) show substantial performance gains on both image and video tasks with minimal memory footprints, making them practically viable for multimodal tasks.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Well-organized, well-structured and clearly written paper.\n2. The paper lays out a set of design choices to reduce the size of multimodal models. Experiments validate these design choices in multimodal settings.\n3. The datasets on which the models are validated are diverse (single-image, multitask, and video) and challenging, lending credibility to the experiments and the significance of the contribution.\n4. The authors make all model weights, datasets, and code, publicly available. Additionally, they include a mobile application to demonstrate inference on a smartphone with the reduced and compact multimodal model.", "reasons_to_reject": "1. Evaluation metric insufficiency: The authors evaluate the resulting efficiency of the reduced model size in terms of the amount of RAM required to run the evaluations. Although the architecture is an important determinant of the computational cost, it appears that they are missing out on other important aspects of efficiency to convincingly argue about the improved efficiency. Why did the authors not consider aspects related to latency, throughput, and energy consumption? Without this, it is difficult to assess the true benefits of the proposed model reduction for on-device applications. Edit: the author response includes throughput evaluations.\n\n2. A comparison with other on-device multimodal models is also missing, which fails to give a complete picture of the benefits of the proposed reduced model.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gj5eQGT1TB", "reviewer": "Reviewer_BnL7", "summary": "The paper presents SmolVLM, a series of carefully engineered small-scale multimodal models. It provides a detailed comparison of different model architectures, tokenization strategies, and data curation methods, analyzing their impact on overall performance. A set of model weights is released, achieving strong performance gains with significantly lower memory consumption.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper is well-structured and presents its findings clearly.\n2. The experimental results are solid and provide valuable engineering insights for the community.", "reasons_to_reject": "1. In Section 2.1, the paper compares the impact of vision encoders with different parameter scales on model performance. However, it is unclear whether the input image resolutions are kept consistent across these encoders. How does the paper isolate the effect of encoder size from that of image resolution?\n2. In Figure 3 (left), the authors state that smaller language models (LMs) benefit less from larger vision encoders. However, since each curve represents a fixed encoder, using curves corresponding to fixed LMs instead might improve interpretability.\n3. In Figure 3 (right), the legend is missing—it's unclear what the different colors represent.\n4. Lines 98–99 claim that smaller VLMs benefit from more aggressive compression, but Figure 3 (middle-right) seems to show the opposite: the 256M model's performance decreases while the 500M model's performance improves.\n5. Line 186 states that training beyond 3.5 minutes yields minimal further gains, yet Figure 7 (right) does not show any data points beyond the 3.5-minute mark.\n6. In addition to the composition of the training data, more details about the training hyperparameters would be helpful—for example, which modules were frozen or trained during different training stages.", "questions_to_authors": "1. In Section 3.1, the term positional tokens is mentioned. Could the authors clarify what this refers to, and how it relates to position embeddings? An example would be helpful to illustrate the distinction.", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces SmolVLM, a suite of compact vision-language models (256M–2.2B) optimized for edge devices under 1GB VRAM. The authors conduct extensive ablations and demonstrate strong performance across a wide range of image and video benchmarks. The work is well-motivated, reproducible (open-sourcing code, models, and a mobile app), and offers practical design insights.

Initial concerns about efficiency metrics and fairness of comparisons were addressed thoroughly in the rebuttal, including added throughput, latency, and broader baseline comparisons. While some techniques are incremental as pointed by the reviewers, the paper provides valuable engineering contributions with strong empirical backing. The Area Chairs believe this work will foster the development of cost-efficient and open-sourced VLM, and thus recommend an acceptance.
Decision: Accept