Recursive Inference Scaling: A Winning Path to Scalable Inference in Language and Multimodal Systems
Authors: Ibrahim Alabdulmohsin
Xiaohua Zhai
Venue: NeurIPS 2025 poster ()
Keywords: Recursive inference
scaling inference time
self-similarity
language models
SigLIP
multimodal systems.
Primary area: deep_learning
TL;DR: We introduce Recursive Inference Scaling (RIS), a plug-in technique that exploits language's fractal structure to boost model performance, unlocking a new dimension of inference scaling.

Abstract: Inspired by recent findings on the fractal geometry of language, we introduce Recursive INference Scaling (RINS) as a complementary, plug-in recipe for scaling inference time in language and multimodal systems. RINS is a particular form of recursive depth that significantly outperforms +55 other variants, including the recent "repeat-all-over" (RAO) strategy in Mobile LLM (Liu et al., 2024) and latent recurrent thinking (Geiping et al., 2025). Unlike prior works, we carry out our comparisons on a compute-matched regime, and demonstrate that for a  fixed model size and training compute budget, RINS substantially improves language modeling performance. It also generalizes beyond pure language tasks, delivering gains in multimodal systems, including a +2% improvement in 0-shot ImageNet accuracy for SigLIP-B/16. Additionally, by deriving data scaling laws, we show that RINS improves both the asymptotic performance limits and the scaling exponents. More importantly, with light-weight (linear) adapters (comprising <1% of model parameters) and stochastic dropout, RINS offers a no-regret strategy, meaning that RINS-enabled pretraining improves performance in language modeling even when recursive depth is not applied at inference time. This corresponds to improving performance on a training compute-, parameter-, and inference-matched regime, suggesting its potential as a viable component of LLM pretraining!

Recursive Inference Scaling: A Winning Path to Scalable Inference in Language and Multimodal Systems
Ibrahim Alabdulmohsin
Xiaohua Zhai Google Deepmind Zürich, Switzerland xzhai Ggoogle. com
Google Deepmind Zürich, Switzerland i ibomohsin@google. com
Abstract
Inspired by recent findings on the fractal geometry of language, we introduce Recursive INference Scaling (RINS) as complementary, plug-in recipe for scaling inference time in language and multimodal systems. RINS is a particular form of recursive depth that significantly outperforms +55 other variants, including the recent "repeat-all-over" (RAO) strategy in Mobile LLM (Liu et al.) (024) and latent recurrent thinking (Geiping et al. 2025). Unlike prior works, we carry out our comparisons on compule-matchee regime, and demonstrate that for fixed model size and training compute budget, RINS substantially improves language modeling performance. It also generalizes beyond pure language tasks, delivering gains in multimodal systems, including a +2% improvement in O-shot ImageNet accuracy for BigLIP-B/1... Additionally, by deriving data scaling laws, show that RINS improves both the asymptotic performance limits and the scaling exponents. More importantly, with light- (linear) adapters (comprising < 1% of model parameters) and stochastic dropout, RINS offers a no-regret strategy. meaning that RINS-enabled pretraining improves performance in language modeling even when recursive depth is not applied at inference time. This corresponds to improving performance on training compute-, parameter-, and inference-matched regime, suggesting its potential as a viable component of LLM pretraining!
Introduction
There has been a proliferation of research in recent years pointing to the pivotal role of scale, and how its benefits could be predicted empirically (Hestness et al. 2017 Kaplan et al. 2020 Alabdulmohsin al. 2022] Bansal et al. 2022; Zhai et al.) (002)) Generally, the performance deep neural networks f(z) (such as its eтror rate or (og-perplexity) often follows power law f(x) ~ = + as one varies a dimension æ, such as the data size or model parameters. These "scaling laws," as they are known today, have been used, among others, to determine the training data size needed for specified level of accuracy (Cho et al., 2015 Beleites al. [2033] Figueroa et al.) (012)) and to optimize the model architecture (Kaplan et al. 2020 Hoffmann et al. 2022 Alabdulmohsin et al. (2224b), with some theoretical justification (Bahri et al. 2021] Hutter 2021 Sharma and Kaplan 2022).
Besides this conventional approach of scaling training compute, the impact of increased inference compute on model performance has emerged as another key scaling dimension. For example, chain- of-thought (CoT) prompting show that eliciting longer inference paths through additional token generation could improve the reasoning capabilities of LLMs (Wei et al., 2024). similar to the success of critiquing before evaluating (Ankner et al.] (024).. Also, AlphaCode (Li et al. (022) and Codex (Chen et al. (021) generate multiple samples during inference to enhance code generation. Remarkably, Brown et al. (024) shows that the benefit of sampling multiple solutions in tasks such
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Block
Block
Band
Iepat
Ourpet
Iales
Irpes
Jales)
x- fales)
Brrges
RI5
Iepat
Orge
to falte s)
(M448)
Figure 1: LEFT: In RINS, the model f X' + 3 is split into two parts: the first block SA X + X applied iteratively to its own output times before passing the output to the second block. RIGHT: Illustrative examples of models with different signatures and degrees. From top to bottom: (I) Baseline (Signature: AB, Degree: I). a feedforward architecture with no recursion. (2) repeat-all-over (RA) (Liu et al. 2024), where the entire model is recursively applied on its output. When recursion is done twice, it has a signature of ABAB. (3) RINS with signature A (4) (A² whose degree is 2, in which the same parameter sharing signature is applied each of the two blocks A and B.
mathematics and coding- when measured by coverage- holds for up four orders of magnitude of inference compute. Thus, inference compute follows systematic scaling patterns that can be leveraged to improve models. Refer to the survey by Welleck et (2024) for more details.
Recently, it has also been noted that language exhibits a "self-similar" (fractal) nature, meaning that similar patterns repeat across different scales of its representation, from individual words to entire sentences and paragraphs (Alabdulmohsin et al., 2024a). Inspired by this finding, we examine if recursive depth, which can be interpreted as a form of scale-invariant decoding, offers complementary approach for scaling inference time in language models. To this end, we examine an extensive set of parameter-sharing strategies and, indeed, identify the best to be a special form of recursion, which we term Recursive INference Scaling (RINS). We show that RINS yields significant performance gains over +55 other methods when controlling for model size and training compute.
RINS builds upon the concept of model recursion but recasts it as a powerful inference- time scaling strategy. It leverages a simple yet profound idea: use your existing architecture and training compute budget as is, but exploit the self-similar structure of language by recursively applying an early portion your network to refine its output. In turn, this simple strategy improves performance significantly.
Recursion has shown promise in language modeling, with recent work by Liu et al. (2024) and Geiping et al. (2025) demonstrating that recursive architectures outperform similarly sized vanilla models trained on similar number of tokens. However, while such works demonstrate the sample efficiency of recursive architectures, in which models are compared when trained on a similar number of tokens, their analysis does not explicitly account for the increased computational cost of recursive operations during training. Hence, it remains unclear whether the performance gains observed in prior work come from the inherent advantages of model recursion or simply from having increased the training compute. Indeed, our findings suggest that for moderately sized models (over 1 billion parameters). the performance gains of "repeat-all-owe" can be matched by training the bascline model longer to consume an equivalent compute. RINS, by contrast, significantly outperforms all other baselines on a compute- and parameter- marameder-matched matched setup, including when scaling inference by increasing the context length (see Figure 2p.
(RAO) in MobileLLM (Liu et al. 2024)
Crucially, a stochastic variant of RINS not only can enhance performance further, such as in multi- modal systems, but also provides the flexibility to optionally forgo increased inference computation at test time with minimal performance degradation. We show that combining stochastic RINS with lightweight (linear adapters), comprising < 1% of parameters, offers no-regret strategy, meaning that LLNS-enabled pretraining improves performance in langauge modeling even when recursive depth is not applied at inference time! See Section 4 for details.
We conduct our experiments mostly on compact models, which are typically intended for deployment environments with stringent memory limitations (Liu et al. (024).. Given the direct relation between a model's memory footprint and its parameter count (e.g. billion parameter model with 16-bi floating-poit precision requires 2GB of DRAM), the ability to enhance accuracy while maintaining a fixed parameter count is highly desirable. RINS achieves this by unlocking significant gains without increasing parameter count for the same training compute. In Section 6 we study the effect of sharing the KV cache during recursion to reduce memory footprint even further. While KV cache sharing diminishes some of the gain, RINS with KV cache sharing still enjoys an advantage.

---

Statement of Contribution. In summary, we introduce Recursive INference Scaling (RINS), a complementary plug-in method for scaling inference time. We:
1. propose a taxonomy of trrameter-sharingg architectures, empirically evaluating their effec- tiveness. Our comprehensive analysis identifies RINS as a powerful approach, outperforming +55 other methods like RAO used in Mobile L.M (Liu et al. (024) and latent recurrent thinking (Geiping et al., 2025), and scaling inference by increasing the sequence length.
2. unlike prior works, we control for training compute FLOPPs in our comparisons.
3. unlike prior works, we study the effectiveness of recursive depth beyond language to multimodal systems that incorporate language in their processing, such as contrastive models. In particular, our SILLI--RINS-B/I outperforms the popular SigLIP-B/II (Zhai et al. 2023) by a wide margin; c.g. improving O-shot accuracy in ImageNet from 77.3% to 79.6% and CIFAR 100 from 70.3% to 80.7%.
4. argue that the performance gain of RINS likely stems from the self-similar (fractal) nature of language, by showing that a similar analysis in vision yields minimal improvements.
5. derive data scaling laws for RINS, revealing improvements in both the asymptotic perfor- mance limit and convergence speed (i.e. scaling exponent).
6. show that stochastic RINS can enhance performance even further, such as multimodal sys- tems, while offering the option to revert to non-recursive inference at test time with minimal performance degradation. In particular, with lightweight (linear) adapters, stochastic RINS offer no-regret strategy, suggesting its potential as viable component of LLM pretraining.
7. analyze the impact of KV cache sharing to reduce memory footprint even further and show that RINS continues to offer an advantage in that setup.
2 Recursive Inference Scaling
Overview. Before describing how RINS works, we formalize definitions. Let X' be a fixed domain, often the space of sequences of soft tokens of embedding dimension d. Let L = {41, l2.... 4n) be a fixed set of n unique blocks, where each block 4 X' + X is a function mapping from the input space X' to the same output space X. By "unique" here we simply imply that such blocks (which typically comprise of multiple layers each) are not constrained to share the same parameters. Let ((L) be the space of all possible computation graphs representing neural network architectures that can be constructed by composing blocks from the set LL while f E G(L) be one specific architecture.
Figure 1 (right) illustrates some examples for the case when [T.| 2. For instance, one can repeatedly apply the entire model, as in the "repeat-all-over" (RAO) approach in Mobile LLM (Liu et al. (2024). or recursively apply a strict subset of the architecture, such as a single a block within the model. The choice of arrangement of blocks can significantly impact the model's performance and efficiency.
Formally, let C(J) be the actual computational cost (in FLOPs) of training f G(L) on a dataset sampled i.i.d. from distribution DD considering only the forward pass. Also, L(J) is performance metric of interest (e.g.. validation loss) for model f. with lower values being better.
Definition 2.1. For a fixed set of blocks L and a training compute budget recursive architecture f* E (L) is called "better" than another f E G(L) if C(ft) S C(f) & and E[C(f*) s B[C(f)]
In other words, search for the architecture f*, constructed only from the set of blocks IL, that minimizes the loss under the constraint of a bounded training compute c.
Model recursion offers simple, plug-in in approach for scaling inference time. By applying some layers iteratively to their own output, we effectively increase the computational path length during inference without altering the underlying model architecture. This allows us to exploit the benefits of increased inference compute. Importantly, it is complementary to other techniques like chain-of-thought (CoT)
prompting (Wei et al. 2024) and repeated sampling (Li et al.. 2022 Chen et al. 2021
Taxonomy. As discussed in Section] I. language has been shown to exhibit self-similarrit, meaning that similar patterns repeat across multiple levels of granularity, from the structure of individual sentences to the composition of entire texts (Alabdulmohsin al., (024a). This observation suggests decoding could be particularly beneficial for language processing.
that recursive
3

---

Mosal au 2006
Macil Sus 620
Macil San 2006
Macel Gust SXM
Rassira
Rasorra MA
Rase
EAD
Lorpdes
torpdes
torpdes
Londdes
27
Taiing FLORS Separd
Taating FLORS Sapard
Takting FLOR Dayarr
Taaling TuDNe Dayortl
Language models are trained on 200B tokens. The z-axis is the training cost in units of layer x step. Notably, the performance advantage of RINS increases with longer training. The long-sequence baseline, using a context length of 1,536 tokens, exhibits lower performance due to processing fewer examples to maintain the same FLOPs count. See Figure Ba for longer training durations and Figure 4 for larger (IB parameter) models, further demonstrating the value of RINS. Sharp drops in perplexity near the end of training are due to learning rate cooldown.
Figure 2:
Model Sizee (4 params)
Recursive Signature
exaamples / core/ sec (1)
Peak Memory Usage (GiB) (J)
Baseline (AB)
22.7
300M
RINS (A²B)
18.1
3.31
RINS BB
13.9
3.56
RAO (ABAB)
12.6
4.35
Baseline (AB)
12.0
3.92
600M
RINS (B²B)
RINS (B²
5,70
6.31
RAD (ABAB)
6.5
7.48
Table I: A summary of latency and memory footprint of recursive architectures. We observe that RINS with A ABBiignature has significantly better latency and memory footprint than RAO and still outperforms it significantly in terms of quality.
To determine the extent which this is true, we systematically explore a vast space of parameter sharing strategies. We use the following taxonomy based on two key concepts: (1) signature and (2) degree. The "signature" of an architecture deseribes how parameter sharing is applied across different blocks of the model. For example, an architecture with the signature A indicates that the model is partitioned into three unique blocks (A, B, and C) of equal size. The processing flow would be: apply block A the input, apply B to A's output, apply B again to its own output (hence the exponent), and apply C to the output of B. Note that while the signature AB²C has the same parameter count as ABC, incurs R 33% more compute due to the repeated use of block B. This highlights the trade-off between computational cost and performance using recursion that we will carefully study in this work.
On the other hand. "degree" specifies if recursive depth is nested, as illustrated in Figure (bottom). degree of 2, for example, means that each of the blocks A, B, and in our example have themselves the same recursive pattern used in the overall model. This adds another dimension to our taxonomy, allowing for finer- -raiing classification of recursive architectures. Degree makes notation concise, but is not necessary; e.g. (ABB), is equivalent to ABB CDD CDD. By systematically varying both signature and degree, we can comprehensively explore the space of recursive models and identify optimal configurations. Appendix A provides detailed pseudocode. With this framework, we now state our main question: Which family of architectures (i.e. signatures and degrees) lead to better performance according to Definition 2.1 under fixed compute budget?
To reiterate, this a non-trivial question because it possible for non-recursive model to outperform all others given that it sees more training examples, since we always match from FLOPs. For instance, increasing the context length can inferior to longer training within compute constraint, as shown in Section 3 Nevertheless, our analysis reveals that Recursive INference Scaling (RINS) emerges as clear winner. Its consistent superiority suggests that captures a fundamental principle for efficiently processing language. We hypothesize that this is due to the self-similar geometry of language.
Definition 2.2. RINS corresponds to architectures with degree and signature A BB, for some 1.
In other words, RINS partitions a model depthwise into two equally-sized blocks A and B. The first block A recursively applied on its own output r times before applying B. See Figure [ for illustration and Appendix for detailed pseudocode. In Section 6 we study the optimal value of r.

---

BL
RAO
RINS
BL
RAD
RINS
OpenBookQA
summary of latency 37.1 1: 0.8
40.0 0.6
39.0 1:
BaoQQ
53.1 1: 4.7
58.3 0.6
59.5 1: 0.8
PIQA
65.8 = 3.4
68.8 0.5
69.8 0.3
SIQA
40.2 0.5
40.0
0.5
40.9
HellaSwag
16.0 =
0.3
0.5
CommenSenseQA
29.0 1.6
313
32.4=10
Table 2: O-shot evaluation in downstream common sense reasoning tasks. All models 600M
parameters, pretrained on the compute-equivalent of 500B tokens in the baseline (BL). The best
signatures for RAO and RINS in Figure 3a are used in this evaluation. See Appendix C for details.
Main Claim. Our claim can be summarized as follows. Once language model is chosen and the training compute budget is planned, one should enable RINS during training, which does not change the model size, and train the new RINS-enables model for the same amount of compute. Our empirical results demonstrate that RINS-enabled models will consistently outperform baseline models. In addition, stochastie RINS, particularly with linear adapters (see Section + offers the option of forgoing scaling inference at test time with little performance degradation.
Experimental Results
In this section, we study the impact of various parameter-sharing strategies in language modeling, following the taxonomy introduced in Section We show how RINS emerges clear winner. All experiments are carried out using the Big Vision codebase (Beyer et al. 2022).
Setup. First, train decoder-only transformer language model (Vaswani al., 2017) with
relative position embeddings and sequence packing. We use C4/English tokenizer with a vocabulary size of 32K. The model is trained on mixture of C4 (Raffel al. (020)) and SlimPajama (Soboleval iet al. 2023) with equal weight using a batch size 1,024 and context length 1,024. The non-recursive baseline is trained for 200K steps, which amounts about 200B training tokens. Other recursive models are trained on fewer tokens in order to match the same total compute FLOPs.
The optimizer is Adam (Kingma and Ba 2014) with learning rate 5 x 10-4 and weight decay x 10-5, using an inverse square root decay schedule with 5K warm-up and 5K cool-down steps. This learning rate was chosen by sweeping across the values Ir (10-3, 5 10-1 10-1, x 10-9) with wd = 1r/10. The reported value yielded the best performance for the non-recursive baseline. The full training configuration along with list of all the architectures we sweep across (i.e. signature and degree) can be found in Appendix [ Each model trained on 16 x 16 TPUv5 chips for approximately 2.2K core-hours. Models that failed OOM were excluded. Overall, we train 59 models of two sizes: 300M and 600M parameters, which include RAO (Liu et 2024) and latent recurrent thinking (Geiping al. 2025) whose signature ABT C. In all models, the embedding dimension is 1, 1,536 and the MLP dimension is 6, 144. In Figure 3a parameters.
we present results using models with billion
Results. Figure 2 shows that RINS outperforms all other approaches. Importantly, the optimal number of recursion rounds in RINS (e.g. whether to use A²B or AB more recursions) depends on the allocated training compute budget. This is seen in Figure 2 in the fact that the nen-recursive baseline AB (a special case of RINS with r = 1) initially outperforms all other models, before its performance saturates and recursive models begin to outperform it for the same training compute FLOPss and parameter count. We study the relation between the optimal number of recursion rounds and compute later in Section 6 In addition, RINS outperforms scaling inference time by only increasing the sequence (context) length (green vs. red curves in Figure and memory footprint is provided in Table
Longer Training Duration. One observation in Figure is that the performance gap seems to widen in favor of RINS as more training compute is allocated. To verify this, we resume training the 60rM-parrmeter models on 500B training tokens. Here, we only compare RINS with non-recursive baseline and RAO, since RAO was identified in prior works to outperform other parameter-sharing strategies (Liu et al.] 2024). Figure 33a shows that the performance gap continues to increase in favor of RINS, This improvement also persists downstream, as shown in Table 2
5

---

Infirite Compute Limit
Inforite Compute Limit
2.4
KG0
2.4
2.7
1.3x
13
2.5
241
6.0.
10:00
2.34
3.30
10:04
(a) Performance of RINS BA ABB, BB vs. RAO (A",
(b) Asymptotic performance of 1 IB-parameter LMs. evaluated in C4 (left) and SlimPajama (right) using stochastic RINS. Dotted line is for baseline. We ob- serve a tradeoff between inference flexibility (gap be- tween Ix & 2x) and potential gain of inference scaling (2x result). We resolve this tradcoff in Section
A", A*) and baseline plotted against increasing compute
budget on 600M-Paaa 600M-parameter parameter models. The performance advantage of RINS grows with the computational bud- get. The long sequence baseline is not shown since underperforms other methods Figure
0.0
0.25
0.50
2.7
2.7
2.7
SimPajams Log Log-Pr.
SimPajana Log-PPL
SimPajama Log-PPL
Baseline
RINS@11x
RINS@1.5
2.5
2.5
2.5
RINS@2X
2.3
0.09
2.08
4.08
8.08 10.08
2.00
4.08 6.08 8.08 10.08
0.08
2.08
4.08 6.08 8.08 10.08
Training (layers)
Trairing (layers)
Training FLOPs (layers)
Figure 4: Performance stochastic RINS (A with varying inference costs for IB parameter LMs. The x-axis represents the training compute cost. The legend indicates the inference cost of each stochastic RINS configuration relative to the baseline; e.g. 1.5x denotes 50% increase in inference cost. For ps = 0, RINS @ 1x is significantly worse, with perplexity scores > 3. As expected, RINS converges performance the baseline as P. + Similar results using C4 are Appendix F
4 Stochastic Recursive Inference Scaling
Next, we investigate the effect of stochastically dropping blocks during training, inspired by the regularization technique of stochastic depth (Huang et al. 2016). Our primary goal is to determine whether this approach can further enhance the performance of RINS while simultaneously offering the flexibility reverting to non-recursive inference without sigmificant degradation in model quality.
To recall, RINS has the signature A A'B for some r > 1. To implement stochastic RINS, we introduce skip probability P. [0, 1) and sample during training the number of recursion rounds in each step to be 1 + 1/. where 77 is a binomial random variable with probability of success trials
Ps and number of Thus, block is always executed at least once. During inference, we are free to choose how to scale compute by setting the value of 2 1. See the detailed pseudocode in Appendix [A] For this, we train bigger models with billion parameters. We use an embedding dimension 2,048 and MLP dimension 8, 8,192. All models have 18 decoder blocks. We train for 500K steps and compare RINS with signature A against the non-recursive baseline.
Figure Aecure4smmmrtti Summarizes the advantages of stochastic RINS. Notably, we observe that as p. > 0 increases, stochastic RINS mitigates the performance degradation incurred when scaling not applied at inference time, while still offering big gains when inference time is scaled. Not surprisingly, though, scaling inference time is less effective when p, increases, suggesting tradeoff between flexibility at inference time and the magnitude of potential gains from scaling. As shown in Figure [3] similar conclusions hold in the asymptotic (infinite-come compute) regime, assuming the loss follows power law relation (Kaplan et al. 2020). We resolve this apparent tradeoff in Section 6 using linear adapters.
5 Multimodal Systems
Setup. Besides language, we study the impact of RINS in vision-language pretraining, motivated by the fact that such models also process natural language. pretrain SigL.I--B/16 models, which are contrastive models trained using the sigmoid loss on English- languugg image- text pairs (Zhai et al.)

---

Datuset
AB
Long-Seg
ABAB
AAB
ImageNet
73.4
73.0
72.7
74.1
0-shot clansification
CIFAR100 Pet
68.9
63.6
65.3
72.2
90.4
COCO img21xt0
62.7
62.0
61.1
62.3
COCO img2
84.8
84.1
82.9
84.1
0000 2mg2
90.7
90.4
89.5
90.2
0000 (x
44.6
44.2
43.2
45.1
COCO 0x22mmg@5 @
69.6
69.4
68.4
70.0
Retrieval
COCO (5
78.8
78.7
77.5
79.0
Flikkr img2
Flickr img21xx085
98.0
97.8
97.5
98.5
Flickr img2txr@
99.1
98.9
Flickr Ext2mm@@1
75.1
73.7
73.2
74,3
Flickr txt2ing@5
92.3
92.1
Flickr 1x22mgg 10
95.6
95.6
95.6
95.8
Table 3: Performance SigLIP-B/16 on ImageNet (Deng et al., 2009). CIFAR100 (Krizhevsky
2009), Pet (Parkhi et al., 2012). COCO (Chen et al. 2015). and Flickr (Young et al. 2014). All models are identical in size to SigLIP-B/16 and have the same training compute FLOPs. Long-Seq is SigLIP-B/16 trained higher resolution of 280 and text length 80 (25% increase in sequence length - 50% increase in inference cost, similar to AAB). Wilcoxon signed rank test (Wilcoxon, 1992), gives
p 0.003 so the evidence in favor of RINS statistically significant at the 99% confidence level.
We follow Zhai et al. (2023) in most aspects. Images are resized to 256 x 256 with 16 x 16 patch sizes. Texts, on other hand, are tokenized using C4 tokenizer (Raffel et al., 2020) with vocabulary size of 32K, and we keep maximum of 64 tokens. The optimizer is Adam with learning rate 10-3 and weight decay 10-A, using an inverse square root decay schedule with 5K warm-up and cool-down steps. For the baseline, we use SigLIP-B/16 pretrained on 10B training examples. Again, recursive models are trained on fewer examples to match the total training compute cost. Due amount of compute involved in these experiments, we only compare the non-recursive baseline (with signature AB) against RAO (with signature ABAB) and RINS (signature B) with degree
2023
Results. Table 3 shows that RINS (with signature A B) outperforms the non-recursive baseline, including with long sequence length, and RAO in zero-shot and retrieval evaluations. Of importance is the impact in ImageNet O-shot classification, where we see an improvement of about +0.7%.
Overtraining. Next, we demonstrate that RINS substantially advance tfate-of-the-ar results in multimodal systems for a given model size in the overtraining regime. We train recursive variant of SigLIP-B/16, denoted SigLIP-RINS-B/16 using the signature BB In light of the findings presented later in Section 6, we increase the number of recursions here given the increase in compute.
We adhere to the training protocol outlined above, with the exception that SigLIP- CHII--RIN-BBII B/16 is now trained on 40B examples. matching the training data scale of the widely-used, publicly available HigLIP-B/16 checkpoint. Note that both models have an identical size. Moreover, following Pouget et al. 2024), we utilize a training mixture comprising both English and multilingual data to enhance cultural diversity, and report the cultural metrics recommended in Pouget et al. (2024)
well as multilinguality evaluations using XM3600 dataset (Thapliyal et al. 2022). ). So, to ensure an appropriate comparison of results, re-train LigLIP-B/16 on 40B examples from the same data mixture, The primary datasets for cultural diversity evaluation are Dollar Street (Rojas et al. 2022)) GeoDE (Ramaswamy al., 2024), and Google Landmarks Dataset v2 (GLDv2) (Weyand al. 2020 We use the Gemma tokenizer (Mesnard et al. 2024).
As shown in Table 6 BINI---INNS-B116 significantly outperforms the standard SigLIP-B/I6 across all benchmarks. In Fact, stochastic RINS with skip probability p. = improves results even further. Crucially, these results demonstrate that RINS offers fundamental advantage in multimodal learning that are not replicated by simply overtraining a non-recursive counterpart.
Further Analysis
Vision. As previously discussed, the performance gains in RINS are consistent with the self-similar nature of language. By performing recursive, scale-invariant decoding, RINS introduces inductive
7

---

Dalasel
SigLIP-B/16
SigLIP-RINS-BRE
P. 0
ImageNet
77.3
79.0
79.6
79.2
O-shot classification
CIFAR100
81.7
Pet
92.6
94.8
94.4
92.7
Dolllar Street
17.6
19.7
19.2
19.3
DeeLooCGieoDDE BRADE--Cuung Country
22.5
23.3
24.7
23.9
Cultural diversity
GeoLoD Region
38.7
Dollar Street GooDE
51.5
52.9
53.1
53.1
93.1
93.7
94.2
GLDv2
51.6
53.7
52.4
52.5
Multilinuulity
XM3600 img21xtd1
53.5
XM3600 1x
39.5
43.0
43.1
41.9
COCO img2xxr@1
67.6
69.4
70.0
69.5
Retrioval
@1
50.3
51.5
52.0
Flickr img2tre1
91.9
92.9
93.5
92.4
Flickr (f
80.1
80.7
81.4
80.5
Table 4: Performance of multilingual SigI.IP models on various datasets under an overtraining regime. All models are identical in size to SigLIP-B/16. As shown in the rightmost columns, stochastic RINS (ps > 0) outperforms the other models. Full retrieval multilinguality results are in Appendix B
bias that encourages the model to recognize and exploit recurring patterns at different scales (see Appendix for further discussion). To test this is likely the source of its advantage, we conduct similar empirical evaluation in vision, a domain lacking relf-similarity. Appendix BeaendxxxGGproor GIprovides the full evaluation results using encoder-only vision transformers (ViT) (Dosovitskiy al. 2021) on SmageNet-IILSRRC SSRCV2012 (Deng et al. 2009). We observe that recursive architectures, including RINS, do not confer any advantage the supervised image classification domain, in agreement with our hypothesis that relates the success of RINS to fractal structure of language.
Data Scaling Laws. Next, we investigate the influence of Recursive INference Scaling (RINS) on the data scaling behavior of language models. Specifically, we fit a power law of the form function of the training FLOPs This allows
E(x) 3x to the reg-erplexitty loss s(x) us to analyze the impact RINS on both the scaling exponent c and the asymptotic performance limit Eoc. revealing whether the performance gains in RINS stem from improved scaling exponent, a lower asymptotic limit, or a combination of both. We use the 200M-paramette language models in Figure 3al whose signature is A A"B, for 2,, 4). where baseline. As shown in Figure Sa RINS improves both the scaling exponent and the asymptotic limit. The improvement in the asymptotic limit provides further evidence that the performance gap in favor of RINS is not closed by overtraining non-recursive models.
corresponds to the non-recursive
Optimal Number of Recursion Rounds. We observe from the scaling parameters in the previous section that if the performance of RINS is modeled as power law f.(r) 3,x - + Er, then G, increases with r while & decreases. Furthermore, the coefficient 8, also increases with r. This implies that while scaling inference increasing r might initially exhibit higher loss due the larger 3r, its faster convergence (c,)) and lower asymptotic limit (8,)) will eventually lead to superior performance. In other words, using the higher recursion level is advantageous eventually, which is consistent with the experimental results. To quantify this more explicitly, we train language models with four signatures A:B: E 2, 4). Then, plot the optimal value of r against training compute. As shown in Figure 5b, the optimal value of monotonically increases with training compute, in agreement with earlier results. Also, smaller models benefit more from RINS.
Adding Linear Adapters. Earlier in Section 4 we showed that enabling stochastic RINS during training exhibits tradeoff between worst-case and best-case performance, depending whether or not RINS is applied at inference time. Next, we introduce additional improvement: when maximum of recursion rounds are used in stochastic RINS, we add lightweight, linear adapters (i.e. linear projection layers) to the output before the projection head. The choice of which adapter to apply is function of how many recursion rounds are used. Specifically, if signature A'B is used, the r-th adapter applied. Empirically, this introduces < 1% more parameters and has a negligible impact FLOPs. Yet, it resolves the tradeoff encountered earlier shown in Figure T (left). With

---

SlimPajama
0.575
0.575
0.550
0.525
Optimal
C4
Pajama
2.500
2.475
2.35
1
300M
600M
2.450
18
0.0
0.2
0.4
0.6 0.0
1.0
Training FLOPs (layers)
1e10
(a) TOP: Scaling exponent e models with signature A B, where r = is the baseline. RINS (r > 1) improves scal- ing exponents. BOTTOM: Asymptotic log-perplexity 6oo plotted against RINS improves S overtraining the baseline cannot match performance gain.
(b) Optimal number of recursions in RINS is plotted vs. training compute for 300M-, 600M- and IB- parameter LMs. Compact, overtrained models benefit more from RINS.
2.9
Sharing
Sharing
2.9
Sharing
Sharing
SUrPPNNee Log
Daneline P.
ca Lop PPL
Sin SimPagpma Log
RINgg 2x NNS0 NNS022
Pp
2.7
2.5
2.5
2.5
1.58 2:30 2.50 Toining (syers)
0.00
0.53 1.00
2,50
0.50
1.58
2.00
2.50
2.50
Traniin (ayers)
Training (ayers)
FLOPs (laees))
Figure 6: LEFT 2 PLOTS: a-axis corresponds to performance when RINS is enabled during training but disabled at inference time 600M -araameee LMs. Ps 2 adapters matches the baseline at 1x the inference cost while P. = 0.8 results in better language model, even though all models have the same training compute, parameter count, and inference cost. We speculate this is because RINS provides a better inductive bias. RIGHT 2 PLOTS: RINS with p. 0.5 improves performance with KV cache sharing, although the improvement diminishes.
1 in stochastic RINS with linear
linear adapters, stochastic RINS can provide no-regret strategy, where performance improves in RINS-enabled pretraining even when RINS is not applied at test time.
Adding KV Cache Sharing. Finally, RINS by itself reduces the memory footprint by matching performance using small models, which is important for compact models as discussed earlier. We now push this memory reduction further by analyzing the effect of KV cache sharing, where queries during recursion attend to the tokens and keys the first block in RINS. Specifically, for models with signature A'B, in every subsequence use of block A, queries attend to the cached keys and values of the first call of A. This ensures that memory footprint does not grow with recursive depth. Figure 6 shows that while KV cache sharing diminishes the gain, RINS with KV cache sharing still offers advantage compared to the baseline.
Discussion and Related Works
The premise that extending inference time enhances the quality of language model outputs finds support in cognitive science. Human deliberation, often associated with System 2 thinking, is linked to improved decision-making capabilities (Lawson et al. 2020). Mirroring this, numerous computational strategies have been developed scale inference within Large Language Models (LLMs). Prompt-based methods like Chain-of Sean--of-Thhughh (CoT) (Wei et al., 2024) and critique-ther generate approaches (Ankner et al. (024)) are prominent examples. Other iterative refinement methods like ReAct (Yao et al.] (023). Self-Retine (Madaan et al.) (2024) and Reflexion, which incorporate feedback and reflection, also improve inference quality. Even simpler strategies, capitalizing on stochastic nature of L.M decoding that generate multiple responses and select among them, such as
self-consistency (Wang et al., 2023),, have shown efficacy that scales well across multiple orders of magnitude of inference compute (Brown et al. 2024).

---

However, the assumption of monotonic improvement with increased inference calls is not guaranteed hold. As argued by Chen et al. (Chen et al., 2024). repeated sampling may lead to convergence towards the most probable, but not necessarily the optimal, solution. This is particularly pertinent for challenging instances where the probability of correct answer below chance (i.e., < 0.5).
In this work, we introduce a complementary approach, called Recursive INference Scaling (RINS), which can be integrated with other techniques. RINS identifies a specific form of model recursion as an effective inference scaling strategy, demonstrating significant performance gains over various other recursive architectures, including latent recurrent thinking (Geiping et al. 2025) and the Repeat All Over (RAO) strategy identified by Liu et al. (2024) as state-of-the art for mobile LLMs.
Recursion is form of parameter sharing, technique that has been explored for while. Illustrative examples include Universal Transformers (Dehghani et al. 2019) and ALBERT (Lan et al. 2020). De- spite their innovative design, their scaling exponents et al. 2022a) so they failed to subsume it (Tay et al. 2022b). RINS, by contrast, improves both the scaling exponents and asymptotic performance limits.
smaller than in the vanilla transformer (Tay
Acknowledgement
The authors would like to thank Alexander Kolesnikov, André Susano Pinto, Michael Tschannen, Jeremiah Harmsen, Olivier Bachem, Jamie Lohoff, Jean-bastien Grill, Mathieu Blondel, Michaël Sander, Tianlin Liu, and Vincent Roulet, for the insightful discussions. We also thank the Google DeepMind unit at large for providing supportive research environment.
References
Alabdulmohsin, I., Neyshabur, B., and Zhai, X. (2022). Revisiting neural scaling laws in language and vision. In NeurlPS.
Alabdulmohsin, I., Tran, V. Q., and Dehghani. M. (2024a). Fractal patterns may illuminate the success of next-token prediction. In NeurlPS.
Alabdulmohsin, I., Zhai, X., Kolesnikov, A., and Beyer, L. (2024b). Getting VIT in shape: Scaling laws for compute-optimal model design.
Ankner, Z., Paul, M., Cui, B., Chang, J. D., and Ammanabrolu, P. (2024). Critique-out lot-uuu- reward models.
Bahri, Y., Dyer, E., Kaplan, J., Lee, J., and Sharma, U. (2021). Explaining neural scaling laws. arXiv preprint arXiv:2 102.:60011
Bansal, Y., Ghorbani, B., Garg, A., Zhang, B., Krikun, M., Cherry, C., Neyshabur, B., and Firat, O. (2022). Data scaling laws in NMT: The effect of noise and architecture. arXiv preprint arXiv:2202.01994.
Beleites, C., Neugebauer, U., Bocklitz, T., Krafft, C., and Popp. J. (2013). Sample size planning for classification models. Analytica chimica acta, 760:25-33.
Beyer, I.,, Zhai, X., and Kolesnikov, A. (2022). Better plain vit baselines for imagenet-
Bisk. Y., Zellers, R., Bras, R. L., Gao, J., and Choi, Y. (2020). Piga: Reasoning about physical commonsense in natural language. In Thirty-Fourth AAAI Conference on Artificial Intelligence.
Brown, B., Juravsky, J.. Ehrlich, R., Clark, R., Le, Q. V., Ré, C., and Mirhoseini, A. (2024). Large language monkeys: Scaling inference compute with repeated sampling.
Chen, L., Davis, J. Q., Hanin, B., Bailis, P., Stoica, I., Zaharia, M., and Zou, J. (2024). Are more llm calls all you need? towards the scaling properties of compound systems. The Thirt-eighth Annual Conference Neural Information Processing Systems.
10

---

Chen, M., Tworek, J., Jun, H.. Yuan, Q. de Oliveira Pinto, H. P., Kaplan, J., Edwards, H., Burda, Y., Joseph, N.. Brockman, G., Ray. A., Puri, R., Krueger, G., Petrov, M., Khlaaf, H., Sastry, G., Mishkin, P. Chan, B., Gray, S., Ryder, N., Paviov, M., Power, A., Kaiser, I.., Bavarian, M., Winter, C., Tillet, P., Such, F. P., Cummings, D., Plappert, M., Chantzis, F., Barnes, E.,, Herbert- Voss, A., Guss, W. H., Nichol, A., Paino, A., Tezak, N., Tang, J., Babuschkin, I., Balaji, S., Jain, S., Saunders, W., Hesse, C., Carr, A. N., Leike, J., Achiam, J., Misra, V., Morikawa, E., Radford, A., Knight, M., Brundage, M., Murati, M., Mayer, K., Welinder, P., McGrew, B., Amodei, D., McCandlish, S., Sutskever, I., and Zaremba, (2021). Evaluating large language models trained code.
Chen, X., Fang, H., Lin, T.-Y,, Vedantam, R., Gupta, S., Dollár, P., and Zitnick, C. L.. (2015). Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv: 150..00355.
Cho, J., Lee, K., Shin, E., Choy, G., and Do. S. (2015). How much data is needed to train medical image deep learning system to achieve necessary high accuracy? arXiv preprint arXiv:1511.06345
Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, T., Collins, M., and Toutanova, K. (2019). BoolQ: Exploring the surprising difficulty of natural yes/no questions. In NAACI.
Dehghani, M., Gouws, S., Vinyals, O., Uszkoreit, J., and Lukasz Kaiser (2019). Universal transform- ers.
Deng, J., Dong, W. Socher, R., Li, L.JJ, Li, K., and Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In CVPR.
Dosovitskiy, A., Beyer, L., Kolesnikov, A.. Weissenborn, D., Zhai. X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. (2021). An image is worth 16x16 words: Transformers for image recognition at scale.
Figueroa, R. L., Zeng-Treitler, Treitler, Q.. Kandula, S., and Ngo. L. H. (2012). Predicting sample size required for classification performance. BMC medical informatics and decision making, 12(1):1-10.
Geiping, J., McLeish, S., Jain, N., Kirchenbauer, J., Singh, S., Bartoldson, B. R., Kailkhura, B., Bhatele, A., and Goldstein, T. (2025). Scaling up test-time compute with latent reasoning: A recurrent depth approach.
Hestness, J., Narang, S., Ardalani, N., Diamos, G., Jun, H., Kianinejad, H., Patwary, M., Ali, M.. Yang, Y., and Zhou, Y. (2017). Deep learning scaling is predictable, empirically, arXiv preprint 1712:0171,
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. d. L., Hendricks, L. A., Welbl, J., Clark, A., et al. (2022). Training compute-optimal large language models. In NeurIPS.
Huang, G., Sun, Y., Liu, Z., Sedra, D., and Weinberger, K. (2016). Deep networks with stochastic depth.
Hutter, M. (2021). Learning curve theory. arXiv preprint arXiv:2102.04074
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. (2020). Scaling laws for neural language models. arXiv preprint arXiv:2001.0830
Kingma, D. P. and Ba, J. (2014), Adam: A method for stochastic optimization. arXiv preprint 112.v.1412.6980.
Krizhevsky, A. (2009). Learning multiple layers of features from tiny images. Technical report.
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., and Soricut, (2020). Albert: lite bert for self-supervised learning of language representations.
Lawson, M. A., Larrick, R. P., and Soll, J. B. (2020). Comparing fast thinking and slow thinking: The relative benefits of interventions, individual differences, and inferential rules. Judgment and Decision making, (566):660-644

---

Li, Y., Choi. D., Chung, J., Kushman, N., Schrittwieser. J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., Hubert, T., Choy, P., de Masson A'ttumme C., Babuschkin, I., Chen, X., Huang, P.-S., Welbi, J., Gowal, S., Cherepanov, A., Molloy, J., Mankowitz, D. J., Sutherland Robson, E., Kohli, P., de Freitas, N., Kavukcuoglu, K., and Vinyals, (2022). Competition-level code generation with alphacode. Science, 378(6624): 1092224)9902199
Liu, Z., Zhao, C., landola, F,, Lai, C., Tian, Y., Fedorov, I., Xiong, Y., Chang, E., Shi, Y., Krish- namoorthi, R., al. (2024). Mobilellm: Optimizing sub-billion parameter language models for on-device use cases. In ICML.
Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye. S., Yang. Y., (2024). Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36.
Mesnard, T. Hardin, C., Dadashi, R., Bhupatiraju. S., Pathak, S., Sifre, L., Rivière, M., Kale, M. S..
Love, J., Tafti, P., Hussenot, L., Sessa, P. G., Chowdhery, A., Roberts, A., Barua, A., Botev, A.. Castro-Ros. Rooto-RRs,AA A. Slone, A., Héliou, A., Tacchetti, A., Bulanova, A., Paterson, A., Tsai, B., Shahriari, B., Lan, C. L., Choquetie-Choo, C. A., Crepy, C., Cer, D., Ippolito, D., Reid, D., Buchatskaya, E., Ni, E., Noland, E., Yan, G., Tucker, G., Muraru, G.-.,, Rozhdestvenskiy, G., Michalewski, Keeling, J., Labanowski, J., Lespiau, J.-B., Stanway, J., Brennan, J.. Chen, J., Ferret, J., Chiu. J., Mao-Jones, J., Lee, K., Yu, Millican, K., Sjoesund, L. L., Lee, L., Dixon, L., Reid, M., Mikula, M., Wirth, M., Sharman, M., Chinaev, N., Thain, N., Bachem, O., Chang. O., Wahltinez, O.,, Bailey, P., Michel, P., Yotov, P., Chaabouni, R., Comanescu, R., Jana, R., Anil, R., Mcllroy, R., Liu, R., Mullins, R., Smith, S. I.., Borgeaud, S., Girgin, S., Douglas, S., Pandya. S., Shakeri, S., De, S., Klimenko, T.. Hennigan, T., Feinberg, V., Stokowiec, W., hui Chen, Y., Ahmed, Z., Gong, Z., Warkentin, T., Peran, L., Giang, M., Farabet, C., Vinyals, O., Dean, J., Kavukcuoglu, K., Hassabis, D., Ghahramani, Z., Eck, D., Barral, J., Pereira, F., Collins, E., Joulin, A., Fiedel, N., Senter, E., Andreev, A., and Kenealy, K. (2024). Gemma: Open models based gemini research and technology.
H., Tenney, Grishchenko, I., Austin,
Mihaylov, T., Clark, P., Khot, T., and Sabharwal, A. (2018). Can suit of armor conduct electricity? new dataset for open book question answering. arXiv preprint arXiv:180..00 11091087897
Parkhi, O. M., Vedaldi, A., Zisserman, A., and Jawahar, C. V. (2012). Cats and dogs. In IEEE Conference on Computer Vision and Pattern Recognition.
Pouget, A., Beyer, L., Bugliarello, E.. Wang, X., Steiner, A. P., Zhai, X., and Alabdulmohsin, I. (2024). No filter: Cultural and socioeconomic diversity in contrastive vision-language models. In NeurlPS.
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. (2020). Exploring the limits of transfer learning with unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67.
Ramaswamy, V. V., Lin, S. Y., Zhao, D., Adcock, A., van der Maaten, L., Ghadiyaram, D., and Russakovsky, O. (2024). Geode: geographically diverse evaluation dataset for object recognition. Advances in Neural Informaionn Processing Systems, 36.
Rojas, W. A. G., Diamos, S., Kini, K. R., Kanter, D., Reddi, V. J., and Coleman, C. (2022). The dollar street dataset: Images representing the geographic and socioeconomic diversity of the world. In Thitty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track.
Sap, M., Rashkin, H., Chen, D., LeBras, R., and Choi, Y. (2019). Socialiqa: Commonsense reasoning about social interactions.
Sharma, U. and Kaplan, J. (2022). Scaling laws from the data manifold dimension. JMLR, 23(9):1-34.
Soboleva, D., Al-Khateeb, F., Myers, R., Steeves, J. R., Hestness, J., and Dey, N. (2023). SlimPajama: A 627B token cleaned and deduplicated version of RedPajama.
12

---

Talmor, A., Herzig, J., Lourie, N., and Berant, J. (2019). CommonsenseQA: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 4159-4158, Minneapolis, Minnesota. Association for Computational Linguistics.
Tay, Y., Dehghani, M., Abnar, S., Chung, H. W., Fedus, W., Rao, J.. Narang, S., Tran, V. Q.. Yogatama, D., and Metzler, D. (2022a). Scaling laws vs model architectures: How does inductive bias influence scaling? arXiv preprint arXiv:2207.1055/
Tay, Y., Dehghani, M., Bahri, D., and Metzler, D. (2022b). Efficient transformers: A survey,
Thapliyal, A. V., Pont-Tuset, J., Chen, X., and Soricut, R. (2022). Crossmodal-3600; massively multilingual multimodal evaluation dataset. arXiv preprint arXiv:220S.12522
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, I.., Gomez, A. N., Kaiser, L.,, and Polosukhin, (2017). Attention is you need. NeurIPS.
Wang. X., Wei, J., Schuurmans, D., Le, Q.. Chi, E.,, Narang, S., Chowdhery, A., and Zhou, D. (2023). Self-connis improves chain of thought reasoning in language models.
Wei, J.. Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E. H., Le, Q. V., and Zhou, D. (2024). Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA. Curran Associates Inc.
Welleck, S., Bertsch, A., Finlayson, M., Schoelkopf, H., Xie, A., Neubig, G., Kulikov, I., and Harchaoui, Z. (2024). From decoding to geta--ennraaton: Inference-time algorithms for large language models. Transactions Machine Learning Research. Survey Certification.
Weyand, T., Araujo, A., Cao, B., and Sim, J. (2020). Google landmarks dataset v2-a large-scale benchmark for instance-level recognition and retrieval. Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 2575-2584,
Wilcoxon, F. (1992). Individual comparisons by ranking methods. In Breakthroughs in statistics: Methodology and distribution, pages 196-202, Springer.
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., and Cao, Y. (2023). React: Synergizing reasoning and acting in language models.
Young, P., Lai, A., Hodosh, M.. and Hockenmaier, J. (2014). From image descriptions visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, 2:67-78.
Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., and Choi, Y. (2019). Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.
Zhai, X., Kolesnikov, A., Houlsby, N., and Beyer, L. (2022). Scaling vision transformers. In CVPR.
Zhai, X., Mustafa, B., Kolesnikov, A., and Beyer, L. (2023). Sigmoid loss for language image pre-training.
Zhang, H., Cisse, M., Dauphin, N., and Lopez-Paz, D. (2018). mixup: Beyond empirical risk minimization.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We support all claims with extensive empirical results stated in the Abstract
and Introduction. See Sections 3 and in addition to the Appendices.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We show that RINS is not effective for pure vision tasks, such as image classification. See Appendix G
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sellsspeciiiationn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
14

---

Answer:
[NA]
Justification:
We do not derive any new theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We describe experimental setup in details (see for example Section 3 and include the full training configurations in Appendix E
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
15

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: Our experiments based the open-sourced Big Vision codebase, but we do not release the code. However, we describe everything needed to reproduce our experiments in detail, and provide the full training configurations in Appendix B
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
mublic/guides CoodeSubmissioooolle.
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips wttpccccccccccueeeeeeeeeeeepeeee
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: We provide all training and test details in the paper (see Sections and 6 and the Appendices) including a description of how the hyperparameters were selected (see Section 3 for language and Section 5 for SigLIP).
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
For SigLIP experiments, we do not provide error bars because each experiment quite expensive compute- wise and cannot train multiple models.
Justification: See the error bars in Table 2
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We describe compute resources (e.g. topology and compute hours) in Section 3
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics httpps:/neuurps....
Answer: [Yes]
Justification:
The research conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work focuses on improving the performance of language models and mul- timodal systems. The technique introduced in this work is not tied to particular applications or deployments.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
17

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does not release data or models.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All assets (papers, codes, datasets, etc) are properly cited in the paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper doesn't 't release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
19

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
Guidelines: the core method development in this research does not involve LLMs any important, original, or non-standard components.
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
20

---

A Recursive Inference Scaling (RINS) Pseudocode
class NecursiveBlock(>
config: single block contig eignature: etr "s"
degree: int
p_skip: Tuple(float, ... # skip prob for blacks
def ...c..l.__salll (salf, x): ""*call nodel on input "."*"
degree
blocks - (c: (ingle confif setlf signatureg
alse:
blocks (c: RocursiveBlock
configraelf config, signaturerself aignature,
degree-self.co degree 1, s.skippself.p.skt.
for in (et(f eignature))
inputs =x
forward pass
for in range (1en((1t (aan(eelt.ssgnatrr)))) signature)):
c self signature [i] choice if randon.unifow u
-- 1: atochastic RIXS
self pelflp)
(c)cka(cl(x)) # else skip
(x)
return x
Figure 7: Numpy-like syntax for models with fixed signature and degree. When no stochastic depth applied, we have Pa where the first and last entries are zero to guarantee they are executed, which is equivalent to sampling the number of recursion rounds from binomial distribution as described in Section 4
RINS, we expand p. into a tuple of the form (0, P............................... P.. .......... 0).
21

---

B Retrieval and Multilinguality Detailed Results
The following tables provide the full retrieval and multilinguality results for SigLIP-RINS-BIA
Language
Iuage-to- Text Retrieval
Texl-to-lmage Retrieval
SigLIP- B/16
SigLIP-RINS-BI16
SigLIP- B/16
SIgLIP-RINS-BI6
P.
Pa
Px
P.
Pa
P.
46.7
7.0
11.2
10.3
8.7
4.8
7.2
6.8
5.1
51.1
55.8
55.8
54.5
40.5
45.0
44.6
43.0
61.3
68.8
68.8
65.2
45.8
51.6
52.6
49.8
81.5
84.1
83.7
83.3
69.3
72.2
72.4
71.5
55.6
56.7
56.0
56.3
53.2
$3.4
53.4
51.2
71.3
71.7
63.9
fa
54.4
61.3
58.6
59.6
48.1
51.6
51.6
51.9
Ii
34.6
43.2
43.0
39.8
21.9
28.4
28.8
25.9
19.0
21.8
20.5
19.4
10.4
12.5
12.3
11.3
fr
74.8
77.6
76.2
76.9
67.0
69.5
69.9
68.0
hr
46.9
57.4
56.2
53.1
33.6
41.2
40.4
37.9
55.2
53.9
42.7
43.1
41.2
74.1
77.8
77.7
78.4
65.3
68.0
68.2
67.0
73.9
79.1
77.5
77.3
67.3
70.4
70.2
69.9
48.2
56.8
54.5
52.3
37.5
43.9
43.1
40.8
in
48.8
57.8
57.9
53.9
38.1
43.2
42,0
39.6
51.7
mi
0,7
0.7
0.8
0.6
0.2
0,4
0,3
0.3
67.8
65.0
59.0
51.7
61.0
68.9
67.1
66.8
45.5
$2.5
520
50,9
pl
62.3
68.9
67.4
67.1
$4.0
57.6
59.0
57.1
71.1
70.1
71.4
63.3
63.2
quz
7.2
7.7
7.9
7.7
3.1
3.0
3.2
2.9
66.7
70.8
69.5
70.3
61.3
64.2
64.3
64.2
sv
71.9
71.9
70.1
51.3
55.3
$6.6
54.4
10.2
10,0
9.4
5.2
5.4
4.4
te
1.4
1.6
1.0
0.5
0.7
0.5
29.5
38.2
35.1
34.5
24.7
23.1
24.0
tr
$2.9
$8.0
57.9
57.6
44.3
47.3
46.9
46.6
58.1
73.8
79.3
78.4
77.8
61.8
65.7
66.4
65.5
55.3
60.2
60.5
58.7
44.9
48.3
49.8
47.1
Table Per-language retrieval evaluations using the Crossmodal-360M dataset Thapliyal (2022)
22

---

Metric
SigLIP-B/16
SigLIP-RINS-EE
(370M params)
(370M params)
Mubtliiguality
XM3600 img21x111
48.4
53.5
52.6
51.8
XM3600 img22tr
68.4
72.6
72.0
70.9
XM3600 img2txt@11 10
74.4
78,1
27.5
76.5
XM3600 ex22mng@1
39.5
43.0
43.1
41.9
XM3600 1x2
59.5
63.0
63.1
61.6
XM3600 Ex22nng @
66.1
69.3
69.3
68.0
COCO img 1
67.6
69.4
70.0
69.5
COCO img22v105
87.2
88.6
88.9
88,3
COCO img22xx0@ 10
92.6
93.2
93.5
23.2
00C0 1x12mmg@1
50.3
51.5
52.4
$2.0
tx22mng@5
74,7
75.2
75.7
COCO ix12img@
82.6
83.0
83.5
83.2
Flickr img2xt@1
92.9
93.5
Flickr img22xx685
99.3
98.7
99.5
98.7
Flickr @mg2xxt@ 10
99.6
99.5
99.7
99.3
Flickr ix
80.1
80.7
81.4
80.5
Flickr txt2img@5
94.6
94.3
95.4
Flickr tx22mng@10 10
97.1
97.0
97.4
97.0
Table 6: Performance of multilingual SigLIP models on various datasets under an overtraining regime. All models are identical in size SigLIP-B/16. As shown in the rightmost columns, stochastic RINS (p. > 0) outperforms the other models. Full retrieval & multilinguality results are in Appendix
C Zero-shot Evaluation on Common Sense Reasoning Tasks
This section describes the evaluation setup for assessing the zero-shot common sense reasoning capabilities of the language model (Table 2
Datasets. Each model is evaluated six benchmarks:
OpenBookQA: A multiple-choice question answering dataset requiring knowledge from an open book of elementary level science facts (Mihaylov al. 2018).
BoolQ: A yes/no question answering dataset requiring the model to determine the truthful- ness of given statement based on a provided passage (Clark et al. 2019).
PIQA: A multiple-choice benchmark focusing on physics-related reasoning, such as how interact with objects in daily life (Bisk et al. 20200).
SIQA: A multiple-choice dataset focusing on social common sense reasoning (Sap et al.
2019).
HellaSwag: A multiple-choice dataset focusing on sentence completion Zellers al.
2019).
CommonSenseeAA- A multiple-choice question answering dataset with distractors TTamorr
ct al. 2019).
Prompting. Each model is evaluated a zero-shot setting, meaning does not receive any training examples specific the task. Instead, since all tasks above have multiple-choice answers, evaluate the log- perplexity of each option, applying a causal mask, after concatenating the contextual information (if any), followed by the prefix and the answer. Then, select the choice that has the lowest per-token log-perplexity score as the model's 's answer.
We do not apply any prompting, except in BoolQ and PIQA. In BoolQ, we use the prompt template:
<data ('aassae ']> }>> Based on this, the answer to the question: <data quuaaaituussi is:
which we have found to improve performance significantly. In PIQA, we formulate each sentence in the form:
23

---

The goal is: {goal} The solution is: (sol}.
We do this PIQA because, otherwise, the sentences can difficult to understand, even for humans. In one example, for instance, the goal is: "Deep clean coffee grinder. and the two possible solutions are: "Scrape with rice." and "Scrape with flour." " Concatenating directly would result in sentences like "Deep clean coffee grinder. Scrape with rice." whose unclear.
Evaluation Metric. The evaluation metric is accuracy. For each example, the model's predictions compared to the ground truth labels, and the accuracy is calculated as the percentage of correct predictions. The model is evaluated a deterministic mode, meaning no randomness is involved in the inference process.
24

---

D Recursion and Seff-Similartty
Cunnccitel Porteheeeet
Clorcstial
Puretmeett
POS; Artide
Iaraltctle fasazo)
Rype: Detiate (enmertir Sucenes
Toleen: "Eodf" yoer
Trives "toi"" POS: Adjatian lygalake fasano)
Tokem: "Extl PUS: Adjacto Seatinere: Nogicie (enventi Surons]]
Noural Notwook boued
Tolom: "Grwer
Tolsen: "Uueee"
your tuns))
Nerma Natonek
lystteitt Bortzee
Tokune 'Quaz"" POS: Nore
POS: Nom
Keelty: Charetse
Posilion: Sabaet
Tolom: "sppoon""
Thlern "appoard"
Tokosz "upperr" POS:
POS: Vuri
Tene::
Same Modell
Figure 8: Caption
How does depth-wise model recursion relate to self-similarity of language across the temporal dimension? We provide an illustration of this Figure
Informally, one might view the early layers a neural network to be transforming the raw linguistic input into a richer language that is appended with useful, high-level details. For instance, in the sentence "the evil queen appears ...", early layers potentially through mechanisms like attention, might modify the token "queen" to incorporate features such part-of-speech (POS) tagging, sentiment, and grammatical information. Subsequent layers can then iteratively incorporate progressively higher-level contextual information.
The concept of self-similarity in temporal sequences, including language, hints that higher-order linguistic features exhibit patterns analogous to those present in the raw input. If this property holds for language, implies that the same neural network block can be recursively applied to generate increasingly abstract features.
However, while this interpretation offers a conceptual framework, essential to acknowledge that remains a hypothesis. Empirical evidence has demonstrated the self-similarity of log-perplexity scores in language (i.e. surprise of information-theorees at the word level are mirrored at sentence and potentially higher levels of linguistic structure. The efficacy of Recursive INference Scaling (RINS) is another evidence favor of the informal picture above.
complexity), indicating that patterns observed
25

---

E
Configuration
E.1 Architecture Sweep
We use the following pseudocode when sweeping across recursive architectures. The goal is to ensure that we only compare architectures that have similar size.
baseline
add (signaturo= 'A", degree=1)
RAD
add (sigusture- degreel)
add (wwggaturee M degreel)
add (adissaatture-aAn 'AMA'' degree1)
other architectures
signature ['ABB', 1ABA', 'AAB', 'ABBC', 'AABC', 'ABCC', 'ABBB', "ARAB", 'AABB *] for degree in [1, 2, 3):
sun_uniquo_blocke (eet (eignatuue) degree unigue mique layere // nun, wwwque.uquuuue.e blocke
mun_leyers_per. block
lsyer_in,
nus auu__layers_por.Dloc. 0:
add addciggattureesssaauuue degree-degree)
26

---

E.2
Training Configuration
E.2.1
Language Modeling
"*"Config training decoder-onl) language model.""!
config. seed 0 config, totel_
avept
32,000
config. input dict()
config.input max_len 1,024 # or 1,536 for long sequence baseline tokenizr 'n__en"
config... iaput. data # equel xeight
'c4/en': 1.0, hagggaggtesscrrrr...........memm
1.0,
for dataset_namd in config. data: data:
config-inpat [dataset_name] {
config input [dataset_neme] data diet( Rame=dataset_
split=/troin',
config input [dataset_ bhuffle_.uffer_eee 250,000
config... Input. wwt.g..appt..aaaae... batch.
1,024
# Optimizer section config. optax_nneee "acale_by_ade config. grad_clip_norm 1.0
config. 1r 5e-4
config, .d 5e-5
config. schedule dict (dwccddtypeeeyeeeeeeeeeeee
sarmup_ Stempp_teep-BB... 6000
caoldovn_
E.2.2
Vision: Image Classification
"*"Config for training a V1T model. n*n
config. seed
config. totsl_epochs config, nun, 1,000 config.init_bend init_
config. input dict()
config bapptt.btchhalsee.. 1,024
config. shufttteefflleeeueeeee
250,000
# preprocessing config input.ppp pp ivalue_range config mixup 2d told_in=Xone)
cropptionn Iilip_ir'
# Optimizer section config, optax_name 'scale_by acale_by_adarr config, grad_clip_norm 1.0
config. optaz diet (mu_dtyee-- bblosstiob bflosti6', b2-0.95)
config. lr 0.001
config. 0.0001
config. schedule dict (dt.:ddcyytyteooooiiiiii. wermup_stepe-10.
27

---

E.2.3 Language-Image Pretraining
"*"Config for training a SigLIP model. config. seed
config. exaale
config. input dict()
config, www.t.patch...............b..... size 1,024 x 32 config. www..tttattt.tlleeoppeee
t preprocessing
config. inppi.ttttt........ tokenize nc4 multilingual c4 config. iappt.tax__lnn 64 for text menory
config www.ttapppteetttt... prefetch
saarig.mmddll. biag_iiit = -10.0
config. modeg. temperature init 10.0
* Optimizer section config optax_ syale_by_adee adam
config. grad_clipp 1.0
config. lr 0.001
config, xá 0.0001
config schedule - diet wypeeeyyyyyp-'cooliiee
vermup_atepe-22_o00 _000
28

---

C4 Evaluation Results with Stochastic RINS
= 0.0
0.255
Py = 0.50
2.8
2.8
2.8
C1 Log-PPL
Log-PPL
2.6
2.6
CA
0.08
2.0B 4.0B 6.08
10.08
0.08
8.08 10.08
0.08 2.08 4.08 6.08 8.08 10.08 Training FLOPs (layers)
Training FLOPs (layers)
Training FLOPs (layers)
Figure 9: Performance of stochastic RINS (A with varying inference costs for IB parameter LMs on C4, similar to Figure reported reporedrepprreddd on SlimPajama. The r-axis represents the training compute cost. The legend indicates the inference cost of each stochastic RINS configuration relative to the baseline; e.g. 1.5x denotes 50% increase inference cost. For p. = 0, RINS@ Ix is significantly worse, with perplexity scores > 3. As expected, RINS converges in performance the baseline ps.
29

---

G Vision
As previously discussed, the performance gains in RINS are consistent with the self-similar nature of language. By performing recursive, acale-invarian decoding, RINS introduces an inductive bias that encourages the model recognize and exploit recurring patterns different scales (see Appendix errendixDfforr for further discussion). To test if this is likely the source of its advantage, we conduct similar empirical evaluation vision, a domain lacking self-similarity.
Setup. We train an encoder-only vision transformer ViT-B/16 Dosovitskiy et al. (021) on ImageNet- IISRCV2012 Deng al. (2009). The non-recursive baseline model is trained for either 300 1,000 epochs using a batch size 1,024. while recursive models are trained on fewer epochs to match the same total training compute FLOPs. We apply MixUp (probability 0.2) during training Zhang et al.) (2018) and use learned position embedding. The optimizer is Adam where we tune the learning rate for each architecture in the set Ir E {10-3 3, 7 10 3 10-4 10-4) with weight decay wd = 1r/10, 100, on small validation split. We use cosine learning rate schedule with 10K warm-up steps. Images are resized to 224 224 and 16
patch sizes are used. The full training configuration in
Appendix B
Results. As presented in Table 7 parameter-sharing techniques, including RINS, do not confer any advantage in supervised image classification. The non-recursive baseline, when trained on longer sequence lengths (i.e., higher image resolution) match the inference cost recursive architectures, surpasses all other methods. This starkly differs from the results observed in language modeling, where RINS provides significant gains even when compared against models that scale inference by increasing the sequence length.
Architecture
Val
Real.
v2
Avg
300 epochs
(ABB),
75.2
$1.4
62.7
73.1
333
75.7
$1.0
62.3
73.0
75.2
80.5
61.4
ABBC
75.1
80.2
61.3
72.2
1,000 epochs
A@336
77.6
82.7
64.6
75.0
ABBC
76.3
81.6
63.2
73.7
AABC
76.0
81.4
62.4
73.2
75.8
$1.4
62.4
73.3
75.7
80.8
62.0
72.8
Table 7: Performance of the top 5 architectures ILSRCV2012 classification. The table compares the performance of recursive architectures with a baseline trained 224- and 336-resolution images. The baseline A (366, trained higher-resolution match the inference of the recursive models, outperforms all parameter-sharing architectures. We use the notation (signature)degree-
30


Reviews:
[{"review_id": "J1HLZLRHo6", "reviewer": "Reviewer_vR9m", "summary": "This paper proposes that the (A^r)B recurrent structure is the best structure for parameter sharing. However, I find several parts of the article confusing and difficult to understand. I hope the author can clarify these points during the rebuttal phase.", "questions": "My questions are listed above.", "limitations": "YES", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "I list some of the points that I find confusing as follows:\n- Why it is a winning path to scalable inference in Language and **Multimodal Systems**, regarding that this paper (Appendix G) concludes that \"parameter-sharing techniques, including RINS, do not confer any advantage in supervised image classification. The non-recursive baseline surpasses all other methods. This starkly differs from the result observed in language modeling.\" Though Section 6 provides some other experiments about multimodality. It is still unclear what the relationship between these two parts is.\n- How is this paper related to **inference scaling**? The experiments focus on the PPL and training FLOPs. I don't find contents related to scalable inference.\n- The experiments are conducted on 300M/600M models. We can hardly say it is a **winning path** based on observations from such small models.", "was_revised": true, "final_justification": "The rebuttal by authors addresses most of my questions. The paper effectively demonstrates the empirical advantages of RINS, showcasing its strong performance across various experimental settings. However, it would benefit from deeper insights into why RINS functions as the most effective structure.", "rebuttal": "We thank the reviewer for the feedback. We believe there is a misunderstanding regarding the scope and key results of our paper, and we hope our clarification below will resolve it.\n\n**The Scope of \"Multimodal Systems\" and the Role of Vision Experiments**\n\nWe suspect that the reviewer is mixing the term “multimodal systems” with the term “all modalities”. Our claim is that RINS is effective for language and multimodal systems that process language (such as contrastive vision-language models). We never claimed that it works for all modalities.\n\nLanguage is a **major** domain and introducing methods that improve language models is an important contribution. It would be unfair to require a method inspired by the beautiful self-similar structure in language to also work for another domain that doesn’t have that structure, such as vision. \n\nThe negative results in pure image classification (Appendix G) are not a contradiction but rather supporting evidence for our hypothesis. Since vision as a modality does not exhibit the same self-similar structure as language, we would not expect a language-inspired method to necessarily confer an advantage in pure vision.\n\nWe demonstrate the success of RINS in multimodal systems where language is a key component. Our experiments with SigLIP-B/16, a vision-language model, show significant improvements across a wide range of tasks, including:\n* +2.3% in 0-shot ImageNet accuracy (from 77.3% to 79.6%).\n* Widespread gains in retrieval, cultural diversity, and multilinguality benchmarks (Tables 2, 3, 4, 5).\n\nThese results strongly support our claim that RINS is a \"winning path\" for systems that process natural language, whether in isolation or as part of a multimodal architecture. We say it is a \"winning path\" because we have compared it against 59 other parameter-sharing architectures.\n\n**The Connection to \"Inference Scaling\"**\n\nRINS is, by definition, an inference-time scaling method. RINS scales the amount of computation at inference time by recursively applying a subset of the model's layers (e.g., using signature $A^rB$). This is achieved while keeping the model parameter count and the total  training compute (FLOPs) fixed relative to baselines.\n\n**The Evaluation and PPL**\n\nOur evaluation is comprehensive and not limited to perplexity (PPL). This includes:\n\n* For language models, zero-shot evaluation on common sense reasoning tasks (Table 1).\n* For SigLIP, zero-shot classification, retrieval, multilinguality, and cultural diversity metrics.\n* Analysis of how RINS improves both the scaling exponent and the asymptotic performance limits (Section 5, Figure 5a).\n\n**The Scale of Models Used in Experiments**\n\nWhile our initial, broad sweep of 59 different architectures was conducted on 300M and 600M parameter models for computational feasibility, we validated our key findings on larger 1B parameter models. See for example: Figure 3b, Figure 4, Figure 5b. Also, as we argue from our scaling experiments in Figure 5b, RINS helps models  with >1B parameters if they are trained on more tokens. \n\nBut to verify this more, we have now trained a 2B parameter language model using either: no recursion (BL), RAO, and RINS with signature AAB. The baseline is pretrained on 200B tokens and models with recursion are trained on fewer tokens to match the total training FLOPs. As expected the baseline is always better than RAO while RINS outperforms the baseline; and the gap continues to increase in favor of RINS. This is in agreement with our scaling law studies. Please see the C4 validation perplexity scores below. \n\nUnfortunately, since we cannot attach images to the rebuttal (as per the new Neurips guidelines), we provide a table of results. We hope this is sufficient to demonstrate that RINS works well even with models >1B parameters. We will add this to the supplementary materials of the paper.\n\n| Training FLOPs (tokens x layers) | BL | RAO | RINS (AAB) | \n|---|---|---|---|\n| 150K | 2.558 | 2.593 | 2.559 |\n| 250K | 2.515 | 2.534 | 2.506 | \n| 350K (after cooldown) | 2.424 | 2.437 | 2.409 |\n\n\n\n**Summary of Key Contributions**\n \nWe feel it is unfortunate that most of our contributions that distinguish our work from others seem to have been overlooked in this review, including :\n\n1. *No-Regret\" Strategy*: We introduce stochastic RINS and show that stochastic RINS with linear adapters offer a **no-regret** strategy. This is a major contribution because using our recipe, one can choose not to use recursive inference at test time with no loss in performance while still having the option to use recursive inference to improve performance significantly *for the same set of weights and same training FLOPs*.\n\n2. *Multimodal and Vision Contexts*: To our knowledge, this is the **first** work to systematically study the effectiveness of recursive architectures in both pure vision and vision-language settings. We demonstrate significant gains in SigLIP.\n\n3. *Memory-Efficient Inference*: We study the impact of KV cache sharing and show that even when cache sharing is used to keep memory footprint fixed, RINS still offers performance gains (Figure 6). \n\n4. *Scaling Law Analysis*: We go beyond simple performance reporting by deriving data scaling laws for RINS, showing it improves both the scaling exponent and the irreducible asymptotic loss, proving that its benefits cannot be matched by simply training a baseline model for longer (Figure 5a). We also study the optimal number of recursion rounds against model size and training compute.\n\nWe hope these clarifications have addressed the reviewer's concerns and clarified the novelty and significance of our findings. We will revise the paper to make our contribution clearer and take your concerns into account. We respectfully ask the reviewer to reconsider their evaluation, and we would be happy to answer any remaining questions during the discussion period."}, {"review_id": "o8xWmpcYs6", "reviewer": "Reviewer_VPZK", "summary": "This paper introduces Recursive INference Scaling (RINS), a method for scaling inference time in language and multimodal models by recursively applying a subset of model layers. The paper splits a model into two blocks A and B, where block A is applied recursively r times to its own output before passing to block B (A^r B). Further combining with stochastically dropping and linear adapter, RINS-enabled pretraining improves performance in language modeling even when recursive depth is not applied at inference time. The paper evaluates RINS against 55+ other parameter-sharing strategies under compute-matched conditions and demonstrate consistent improvements in language modeling and multimodal tasks.", "questions": "- Could you explain more the hypothesize about self-similar geometry and how it supports RINS theoretically.\n- Why some results show very large error bars?", "limitations": "See weaknesses.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengthes:\n- The paper proposes a simple, elegant and effective A^r B parameter-sharing method, and systematically studies the scaling law of it, across model size, modalities, tasks, and data size.\n- The comparison with the control of training budget is fair and meaningful.\n- Provide a stochastic RINS which improves performance even when recursion isn't used at inference, making the method more practical feasible.\n- Analyze the KV cache sharing for reduced memory footprint.\n\nWeaknesses:\n- The paper targets the deployment environments with stringent memory limitations, but does not include latency analysis or profiling memory footprint.\n- Evaluations are focused primarily on C4/SlimPajama and perplexity for language modeling.", "was_revised": true, "final_justification": "The paper proposes an elegant yet effective method, and systematically studies the scaling law of it, across model size, modalities, tasks, and data size and shows the benefits of efficiency (memory foot print and FLOPs), downstream quality, practical feasibility.", "rebuttal": "We thank the reviewer for the feedback. We are very pleased that you have found our work elegant and insightful. Please see our response to your questions below. We would be happy to answer any further questions during the discussion period. \n\n**Latency and Memory**\n\nThe actual wall-clock time depends on the type of hardware. To make our results more general, we only compared theoretical FLOPs in the main paper. Nonetheless, we report below training-time statistics on TPUv5:\n\n|Model Size (# params)| Recursive Signature | # examples / core / sec | Peak Memory Usage (GiB) |\n|---|---|---|---|\n|300M| Baseline | 22.7 |  2.60 |\n|| RINS (A$^2$B) | 18.1  |  3.31 |\n|| RINS (A$^3$B) |  13.9 |  3.56 |\n|| RAO (2x) |  12.6 | 4.35  |\n|||||\n|600M| Baseline | 12.0 | 3.92 |\n|| RINS (A$^2$B) | 8.5 | 5.70 |\n|| RINS (A$^3$B) | 6.4 | 6.31 |\n|| RAO (2x) | 6.5 | 7.48 |\n\nNote that RINS with A$^2$B signature has a significantly better latency and memory footprint than RAO and still outperforms it significantly in performance. We will add these results to the main paper for completeness.\n\n**Evaluation**\n\nWhile we do focus on perplexity (PPL) during the big sweep (where we compare 59 parameter-sharing architectures), we also evaluate RINS against popular baselines in downstream tasks. This includes:\n\n* For language models, zero-shot evaluation on common sense reasoning tasks (Table 1).\n* For SigLIP, zero-shot classification, retrieval, and cultural diversity metrics (Tables 2, 3, 5) and multilinguality (Appendix B, Table 4).\n* Analysis of how RINS improves both the scaling exponent and the asymptotic performance limits (Section 5, Figure 5a).\n\nIn addition, we will also include the latency and memory footprint results above in the main paper for completeness. \n\n**Recursive Inference and Self-Similarity**\n\nWe provide more details about it in Appendix D. Informally, self-similarity means that similar patterns exist at multiple scales in language (e.g. across paragraphs vs. across larger scopes). Since deeper layers encode high-level contextual information (which correspond to higher-level representations), we expect that a portion of what the early layers learn can be directly used in higher layers as well. But this is precisely what recursive inference enables. \n\n**Variance in Results**\n\nA few downstream evaluations have a large variance. In BoolQ, for example, if you compare Table 10 in MobileLLM paper, the 350M baseline gets 53.8% while the much smaller 125M baseline gets 57.5%, despite the fact that larger models tend to do better overall. This is most probably due to variance. Interestingly, recursive inference mitigates this noise as seen in Table 1 in our paper.\n\n**Summary**\n\nWe hope these clarifications have addressed the reviewer's questions. We will revise the paper to make our contribution clearer and take your questions into account. If our answers are satisfactory, we respectfully ask the reviewer to update their score, and we would be happy to answer any remaining questions during the discussion period."}, {"review_id": "e2U3so8tlP", "reviewer": "Reviewer_ATEM", "summary": "Recursive Inference Scaling (RINS) work has taken inspiration from the self-similar(fractal) nature of language modality and seeks to exploit scale-invariant decoding for better generalization. The methodology decomposes a model into two sequential blocks (A & B), and then recursively applies first block A multiple times (A^rB structure). This helps in increasing the inference time depth without increasing parameter count. An exhaustive ablation and empirical study have been performed using a compute-matched evaluation framework. RINS consistently outperforms most of the baselines.\nRINS also showcase better scaling exponents and asymptotic limits, demonstrating superior sample efficiency and generalization.\nFurther, the paper introduces stochastic RINS, where recursion depth is randomized during training, and shows that incorporating lightweight linear adapters(<1%) enables no-regret-training. RINS trained models retain improved performance even when recursive inference is skipped at test time. Additionally, KVcache sharing study has been performed, to reduce memory cost while maintaining performance gains. Overall, RINS aims to provide plug and play method for scaling inference in compact models, with benefits validated across both language and multimodal tasks.", "questions": "1. It would be nice to get some theoretical insights on what exactly RINS is benefitting from, or what is the recursive blocks optimizing or learning.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths-\n1. The paper provides a highly empirical study with evaluation on more than 55 architectural variants, multiple model sizes(300M-1B), and tasks. All experiments are compute and parameter matched, ensuring fair comparisons. It is great to see alot of common downstream tasks were covered including but not limited to OpenBookQA, PIQA, 0-shot Imagenet classification.\n2. The authors have presented the method, motivation and results in a well-structured and accessible manner. The recursion signature, degree, adapter use, KVcache sharing provide insights into the trade offs and behavior of RINS. The notation, taxonomy and diagrams aid understanding.\n3. The training details, datasets, FLOPs and failure cases(OOM for some models) are explicitly detailed.\n4. The use case is highly relevant for deployment constrained environments.\n5. The no-regret training setup, that provides training time benefits even without using recursion at test time is an important and practical innovation that lowers adoption cost. \n6. The motivation from fractal hypothesis of language and the idea of recursively applying a subset of model instead of full model seems quite novel and interesting.\n\nWeakness-\n1. While the paper is extremely insightful and provides strong empirical justification, it is not clear what is the deep theoretical framework explaining why RINS work so great. It isn't very clear about what exactly the recursive A block learning or refining over iterations.\n2. While inference compute(FLOPs) is matched, real world wall-time or memory benchmarks which are crucial for deployment are not presented.\n3. Most of the experiments are limited to 300M-1B parameter model, would it provide gains for actual LLM use case models.", "was_revised": true, "final_justification": "The rebuttal addresses most of the weakness and the recursion signature, degree, adapter use, KVcache sharing provide insights into the trade offs and behavior of RINS which has been proved through extensive empirical study.", "rebuttal": "We thank the reviewer for the detailed feedback. We are very pleased that you have found our work insightful. Please see our response to your questions below. We would be happy to answer any further questions during the discussion period.\n\n**Theoretical Analysis**\n\nOur hypothesis for why RINS works well in language is discussed at length in the paper using self-similarity (see for example Appendix D). Besides showing that RINS indeed works for language, we also present another important evidence: vision (Appendix G). Since vision does not exhibit the same self-similar structure as language, we would not expect a language-inspired method to necessarily confer an advantage there, which we show to be the case indeed.\n\nWe do acknowledge that rigorous theoretical proofs would be helpful but it is difficult to prove results theoretically for deep neural networks. The alternative is to treat it as a scientific phenomenon: formulate hypotheses, make predictions based on our hypotheses and test those predictions experimentally, which is what we do in the paper.\n\n**Latency and Memory**\n\nThe actual wall-clock time depends on the type of hardware. To make our results more general, we compared theoretical FLOPs in the main paper. Nonetheless, we report below training-time statistics on TPUv5, which we will add to the paper for completeness.\n\n|Model Size (# params)| Recursive Signature | # examples / core / sec | Peak Memory Usage (GiB) |\n|---|---|---|---|\n|300M| Baseline | 22.7 |  2.60 |\n|| RINS (A$^2$B) | 18.1  |  3.31 |\n|| RINS (A$^3$B) |  13.9 |  3.56 |\n|| RAO (2x) |  12.6 | 4.35  |\n|||||\n|600M| Baseline | 12.0 | 3.92 |\n|| RINS (A$^2$B) | 8.5 | 5.70 |\n|| RINS (A$^3$B) | 6.4 | 6.31 |\n|| RAO (2x) | 6.5 | 7.48 |\n\nNote that RINS with A$^2$B signature has a significantly better latency and memory footprint than RAO and still outperforms it significantly in terms of quality. We will add these results to the main paper.\n\n**Model Size**\n\nAs we argue from our scaling experiments, RINS helps models  with >1B parameters if they are trained on more tokens. \n\nBut to further confirm this, we have trained a 2B parameter language model using either: no recursion (BL), RAO, and RINS with signature AAB. The baseline is pretrained on 200B tokens and models with recursion are trained on fewer tokens to match the total training FLOPs. As expected the baseline is always better than RAO while RINS outperforms the baseline; and the gap continues to increase in favor of RINS. This is in agreement with our scaling law studies. Please see the C4 validation perplexity scores below. \n\nUnfortunately, since we cannot attach images to the rebuttal (as per the Neurips guidelines), we provide a table of results. We hope this is sufficient to demonstrate that RINS works well even with models larger than 1B parameters. We will add this to the supplementary materials of the paper.\n\n| Training FLOPs (tokens x layers) | BL | RAO | RINS (A$^2$B) | \n|---|---|---|---|\n| 150K | 2.558 | 2.593 | 2.559 |\n| 250K | 2.515 | 2.534 | 2.506 | \n| 350K (after cooldown) | 2.424 | 2.437 | 2.409 |\n\n**Summary**\n\nWe hope these clarifications have addressed the reviewer's questions and strengthened our contribution. We will revise the paper to take your questions into account. We would be happy to answer any remaining questions during the discussion period."}, {"review_id": "k4HDj4TSOd", "reviewer": "Reviewer_oAMw", "summary": "The paper proposes Recursive INference Scaling (RINS), a recursive architectural design aimed at improving inference performance in language and multimodal models. RINS recursively applies early parts of the model (Block A) multiple times before forwarding to the later blocks (Block B), with the idea of leveraging the self-similar (fractal) structure of language. The authors benchmark this against other recursive methods like Repeat-All-Over (RAO) and show gains in both language modeling and multimodal tasks under compute-matched settings.", "questions": "- Would RINS still outperform simpler sampling-based ensemble methods if evaluated under equal wall-clock time constraints?\n\n- How do you recommend partitioning a large pretrained model into A and B blocks in real applications—what are the criteria?\n\n- Can you provide evidence that RINS remains beneficial when scaling to models >1B parameters with real deployment constraints?", "limitations": "No — The discussion on societal impact and practical deployment tradeoffs is underdeveloped. Authors should include analysis of runtime cost, carbon footprint, or fairness implications of enabling RINS, especially for multimodal systems used in global-scale platforms.", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n- Compute-Controlled Experiments: The paper controls for training compute, which is commendable and eliminates a common confounding factor in model comparisons.\n\n- Multimodal Evaluation: The application to SigLIP and multimodal benchmarks adds empirical breadth.\n\n- Stochastic RINS Design: Introduces inference flexibility and reduces regret via lightweight adapters, potentially improving deployment flexibility.\n\n\n\n\n**Weaknesses**\n\n***Overhyped Novelty:***\n- The idea of recursion or repeated inference is not new—RAO, latent recurrent thinking, and universal transformers have explored similar ground.\n\n- RINS is presented as a novel and superior alternative, but it essentially tunes recursive patterns and partitions, lacking substantial conceptual innovation.\n\n***Heavily Empirical, Light on Insight:***\n\n- While the experiments are abundant, they mostly confirm what could be expected: more compute during inference can help, especially with training-time regularization.\n\n- There is little theoretical grounding or mechanistic insight into why RINS specifically works better than RAO, aside from the fractal analogy.\n\n***Limited Generalization Beyond Specific Settings:***\n\n- The improvements are only observed in language and vision-language setups. In pure vision tasks, RINS offers no benefit—this suggests the method is brittle and task-specific.\n\n- It is unclear if RINS scales well to larger LLMs (e.g., >7B), especially given the overhead of recursive inference.\n\n***Deployment Realism Unclear:***\n\n- While the paper proposes stochastic variants to mitigate inference costs, real-world deployment often requires strict latency budgets, and recursive inference is not easily compatible with that.\n\n- The actual wall-clock inference latency and memory overhead are not reported, which is crucial for a method claiming \"scalable inference\".\n\n**Lack of Simpler Baseline Comparisons:**\n\n- Comparisons are primarily made against RAO and long-sequence baselines, but not against simpler strategies like output sampling ensembles or basic CoT prompting under same FLOPs.\n\n- It's unclear if the gains come from architectural recursion or simply from deeper, repetitive computation paths.", "was_revised": true, "final_justification": "I maintain the score.", "rebuttal": "We thank the reviewer for the feedback. We hope our clarification below resolves your concerns.\n\n**Novelty**\n\nWe agree that recursion is an established concept, and we have compared against prior work like RAO. In fact, our big sweep covers 59 other parameter sharing architectures. Nevertheless, we believe there is significant novelty in our work:\n\n* A strict control for training compute (FLOPs): This is a critical difference from prior work, as it allows for fair comparisons. For instance, we demonstrate that the gains from the RAO strategy in MobileLLM disappear when a non-recursive baseline is trained longer to match the same compute budget.\n* We believe we are the first to offer a **no-regret** strategy, which uses two ingredients that we propose: (1) stochastic RINS, and (2) linear adapters. This is a major contribution because with our recipe, practitioners do not lose anything by enabling RINS during training.\n* To the best of our knowledge, we are the first to study the effectiveness of recursive architectures in pure vision and vision-language settings. We demonstrate significant gains in SigLIP.\n* We carry out an extensive set of ablations. For example: \n  - we study the impact of KV cache sharing and show that even when cache sharing is used to keep memory footprint fixed, RINS still offers performance gains (Figure 6). \n  - we go beyond simple performance reporting by deriving data scaling laws for RINS, showing it improves both the scaling exponent and the irreducible asymptotic loss, proving that its benefits cannot be matched by simply training a baseline model for longer (Figure 5a).\n  - We also study the optimal number of recursion rounds against model size and training compute.\n\nIn summary we do not just propose a minor variation of recursive inference. We study it rigorously on a fair compute-matched regime, resolve limitations in prior works, and compare RINS against 59 other parameter-sharing architectures. \n\n\n**Mechanistic Insight**\n\nOur hypothesis for why RINS works well in language was discussed at length in the paper using self-similarity (see for example Appendix D). Besides showing that it indeed works for language, we also present another evidence: vision (Appendix G). Since vision does not exhibit the same self-similar structure as language, we would not expect a language-inspired method to necessarily confer an advantage there, which we show to be the case indeed.\n\nWe do acknowledge that rigorous theoretical proofs would be helpful but it is difficult to prove results for deep neural networks. The alternative is to treat it as a scientific phenomenon: formulate hypotheses, make predictions based on our hypotheses and test those predictions experimentally, which is what we do in the paper.  \n\n**Generalization**\n\nLanguage is a **major** domain and introducing methods that improve language models is an important contribution. It would be unfair to require a method inspired by the beautiful self-similar structure in language to also work for another domain that doesn’t have that structure, such as vision. As we said above, the negative results in pure image classification (Appendix G) are not a contradiction but rather supporting evidence for our hypothesis regarding self-similarity.\n\nBesides, we demonstrate the success of RINS (beyond pure language) in multimodal systems where language is a key component. Our experiments with SigLIP-B/16, a vision-language model, show significant improvements across a wide range of tasks, including:\n\n* +2.3% in 0-shot ImageNet accuracy (from 77.3% to 79.6%).\n* Widespread gains in retrieval, cultural diversity, and multilinguality benchmarks (Tables 2, 3, 4, 5).\n\n**Deployment**\n\nThis is an excellent point, and we are happy to provide clarification.\nWe agree that latency is a limiting factor. But, memory footprint is also another major constraint (please see the discussion about why memory is a major bottleneck in Section 1 of the MobileLLM paper). For memory, a significant portion is due to the model parameters and another portion is due to the KV cache. RINS improves performance without increasing the model size. In addition, we show that it helps even when KV cache sharing is used, which keeps the memory footprint fixed.\n\nThe actual wall-clock time depends on the type of hardware. To make our results more general, we compared theoretical FLOPs in the paper. Nonetheless, we report below training-time statistics on TPUv5, which we will add. to the paper for completeness:\n\n|Model Size (# params)| Recursive Signature | # examples / core / sec | Peak Memory Usage (GiB) |\n|---|---|---|---|\n|300M| Baseline | 22.7 |  2.60 |\n|| RINS (A$^2$B) | 18.1  |  3.31 |\n|| RINS (A$^3$B) |  13.9 |  3.56 |\n|| RAO (2x) |  12.6 | 4.35  |\n|||||\n|600M| Baseline | 12.0 | 3.92 |\n|| RINS (A$^2$B) | 8.5 | 5.70 |\n|| RINS (A$^3$B) | 6.4 | 6.31 |\n|| RAO (2x) | 6.5 | 7.48 |\n\nNote that RINS with AAB signature has a significantly better latency and memory footprint than RAO and still outperforms it significantly in terms of quality. We will add these results to the main paper.\n\n**Other Baselines**\n\nAs we mention in the paper, RINS is not competing with other techniques such as CoT and repeated sampling because they can both be implemented together. In addition, CoT is specific to decoder-only architectures, while RINS can be used in encoder-only architectures, such as SigLIP.\n\n**More Inference is Better?**\n\nThe claim that “more inference compute is better” so our findings aren't surprising is not accurate in our opinion. Yes, this often holds when models are trained *on the same number of tokens* but not necessarily when training compute FLOPs is controlled for. As we show in the paper, RAO and recursive inference in pure vision don’t help when we remove those confounding factors (even though we still do more inference compute at test time).\n\n**Partitioning the Model**\n\nAs we discuss in Lines 151-154, we always partition the model depthwise into two equally sized blocks. It is a simple, straightforward rule that does not need further tuning.\n\n**Beyond 1B-parameter Models**\n\nAs we argue from our scaling experiments, RINS helps models  with >1B parameters if they are trained on more tokens. \n\nBut to further confirm this, we have trained a 2B parameter language model using either: no recursion (BL), RAO, and RINS with signature AAB. The baseline is pretrained on 200B tokens and models with recursion are trained on fewer tokens to match the total training FLOPs. As expected the baseline is always better than RAO while RINS outperforms the baseline; and the gap continues to increase in favor of RINS. This is in agreement with our scaling law studies. Please see the C4 validation perplexity scores below. \n\nUnfortunately, since we cannot attach images to the rebuttal (as per the Neurips guidelines), we provide a table of results. We hope this is sufficient to demonstrate that RINS works well even with models larger than 1B parameters. We will add this to the supplementary materials of the paper.\n\n| FLOPs (tokens x layers) | BL | RAO | RINS (A$^2$B) | \n|---|---|---|---|\n| 150K | 2.558 | 2.593 | 2.559 |\n| 250K | 2.515 | 2.534 | 2.506 | \n| 350K (after cooldown) | 2.424 | 2.437 | 2.409 |\n\n\nWe hope these clarifications have addressed the reviewer's concerns and clarified the novelty and significance of our findings. We will revise the paper to make our contribution clearer and take your concerns into account. We respectfully ask the reviewer to reconsider their evaluation, and we would be happy to answer any remaining questions during the discussion period."}]

Author rebuttal:

Meta-review: This paper proposes a framework that recursively applies a subset of a model to scale inference time in LLMs and multimodal LLMs, supported by extensive empirical results. Initial reviews were polarized, and even after rebuttal and discussion, the reviewers did not reach full consensus. Most reviewers appreciated the elegance of the key idea and the strength of the results. One reviewer continued to recommend rejection without providing convincing justification or engaging with the rebuttal despite repeated reminders. After carefully considering the paper, reviews, and discussion, the AC concurs with the majority and recommends acceptance. The authors are encouraged to incorporate the rebuttal clarifications into the camera-ready version.
Decision: Accept (poster)