Title: Input-Time Scaling
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: Input-Time Scaling, train-test co-design, reasong, less is more phenomenon, training with random selection and augmentation, exceptional and unexpected math reasoning performance

Abstract: Current Large Language Models (LLMs) have achieved impressive performance in math and other reasoning tasks. They are usually  post-trained on large-scale carefully curated datasets (data & training scaling) and doing reasoning in test time (inference time scaling). In this work, we present a new scaling paradigm, Input-Time Scaling. It can effectively boost reasoning abilities, and it complements previous scaling methods by putting resources on queries (input time). During training and testing, we utilize meta-knowledge from LLMs to refine inputs with different strategies. During the exploration of this paradigm, we also discover a new phenomenon,  train-test co-design.  It requires us to apply query strategies during training and testing as a whole. Only applying strategies on training or testing would seriously degrade the performance gained. 
Through our experiments, we are surprised to find that seemingly low data quality datasets can perform better. We can get the best performance even by adding irrelevant and out-of-distribution information to the queries, with randomly selected 1k examples from a minimally filtered dataset. These findings contradict the previous widely held inductive bias,  "garbage in, garbage out". It further raises 
concerns about the inductive biases we use during data curation. We show that curating datasets with seemingly high-quality data can potentially limit the performance ceiling. In addition, models trained on more data with similar quality (15k VS 1k) perform worse, the intuition of simply scaling the size should also be carefully inspected. The good news is that our findings are compatible with the Less is More phenomenon. 1K examples are enough to invoke high-level reasoning ability, only replacing the quality requirement potentially with diversity.
With comprehensive experiments on Qwen2.5-32B-Instruct, we show using the simplest form of Input-Time Scaling is able to be comparable or reach SOTA performance among 32B open-source models with AIME24(76.7%) and AIME25(76.7%) pass@1, greedy decoding. A majority vote of three models can further achieve AIME24(76.7%) and AIME25(80%). If we start from DeepSeek-R1-Distill-Qwen-32B, the best result would be the among the SOTA performance of open-source models, 90.0% on AIME24 and 80.0% on AIME25. Our method is extremely simple and clear, without using tedious data\&training pipelines and with little human labor. Such transparency and efficiency are our core contributions. To facilitate reproducibility and further research, we are working on open-source our datasets, data pipelines, evaluation results, and checkpoints.

TL;DR: We introduce the Input-Time Scaling paradigm and the train-test co-design phenomenon. Quality and quantity intuitives may lower the performance. SFT with 1k randomly selected and augmented data can get 90.0% on AIME24 and 80.0% on AIME25 (32B model).