Nemotron-CC-Math: A 133 Billion-Token-Scale High Quality Math Pretraining Dataset
Authors: Rabeeh Karimi mahabadi
Sanjeev Satheesh
Shrimai Prabhumoye
Mostofa Patwary
Mohammad Shoeybi
Bryan Catanzaro
Venue: ICLR 2026 Poster (2026)
Keywords: Mathematical Reasoning
Web-Scale Data Curation
LLM-Based Cleaning
Pretraining datasets
Deduplication
Primary area: datasets and benchmarks
TL;DR: We build Nemotron-CC-Math, a high-quality math corpus from Common Crawl that outperforms prior math corpora, boosting LLM performance on math, code, and reasoning tasks.

Abstract: Pretraining large language models (LLMs) on high-quality, structured data such as mathematics and code substantially enhances reasoning capabilities. However, existing math-focused datasets built from Common Crawl suffer from degraded quality due to brittle extraction heuristics, lossy HTML-to-text conversion, and the failure to reliably preserve mathematical structure. In this work, we intro-
duce Nemotron-CC-Math, a large-scale, high-quality mathematical corpus constructed from Common Crawl using a novel, domain-agnostic pipeline specifically designed for robust scientific text extraction. Unlike previous efforts, our pipeline recovers math across various formats (e.g., MathJax, KaTeX, MathML) by leveraging layout-aware rendering with lynx and a targeted LLM-based cleaning stage. This approach preserves the structural integrity of equations and code blocks while removing boilerplate, standardizing
notation into L A T EX representation, and correcting inconsistencies. We collected a large, high-quality math corpus, namely Nemotron-CC-Math-3+(133B tokens) and Nemotron-CC-Math-4+ (52B tokens). Notably, Nemotron-CC-Math-4+ not only surpasses all prior open math datasets-including Mega-Math, FineMath, and OpenWebMath-but also contains 5.5× more tokens than FineMath-4+, which was previously the highest-quality math pretraining dataset. When used to pretrain a Nemotron-T 8B model, our corpus yields +4.8 to +12.6. 
gains on MATH and +4.6 to +14.3 gains on MBPP+ over strong baselines, while
also improving general-domain performance on MMLU and MMLU-Stem.
We present the first pipeline to reliably extract scientific content—including
math—from noisy web-scale data, yielding measurable gains in math, code, and
general reasoning, and setting a new state of the art among open math pretraining
corpora. To support open-source efforts, we release our code1 and datasets 2
.

Published as a conference paper at ICLR 2026
NEMOTRON-CC-MATH A 133 BILLION-TOKEN- SCALE HIGH QUALITY MATH PRETRAINING DATASET
Rabech Karimi Mahabadi Sanjeev Satheesh', Shhimai Prabhumoye Mostofa Patwary',
Mohammad Shoeybi', Bryan Catanzaro'
'NVIDIA 'Boston University
rkarimi #Karim/nahab@eriiii..ccm
saattseshherviiia.com
ABSTRACT
Pretraining large language models (LLMs)) high-quality, structured data such as mathematics and code substantially enhances reasoning capabilities. However, existing math-focused datasets built from Common Crawl suffer from degraded quality due to brittle extraction heuristics, lossy HTML-tt-ttet conversion, and the failure to reliably preserve mathematical structure. this work, we intro- duce Nemotron CCrrrnn-CC--Mt,,, large-scale, high-quality mathematical corpus con- structed from Common Crawl using a novel, domain-agnostic pipeline specifically designed for robust scientific text extraction.
Unlike previous efforts, our pipeline recovers math across various formats (e.g.. MathJax, KaTeX, MathML) by leveraging layout-aware rendering with lynx and targeted LLM-based cleaning stage. This approach preserves the structural in- tegrity of equations and code blocks while removing boilerplate, standardizing notation into 18TІX representation, and correcting inconsistencies.
We collected a large, high-quality math corpus, namely Nemotron-CC-Matth3 (133B tokens) and Nemotron-CC-Matth CC-Math-4+ not only surpasses all prior open math datasets-including Mega- Math, FineMath, and OeenWebMath-bbu also contains 5.5x more tokens than FineMath-4+, which was previously the highest-quality math pretraining dataset. When used to pretrain Nemotron-T 8B model, our corpus yields +4.8 to +12.6 gains on MATH and +4.6 to +14.3 gains on MBPP+ over strong baselines, while also improving general-domain performance on MMLJ and MMLU-SSent
(52B tokens). Notably, Nemotron-
We present the first pipeline to reliably extract scientific content-including math- from noisy web-scale data, yielding measurable gains in math, code, and general reasoning, and setting new state of the art among open math pretraining corpora. To support open-source efforts, we have released our code and datasets
INTRODUCTION
The rapid advancement of large language models (LLMs) has sparked growing interest in improv-
ing their mathematical reasoning capabilities. Recent studies indicate that pretraining on carefully
curated domain-speciallzed mata-such mathematics (Paster et al. 2024 Han et al. 2024 Wang
let al. 2024b; (Azerbayev et al. 2024) and code (Kocetkov et al.) [202] Lozhkov et al.) 2024 Li
al. 2023) substantially improves domain-specifh accuracy, general knowledge and reasoning
abilities (Muennighoff' et al.] 2203)) Aryabumi et al.] [2024] Lewkowycz et al.] [022] Shao et al.,
2024). This suggests that high-quality mathematical data plays pivotal role only in solving
math problems but also in strengthening broader reasoning skills.
Math capabilities of models like O1 (OpenAl (024) and DeepSeek-R1 (Guo et al. 2025 critically depend on access to large-scale, high-quality mathematical pretraining data. Unfortunately, datasets used in pretraining SOTA models like DeepSeekMath (Shao et al. 2024). Minerva (Lewkowycz
"Rabeeh and Sanjeev the primary authors and contributed equally.
https: httpre///.ttiiaoooooo//o///nnoea
s/math
http://

---

Published as a conference paper at ICLR 2026
Ophar (2222))
Chamistry
Stanittis ( 16%)
Esgremics
LLM
(1 (1.7%%)
50%)
Math URLs (Megemeth, Finemsth, OWN, InfilMM- WebMath)
Tr
</>
HTML
TXT
Mathematics (60. 28%)
Physis 01.2%)
Computer science
Common crow!
(11 99%)
(98 snapshots)
Cleaned text
Deduplication
Decontamination
Filtering Math pages (229 54M Docs)
Lynx processing
Refining
Nomotron-co CC--hthhhhhhhhhhh (101.15M) 150.15M Decs, 133.268 tokens)
Figure 1: Overview of the Nemotron- Cemmotron-CCMaale CC-M construction pipeline. Starting from Common Crawl snapshots, we extract math-related URLs using curated datasets (e.g.. MegaMath, FineMath). After fetching 229.54M wehpages, we render pages through Lynx and apply L.L--based cleaning. qual- ity filtering, and deduplication (see 1 We visualize the topic distribution of our data (Right).
al. 2022) and Qwen-2.5-Math Yang al. 2024 are not publicly released. Meanwhile, open- (OWM) (Paster et al.] (022). FineMath (Allal et al. 2025]. InfiMMWeeMMtth (Han al., (024) and MathPile (Wang et al. 2024b) remain limited in both scale and adelity-largely due to brittle extraction pipelines that degrade content quality and
source alternatives such as
fail to preserve mathematical equations and structure (see Appendix A.3).
While Common Crawl forms primary source for large-scale pretraining (Penedo et al. 2023 Tang ct al. 2024 Su et al. 2025). its value for mathematical pretraining remains underexploited. Existing math-specific extraction pipelines (Paster al.] 2024 Zhou et al., 2025) are not well-suited to fully leverage this resource. In particular, current methods freeueetly fail to detect or accurately extract equations, either omitting them altogether corrupting their structure (Han al.) [024] Allal al., 2025). This severely compromises content fidelity. Mathematical notation on the web appears in a wide range of formats-including including MathML, ЦЯT₃,, and dynamically rendered scripts-whor rep- resentations continue to evolve over time (see Figure Compounding this challenge, HTML pages in Common Crawl often lack associated stylesheets and JavaScript resources, preventing proper ren- dering and further obstructing reliable equation recovery. These limitations collectively hinder the construction of high-quality mathematical pretraining corpora that capture the breadth and variety of real-world mathematical content.
To bridge this gap, we propose modular, scalable, and domain-agnostic framework for reliably ex- tracting mathematically rich content from raw web data, enabling the construction of a large-scale, diverse, and high-fidelity math corpus. Our multi-stage extraction and filtering pipeline ensures quality at scale (see Figure IT. In the first stage, HTMI documents are rendered into text using the Lynx text-based browsef which preserves mathematical equations and symbols with high accu- racy. In the second stage, lightweight LLM normalizes heterogeneous math representations into ETTXX while discarding boilerplate and irrelevant content. This LLM-based approach allows us to avoid the brittle, heuristic-basee based rules employed in previous pipelines (Paster al., 2024), resulting in more reliable and consistent extraction of mathematical content. Subsequently, we apply a qual- ity classifier retain the high-quality pages, followed by deduplication eliminate redundancy. In addition, we perform thorough contamination detection against downstream benchmarks (see $2.4). ensuring that any overlapping or duplicated samples are identified and removed from the corpus.
By leveraging the scale of Common Crawl and the rigor of our pipeline, we present Nemotron-CC- Math-the highest quality open-source math corpus to date, comprising of 133B tokens, where its highest quality subset (Nemotron-CCs -Coorroo--C-MM--444 totals 53B tokens. Our pipeline is optimized for performance using Polars and Ray, enabling us to process terabytes HTML content efficiently. To facilitate future research, we release both the dataset and our full pipeline implementation.
Our contributions are as follows:
We reviewed prior data extraction pipelines, and show that they f'il to accurately extract math and code content, often stripping math equations and code snippets (Appendix A.3).
We introduce scalable and modular pipeline for extracting high-quality mathematical content from Common Crawl, explicitly addressing the longstanding challenge of HTML math variabbliny--includnn LaTeX, MathML, Unicode, and inline malformed equations.
https://lynx. invi
2

---

Published as a conference paper at ICLR 2026
LaTeX Delimiters:
Matholi Tags:
[iatod] [ixpae)
[MA egemantics class="M.IX- TeXAton- ORD's emstyle ballyystyll="traa scriptlevel: -miff//mi cmw:(//mo-mixxr///e/ http://///mmmm////// smpre/me ensupe
[eex]] (20maa)2)
(| Rxj x*22 {
/mbp snrow oless MJX- Tex--eXtoommOOe ORDD c2/22/mm </trou </msup d/mstyli </mrowe camatation encoding= application/x--txx [idioplnystyle [(x)=x([2)
M =60)=xt2)))
(m
Pre Togs:
</onmatation /ssemnntics MATH] (tdisplyystyle f(4)xx (2))
LLM
Latex Format
soee xmi lntex"s lntex'>> (20xpx(2)/ppr>
mg arc= httpo///////// www.coog..../////oooooooooo wttp://w....ooooooooooo////00000 (ngs7((0=xx(()) dpxex (2)7
Image Tags:
Figure 2: Mathematical expressions HTML pages appear in diverse formats HammatsLaTeeX LaTeX within cus- tom delimiters, <pre> blocks, image tags, and MathML. These variations challenge standard text extraction pipelines, which often fail to recover the underlying 18T₃X equations correctly. To address this, we use an LLM to standardize all mathematical representations into a unified ITTEX format.
To our knowledge, this is the first work to employ the text-based browser Lynx for HTMM--to-plinn-eex conversion with preservation of math and code formatting, and to in- troduce LLM-based standardization of mathematical representations across the content.
We will release math-rich web documents extracted from Common Crawl, whose 4+ subset contains 5.5 more tokens than FineMath-4+, the previous best math pretraining dataset.
CC-Math, a dataset of 133B tokens of high-quality and diverse
We will open-source our full pipeline (extraction, processing and scoring) to ensure repro- ducibility and support its application to other domains.
We thoroughly analyze Mimotron-CC-Mate by examining its composition, including statistics on webpage types, subject areas, and the most frequent source domains.
Through extensive experiments and detailed quality analysis ($3.)) we demonstrate that dataset outperform those trained existing pretraining math
models pretrained datasets across range of benchmarks, including math, code, and general knowledge tasks.
2 THE COLLECTION OF NEMOTRON-CC-MI
We construct the Wemotron-CC-MA corpus from Common Craw large-scale web archive ex- tensively used in recent LLM training (Dubey et al., 2024 Hui et al. 2024 DeepSeek-Al et al. 2025). Common Crawl contains over 300B documents across more than 6.M WARC files (each contains over 1GB of compressed content). Our goal is to build pipeline that can process the tech- nical content from Common Crawl correctly. We apply our pipeline to math domain to assemble high-quality, large-scale corpus of mathematical content from Common Crawl. To achieve this, we designed robust and highly scalable data processing pipeline capable of operating at the full scale of Common Crawl, illustrated in Figure
Prior efforts such as OWM (Paseer et al. 2024) and DeepSeekMath (Shao et al. 2024) rely on lightweight classifiers to identify technical pages. We initially explored a similar approach but found fundamental limitations in achieving meaningful improvements: first, mathematical content constitutes 1% of Common Crawl, making manual ground truth annotation extremely difficult; second, since classifiers must run on all Common Crawl documents, only very efficient methods like FastText with simplified HTML parsing are viable. This creates inherently high-bias setup with no straightforward path to improvement-attoone ably lead drastic drops precision. Rather than refining such classifiers for marginal gains, leverage community-flletn filtered datasets: extracting URLs from OWM, InfiMM-Weh
to increase recall for technical content invari-
(Han et al., 2024]. FineMath (Allal et al., 2025), and MegaMath (Zhou al.] (025). including all major sub- sets. This approach allows us to benefit from the diverse filtering strategies employed by different research groups while avoiding the limitations of any single classifier.
We then retrieve the original HTML from 98 Common Crawl snapshots (2014 2014-2024) for these URLss, cnabling ine-grained extraction that preserves mathematical expressions, symbols, and formatting- often degraded in prior processing (Appendix A.3). This process yields 229. 229.54M high-quality web- pages spanning a diverse range mathematical content.
https:: https://ccmmoncraa// .org/
3

---

Published as a conference paper at ICLR 2026
2.1
RELIABLE TEXT EXTRACTION FOR SCIENTIFIC CONTENT
2.1.1 LIMITATIONS OF PRIOR WORK
Extracting mathematical content from raw HTML presents a significant challenge for text extraction pipelines. Unlike natural language, which often follows consistent structural patterns, math equa- tions appear in highly variable forms across the web (see Figure(2). These variations stem from the absence of standardized conventions for embedding math in HTML, as well as the diversity of ren- dering engines (e.g., MathJax, KaTeX, MathML, image-based representations, and custom plugins). Moreover, websites frequently evolve their rendering strategies, making any fixed set of heuristics fragile in practice. As result, existing extraction pipelines often fail to reliably extract scientific content, with equations either missed entirely. mis-parsed, or distorted. Preserving formatting is equally important: the indentation and layout of code blocks and the placement of mathematical symbols often carry semantic meaning, and losing this structure severely degrades the value of the extracted content for downstream modeling.
Existing content extraction tools such as JUSTEXT (Endredy & Novák 2013). TRAFILATURA (Bar-
baresi 2021). and RESILIPARSE (Bevendorff et al. 2018) used in large-scale dataset construction
pipelines including The Pile (Gao et al.] 2020), FineMath (Allal et al.] 2025). and RefinedWeb
(Penedo et al. 2023) -were designed primarily for general-purpose boilerplate removal and narra- tive text extraction. While effective for general documents, they often strip or corrupt equations, miss inline 18Tς equations changing semantics, and flatten (or miss) code blocks requiring strict indentation (e.g., Python). These shortcomings limit their usability for building high-quality math or code datasets. Examples are provided in Appendix A.3
2.1.2 OUR TEXT EXTRACTION PIPELINE
The diversity of mathematical representations on the web necessitates using large language models to faithfully convert technical HTML content into format suitable for LLM pretraining. Since raw HTML from WARC files is too verbose for direct LLM processing, and traditional parsers risk losing critical information, we employ lynx, a text-based browser that renders web pages into plain text while preserving mathematical equations and code formatting. Unlike DOM- parsers used in prior work (Paster et al. 2024; Zhou et al. 20255] Allal et al. 2025). lynx executes HTML layout rules to produce output that mirrors the human peeneivee page structure, reliably capturing equations and maintaining code indentation.
While yynx preserves the structural layout of web pages, its output includes boilerplate elements such as navigation bars and redundant headers. To refine this output, we apply an LLM cleanup pass using Phi-4 (Abdin et al. 2024a) (1BB parameters), which preserves primary content and references while removing non-essential content. LLM additionally standardizes mathematical expressions into consistent 18TEX format (Figure and corrects typographical errors. This two-stage pipeline- structural preservation via Lynx followed by semantic refinement via an LLM-yields high-quality, coherent text suitable for large-scale mathematical corpora. Ablation studies ($3.2) show that this cleanup task is simple enough for smaller models to perform effectively. Qualitative comparisons with prior work and the full cleanup prompt are provided in Appendices and A.5 respectively.
2.2 QUALITY CLASSIFICATION
To support the later stages of training where data fidelity is especially important (Hu et al. 2024 Ab-
din al., 2024b), we further filtered our Demotron-CC-Mann to retain only its gighest-scoring sub- set, Commotro-CCC-Mtth-ee
We employed the FineMath classifier (Allal et al. (025) which assigns a 5-point scale score to each page, focusing on identifying content with mathematical reasoning and material suited to middle- and high-school levels. The data distribution for each quality score is provided in Appendix $A.7 7 After classification, we also performed deduplicatication and decon- tamination (sce $2.3 and $2.4). We developed two variants of Nemotron-CC-Matte CC-Math: Nemotron-CC Math-4+ (52.32B tokens, 45M documents) with scores 4-5 and Nemotron- Secootron-CC-MMe CC-Math- 3+ (133.26B tokens, 101M documents) with scores 3-5.
4

---

Published as a conference paper at ICLR 2026
Table 1: Comparison of Nemotron- -C-ooon-CC---aht with math pretraining datasets. Nemotron-CC-Me 4+ is 5.5x larger than the highest-quality open math dataset (FineMath-4+) with a permissive li- cense, and substantially outperforms FineMath across math, code, and knowledge tasks (Table
Open Source
Documents
#Tokens
Dataset
(M)
Source
Minerva Lewkowycz 20122
X
arXiv,
MathMix Lighiman 2025
X
1,50
Unknown
DeepSeckMaln (Slião et al. 2024
X
120
CommonCraw]
ProofPile Azerbayev of 2025
2.04
8,30
arXiv, Textbooks, Formal Math Libraries, StackExchange, ProofWiki. MATH OpenWebMath. ArXiv, AlgebraicStuck Khan Academy, Synthetic data arXiv, Textbooks, ProotWiki, Wikipedia,
ProofPile-2 2 (Azerbuyev a. 2024
11.20
55
AMPS (Hendrycks et al. 10
5.10
0,70
MuthPile Wang et al. 2024b
0.73
9.50
StackExchange, CommonCrawl
Math (Paster al. 2024
6.30
14.70
CommonCraw
InfiMM-WebMalh-- (Han et 2024
6.30
8,50
CommonCraww
FineMath-4+ Allal et il. 2025
6.70
9.60
CommonCraaw
MegaMath- Pro Zioure
15
15.10
CommonCrawl
Nemotmo- CC- Math-4+ (Ours)
52.32
CommonCrawi
InfiMM-Weh Mathh 3+ et 2024
CommonCraww
FineMath- 3+ (Allal et at, 2025
/
21.40
34
CommonCrawl
MegaMath- Weo Zhoure 2025
/
106.50
263.90
CommonCrawl
Nemotron- CC- Math-3+ (Ours)
/
101.15
133.26
CommonCrawl
2.3 FUzZY DEDUPLICATION
Removing near-duplicate documents is essential for efficient and stable model training, and reduc- ing the risk of memorization (Lee et al. 2022, Tokpanov et al. 2024). We applied fuzzy deduplica- tion using the NeMo-Curator framework, which uses a Min lash-based Locality Sensitive Hashing (LSH) (Broder 2000) to efficiently detect duplicates. The probability that two documents with Jac- card similarity hash to the same bucket P = 1- (1 5*) where b is the number of hash functions per band and is the number of bands. With r-20 bands and b=13 hash functions per band, our setup targets a Jaceard similarity threshold of 0.8. Pairwise similarity is computed using 24-character n-grams, and LSH uses concurrent shuffling of five bands identify duplicates.
2.4 DECONTAMINATION
The source documents used in Nemotron-CC-Mo are from mostly pre-decontaminated datasets. However, we follow a more thorough decontamination procedure as outlined in Yang et al. (2023) We embed all the documents in Nemotron-CC-Ne
using the Qwen2.5B 32B model (Qwen al.
2025) as well as all the prompts and answers from our evaluation benchmarks: MMLU (Hendrycks) al. 2021a). MMLU-Pro (Wang et al.] 2024a). MATH (Hendrycks
al. 20211), and
GSM8K (Cobbe et al., 2021). We remove all documents with a cosine similarity above 0.9 to any benchmark prompt answer, resulting in the removal of less than 0.002% of all documents.
3 EXPERIMENTS
Datasets We compare Nemotron-CC-Male to existing prior math pretraining datasets, including Megamath, OWM, and FineMath. Table summarizes the dataset statistics.
Experimental Setup Math and code abilities generally arise only after extensive training; follow- OLMo
ing Blakeney et al. (2024); Dubey et al.) (2LM et al. (2024); Allal et al. (2025) to estimate the quality of different math pretraining datasets, we run annealing ablations on a mid training checkpoint of Nemotron-T 8B model (NVIDIA et al. 2025). The base model was pretrained on 9T tokens using a mixture general-domain and math-focused corpora (see Appendix A.8] for a detailed breakdown). In each ablation, the target math dataset is upweighted to constitute 30% of the total data blend, while the weights of all other datasets are correspondingly downweighted to make up the remaining 70%. This controlled adjustment isolates the contribution of the math data while preserving overall blend composition (See Appendix for hyper- parameters). We consider two controlled ablations:
5

---

Published as a conference paper at ICLR 2026
100B Token Ablation: This setting targets compact, high-quality math datasets, typically below 30B tokens. For each run, the mathematical portion of the blend replaced with a single candidate dataset-such as FineMath-4+, MegaMath- -tro-Proo.. OWM-enabling direct comparison with Nemotron-CCe CCrroon--C-MMah-44+ The modified blends are trained for 100B tokens evaluate performance under consistent compute budget.
To fairly assess larger math datasets, including FineMath-3+ and MegaMath-Wais we apply the same replacement and proportional adjustment strategy but extend the total annealing budget to 300B tokens. This configuration also tests whether increased scale can offset dataset quality differences.
300B Token Ablation:
Benchmarks We evaluate model performance across a diverse suite of benchmarks spanning knowledge understanding, code, and mathematical reasoning tasks. Knowledge understanding is assessed using MMLU datasets, including MMLU-Pro (Wang et al.] (022a)) MMI.U, and MMLU- STEM (Hendrycks et al. 2021a) with results reported exact match (EM) accuracy. Code gener- ation quality is measured on four tasks-MBPP (Austin et al. 2021). and HumanEval (Chen et al.) 2021] and their EvalPlus variants, HumanEval+ and MBPP+ (Liu al. 2023). For code tasks, fol- lowing Guo et al. (2025). to improve the stability. we report the avg02 which reports the average accuracy from generating 20 samples for each prompt. To produce these samples, we apply nucleus sampling with a temperature of 0.6 and a top-p value of 0.95. Mathematical reasoning is evalu- ated on the GMS8K (Cobbe et al., (02T] and MATH (Hendrycks et al.) 2021b) benchmarks, with greedy decoding and using Math- Verify for symbolic matching. We run evaluations of all models ourselves using Am-evaluat on harness
3.1 PRETRAINING EXPERIMENTS RESULTS
Results Tables cantess Nemotron-T 8B models pretrained with different math datasets 100B and 300B tokens, respectively. Across all benchmarks, models using the curated Nemotron-CC. Math consistently match or outperform competing datasets including OWM, MegaMath, and FineMath-on knowledge, code, and math tasks.
At 100B tokens, Cemotron-CC-Mathh-ee achieves the top results in every math task-e.g.. 40.6 on MATH (+4.8 vs. FineMath-4+, +6.6 vs. MegaMath-Pro) and 76.27 on GMS8K (+0.3 vs. FineMath- 4t, +4.85 vs. OWM). It also leads most code benchmarks (c.g., 34.82 on HumanEval+, +2.3 vs. OWM) and all knowledge tasks (e.g., 38.49 on MMLU-Pro, +2.1 vs. MegaMath-Proo.
At 300B tokens, Cemotron-CCC-Math-33 extends these gains-reaching 44.2 on MATH (+9.6 vs. FineMath-3+, +12.6 vs. MegaMath- Webb and 80.06 GMSSK (+0.6 FineMath-3+, +3.6 OWM). Code scores also improve substantially, with 37.16 on HumanEval+ (+3.0 vs. FineMath- 3+) and 43.51 MBPP+ (+4.6 MegaMatth-War + 14.32 Finemath-3+). Knowledge remains best or near-best across MMLU variants, with a top score of 64.26 on MMLU-STEM.
Although do not explicitly target the code domain, it is noteworthy that the curated Nemotron- CC-Math datasets substantially improve code performance. Upon analysis, we find that Nemotron- contain approximately 4.3M and 1.44M samples with code s In contrast to prior datasets, which often fail to capture code content, our curation pipeline retains code snippets in full, preserving syntax and structure. We attribute the observed code improvements to this incidental yet high-quality code data. Overall, results show that high-quality curated math data in pretraining boosts performance in math reasoning, code, and general knowl- edge. Comparing 100B and 300B token results, gains scale with more pretraining. This highlights the value of high-quality math data for improving LLMs across specialized and general domains.
CC-Math-3+ and Nemotron-CC--atthe
3.2 ABLATION ON MODEL CHOICE
To ablate the model choice for the task of boilerplate removal from rendered web pages, we sampled 7M documents and evaluated several instruction-tuned LLMs including DeepSeck-V3 (Liu et al.
https: wtt////ttttt com/ /1 he /1mm eval vat on ao--aarne harness
We filter examples enclosed within triple backticks indicating code block (e.g., "'yytoon... ... ").

---

Published as a conference paper at ICLR 2026
Models Trained on 100B Tokens
MegaMath FineMath
Nemotron-CC-MMn
Renchmark (Metrie)
OWM
(Pro)
(4+)
(4+)
MMI.UPrr (FM)
35.49
36.41
36.74
38.49
Knowledge MMLU (EM)
65.62
66.81
66.73
67.55
MMLU-Stem (EM)
58.83
60.86
61.62
62.67
HumanEval+ (10g0220)
32.53
31.01
32.16
34.82
MBPP+ (avger
43.76
46,03
28.88
45.11
Code
MBPP (avg@220)
53.11
52.51
53.42
53.48
HumanEval (mvg@200)
37.07
35.91
37.77
38.93
MATH (EM)
29.20
34.00
35.80
40.60
Math
GMS8K (EM)
71.42
73.46
75.97
76.27
Models Trained on 300B Tokens
MegaMath
FineMath (3+)
Mamotron-CC-Marn
Benchmark (Metric)
OWM
(Web)
(3+)
MMLU-Pro (FM)
35.00
36.33
39.57
39.32
Knowledge MMLU (EM)
65.20
65.44
67.92
68.20
MMLU-Stem (EM)
59.20
59.88
62.29
64.26
HvmanEval+ (arge220)
33.54
32.29
34.18
37.16
37.59
38.89
29.19
43,51
Code
MBPP (2vg@20)
52.22
53.05
57.57
56.15
HumanEvalooze200
38.32
36.34
37.80
40.30
MATH (EM)
34.20
31.60
34.60
44.20
Math
GMS8K (EM)
76.42
78.24
79.45
80.06
Table 2: Evaluation results for models trained with different math datasets using either 100B or 300B tokens. NEMOTRON-CC-ME OpenWebMatte MegaMath, and FineMath baselines across knowledge, code, and math tasks. Math performance improves with longer token horizon, showing Nemotron-CC-Mate continues to scale effectively with increased training. Code results use average accuracy over 20 generations; all other tasks use exact match (EM). Bold indicates the best result in each row.
variants consistently outperform obtain comparable results to
2024), Owen2.5-32BB, 32t//ttttts Qwenn..5-7BBInssruut (Team 2024); and Phi-4 across knowledge, coding, and math benchmarks.
Table presents the results. Surprisingly, despite significantly smaller size (14B parameters), Phi-4 performs competitively across all domains, often matching exceeding the results of much larger models such as DeepSeek-V3 V3 (671B) and Qwen2.5-72B-Inse
(72B). In particular, Phi-4 achieves the best performance on math tasks (e.g.. 79.98 EM on GMS8K and 40.6 EM on MATH) and leads or matches the performance in several code-related benchmarks.
Given the marginal differences in performance and the substantial gap in computational cost, selected Phi-4 as the default model for all experiments in this paper. Our findings indicate that the task of webpage boilerplate removal does require excessively large models, and smaller instruction-tui models can yield efficient and effective results.
3.3 LLM-AIDED QUALITY ASSESSMENT SCIENTIFIC CONTENT FIDELITY
We first performed an overlap analysis across OWM, MegaMath-Pro FineMath, and our Nemotron- CC-Math datasets, identifying 97,788 shared samples. As this joint subset is far too small for mean- ingful side-by-side pretraining experiments, quality assessment. To complement standard benchmark results, we conduct an LLM-aided qual- ity assessment directly measure how effectively Common-Crawl-deen preserve scientific content during extraction and cleaning. This assessment focuses on four critical dimensions essential for high-quality mathematical pretraining corpora:
use it as rigorous, shared basis for comparative
mathematical datasets

---

Published as a conference paper at ICLR 2026
3.5
MagaMathh-Pi
Nemotron-CC-Matheee
FineMath-4+
3.0
2.07
2.90
2.5
12.55
2.53
2:28
2.30
2.44
2.79
2.23
Arrorage Scare
2.0
1..85
1.5
0.0
Marth
Codee
Polthrulnees
Readsbility
Metric
Figure 3: LLM-Aided quality assessment results comparing the cleaned webpage quality across four datasets: Ours (Nemotron-CC-Mathh, OWM, FineMath, and MegaMath. The LLM judge scored 100 randomly sampled documents from each dataset on four dimensions. Our method achieves the strongest performance in preserving mathematical expressions and code structure, with competitive faithfulness and the highest readability. These results highlight the effectiveness the Lynx > LLM pipeline in retaining scientific structure while improving textual clarity.
Math Preservation (0-3 or N/A): Correctness and completeness of mathematical expres- sions unified to BTEX.
Code Preservation (0-3 or N/A): Structural and semantic integrity, syntax, and functional behavior preserved code blocks.
Faithfulness (0-3): Preservation of core scientific content integrity without omission or meaning alteration.
Readability (0-3): Overall clarity, organization, coherence, and textual flow of the output document.
Methodology We randomly sampled 100 webpages from the shared subset. We employed OpenAI gpt-5.1 1 as automated judge assess the conversion quality. The judge provided with the original raw text (extracted via Lynx) ) and the converted document from each dataset, guided by a detailed scoring rubric and evaluation instructions (see Appendix A.6). We report the mean score for each dataset across the 100 samples and four dimensions.
Results Figure summarizes the assessment results. Nemotron-CCo -Crtoon-CC-MMthhhh demonstrates superior fidelity in preserving the underlying scientific structure, achieving the highest scores both Math and Code Preservation:
Benchmark (Metric)
DeepSeek-V3 Qwen2.5-32B Qwen2.5-728 Phi-4
MMLU-Pre Prr (EM)
38,82
39.65
39.65
38.40
MMLU (EM)
67.68
67.01
67.73
67. 67.54
Knowledge MMLU-Ster
62.96
61.88
62.73
63.24
HumanEval+ (2)g@20)
28.54
28.35
29,63
30.40
MBPP+ (avg@220)
45.99
38.58
41.34
41.88
Code
MBPP (avg)r20)
53.91
$3.83
54.09
55.39
HumanEvar (1gg0200)
32.10
31.92
35,21
34.09
MATH (EM)
36.60
38.80
38.60
40.60
Math
GMSSK (EM)
75.51
74.00
73.92
79.98
Table 3: Model choice ablation. We compare DeepSeek-V3 V3 (671B), Qwen2.5-32B/72BB and Phi-4 (14B) across benchmarks. Despite its smaller size, Phi-4 performs competitively-on -otenn leading in math demonstrating smaller models can efficiently clean webpages without losing performance.
8

---

Published as a conference paper at ICLR 2026
#Documents
Document
#Characters (B)
Characters
Domain
Domain
(M)
%
mathhelpforum.com
8.54
8.44
mathhclpforum.ccm
17.11
3.67
jiskha.com
5.26
jiskha.com
12.52
2.69
physiesforums.com
2.82
2.78
mathforum.ow
8.96
1.92
math wtackexchange.ccm..
2.38
2.35
morsicsforums.cco com
8.19
1.76
mathforum.org
2.38
2.35
math wwrchhtttcccchgee.coom
6,96
1.49
openstudy.com
1.88
1.86
mathoverflow.net
6.78
forums.
1.51
1.49
wrichmmahs.oog
6.24
1.34
mathoverflow.nch
1.33
1.32
scribd.com
4.47
0.96
nrich.maths.org
1.13
1.12
educator.com
3.58
0.77
mathisfunforum.comm
0.76
0.75
forums, wolfram.com
3.35
0.72
coursehero.cov
0.75
docplayer.net
3.20
brilliant.org
0.68
0.67
wn.wiippddi..om
3.07
0.66
gmatclub.com
0.65
0.65
openstudy.com
3.10
0.66
chegg.com
0.58
0.57
gmatcluh.com
2.73
0.59
gradesaver.com
0.54
0.53
mathisfunforum.
2.72
0.58
socratic.org
0.49
0.49
voursehero. com
0,50
purplemath.com
0.45
0.44
slideplayer.comg hindowi.com softmath.com com archive.org
2,06
0.44
srssiisstaacexxcanng..com
0.44
0.43
1.98
0,42
bellerlesson.com
0.41
0.41
1.95
0.42
brainmass.com
0,40
0.40
1.96
0.42
Table 4: Comparison of Most Common Domains by Document (left) and Character Count (right).
Math Preservation (2.55): Our score significantly surpasses all baselines (OWM: 2.28, FineMath: 2.31, MegaMath: 1.85). This validates that our combined Lynx-based extrac- tion and L.LM-driven и8パд normalization more reliably preserves mathematical content than prior heuristic ML-totext pipelines.
Code Preservation (3.00): Nemotron-CC -C-oon--CCMMiti achieved the maximum score, substan- tially outperforming all competitors (OWM/FineMath: 2.50, MegaMath: 1.75). This firms the effectiveness of the lynx-based approach in retaining code formatting and in- dentation, often lost by DOM-based extractors.
Faithfulness (2.44) and Readability (2.90): Although faithfulness score is marginally lower than FineMath (2.63) and OWM (2.59)-a a consequence of the intentional LLM- based cleanup that may involve compressing rewriting contextual details Nemotron- CC-Math achieved the highest readability. This demonstrates that our pipeline successfully trades minimal loss in literal text preservation for superior textual coherence and clarity.
In conclusion, these results confirm that Nemotron-CC-Matte offers the optimal balance between structural fidelity and textual clarity. By moving beyond brittle heuristic pipelines, our Lynx LLM approach captures essential scientific content and structure while yielding coherent, high- quality text, establishing Ceemotron-C-Maile as substantially higher-quality scientific corpus com- pared existing Caaunon-Crawl-baaee math pretrainig datasets.
3.4 DATASET ANALYSIS
Data Composition We measured domain distribution by document and character count. Table shows the top twenty domains by each metric. Similar to prior work (Paster et al. 2024), the most common sources are discussion forums, Q&A sites, and educational resources. Overall, the dataset spans 980,922 unique domains, with the top 100 domains accounting for 36.46% % of characters and 43.22% of documents.
Topic Distribution To characterize the dataset, we randomly sampled 150,000 documents and classified them into mathematics, physics, statisties, chemistry, economics, computer science, or other using the Boem3-30B-AA3--Intttueee
model (see Appendix A.2 for the prompt). Figure I shows the results. Most documents pertain mathematics, with the remainder distributed across the other scientific domains; approximately 4.12% fall outside these categories.
RELATED WORKS
High-quality math pretraining datasets are essential for improving LLM reasoning. OWM compiles 14.7B tokens from Common Crawl but relies brittle heuristics and Resiliparse for HTML ren- dering, often stripping or corrupting formulas and code. FineMath inherits these issues, building its 54B-token corpus using the OWM pipeline. Similarly, MegaMath faces similar challenges.

---

Published as a conference paper at ICLR 2026
MathPile (Wang et al.) 2024b) aggregates 9.5B tokens from sourees including arXiv (85%), text- books, and forums but much of the content remains in raw LTEX form, limiting usability for LLM pretraining. InfiMM-Web-MMe
(Han et all 2024) is 40B tokens multimodal dataset pairing im- ages with math text. Proof-Pile (Azerbayev et al. 2023) is a 8.3B-token dataset collected from various sources such arXiv, formal math libraries, Wikipedia and Stack Exchange. Proof-Pile- 2 (Azerbayev et al., (2024) is a 55B-token dataset combining arXiv, OWM, and Algebraic-Staed mathematical code. Additionally, auxiliary datasets include AMPS (Hendrycks 20.I)) with Khan Academy problems and Mathematica--enne generated content, and NaturalProof's (Welleck et al.
2021 covering theorems and proofs from ProofWiki and the Stacks project.
Proprictary datasets like WebMath (OpenAI) (Polu & Sutskever (2020), MathMix (Lightman et al.
2023), DeepSeekMath (Shao et al., 2024), and Minerva's Math Web Pages (Lewkowyez al. 2022) advance math reasoning but lack public access, limiting transparency. We release Nemotron-CC Math openly to foster community progress.
5 CONCLUSION
We present scalable, domain-agnostic pipeline for extracting high-quality technical content from Common Crawl, focusing here on the mathematical domain. By integrating robust HTML-to-tex conversion with LLM-based domain- aware cleaning, our approach addresses longstanding chal- lenges in web-scale extraction of structured technical data.
Applied to the math domain, our pipeline produced Cemotron-CC-Mann whose 4+ subset is 5.5 larger than the previous highest-quality math set, FineMath-4+. Pretraining on Nemotron-CC- Matt improves math reasoning (+ 12.6 MATH), code generation (+14.3 MBPP+), and general knowledge (+5.1 MMLU-Stem), outperforming prior math datasets.
Importantly, the modular, domain-agnostic design enables application to other technical fields. As LLMs advance, our pipeline offers a crucial tool for generating targeted, high-quality pretraining data to drive model capabilities.
ACKNOWLEDGMENTS
We would like to thank Markus Kliegl for the original suggestion to explore text based browsers, and Ying Lin for their helpful comments and suggestions.
REFERENCES
Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett Mojan Javaheripi, Piero Kauffmann, et al. Phi-4 techni- cal report. arXiv preprini 24Xi::242.008005 2024a.
Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat S. Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Caio César Teodoro Mendes, Weizhu Chen, Vishrav Chaudhary, Parul Chopra, Allie Del Giorno, Gustavo de Rosa, Matthew Dixon, Ro- nen Eldan, Dan Iter, Amit Garg, Abhishek Goswami, Suriya Gunasekar, Emman Haider, Junheng Hao, Russell J. Hewett, Jamie Huynh, Mojan Javaheripi, Xin Jin, Piero Kauffmann, Nikos Karam- patziakis, Dongwoo Kim, Mahoud Khademi, Lev Kurilenko, James R. Lee, Yin Tat Lec, Yuanzhi Li, Chen Liang, Weishung Liu, Eric Lin, Zeqi Lin, Piyush Madan, Arindam Mitra, Hardik Modi, Anh Nguyen, Brandon Norick, Barun Patra, Daniel Herez-Becken Thomas Portet, Reid Pryzant, Heyang Qin, Marko Radmilac, Corby Rosset, Sambudha Roy, Olatunji Ruwase, Olli Saarikivi, Amin Saied, Adil Salim, Michael Santacroce, Shital Shah, Ning Shang, Hiteshi Sharma, Xia Song, Masahiro Tanaka, Xin Wang, Rachel Ward, Guanhua Wang, Philipp Witte, Michael Wyatt, Can Xu, Jiahang Xu. Sonali Yadav, Fan Yang, Ziyi Yang. Donghan Yu, Chengruidong Zhang, Cyril Zhang, Jianwen Zhang, Li Lyna Zhang, Yi Zhang, Yue Zhang, Yunan Zhang, and Xiren Zhou. Phi-3 technical report: A highly capable language model locally on your phone. CoRR, abs/2404.1 14219, 2024b. URL (ntt/:ttts://eec htta:://dooooor////// 48550/ arxiv. 2404. 14219
Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, Gabriel Martín Blázquez, Guilherme Penedo, Lewis Tunstall, Andrés Marafioti, Hynek Kydliček, Agustín Piqueres Lajarín, Vaibhay Srivastav,
10

---

Published as a conference paper at ICLR 2026
Joshua Lochner, Caleb Fahlgren, Xuan-Son Nguyen, Clémentine Fourrier, Ben Burtenshaw, Hugo Larcher, Haojun Zhao, Cyril Zakka, Mathieu Morlon, Colin Raffel, Leandro von Werra, and Thomas Wolf. Smollm2: When smol goes big data-centric training of small language model, 2025.
absp.bs///aaaaoo.oooooo.....ssar hbtpr///aaa///2/...22/// 2502. 02737
Viraat Aryabumi, Yixuan Su, Raymond Ma, Adrien Morisot, Ivan Zhang, Acyr Locatelli, Marzieh Fadaee, Ahmet Üstün, and Sara Hooker. To code or not to code? exploring impact of code in pre-training. In ICLR, 2024.
Jacob Austin, Augustus Odena, Maxwell Nye. Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoe Le, and Charles Sutton. Program Synthesis with Large Language Models, 2021. URL
Zhangir Azerbayev, Bartosz Piotrowski, Hailey Schoelkopf, Edward W Ayers, Dragomir Radev, and Jeremy Avigad. Proofnet: Autoformalizing and formally proving undergraduate-levw mathemat- ics. arXiv preprint arXiv:2302.12433 2023.
Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen Marcus McAleer, Albert Q. Jiang, Jia Deng, Stella Biderman, and Sean Welleck. Llemma: An open language model for mathematics. In ICLR, 2024.
Adrien Barbaresi. Trafilatura: A Web Scraping Library and Command-Line Tool for Text Discov- ery and Extraction. In Proceedings of the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natu- ral Language Processing: System Demonstrations, pp. 122-131. Association for Computational Linguisties, 2021. URL wttpt::t//aaaae www.hlllnnoooooyoooooooooo wwwoorgg/2222 -cco-demo . 15
Janek Bevendorff, Benno Stein, Matthias Hagen, and Martin Potthast. Elastic ChatNoir: Search En- gine for the Clue and the Common Crawl. In Leif Azzopardi, Allan Hanbury, Gabriella Pasi, and Benjamin Piwowarski (eds.), Advances in Information Retrieval. 40th European Conference on IR Research (ECIR 2018), Lecture Notes in Computer Science, Berlin Heidelberg New York, March 2018. Springer.
Cody Blakeney, Mansheej Paul, Brett W Larsen, Sean Owen, and Jonathan Frankle. Does your data spark joy? performance gains from domain upsampling at the end of training. arXiv preprint arXiv:2406.034760 2024.
Andrei Broder. Identifying and filtering near-duplicate documents. In Annual symposium on combinatorial pattern matching, pp. 1-10. Springer, 2000.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, et al. Evaluating Large Language Models Trained Code, 2021. URL https://
axxiv.org/abs. 2107 23377 03374
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reicchiro Nakano, Christopher Hesse, and John Schulman. Training Verifiers Solve Math Word Problems, 2021. URL https://arxix:
org/abs/ 22100/1000111 14168
DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Cheng- gang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji. Erhang Li, Fangyun Lin, Fucong Dai. Fuli Luo, Guangbo Hao, Guanting
Chen, Guowei Li, H. Zhang, Han Bao. Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui
Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J. L. Cai, Jian Liang, Jianzhong Guo, Jiaqi Ni, Jiashi Li, Jiawei Wang, Jin Chen, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, Junxiao Song. Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Lei Xu, Leyi Xia, Liang Zhao, Litong Wang, Liyue Zhang, Meng Li, Miaojun Wang, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingming Li, Ning Tian, Panpan Huang, Peiyi Wang, Peng Zhang, Qiancheng Wang, Qihao Zhu, Qinyu Chen, Qiushi Du, R. J. Chen. R. L. Jin, Ruiqi Ge. Ruisong Zhang. Ruizhe Pan, Runji Wang. Runxin Xu, Ruoyu Zhang, Ruyi Chen, S. S. Li, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shaoqing Wu, Shengfeng
11

---

Published as a conference paper at ICLR 2026
Ye, Shengfeng Ye, Shirong Ma, Shiyu Wang, Shuang Zhou, Shuiping Yu, Shunfeng Zhou, Shut- ing Pan, T. Wang, Tao Yun, Tian Pei. Tianyu Sun, W. L. Xiao, Wangding Zeng, Wanjia Zhao, Wei An. Wen Liu, Wenfeng Liang, Wenjum Gao. Wenqin Yu, Wentao Zhang, X. Q. Li, Xiangyue Jin, Xianzu Wang, Xiao Bi, Xiaodong Liu, Xiaohan Wang, Xiaojin Shen, Xiaokang Chen, Xi- aokang Zhang. Xiaosha Chen, Xiaotao Nie, Xiaowen Sun, Xiaoxiang Wang, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xingkai Yu, Ximnan Song, Xinxia Shan, Xinyi Zhou, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, Y. K. Li, Y. Q. Wang, Y. X. Wei, Y. X. Zhu, Yang Zhang, Yanhong Xu, Yanhong Xu, Yanping Huang, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Li, Yaohui Wang, Yi Yu, Yi Zheng, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Ying Tang, Yishi Piao. Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yu Wu, Yuan Ou, Yuchen Zhu, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yukun Zha, Yunfan Xiong, Yunxian Ma, Yuting Yan, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Z. F. Wu, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhen Huang, Zhen Zhang, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhibin Gou, Zhicheng Ma, Zhigang Yan, Zhihong Shao, Zhipeng Xu, Zhiyu Wu, Zhongyu Zhang, Zhuoshu Li, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Ziyi Gao, and Zizheng Pan. Deepseek-v3 technical report, 2025. URL https://arxiv> www.oogg//aaoo. 2412 2 19437
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, et al. The Ilama 3 herd of models. arXiv preprint arXiv:2407.2173 1783, 2024.
István Endrédy and Attila Novák. More effective boilerplate removal-the goldminer algorithm. Polibits, 48:79-83, 12 2013. doi: 10.17562/PB-48-10.
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling, arXiv preprini urXiv:2101.0002) 2020.
Daya Guo, Dejian Yang, Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in lims via reinforcement learning. arXiv preprint arXiv:2501.12944 2025.
Xiaotian Han, Yiren Jian, Xuefeng Hu, Haogeng Liu, Yigi Wang, Qihang Fan, Yuang Ai, Huaibo Huang. Ran He, Zhenheng Yang, and Quanzeng You. modal pre-training for enhanced mathematical reasoning, 2024. ttl.hhtps://arxiv
Advancing multi-
abs 0bs/2409.20 12569
Dan Hendrycks. Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Ja- cob Steinhardt. Measuring massive multitask language understanding, 2021a. URL https: 03300
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song. and Jacob Steinhardt, Measuring mathematical problem solving with the math dataset. NeurIPS, 2021b.
Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang. Yuxiang Huang, Weilin Zhao, et al. Minicpm: Unveiling the potential of small language models with scalable training strategies. arXiv preprint arXiv:2404.0633) 063955 2024.
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang. Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, Kai Dang, Yang Fan, Yichang Zhang, An Yang, Rui Men, Fei Huang, Bo Zheng, Yibo Miao, Shanghaoran Quan, Yunlong Feng, Xingzhang Ren, Xu- ancheng Ren, Jingren Zhou, and Junyang Lin. Qwen2.5-coder technical report, 2024. URL https:
org/ abs/ 2409. 12186
Denis Kocetkov, Raymond Li, Loubna Ben Allal, Jia Li, Chenghao Mou, Carlos Muñoz Ferran- dis, Yacine Jernite, Margaret Mitchell, Sean Hughes, Thomas Wolf, et al. The stack: 3 th of permissively licensed source code. arXiv preprint arXiv:2211.1555 2022.
Vijay Korthikanti, Jared Casper, Sangkug Lym, Lawrence McAfee, Michael Andersch, Moham- mad Shocybi, and Bryan Catanzaro. Reducing Activation Recomputation in Large Transformer Models, 2022. URL htt.stt:: http:://arxvvv... abs 2205. 051
12

---

Published as a conference paper at ICLR 2026
Katherine Lee. Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison- Burch, and Nicholas Carlini. Deduplicating training data makes language models better. In ACL, pp. 1424-8445, 2022.
Aitor Lewkowyez, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ra- masesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. Solving quantitative reasoning problems with language models. NeurIPS, 35:3843-38577 3857,, 2022.
Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Mare Marone, Christopher Akiki, Jia Li. Jenny Chim, et al. Starcoder: may the source be with you! arXiv preprint urXiv:2305.06161, 2023.
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, lly Sutskever, and Karl Cobbe. Let's verify step by step. CoRR, abs/2305.20050, 2023. doi: 10.485500urXx..2355.20000 18550/arX10
URL hattpss//doi.... org/111...0..
2.055220050
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024.
Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation.
arXiv preprint wrXiv:2305.011111
2023. doi:
URL
https: wrg/ abs/ 2tt.//23300.1211
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 1rXiv:1711.05101, 2017.
Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poiries Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, et al. Starcoder and the stack v2: The next generation. arXiv preprint 200:2240.19975 2024.
Niklas Muennighoff, Alexander M. Rush, Boaz Barak, Teven Le Scao, Aleksandra Piktus, Noua- mane Tazi, Sampo Pyysalo, Thomas Wolf, and Colin Raffel. Scaling data-constrained language models, 2023.
NVIDIA, :. Aaron Blakeman, Aarti Basant, Abhinav Khattar, Adithya Renduchintala, Akhiad Bercovich, Aleksander Ficek, Alexis Bjorlin, Ali Taghibakhshi, Amala Sanjay Deshmukh, Ameya Sunil Mahabaleshwarkarr Andrew Tao, Anna Shors, Ashwath Aithal, Ashwin Poojary, Ayush Dattagupta, Balaram Buddharaju, Bobby Chen, Boris Ginsburg, Boxin Wang. Brandon Norick, Brian Butterfield, Bryan Catanzaro. Carlo del Mundo, Chengyu Dong, Christine Har- vey, Christopher Parisien, Dan Su, Daniel Korzekwa, Danny Yin, Daria Gitman, David Mosal- lanezhad, Deepak Narayanan, Denys Fridman, Dima Rekesh, Ding Ma, Dmytro Pykhtar, Dong Ahn, Duncan Riach, Dusan Stosic, Eilcen Long. Elad Segal, Ellic Evans, Eric Chung, Erick Galinkin, Evelina Bakhturina, Ewa Dobrowolska, Fei Jia, Fuxiao Liu, Gargi Prasad, Gerald Shen, Guilin Liu, Guo Chen, Haifeng Qian, Helen Ngo, Hongbin Liu, Hui Li, Igor Gitman, Ilia Kar- manov, Ivan Moshkov, Izik Golan, Jan Kautz, Jane Polak Scowcroft, Jared Casper, Jarno Seppa- nen, Jason Lu, Jason Sewall, Jiaqi Zeng, Jiaxuan You, Jimmy Zhang, Jing Zhang, Jining Huang, Jinze Xue, Jocelyn Huang, Joey Conway, John Kamalu, Jon Barker, Jonathan Cohen, Joseph Jennings, Jupinder Parmar, Karan Sapra, Kari Briski, Kateryna Chumachenko, Katherine Luna, Keshav Santhanam, Kezhi Kong, Kirthi Sivamani, Krzysztof Pawelec, Kumar Anik, Kunlun Li, Lawrence McAfee, Leon Derczynski, Lindsey Pavao, Luis Vega, Lukas Voegtle, Maciej Bala, Maer Rodrigues de Melo, Makesh Narsimhan Sreedhar, Marcin Chochowski, Markus Kliegl, Marta Stepniewska-Dziuuinen Dziubinsaa Matthieu Le, Matvei Novikov, Mehrzad Samadi, Michael Ander- sch, Michael Evans, Miguel Martinez, Mike Chrzanowski. Mike Ranzinger, Mikolaj Blaz, Misha Smelyanskiy, Mohamed Fawzy Mohammad Shoeybi, Mostofa Patwary, Nayeon Lee. Nima Tajbakhsh, Ning Xu, Oleg Rybakov, Oleksii Kuchaiev, Olivier Delalleau, Osvald Nitski, Parth Chadha. Pasha Shamis, Paulius Micikevicius, Pavlo Molchanov, Peter Dykas, Philipp Fischer, Pierre-Y Yves Aquilanti, Piotr Bialecki, Prasoon Varshney, Pritam Gundecha, Przemek Tredak, Rabeeh Karimi, Rahul Kandu, Ran Yaniy, Raviraj Joshi, Roger Waleffe, Ruoxi Zhang, Sabrina
13

---

Published as a conference paper at ICLR 2026
Kavanaugh, Sahil Jain, Samuel Kriman, Sangkug Lym, Sanjeev Satheesh, Saurav Muralidha- ran, Sean Narenthiran, Selvaraj Anandaraj. Seonmyeong Bak, Sergey Kashirsky, Seungju Han, Shantanu Acharya, Shaona Ghosh, Sharath Turuvekere Sreenivas, Sharon Clay, Shelby Thomas, Shrimai Prahhumoye, Shubham Pachori, Shubham Toshniwal, Shyamala Prayaga, Siddhartha Jain, Sirshak Das, Slawek Kierat, Somshubra Majumdar, Song Han, Soumye Singhal, Sriharsha Niverty, Stefania Alborghetti, Suseella Panguluri, Swetha Bhendigeri. Syeda Nahida Akter, Szy- mon Migacz, Tal Shiri, Terry Kong, Timo Roman, Tomer Ronen, Trisha Saar, Tugrul Konuk, Tuomas Rintamaki. Tyler Poon, Ushmish De, Vahid Noroozi, Varun Singh, Vijay Korthikanti, Vitaly Kurin, Wasi Uddin Ahmad, Wei Du, Wei Ping, Wenliang Dai, Wonmin Byeon, Xiaowei Ren, Yao Xu, Yejin Choi, Yian Zhang, Ying Lin, Yoshi Suhara, Zhiding Yu, Zhiqi Li, Zhiyu Li, Zhongbo Zhu, Zhuolin Yang, and Zijia Chen. Nemotron-h: A family of accurate and efficient hybrid mamba-transforme models, 2025. URL nntpps://ars
org/ / aww./aaa.00 03624
Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, et al. 2 olmo 2 furious. arXiv preprint arXiv:2501.00656, 2024.
OpenAL. Learning to reason with 1lms, 2024. URL
Tearminggto-rreson-- 11ms//
Keiran Paster, Marco Dos Santos, Zhangir Azerbayev, and Jimmy Ba. Openwebmath: An open dataset of high-quality mathematical web text. ICLR, 2024.
Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Hamza Alobeidli, Alessandro Cappelli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. The refined- web dataset for falcon Ilm: Outperforming curated corpora with web data only. NewrlPS, 36: 79155-79172, 2023.
Stanislas Polu and Ilya Sutskever. Generative language modeling for automated theorem proving. arXiv preprint arXiv:2009.038 00393,, 2020.
Qwen, :, An Yang. Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang. Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yugiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025. URL wttps::/tsettt. /ttp://axvvvv.. org/ abs/ 2412. 15115
Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. ZeRO: Memory Optimiza- tions Toward Training Trillion Parameter Models, 2020. URL https:t///xxii.....////////// /bs/
02054
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseckmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint 2.Xi:224422033000 2024.
Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Mcgatron-LM: Training Multi-Billion Parameter Language Models Using Model Par- allclism, 2020. httptttss///aatt//
org/ als / 08053
Dan Su, Kezhi Kong, Ying Lin, Joseph Jennings, Brandon Norick, Markus Kliegl, Mostofa Patwary, Mohammad Shoeybi, and Bryan Catanzaro. Nemotron-CC: Transforming Common Crawl into refined long-horizon pretraining dataset. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2459 2475, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN wtgpo//gooooggo...oo//.// aln.g 123/
URL https:
Liping Tang, Nikhil Ranjan, Omkar Pangarkar, Xuezhi Liang, Zhen Wang, Li An, Bhaskar Rao, Linghao Jin, Huijuan Wang, Zhoujun Cheng, Suqi Sun, Cun Mu, Victor Miller, Xuezhe Ma, Yue Peng, Zhengzhong Liu, and Eric P. Xing. Txt360: A top-quality 11m pre- pre-tiaining dataset requires the perfect blend, 2024. URL WRI.ttts:: /hugg ngface. wwtp/////pppcccocoeeeeeeeee
14

---

Published as a conference paper at ICLR 2026
Qwen Team. Qwen2 technical report. arXiv preprint arXtv:2407.10671, 2024.
Yury Tokpanov, Paolo Glorioso, Quentin Anthony, and Beren Millidge. Zyda-2: 5 trillion token high-quality dataset. arXiv preprint arXiv:2411.06066 2024.
Yubo Wang, Xueguang Ma, Ge Zhang. Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang. Tianle Li, Max Ku, Kai Wang. Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark, 2024a. http::::: /ww.vv.rvvvvvvvv//////.a./aavivv org/ abs 2406. 01574
org/abs/
Zengzhi Wang, Xuefeng Li, Rui Xia, and Pengfei Liu. Mathpile: A billion-token-scal pretraining
corpus for math. NeurlPS, 37:25426-25405 25468, 2024b.
Sean Welleck, Jiacheng Liu, Ronan Le Bras, Hannaneh Hajishirzi, Yejin Choi, and Kyunghyun Cho. Naturalproofs: Mathematical theorem proving in natural language. arXiv preprint 1XXi:2204.001122. 2021.
Yang. Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jian- hong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2.5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409. /2122, 2024.
Shuo Yang, Wei-Lin Chiang, Lianmin Zheng, Joseph E. Gonzalez, and lon Stoica. Rethinking benchmark and contamination for language models with rephrased samples, 2023.
Fan Zhou, Zengzhi Wang, Nikhil Ranjan, Zhoujun Cheng, Liping Tang, Guowei He, Zhengzhong Liu, and Eric P Xing. Megamath: Pushing the limits of open math corpora. arXiv preprint arXiv:2504.022 028007 2025.
15

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 USE OF LARGE LANGUAGE MODELS (LLMS)
We used a large language model (LLM) solely to aid in polishing the writing of this paper, includ- ing improving grammar, clarity, and flow. The LLM was not involved in the research ideation, methodology, experiments, analysis, or interpretation of results.
A.2 PROMPT USED FOR TOPIC CLASSIFICATION
We employ the following prompt to classify documents into a predefined set of categories. Dur- ing classification, the large language model (LLM) occasionally produces category labels that fall outside the predefined taxonomy: mathematics, computer science, physics, statistics, economics, chemistry, and other. To maintain consistency and reduce label fragmentation, any out-of-taxonomy label is reassigned to the category other, ensuring coherent and structured category distribution.
You are topic classification assistant.
Given the following document text, identify its main topic from this list only:
Mathematics
Computer Science
Phyysiss
Statistics
Chemistry
Economies
Other
Choose the single most relevant category from the list.
Document:
(text)
Your output should be only word. Finish your response right after category and do not add any explanation.
A.3
QUALITATIVE EXAMPLES
This section presents qualitative comparisons among OpenWeMMath (OWM), MegaMath-Pro, FineMath-4+, and Nemotron-CC-Maath4 highlighting differences in content quality.
A.3.1 DEGENERATE CASES IN MEGAMATH-PRO DATASET
We identified subset of degenerate generations within the MegaMath-Pro dataset. Representative examples are presented below illustrate this phenomenon. Notably, these samples achieve unex- pectedly high scores on both mathematical and language scores, raising concerns about the dataset's s overall reliability for pretraining LLMs.. For each example, we provide the associated metadata. The excerpts shown correspond to the initial portion of each generation; in every case, the text extends over several additional pages, repeating the final sentence displayed.
A.3.2 SIDE BY SIDE COMPARISON BETWEEN OUR DATASET AND PRIOR WORK
We observe that our pipeline not only keep the math equations but also keep the codes and their formatting. We observe that previous pipeline in most cases are not keeping codes or lose their formatting. This is specifically important for languages like python. To highlight this difference, provide two sets of examples demonstrating both code and mathematical equations. Notably, inline equations often removed in prior work, such MegaMath.
16

---

Published as a conference paper at ICLR 2026
degenerate sample from MegaMath-Pro
The Integral Calculator is able to calculate integrals online of the composition of common functions, using integral properties, the different mechanisms of integration and calculation online. The Integral Calculator is simple online calculator that computes the definite and indefinite integrals. The Integral Calculator will show you graphical version of your input while you type.
The Integral Calculator is free online tool for calculating the value of definite integral. The Integral Calculator, part of the graphing calculator, helps with one variable calculus. The Inte- gral Calculator supports definite and indefinite integrals (antiderivatives) as well as integrating functions with many variables.
The Integral Calculator is able to calculate integrals online of the composition of common functions, using integral properties, the different mechanisms of integration and calculation online. The Integral Calculator is simple online calculator that computes the definite and indefinite integrals. The Integral Calculator will show you graphical version of your input while you type.
The Integral Calculator is a free online tool for calculating the value of definite integral. The Integral Calculator, part of the graphing calculator, helps with variable calculus. The Inte- gral Calculator supports definite and indefinite integrals (antiderivatives) as well as integrating functions with many variables.
The Integral Calculator is able to calculate integrals online of the composition of common functions, using integral properties, the different mechanisms of integration and calculation online. The Integral Calculator is simple online calculator that computes the definite and indefinite integrals. The Integral Calculator will show you graphical version of your input while you type.
The Integral Calculator a free online tool for calculating the value of definite integral. The Integral Calculator, part of the graphing calculator, helps with variable calculus. The Inte- gral Calculator supports definite and indefinite integrals (antiderivatives) as well as integrating functions with many variables.
The Integral Calculator is able to calculate integrals online of the composition of common functions, using integral properties, the different mechanisms of integration and calculation online. The Integral Calculator is simple online calculator that computes the definite and indefinite integrals. The Integral Calculator will show you graphical version of your input while you type.
The Integral Calculator a free online tool for calculating the value of definite integral. The Integral Calculator, part of the graphing calculator, helps with one variable calculus. The Inte- gral Calculator supports definite and indefinite integrals (antiderivatives) as well as integrating functions with many variables.
The Integral Calculator is able to calculate integrals online of the composition of common functions, using integral properties, the different mechanisms of integration and calculation online. The Integral Calculator simple online calculator that computes the definite and indefinite integrals. The Integral Calculator will show you graphical version of your input while you type.
The Integral Calculator free online tool for calculating the value of definite integral. The Integral Calculator, part of the graphing calculator, helps with one variable calculus. The Inte- gral Calculator supports definite and indefinite integrals (antiderivatives) as well as integrating functions with many variables.
The Integral Calculator is able to calculate integrals online of the composition of common functions, using integral properties, the different mechanisms of integration and calculation online. The Integral Calculator simple online calculator that computes the definite and indefinite integrals. The Integral Calculator will show you graphical version of your input while you type.
The Integral Calculator is free online tool for calculating the value of definite integral.
Meta:
URL: 31 e https//see.........o..e///eeeeer
Math Score: 13998773399887085
Lang Score: 0.789376616477966
WARC Filename: B4taa//ww...aa/////////////////n
17

---

Published as a conference paper at ICLR 2026
degenerate sample from MegaMath-Pro
The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed.
The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles.
The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed.
The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed. The angle will be calculated and displayed. Use the law of cosines to find one of the angles. The angle will be calculated and displayed. The angle will be calculated and displayed.
Meta:
URL:: SAL::bepp:/// http:://122thhe http://tttkkkn..oooo wwtp:///ooo///oeeeeeooe waw---n-wwwa---d- wwt.a//wwwwaa-oooooooooo------ee wwwpo///wwww.p-aaddaaaaoa----eee
ancie- (Lnder-oba031
Math Score: 0.9885214567184445
Lang Score: 0.864239931 10055674
WARC Filename: CC-NAIN 2021-31 (ikegnnen/1662700065557776.65530 MAIN 202 10804445266 00255.
18

---

Published as a conference paper at ICLR 2026
degenerate sample from MegaMath-Pro
The equation of the axis of symmetry in vertical parabola is equal the x-coordinate of the vertex. The axis of symmetry always passes through the vertex of the parabola. The x- coordinate of the vertex is equal to the formula.
To learn about the axis of symmetry, watch this tutorial! The axis of symmetry is the line that divides the graph into two perfect halves. The axis of symmetry is always a vertical line of the form x where is a real number. A parabola is the graph of a quadratic function. Each parabola has a line of symmetry. Also known as the axis of symmetry, this line divides the parabola into mirror images. The line of symmetry is always vertical line of the form x = n, where n is real number.
When graphing, we want to include certain special points in the graph. The y-intercept is the point where the graph intersects the y-axis. The x-intercepts are the points where the graph intersects the x-axis. The vertex is the point that defines the minimum maximum of the graph.
The axis of symmetry for equation with 2 is the vertical line that passes through the vertex. The axis of symmetry is the line x = h, where (h, k) is the vertex of the parabola. The axis of symmetry is the line that divides the graph into two perfect halves. The axis of symmetry is always a vertical line of the form x n where n is real number. The equation of the axis of symmetry in a vertical parabola is equal the x-coordinate of the vertex. The axis of symmetry always passes through the vertex of the parabola. The x- coordinate of the vertex is equal to the formula.
To learn about the axis of symmetry, watch this tutorial! The axis of symmetry is the line that divides the graph into two perfect halves. The axis of symmetry is always a vertical line of the form n, where is real number.
A parabola is the graph of a quadratic function. Each parabola has a line of symmetry. Also known the axis of symmetry, this line divides the parabola into mirror images. The line of symmetry is always a vertical line the form x = n, where n is a real number.
include certain special points in the graph. The y-intercept is the point where the graph intersects the y-axis. The x-intercepts are the points where the graph intersects the x-axis. The vertex is the point that defines the minimum or maximum of the graph.
When graphing,
The axis of symmetry for an equation with 2 is the vertical line that passes through the vertex. The axis of symmetry is the line x = h, where (h, k) is the vertex of the parabola. The axis of symmetry is the line that divides the graph into two perfect halves. The axis of symmetry is always vertical line of the form x = where is real number.
The equation of the axis of symmetry in a vertical parabola is equal to the x-coordinate of the vertex. The axis of symmetry always passes through the vertex of the parabola. The coordinate the vertex is equal to the formula.
To learn about the axis of symmetry, watch this tutorial! The axis of symmetry is the line that divides the graph into two perfect halves. The axis of symmetry is always a vertical line of the form = n, where is real number.
A parabola is the graph of a quadratic function. Each parabola has a line of symmetry. Also known the axis of symmetry, this line divides the parabola into mirror images. The line of symmetry is always a vertical line of the form x = where n is real number.
When graphing, we want include certain special points in the graph. The y-intercept is the point where the graph intersects the y-axis. The x-intercepts are the points where the graph intersects the x-axis. The vertex is the point that defines the minimum or maximum of the graph.
The axis of symmetry for an equation with 2 is the vertical line that passes through the vertex. The axis of symmetry is the line h, where (h, k) is the vertex of the parabola. The axis of symmetry is the line that divides the graph into two perfect halves. The axis of symmetry is always vertical line of the form n, where is real number. The equa- tion of the axis of symmetry in a vertical parabola is equal to the x-coordinate of the vertex.
Meta:
Math Score: 0199987573719024666
Lang Score: 0.9151935789253235
WARC Filename: CC-MAIN- 2021 h2 MAIN 202 224456 D0277.
19

---

Published as a conference paper at ICLR 2026
Lynx output
#10000 Terabyte
10000 Terabyte
about opensource disclaimer
(BUTTON) about opensource disclaimer
Detailed explanation of smart solution to algo problem beating 99.9% submission
Written on January 7th, 2018 by 10000TB stttachmmnntsartiic_aggggriiii _acl__ooo___haa _side Jamppra---llaaat-----onnnnn-ni--
This post is about coding problem and why the solution pasted down below smart. Problem:
Given two sparse matrices A and B, return the result of AB.
You may assume that A's column number is equal to B's row number.
Example:
A
[1. 0, 0]. 0.3, ]
B=[
[7,0,0],
[0,0,0];
[
100
770
700
AB x |00|
1001
If it is of your interest, I would recommend you take a few minutes to think about how you would approach this problem!
The main focus of this post is 1eexplain detail why the provided solution some improvements/twcaks in the code the smart solution to show you which part is really essential, 3) also will briefty mention why Sparse Matrix Manipulation can help make some improvements top of the smart solution.
and maka
a) Originally, the normal way to calculate the multiplication of metrics A, and B follow: We the
values from the first line of A, and all values from the first column B, and multiply the corresponding values and sum them up. the final sum is the value for the location of first column and first row in final result matrix. Similarly, the value at [ J of result matrix C, which is Ci j calculated as:
[[i]i]= A[ ][O]B[O]Í] [[i][]][[]]]] + A[i]2]BB2200] + ... AÍ]]K]][[J
(which the sum each multiplication corresponding values from row
values from
column of B)
The Key is: if we calculate it this way, finishing calculating the final value for the result matrix once
Then brute force solution as follow:
20

---

Published as a conference paper at ICLR 2026
public class Solution
public int[]]] [ultiply(in(] A, int] B) { int m A.length, A[0].length, nB = B[O].length; in[] C= new in[[m][nB]:
for(int < m: i++) { for (int 0; < (B:j++)) {
for(int k = 0; k < k+)
Cíilj]] += A[i][k] B[klj]:
} return C;
b) The smart solution: the key part of smart solution is that: it does not calculate the final result at once, and it takes each value from A, and calculate and partial sum and accumulate it into the final spot:
For example, for each value ][i][k], is not zero, will used at most times is B[O].
length ), which can be illustrated as follow: Generally for the following equation:
A[i][k][[K[i] A[][K][[K]i] j can be from to nB, if we write all of them down, it will like following:
C[ili] = [[i][]][OO]i] + A[][[]][[][]] A[][[]][[][]]
For from 0
C [[ |= [[о]в[о][j] + A[i[[]][[]O) + A[]]2]B[210] + A[i[[]][[][0] A[i][K///K
[[]]
C[ [[ ]=A[ i [[о]][]][] + [[][]][[]]]] + [[][]][[1]] + A[][K][ÍI Ai][K]][K
[[]]
C J[ nB |=A[i][0//((]]BB] + A[i[L]][[][[B]] + A[]]2]B[[][nB]] + A[j[K]/[K][B)] AjjjK ]+B[K][nB]
As you
from above: for the same value A[i][k] from the first matrix, it will be used most nB times if Afi][k] not zero. And the smart solution is taking advantage of that!!! the
solution can be described as:
For each value A[i][]] in matrix A, if it is not zero, we calculate A[i][k] * B[k][j] and accumulate it into C[ i ][ (Key part: the C[ [[][]] ][ by now is not the final value in the result matrix !!
Remember, in the brute force solution, the final value of C(i]i]. takes of all multiplication B? here CI ][ is only sum some
values K corresponding values from
multiplication values, NOT ALL until the program done) BY NOW, it is very clear that, if the value A[ ][ k from matrix is zero, we skip For-loop- calculation, which is loop iterating nB times, and this is the key part of why the smart solution smart!!!
The smart solution code as follow:
public class Solution {
public im[[[ milt A, int[l] B) { int A.length, length,, nB lengt;; int( [[] C= new int/m [[B]]
for(int i= 0; i m; i++) {
for(int k 0; < n; k++)
if [A[i][k] != 0)
for (int j = 0: nB:
if (B[k]]] : Clillii += A[j[[]] B[k]j]:
return C:
}
(Credit: @ yavinci; am having different version of the solution, so am directly referencing the original version as reference to demonstrate how mine is different).
21

---

Published as a conference paper at ICLR 2026
Based the discussion above, the inner checking (if (B[k]j] OJ) actually not necessary.
because whether or not we have that check, we still iterate times, since the operation Ciilj B[k][]]: inside the if-check is O(1) time)
+= A[i][k]
So the smart solution also be written follow by removing the check which is my version public class Solution {
public int[If] multiply(int[]] A, intfil B) { int m A.length, = A[0].ennth, nB = BeO).hn int(]]] [[] C= new [nt[m][nB]
for(int i < m; i++) {
for(int k = 0; < n; k++) if [A[i][]] t= 0)
for [int (:j nB: {
if [B[k]]] Clilli) A[j[[]] B[k]ü]:
} return C:
}
C) "Sparse matrix manipultion" helps, if compress the first sparse matrix rows of lists( each row list, it contains vaalue, index pair ), we actually don't need to go over all values in a row in matrix A when are calculating the final result matrix. But Overall, it does not help improve run time algorithmatically!!
References:
Image credit: attachments att.tiiee.giiiitttto...iiiaaaaaa
jpg
Please enable JavaScript view the comments powered by Disqus. Feel free to share!
You may also enjoy...
Please enable JavaScript to view the comments powered by Disqus. comments powered by Disqus
10000 Terabyte An unserious blog 10000th.org xuchaohu.com david-hu.org
Meta:
URL:
Sctp:/ Tr
WARC Filename:
MAIN-22211 31l.://ggeeenn//6222666555555000
22

---

Published as a conference paper at ICLR 2026
sample from Nemotron- Manotron-CCC-Mahh CC-Math
10000 Terabyte
## Detailed Explanation of a Smart Solution to an Algorithm Problem
*Written on January 7th, 2018 by @ 10000TB
This post is about coding problem and why the solution provided is smart. The problem is as follows:
++ Probleem:** Given two sparse matrices \( A \) and \( [ V. return the result of \( (AB \). You may assume that (( columm column number equal '(B \)'s number.
* Example:=*
M
A (hegin (bmatrix
& -1 & &
jnd (bmatrix)
M B = (bbggi/ bmatrix) & 0 & 0 O 0 & O& & end bmatrix]
MI (egiir/bmarrix
& 0 & 0 & & (bmatrixx
"times
(begin(bmatrix]
7 & &
0 0& & end (bmatriix
\begin ( bmatrix}
& & {
-7 & cnd (bmatrix)
If you interested, consider how you would approach this problem!
The main focus of this post is to:
I. Explain in detail why the provided solution is smart.
2. Make some improvementstteeas in the code of the smart solution to show which part really essential.
3. Brieffy mention why Sparse Matrix Manipulation can help make improvements.
### a) Brute Force Solution
The normal way calculate the multiplication of two matrices ((A \BB is as follows: We take all values from the first row ) A \) and all values from the first column of (B V), multiply the corresponding values and sum them up. The final sum is the value for the location of the first row and first column in the final result matrix \ C Similarly, the value A Cfilfi) i) calculated as:
23

---

Published as a conference paper at ICLR 2026
V
cnii-= А[][ A A[]]2]][[1 Aldots ][][]]][[][]
The brute force solution is follows:
"'java
public class Solution { int A.length, Aln].length, nB length; int[ C new int[m][[B]]:
public int[][] multiply(in(]]] A, int[]]] B) {
for (int = 0; < m; i++) {
for (int = 0; nB: j+++ {
for =
k++) {
C[i]l]] += A[i][k] * B[k]ij:
return C:
}
)
### b) The Smart Solution
The key part of the smart solution is that it does not calculate the final result at once. Instead, it takes each value from \). calculates partial sum, and accumulates into the final spot.
For example, for each value ( A[i][]] V), it is not zero, will be used at most (( [BB \) times (V is \ (B[O).ennthh 1)). Generally, for the following equation:
Chilil= = АÍÍ][ов[о1í + 10) A[1[21/[210] |ldots A[ilk]/[klÍj] + Ildots + Afi][K] B[K][]]
V
\G 1) can from to \(nB )). we write all them down, will look like this:
For \(i)) from 0 to \(nB \):
Cí1O] = {[lli]][ii1 + А]][1][[[11] + 1 + Aldots + A[i][K][[][[] |ldots Afi][ K]B[K][O]
V
N
ChI + + А]]П]][[1]]] + A[1[2[/[[1[1] + |ldots + A[]][]][[[][]]] + |lotss Afil[
U
I
Idots
N
CD] = А[Л]О]/[[[]ы)] + A[][]][[1][B]] A[][2[B/2[[nB] Vldots + A[]][]][[][[B]]
Idots + [[]][]][[K[[B]]
24

---

Published as a conference paper at ICLR 2026
As you sec, for the same value ( (Afi][]] \) from the first matrix, it will be used most \(BB times if M A[i][k] () not zero. smart solution takes advantage of this by calculating \ Af i][k] (times B[k][]] \) and accumulating it into \ C(ili V.. Note that ( Cfillji \) is not the final value in the result matrix at this point. In the brute force solution, the final value of \ C[ill the sum of all multiplication values of corresonningg values from M A \) and / V here, (( (Cilli \) only the sum some multiplication values, not all, until program done.
The smart solution code is as follows:
"java public imt[[ mul A, im[0 B) {
public class Solution { int A.length, Aegg.length, nB Lengh:; int(][] C new int[m]]]B]:
for (int = 0; i m; i++)
for (int k = 0; < k++) {
if (Afi][k] :=
(int = nB; j++)
if (B[k]0] := Ctilli) += A[i][]] B[k]ii:
return C;
(Credit: @ @yavinci; am having a different version of the solution, so am directly referencing the original version as reference to demonstrate how mine is different.)
Based on the discussion above, the inner checking ( (if (Bíklj) \neq 0)) \) is actually not necessary because whether or not we have that check, we still iterate \( nB \) times. The operation '( Cli I[i += A[i][k] times B[k][j] \) inside the if-check is \ O(1) \) time. So the smart solution
also be written follows by removing the check (which my version):
''aava
public class Solution {
public imt[[[] multiply(in(]0 A, in B) { int A.length, Aength, nB Ben)th; int C new int[m][nB]]
for (int i= 0: m; i++) {
for (int k 0; < k++) {
if (Afil[k] != 0)
for (int = < <B; { Clillj] += A[i][k] B[kli]:
}
return C;
L..
### c) Sparse Matrix Manipulation
Sparse matrix manipulation helps if we compress the first sparse matrix into rows of lists (in each row list, it contains (value, index) pairs). We actually don't need to go over all values in row in matrix \ A \) when calculating the final result matrix. However, overall, it does not help improve runtime algorithmically.
25

---

Published as a conference paper at ICLR 2026
#### References
1. Image credit: attachments article algorithm col slide lamprras-oooggeem----mmtmmhhiii-
Meta:
URL:
Sctp: B
WARC Fllename:
CC MAIN 2021 htwp://mwwm./..........ooo///aoa 900A M4
26

---

Published as a conference paper at ICLR 2026
sample from Open Wattahh
Detailed explanation of smart solution algo problem beating 99.9% submission This post is about coding problem and why the solution pasted down below smart. Problem:
Given two sparse matrices A and B, return the result of AB.
You may assume that A's column number is equal B's number.
Example:
=
[1, 0, 01, [,1,0,3]
]
B=[
17,0,01,
[0,0,0],
10 ]
[100]
1700
700
AB -103 |000||/000]||-00
1001
If it is of your interest, I would recommend you take few minutes to think about how would approach this problem!
The main focus this post is to Dexplain detail why the provided solution smart and 2)make some improvements/tweaks in the code the smart solution to show you which part is really essential, 3) also i will briefly mention why Sparse Matrix Manipulation can help make some improvements top of the smart solution.
a) Originally, the normal way to calculate multiplication of two metrics A, and as follow: take the the all values from the first line of A, and all values from the first column of B, and multiply the corresponding values and sum them up. the final sum is the value for the location of first column and first row in final result matrix. Similarly. the value ]Li ] of result matrix C, which is C j calculated as:
C == Al доо]вгодí] + A [ A]][2//[11]] + АДІК
(which is the sum of each multiplication of corresponding values from row of A and K values from
column of
The Key we calculate this way, we finishing calculating the final value for the result matrix at once
Then brute force solution is follow:
b) The smart solution: the key part of smart solution is that: it does not calculate the final result at once, and it takes each value from A, and calculate and partial sum and accumulate it into the final spot: For example, for each value A[i][k], if it is not zero, it will be used at most nB times is B[O].length which be illustrated follow: Generally for the following equation:
Cüilli= = Δ[jlo]][(Ji] + А[][ + A[i][]][[][]] + AÍjJK]B[[] [[i]KK][[Kii]
j can be from to nB, if we write all of them down, it will like following:
For from 0 nB:
27

---

Published as a conference paper at ICLR 2026
]=
B[O][0] +
+ ATII]821100 + А[][]]][III АБ]]]]и[К
C ] ][[]
B[O][1] A[][]][[[[]]] +112)BB11 А[]]K][[iО| АЦ]]]][[][
C }=A[ [[O] B[0]/B]] Afill [ [[]B] Afil2]B//2[nB] + Afj]K]B|R]InB]-.. .... A
As you see from above: for the value A from the first matrix, it will be used at most nB times if A[i][k] not And the smart solution is taking advantage of that!!! the smart solution can be described as:
For each value [[]][k] in matrix A, if it is not zero, we calculate A[i][k] B[k][j] and accumulate it into C ][ (Key part: the C[ by now is not the final value in the result matrix !! Remember, in the brute force solution, the final value of C[i]li.. takes sum of multiplication values of K corresponding values from A and B?? here CJ i is only sum of some multiplication values, NOT ALJ. until the program is done)
BY NOW, it is very clear that, the value A[ from matrix zero, skip For-loop- calculation, which loop iterating times, and this the key part of why the smart solution is smart!!!
The smart solution code is as follow:
(Credit: @ yavincii; am having a different version of the solution, so am directly referencing the original version as reference to demonstrate how mine different).
Based the discussion ahove, the inner checking (if (B[k]j) !=
is actually necessary,
because whether or not we have that check, we still iterate nB times, since the operation C[i]li + A[i][k] [[k]ij]: inside the if-check is O(1) time)
So the smart solution can also be written follow by removing the check (which is my version
"Sparse matrix manipultion" helps, if we compress the first sparse matrix into rows of lists( in each row list, it contains (value, index ) pair) ), we actually don't need to go over all values in a row in matrix A when are calculating the final result matrix. But Overall, it does not help improve time algorithmatically!!
References:
1. credit: attachments.ariillc.gggoiiiii htt.tttwww........oooooooooooeie
Meta: URL:
WARC Filename:
CC MAIN 2021-31441 31lgggeeeee////77706655550020220 2AA.::/000..............A.AAA010
28

---

Published as a conference paper at ICLR 2026
sample from MegaMath-
Detailed Explanation of Smart Solution Matrix Multiplication Problem
The problem involves multiplying two sparse matrices A and B. We are given that A's column number is equal to B's s row number.
## Problem Statement
Given two sparse matrices A and B, return the result of AB.
Example: A=[
[1. 0, 0j. [,],0,3]
B=[
[7,0,0],
[0.0,0].
[0
The result AB calculated as follows: =B
[,,0,0),
[-7,0,3]
## Normal Approach to Matrix Multiplication
The normal way calculate the multiplication of two matrices A and B is take the values from the first line of and the first column of multiply the corresponding values, and them up. The final sum the value for the location of the first column and first row the final result matrix.
c[ili [[j][]JB[[j]] A[][I)-[[U]]] A[[[2)-B(1I)) +... A[][K]-B[K][i
## Brute Force Solution
The brute force solution involves calculating the final result at once.
## Smart Solution
The smart solution does not calculate the final result at once. Instead, it takes each value from A, calculates a partial sum, and accumulates it into the final spot.
each value A(i][k], if not zero, it will be used most times (where n Be0).e)g The smart solution can described as follows:
For each value A[i][k] in matrix A, if is not zero, calculate Afi][k] * B[k]j] and accumulate it into Cli I[i].
The key part of the smart solution that it skips a For-loop calculation if the value A[i][k] from matrix A is zero. This is the key part of why the smart solution is smart.
## Smart Solution Code
The smart solution code can he written as follows:
## Improvement Using Sparse Matrix Manipulation
Sparse matrix manipulation can help improve the solution by compressing the first sparse matrix into rows of lists (in each row list. it contains (value, index) pair). This allows us to avoid going over all values in row in matrix A when calculating the final result matrix. However, this does not improve the runtime algorithmatically.
29

---

Published as a conference paper at ICLR 2026
## Key Takeaways
The smart solution does not calculate the final result at once.
The smart solution skips For-loop calculation if the value A[i][K] from matrix A is zero.
Sparse matrix manipulation help improve the solution, but does not improve the runtime algorithnatically
## Example Multiple Choice Question
What the key advantage of the smart solution?
A) It calculates the final result at once.
B) It skips For-loop calculation if the value A[i][k] from matrix A is zero.
C) It uses sparse matrix manipulation improve runtime.
D) a higher time complexity than brute force solution.
Answer: B) skips For-loop calculation if the value A[i][k] from matrix A is zero.
Meta:
URL:
Sctp::
CC MAIN 2021 31/e SAA.://CCCCC.AAAO00000000000 900.MN.0222220000119999000000000
00074,wanc.y
30

---

Published as a conference paper at ICLR 2026
sample document from Finemath-4+
Detailed explanation of smart solution to algo problem beating 99.9% submission This post is about coding problem and why the solution pasted down below smart.
Problem:
Given two sparse matrices A and B, return the result of AB.
You may assume that A's column number is equal B's number.
Example:
=
[1, 0, 01, [,1
]
B=[ 17,0,01,
[0,0,0],
10 ]
[100]
1700
700
AB -103 |0000||000]||-000
1001
If it is of your interest, would recommend you take few minutes to think about how would approach this problem!
The main focus this post is to Dexplain detail why the provided solution smart and 2)make some improvements/tweaks in the code the smart solution to show you which part is really essential, 3) also i will briefly mention why Sparse Matrix Manipulation can help make some improvements top of the smart solution.
a) Originally, the normal way to calculate multiplication of two metrics A, and as follow: take the the all values from the first line of A, and all values from the first column of B, and multiply the corresponding values and sum them up. the final sum is the value for the location of first column and first row in final result matrix. Similarly. the value ]Li ] of result matrix C, which is C j calculated as:
C == Al доо]вгодí] + A [ A]][2//[11]] + АДІК
(which is the sum of each multiplication of corresponding values from row of A and K values from
column of
The Key we calculate this way, we finishing calculating the final value for the result matrix at once
Then brute force solution is follow:
b) The smart solution: the key part of smart solution is that: it does not calculate the final result at once, and it takes each value from A, and calculate and partial sum and accumulate it into the final spot: For example, for each value A[i][k], if it is not zero, it will be used at most nB times is B[O].length which be illustrated follow: Generally for the following equation:
Cüilli= = Δ[jlo]][jji] + А[] + A[i][]][[][]] + AÍjJK]B[[] [[i]KK][[Kii]
j can be from to nB, if we write all of them down, it will like following:
For from 0 nB:
31

---

Published as a conference paper at ICLR 2026
]=
B[O][0] А[] ATI1]821100 + А[][]]][III АБ]]]]и[К
C ] ][[] B[O][1] A[][]][[[[]]] +[12)BBB1) А[]]K][[iО| АЦ]]]][[][
C }=A[ [[O] B[0]/B]] Afill [ [[BB] Afi]2]B//2[nB] + Afj]K]B|R]InB]-.. .... A
As you see from above: for the value A from the first matrix, it will be used at most nB times if A[i][k] not And the smart solution is taking advantage of that!!! the smart solution can be described as:
For each value [[]][k] in matrix A, if it is not zero, we calculate A[i][k] B[k][j] and accumulate it into C ][ (Key part: the C[ J by now is not the final value in the result matrix !! Remember, in the brute force solution, the final value of C[i]li.. takes sum of multiplication values of K corresponding values from A and B?? here CJ i is only sum of some multiplication values, NOT ALJ. until the program is done)
BY NOW, it is very clear that, the value A[ from matrix zero, skip For-loop- calculation, which loop iterating times, and this the key part of why the smart solution is smart!!!
The smart solution code is as follow:
(Credit: @ yavincii; am having a different version of the solution, so am directly referencing the original version as reference to demonstrate how mine different).
Based the discussion ahove, the inner checking (if (B[k]jj != is actually necessary, because whether or not we have that check, we still iterate nB times, since the operation C[i]li + A[i][k] [[k]ij]: inside the if-check is O(1) time)
So the smart solution can also be written follow by removing the check (which is my version
"Sparse matrix manipultion" helps, if we compress the first sparse matrix into rows of lists( in each row list, it contains (value, index ) pair) ), we actually don't need to go over all values in a row in matrix A when are calculating the final result matrix. But Overall, it does not help improve time algorithmatically!!
References:
1. Image credit: attachments,arricee
Meta:
URL:
Sctp: or
WARC Filename:
MAIN-22222 31l.3///gnnenn///227065555555555
32

---

Published as a conference paper at ICLR 2026
sample document from MegaMath- Web
### Example I: Calculating Heat Transfer Through Conduction: Conduction Rate Through Ice Box Styrofoam ice box total area of 0.950 m20.950 m2 walls with an average thickness of 2.50 cm. The box contains ice, water, and canned beverages afThe inside of the box is kept cold by melting ice. How much ice melts in one day if the ice box is kept in the trunk of a car
Strattegy+**
This question involves both heat phase change (melting of icc) and the transfer of heat by conduction. To find the amount of ice melted, we must find the net heat transferred. This value can be obtained by calculating the rate of heat transfer by conduction and multiplying by time.
Soooution
Identify the knowns.
Identify the unknowns. We need to solve for the mass of the ice, We will also need to solve for the net
heat transferred to melt the ice,
Determine which equations use. The rate of heat transfer by conduction given by
[latex] [y boldsymbol [=) [/latex]]
The heat is used to melt the ice:
Insert the known values:
[latex]] boldxx]bboldymmbbl [late] b0dd
J/s) [[[atex]]
Multiply the rate heat transfer by the time
Set this equal to the heat transferred to melt the ice:Solve for the mass
|latex sizc="2")) boldsymbol frac timess00 textbf J}) (334%times10) times10"3) textbff J/kg] [[latx]]
H/ H/latex]]]]]]/ |/latxx "xx=""""[boldsymbbbb (frac
Discussion
The result of 3.44 kg. or about 7.6 lbs, seems about right, based on experience. You might expect to use about a 4 kg (7-10 Ib) bag of ice per day. A little extra ice is required if you add any warm food or beverages.
Inspecting the conductivities Table shows that Styrofoam a very poor conductor thus good insulator. Other good insulators include fiberglass, wool, and goose-down feathers. Like Styrofoam, these all incorporate many small pockets of air, taking advantage of air's poor thermal conductivity.
Substance
Thermal conductivity k (J/s.m. PC)
---
Silver 420
Copper 390
Gold 318
Aluminum 220
Steel iron 80
Steel (stainless) 14
Ice 2.2
Glass (average) 0.84
Concrete brick 0.84
Water 0.6
Fatty tissue (without blood) | 0.2
Asbestos 0.16
Plasterboard 0.16
Wood 0.08-0.16
Snow (dry) 0.10
Cork 0.042
Glass wool 0.042
Wool 0.04 Down feathers 0.025
Air 0.023
Styrofoam 0.010
Table 3. Thermal Conductivities of Common Substances" (1}
33

---

Published as a conference paper at ICLR 2026
combination of material and thickness is often manipulated to develop good insulators-the smaller the conductivityand the larger the thicknessthe hetter. The ratio ofwill thus large good insulator. The ratiois called thefactor. The rate of conductive heat transfer is inversely proportional The larger the value ofthe better the insulation. factors are most commonly quoted for household insulation, refrigerators. and the aik--unfottunately. it is still in non- metric units ft' (2). ( F.h/Btu, although unit usually goes unstated (1 British thermal unit Btu] is the amount energy needed to change the temperature 1.0 lb of water by 1.0 F). couple of representative values are anfactor of 11 for 3.5-in-thick fiberglass batts (picces) of insulation and anfactor of 19 for 6.5-in-thick fiberglass batts. Walls usually insulated with 3.5-in batts, while ceilings usually insulated with 6.5-in batts. In cold climates, thicker batts may be used ceilings and walls.
Note that in Table 3, the best thermal conductoors-ilverr copper, gold, and aluminum- are also the best electrical conductors, again related the density of free electrons in them. Cooking utensils are typically made from good conductors.
#### Example 2: Calculating the Temperature Difference Maintained by Heat Transfer: Conduction Through an Aluminum Pan
Water is boiling in an aluminum pan placed on an electrical element stovetop. The sauce pan has a bottom that is 0.800 em thick and 14.0 em in diameter. The boiling water is evaporating at the rate of 1.00 g/s. What is the temperature difference across (through) the bottom of the pan?
=*Strategy**
Conduction through the aluminum is the primary method of heat transfer here, and so we use the equation for the rate of heat transfer and solve for the temperature difference {-}
.*Solution
Identify the knowns and convert them the SI units. The thickness of the the, area of the pan,and the thermal conductivity,
Calculate the necessary heat vaporization of g of water:
Calculate the rate of heat transfer given that 1 g of water melts in one second:
Insert the knowns into the equation and solve for the temperature difference:
[latex size=""""] bolddyybbll (Q) (left frac (d] (kA) ([kA)rrrgg rrigh))))
size="2"J]
(tx)/txxtff] textbff (220) textbf( J/s) (ddopp' (extbf(m)) cdotp" { ( (1.5445)mmmmmm1 (-2} textbff 222) //late)]
boldsymbol (frae(8.0 (.000) timesl0"(33) (-3)
** ==Disuussiones
The value heat transferis typical an electric stove. This value gives a remarkably small temperature difference between the stove and the pan. Consider that the stove burner red hot while the inside of the is nearly because of its contact with boiling water. This contact effectively cools the bottom of the pan in spite of its proximity the very hot stove burner. Aluminum is such good conductor that only takes this small temperature difference produce heat transfer 2.26 kW into the pan.
Conduction is caused by the random motion of atoms and molecules. As such, it is ineffective mechanism for heat transport over macroscopic distances and short time distances. Take, for example, the temperature on Earth, which would unbearably cold during the night and extremely hot during the day if heat transport in the atmosphere was to be only through conduction. In another example, car engines would overheat unless there was more efficient way remove excess heat from the pistons.
Check Your Understanding
#+ 1ew How does the rate of heat transfer by conduction change when all spatial dimensions are doubled?
34

---

Published as a conference paper at ICLR 2026
Summary
Heaat conduction is the transfer of heat between two objects in direct contact with each other.
The rate of heat transfer(energy per unit time) is proportional to the temperature differenceand the contact areaand inversely proportional to the distancebetween the objects:
[latex] boldsymmoll (=) [/latex]
Meta: URL:
Sttp::/ httper//peeaaae wttppppssabaaaaeeeeeeppop www..o....wa....ooo.oooooo http://wwww....a.aaaeeeaoeeeeea.
WARC Filename:
crawl data/CC MAIN 2019 M.legpmm///55555555444222.CCC.C 201901 17062012-20115 708401 00486.066.0 ware-gz
0.767403781414052
Math score: 0031407379555577555
35

---

Published as a conference paper at ICLR 2026
sample document from Cemotron-CC-MM
#** Example I: Calculating Heat Transfer Through Conduction: Conduction Rate Through Ice Box
Styrofoam ice box has total of 0.950 m² and walls with average thickness of 2.50 The box contains ice, water, and canned heverages \(0 ((irr)))) (CX) V. The inside box is kept cold by melting ice. How much ice melts in one day the ice box is kept in the trunk of a car at ((35.0 Of } (extbf(C))
Sttatteeey**
This question involves both heat phase change (melting of ice) and transfer of heat by conduction. To find the amount of ice melted, we must find the net heat transferred. This value can be obtained by calculating the rate of heat transfer by conduction and multiplying by time.
Solution=a
1. Identify the knowns.
M
A 0.950), /text/m ) d 2.50%, ( (text//m \. T.1 ("()eerre(ttexttieeeee 86,400), (text(().
35.0 T.2 35.0 ((cire)) 1, 1/, 24), (tex((hoouss] -tt.://e//h///o//oooll
\
2. Identify the unknowns. We need to solve for the mass of the ice, \(m \). We will also need to solve for the net heat transferred melt the ice, X Q ()).
Determine which equations to use. The rate of heat transfer by conduction is given by A
(rac ffac (-T.)))))
4. The heat is used to melt the ice: \(Q mL. ((exx() V).
5. Insert the known values:
\[
(Qaa) \rrac /dicc
[text(m)
(35.0" ((cire) (exx((C-(0()(iii (eeet(CCC/////eeeee/eccc((C///o) (0.050)),, (tex/(mm) 13.3), (text((/s)
\
Multiply the rate of heat transfer by time ((x()\)exx(day)) 86,400), (tex((s)))) V)): M
Q
(Q/jjt = (13.3), (10 ((xx)))))))))) = 1.15 (times (exx),
Set this equal to the heat transferred to melt the ice: \ Q= mL. )), Solve for the mass ((
m Q):
\[
|frac (L. ((exxbb((1)) (1.1 \times 10'6), (ex) \times (ex))))))
=3.44), (kxt
V
*+ Discussionss
The result of 3.44 kg, about 7.6 lbs, seems about right, based experience. You might expect about a 4 kg (7-10 lb) bag of ice per day. A little extra ice is required if you add any warm food or beverages.
Inspecting the conductivities in Table shows that Styrofoam is very poor conductor and thus good insulator. Other good insulators include fiberglass, wool, goose-down feathers. Like Styrofoam, these all incorporate many small pockets of air, taking advantage of air's 'i's poor thermal conductivity.
Substance Thermal Conductivity
Substance
Thermal conductivity \(k) (I/s-m-PC)
Silver
420
36

---

Published as a conference paper at ICLR 2026
Copper
|90
Gold
318
Aluminum
220
Steel iron
80
Steel (stainless)
14
Ice
2.2
Glass (average)
0.84
Concrete brick
0.84
Water
0.6
Fatty tissue (without blood) 0.2
Asbestos
0.16
Plasterboard
0.16
Wood
0.08-0.16
Snow (dry)
0.10
Cork
0.042
Glass wool
0.042
Wool
0.04
Down feathers
0.025
Air
0.023
Styrofoam
0.010
Table 3. Thermal Conductivities of Common Substances#
combination of material and thickness is often manipulated to develop good insulators-the smaller the conductivity \ ((k] \) and the larger the thickness (( d (), the better. The ratio of V d/k V) will thus be large for good insulator. The ratio (( d/k \) is called the ( R \) factor. The rate of conductive heat transfer is inversely proportional to \ R \). The larger the value of \( R \). the commonly quoted for household insulation,
better the insulation. \( \) factors refrigerators, and the like-unfortunately. it is still in non-metric units of ft FF.h/Btu, although the unit usually goes unstated (1 British thermal unit [Btu] is the amount of energy needed change temperature of 1.0 water by 1.0 FF) couple of representative values are an ( R \) factor of for 3.5-in-thick fiberglass batts (pieces) of insulation and an )( R \) factor of 19 for 25-in-thick fiberglass batts. Walls are usually insulated with 3,5-in batts, while ceilings usually insulated with 6.5-in batts. In cold climates, thicker batts may be used in ceilings and walls.
Figure 4.* The fiberglass batt is used for insulation of walls and cailings to prevent heat transfer between the inside of the building and the outside environment.
Note that in Table 3,
thermal conductors-siin silver, copper, gold, and aluminum-an also the best electrical conductors, again related to the density of free electrons in them. Cooking utensils are typically made from good conductors.
### Example Calculating the Temperature Difference Maintained by Heat Transfer: Conduction Through an Aluminum
Water is boiling in an aluminum placed on electrical element stovetop. The saucepan has bottom that is 0.800 thick and 14.0 em in diameter. The boiling water evaporating the rate of 1.00 g/s. What the temperature difference across (through) the bottom of the pan?
=*SStrategy**
Conduction through the aluminum is the primary method of heat transfer here, and we the equation for the rate heat transfer and solve for the temperature difference.
\I
T2-T = (rac (00 (kA))
**Solution
1. Identify the knowns and convert them to the SI units.
The thickness of the pan, \(d= 0.800%.)text(emm = 8.0 times 10'(-3] (extt(m) !).
The of the pan, {{ A (pi(0. 14/2222/
1.54 \iimes 10 (-2)1, (text(m)
The thermal conductivity, ( 220), (tex(()e))JJ (exm/ (ccoo") (ccoot((cirre (text(C) 1).
37

---

Published as a conference paper at ICLR 2026
Calculate the necessary heat of vaporization g of water:
M[
==mL mL [[eexbf[v]] = (1.00 (times (e((-)) \times (tex)))))) 2256), 2256), (Jex
V
3. Calculate the rate of heat transfer given that of water evaporates in one second:
M
Qh 2256), (extt }2.26), (text((WW)
V
4. Insert the knowns into the equation and solve for the temperature difference:
T2-T-11 (ta [ef((ffrae(]] (kA) wrght) (2256), (ext( ( (8.00 \times
10 (. ((xxt/m) ( (tex cedot text m} (edl)edooo"" (door((eirc) (exree, ttimes
(0-(22 (text/mm) 5.33" firic)) [ext[C].
*.Diicussionss
The value for the heal transfer \ Q/t 2.26', (Ke or )2256), (Jest((J/s) V) is typical for electric stove. This value gives remarkably small temperature difference between the stove and the pan. Consider that the stove burner is red hot while the inside of the pan is nearly \(100' (1100*))ar (iire)) (text(CC) because its contact with boiling water. This contact effectively cools the bottom of the pan in spite of its proximity to the very hot stove burner. Aluminum such good conductor that it only takes this small temperature difference to produce heat transfer of 2.26 kW into the pan.
Conduction is caused by the random motion of atoms and molecules. As such, it is an ineffective mechanism for heat transport over macroscopic distances and short time distances. Take, for example, the temperature extremely hot during the day if heat transport in the atmosphere was be only through conduction. another example, car engines would overheat unless there was a more efficient way to remove excess heat from the pistons.
the Earth, which would be unbearably cold during the night and
### Check Your Understanding
1: How does the rate of heat transfer by conduction change when all spatial dimensions are doubled?
#### Summary
Heat conduction the transfer heat between two objects direct contact with each other.
The rate of heat transfer ) Q/t \) (energy per unit time) is proportional to the temperature difference (TT22-_11 ()) and the contact area (( A and inversely proportional to the distance
\(d between the objects:
frac (Q) (fae//===)(ff(AAAATTTTTT])))
Meta: URL:
WARC Filename:
00486.warde warc.
38

---

Published as a conference paper at ICLR 2026
sample document from OpenWehMart
#** Example I: Calculating Heat Transfer Through Conduction: Conduction Rate Through Ice Box
A Styrofoam ice box has a total area of 0.950 m20.950 m2 and walls with an average thickness of 2.50 cm. The box contains ice, water, and canned beverages atThe inside of the box is kept cold by melting ice. How much ice melts in one day if the icc box is kept in the trunk of a car
Strategy
This question involves both heat for phase change (melting of ice) and the transfer of heat by conduction. To find the amount of ice melted, we must find the net heat transferred. This value can be obtained by calculating the rate of heat transfer by conduction and multiplying by time.
Solution
1. Identify the knowns.
2. Identify the unknowns. We need to solve for the mass of the ice, We will also need to solve for the net
heat transferred to melt the ice,
Determine which equations to use. The rate of heat transfer by conduction is given by
$ bbolsyymoll==$S
4. The heat is used to melt the ice:
Insert the known values:
$ boldsymbol
J/s] J.$
Multiply the rate of heat transfer by the time O):
7. Set this equal to the heat transferred to melt the ice:Solve for the mass
[latex size=""""\\ boldsymbol (ffraaaaaaar (L. (Lextbb ([an(
(ollss"ymbol {1.15% timmss textbt J)} times J/kg] [[latex]
Discussion
The result of 3.44 kg. about 7.6 lbs, about right, based experience. You might expect about 4 kg (7-10 Ih) bag of ice per day. A little extra ice is required if you add any warm food beverages.
Inspecting the conductivities in Table shows that Styrofoam is very poor conductor and thus good insulator. Other good insulators include fiberglass, wool, and goose-down feathers. Like Styrofoam, these all incorporate many small pockets of air, taking advantage of poor thermal conductivity.
Substance Thermal conductivity
(J/s.m. (C)
Silver
Copper 390
Gold 318
Aluminum 220
Steel 80
Steel (stainless) 14
Ice 2.2
Glass (average) 0.84
Concrete brick 0.84
Water 0.6
Fatty tissue (without blood) 0.2
Asbestos 0.16
Plasterboard 0.16
Wood 0.08-0.16
Snow (dry) 0.10
Cork 0.042
Glass wool 0.042
Wool 0.04
Down feathers 0.025
Air 0.023
Styrofoam 0.010
Table 3. Thermal Conductivitics of Common Substances
A combination of material and thickness often manipulated to develop good insulators-the smaller the conductivityand the larger the thicknessthe better. The ratio ofwill thus be large for a good insulator. The ratiois called thefactor. The rate of conductive heat transfer is inversely proportional to
39

---

Published as a conference paper at ICLR 2026
The larger the value ofthe hetter the inallation..actorn insulation, refrigerators,
commonly quoted for household tike-unfortunately is still in non-metric units ft2. F.h/Btu although the unit usually goes unstated (1 British thermal unit [Btu] is the amount of energy needed to change the temperature of 1. lb of water by 1.0 "F). A couple of representative values anfactor of 11 for 3.S-in-thick fiberglass batts (pieces) of insulation and anfactor of 6.5-in-thick fiberglass balts. Walls are usually insulated with 3.5-in batts, while ceilings are usually insulated with 6,5-in batts. cold climates, thicker batts may be used ceilings and walls.
Note that in Table 3, the best thermal conductors- silver, copper, gold, and aluminum-ar also the best electrical conductors, again related the density of free electrons them. Cooking utensils are typically made from good conductors.
###l Example 2: Calculating the Temperature Difference Maintained Heat Transfer: Conduction
Through an Aluminum
Water is boiling in an aluminum pan placed on an electrical element on stovetop. The sauce pan has a bottom that is 0.800 cm thick and 14.0 cm in diameter. The boiling water is evaporating at the rate of 1.00 g/s. What is the temperature difference across (through) the bottom of the pan?
Strategy
Conduction through the aluminum is the primary method of heat transfer here, and so we use the equation for the rate of heat transfer and solve for the temperature difference. frac (QHt) (eft( (rac (rac (ightt) (rightt)([/aateee
[latex "ra===""]]
Solution
1. Identify the knowns and convert them to the SI units. The thickness of the pan,the area of the pan,and the thermal conductivity,
Calculate the necessary heat of vaporization of g of water:
Calculate the rate heat transfer given that
water melts one second:
Insert the knowns into the equation and solve for the temperature difference: [Matexx)) (Q))) left() frac (raacd)) (aatt boldsymbol (fraee8.000 (0mes10((33)) textbff m) ((220) textbf( J/s) |cdotp' (cdotp) (m) cdotp" { cire)) textbf((C) (1.5ee
[latex ""oe=""""""ollyymmm"
"2=]
textbff m))))} }
Discussion
The value for the heal transferis typical for an electric stove. This value gives remarkably small temperature difference between the and the pan. Consider that the stove burner red hot while the inside the pan nearly because of contact with boiling water. This contact effectively cools the bottom of the pan in spite its proximity to the very hot stove burner. Aluminum is such good conductor that only takes this small temperature difference to produce heat transfer of 2.26 kW into the pan.
Conduction caused by random motion atoms and molecules. As such, an ineffective mechanism for heat transport over macroscopic distances and short time distances. Take, for example, the temperature extremely hot during the day if heat transport in the atmosphere conduction. another example, car engines would overheat unless there was more efficient way to remove excess heat from the pistons.
the Earth, which would be unbearably cold during the night and be only through
1: How does the rate of heat transfer by conduction change when all spatial dimensions are doubled?
Summary
Heat conduction is the transfer of heat between two objects in direct contact with each other.
The rate of heat transferrenergy per unit time) is proportional to the temperature differenceand the contact areaand inversely proportional to the distancebetween the objects:
S bolldyymbol
Meta:
URL:
Sctp:/ /ttppspbsoossdee..oooooooooopp..
WARC Path:
x3 Riccommnnnawwwlkaraww datplllaaaCCCCC MAIN 2AA.MMAIN------AAAM 4AA.AA/AAAAAAAAA000A404AAAA5AA50 MAIN 201901 17062012.
201901 17084012 00486 warcg.
40

---

Published as a conference paper at ICLR 2026
sample document lynx output
Example 1: Calculating Heat Transfer Through Conduction: Conduction Rate Through Ice Box
A Styrofoam ice box has a total area of 0.950 m20.950 m2 and walls with an average thickness of 2.50 cm. The box contains ice, water, and canned beverages at |boldsymbol ('iirc)) (txxbf((C) The inside of the box is kept cold by melting ice. How much ice melts in one day if the ice box is kept in the trunk of car (boldsymbol (asssymmol(33.00) (35.0"() ((cire) (exttf((C]]
Strategy
This question involves both heat for phase change (melting of ice) and the transfer of heat by conduction. To find the amount of ice melted, must find the heat transferred. This value obtained by calculating the rate of heat transfer by conduction and multiplying by time.
Solution
1. Identify the knowns.
[boldsymbol A=0.950 textbff -d=:::d==2.00) cm)==002550 m):A.T 1=0" { ((eire)) textbf(C):):T 2=35.0 (fcrre((rrreeeeeee (exiif/(e. (= day)=24 texthf (oour))86,,400) te0tte $}}.
2.
Identify the unknowns. We need to solve for the mass of the ice, [mo]dsymbollm] We will also need solve for the net heat transferred melt the ice. (boldsymbbol()..
Determine which equations to use. The of heat transfer by conduction is given by bboldsymbol (QH } H..
(boldsymbol( (f
4. The heat is used to melt the ice: (boldsymbol (Q=mL_[(exxx(((]]]
Insert the known values:
( (Qa boldkymbol
boleee))bblddmmboo/))//o (holdkymbol (0.0000 J/s
Acdotp textbf(mn m| ,edotp" ( ('ciire) (extbf(C) (0.95) textbff (2)300033((()iim (trrs00t))tttettttttttCCCCCCCCC Atextbr(C
-00 ('((cirr)) textbf (0.0250)) m } [latex]]
3/s}
(os))-[[aa J-L/latex
Multiply the of heat transfer by the time (aoldsymbo((1))exxtttt day ): ) textbff s}}
bolldyybol (illsyymoo(Q(((ppppp Q=((Q/ttt((11 13.3 t.xttff
J/s (85)))8864000eeee (otpe//ber(sssss.....k//nn100100 JH}-
7. Set this equal to the heat transferred to melt the ice: (atpllyymmol(mmmmmeeeeetttto/oee (()exxb((r))) Solve for the mass (boldsymbol(m))
(mols [latex size="2"J boll==/2"bbollsymbboll boldsymbol (LL
boldsymbol [latex sizc="2"]] boldsymbol (frac{ 1.15) 10'6 J)) (334333441m111000 10mms textbf( J/kg] [/latex]
kg})
Discussion
The result 3.44 kg. or about 7.6 lbs, seems about right, based on experience. You might expect to use about 4 kg (7-10 Ib) bag of ice per day. A little extra ice is required if you add any warm food or beverages.
Inspecting conductivities in Table shows that Styrofoam very poor conductor and thus good insulator. Other good insulators include fiberglass, wool, and goose-down feathers. Like Styrofoam, these all incorporate many small pockets of air, taking advantage of air's poor thermal conductivity.
Substance Thermal conductivity
(J/s-m-C) Silver 420
Copper 390
Gold 318
Aluminum 220
Steel iron 80
Steel (stainless) 14
Ice 2.2 Glass (average) 0.84
Concrete brick 0.84
Water 0.6
Fatty tissue (without blood) 0.2
Asbestos 0.16
Plasterboard 0.16
Wood 0.08-0.16
Snow (dry) 0.10
Cork 0.042
Glass wool 0.042
41

---

Published as a conference paper at ICLR 2026
Wool 0.04
Down feathers 0.025
Air 0.023 Styrofoam 0,010
Table Thermal Conductivities of Common Substances"1
A combination of material and thickness is often manipulated to develop good insulators-the smaller the conductivity [boldsymbol(k] and the larger the thickness |boldkymbolfd) the better. The ratio of [bolssymbol will thus be large for good insulator. The ratio \ (d/k)
boldsymbol [d/k) is called the /boldsymbol factor. The rate of conductive heat transfer inversely proportional to boldsymbol [R]. The larger the value of bollssybol (R), the better the insulation. (boldkymbol factors are most commonly quoted for household insulation, refrigerators, and the tike-unfortunately. it is still in non-metric units of ft FF.h/Buu, although unit usually goes unstated (1 British thermal unit [Btu] is amount energy needed to change the temperature of 1.00 lb of water by 1.0 'F). couple of representative values are an boldsymbol IR) factor of 11 for 3.5-in-thick fiberglass hatts (pieces) of insulation and an boldsymbol factor of 19 for 6.5-in-thick fiberglass hatts. Walls are usually insulated with 3.5-in batts, while ceilings are usually insulated with 6.5-in batts. In cold climates, thicker batts may be used ceilings and walls.
The figure shows two thick rectangular pieces of fiberglass batt lying one upon the other.
Figure 4. The fiberglass batt is used for insulation of walls and ceilings prevent heat transfer between the inside the building and the outside environment.
Note that in Table 3, the best thermal conductors-ssilve, copper, gold, and aluminum-are also the best electrical conductors, again related the density of free electrons in them. Cooking utensils typically made from good conductors.
Example 2: Calculating the Temperature Difference Maintained by a Heat Transfer: Conduction Through an Aluminum Pan
Water is boiling in an aluminum pan placed an electrical element stovetop. The sauce pan has a bottom that 0.800 em thick and 14.0 cm in diameter. The boiling water is evaporating at the rate of 1.00 g/s. What is the temperature difference across (through) the bottom of the pan?
Strategy
Conduction through aluminum the primary method of heat transfer here, we use the equation for the rate of heat transfer and solve for the temperature difference(.] (olddyymol(TT2-T.I.. [latex size="2""J boldsymbol (oarff((raaffaa (frac (fd (ra)(d)((k)))) right) }-[[latex]
Solution
Identify the knowns and convert them to the SI units.
The thickness of the pan, (boldsymbol(d-008000 textt cm) c =8.0 times 1imees10((33 (1 m)),
the of the pan, (boldsymbol (A=), 14 textbf( m| 2==/1=1=1)mmmm000 (0mees000000 textbff
m)² and the thermal conductivity, (boldsymbol( textbf( J/s) cdotp' tcxtbf(m))
edotp" { f'riee //cirre//extbf((C)) (extbf((C)..
2. Calculate the necessary heat of vaporization of g water:
(boldyymoll
times 10 (-3) textbf [ (2256) times (imee 10'(
J/kg))22256 textbf( J}}-
Calculate the rate heat transfer given that g water melts one second:
(boldsymool (=)::226'( J/s or 12.26',textbf kW}).
4. Insert the knowns into the equation and solve for the temperature difference:
(.2lsyymbl((T2-TT1333. [latex "=e=="2"j.ooldsymbbo
(t) (eff())))ee(t(facc((((()) (kA)))
right) [boldsymbol|=(2255 textfff 3/s})) [latex "ra=="""T.boldsymbii
(8.00)
(imes10'(33)) m)) ((20 textbff J/s) (cd)tcp)))) texthffm (cdopp' ( ('vcire) (C)) (1.5. (i.4e textbff m) [[latex]] m}}2))H([lttxxx
(=):))mmol(=)5..33300 ((ire (eirc)textbb((C (textb((C))
Discussion
The value heat transfer (boldsymbol (Q)): textbff &W or (2256)ttxxbll } typical for an electric stove. This value gives a remarkably small temperature difference between the stove and the pan. Consider that the stove burner is red hot while the inside of the pan is nearly boldsymbol 100 ('irr)) ((eiic))texttee (extbf((C) because of its contact with boiling water.
42

---

Published as a conference paper at ICLR 2026
This contact effectively cools the bottom of the pan in spite of its proximity the very hot burner. Aluminum is such good conductor that only takes this small temperature difference to produce heat transfer of 2.26 kW into the pan.
Conduction is caused the random motion of atoms and molecules. As such, it is ineffective mechanism heat transport over macroscopic distances and short time distances. Take, for example, the temperature on the Earth, which would be unbcarably cold during the night and extremely hot during the day if heat transport in the atmosphere was to be only through conduction. In another example, car engines would overheat unless there was more efficient way remove heat from the pistons.
Check Your Understanding
I: How does the rate of heat transfer by conduction change when all spatial dimensions doubled
Summary
Heat conduction is the transfer of heat between objects in direct contact with each other.
rate heat transfer (boldsymbol (Q/t) (energy per unit time) is proportional to the temperature difference ( (T.2-T.1) and the contact area (boldsymbol [A} and inversely proportional to the distance (dl between the objects: (bldsymboll (t)) [latex]] boldsymbol(=)/Aaeee (boldsymbol( frac (KA(T d}}.
Meta: URL:
hctp: pptesabbokk
WARC Path:
crawl-datw
warc.gz
A.4 HYPER-PARAMETTRR
For phase training, we trained transformer model token horizon of trillion tokens. We used a sequence length of 8192 and global batch size of 768 (629 1456 tokens per batch). we used a peak learning rate of 6 x 10-4 and warmup over 8.3 billion toknes; we used cosine learning rate decay with a minimum value equal to 1% of the peak value, and weight decay of 0.1. We use Adam W optimizer (Loshchilov Hutter, 2017) with parameters 31 0.9 and 32 0.95, and a gradient clipping threshold of 1.0.
We pre-train our model using Megatron-I.M we rely on Transformer Engine for FP8 support. We use 8-way tensor model parallelism (Shocybi et al. 2020) with sequence parallelism (Korthikanti ctt al. 2022) for additional memory savings, and 768-way data parallelism with optimizer state dis- tributed over the data-paralle parallel replicas (Rajbhandari et al. 2020). We trained the Nemotron-T 8B transformer model on 2048 NVIDIA H100 GPUs.
In Phase training, annealing experiments were conducted with total token counts of 100 billion and 300 billion. We employed a linear learning rate decay schedule with no warmup phase, using initial learning rate of 2 10-1, Optimization was performed using the AdamW optimizer with 81 = 0.9, 32 = 0.95, and a gradient clipping threshold set to 1.0 to ensure stability during training.
A.5 PROMPT FOR HTML DUMP CLEANUP AND MATH NORMALIZATION
During the LLM-based cleanup stage, we employ the following prompt template to remove boil- erplate content from raw HTML dumps. Specifically, we utilize the Phi-4 model to identify and
43

---

Published as a conference paper at ICLR 2026
extract meaningful content while discarding irrelevant HTML artifacts. Additionally, it also guide the model to unify math representation latex. The template used is as follows:
You are given raw text extracted from an HTML page. Process this text to extract only the meaningful content, following these strict guidelines:
1. Retain only the main content and its associated titles. Remove all boilerplate, navigation menus, sidebars, footers, headers, related articles, spam comments, in- teractive elements, and advertisements.
2. Preserve all mathematical content-this includes theorems, formulas, proofs, def- initions, explanations, and any mathematical references.
3. Retain relevant comments and references if they contribute meaningfully to the understanding of the content (e.g., clarifications, citations, author notes). Discard irrelevant or low-quality comments.
4. Format all mathematical expressions using LaTeX enclosed single dollar signs on each side ($), not [l, O, or other variants.
5. Do NOT answer or respond to any questions or prompts that appear in the document. If a question is part of the content, keep verbatim, but do not generate answer or explanation.
6. Do not remove or discard any part of the code. any code blocks contain errors formatting issues, make minimal changes to make them runnable, but otherwise leave them exactly as they are.
7. Fix typos, grammatical mistakes, and unclear phrasing. Rewrite sentences when necessary to improve clarity, coherence, and flow, while preserving the meaning and style of the original content.
8. Ensure the output clean, well-structured, and natural. Format titles, sections, equations, and tables to produce high-quality, publication-ready text.
9. If the page contains no meaningful content (e.g.. it's entirely boilerplate, menus, or ads), return exactly: "NO USEFUL CONTENT"
[tet:
Task: Start directly with the processed text. DO NOT include any introductory framing phrases such as "Here is the cleaned content", "Processed output", or similar. End your response after the cleaned content.
A.6 PROMPT FOR SCIENTIFIC CONTENT ASSESSMENT
For clarity and reprodueibility, we provide the full prompt and scoring rubric utilized for the LLM- aided quality assessment described in 3.3 This detailed prompt defines the exact criteria used by the automated judge (gpt-5.. 1) for scoring math preservation, code preservation, faithfulness, and readability, ensuring standardized and objective quality comparison across all evaluated datasets.
44

---

Published as a conference paper at ICLR 2026
You expert evaluator. Your primary task is compare the ORIGINAL. DOCUMENT against the CONVERTED DOCUMENT to assess how well the core scientific content is preserved.
The converted document's goal to retain only the core scientific content while uni- fying math to the target LaTeX format.
Acceptable Conversion Outcomes (No Penalty)
The CONVERTED DOCUMENT's goal is Content Cleanup and Math Unification into Latex. You must NOT penalize the converted document for the following:
Removal of boilerplate, footers, navigation, references, bibliographies, etc.
Paraphrasing: Rewording of main content or headings, provided the original meaning is preserved.
Criteria for Penalization
Penalties are applied only when the integrity of the core scientific content com-
promised:
Essential Content Missing: Core scientific text, math, or code is missing.
Meaning Altered: Changes to text, math, or code that fundamentally change the meaning alter program behavior.
You must evaluate dimensions using this scoring scale:
Not preserved severely corrupted
Partially preserved/ major issues
2 Mostly preserved / minor issues
3 Perfectly preserved
N/A Category not applicable because the ORIGINAL contains no content of that type
Definitions:
Math Preservation (0-3 or "N/A")): Evaluate the correctness AND retaining of the math equations and expressions. Note that original math web can appear in various forms such as:
MathJax KaTeX
MathMI.
SVG/PNG equation images
Inline symbolic expressions text
All math must be converted to proper LaTeX in the CONVERTED DOCUMENT and retained with its meaning intact.
Do NOT penalize:
harmless formatting differences (whitespace, line breaks, equivalent LaTeX forms, etc)
conversion from any math format (MathML, KaTeX, images, etc.) into LaTeX
Penalize when:
math equations missing, incomplete, replaced with prose, altered in meaning.
MathML, MathJax, or other math formats are not retained or not converted to LaTeX
45

---

Published as a conference paper at ICLR 2026
Non-standard LaTeX unnecessary commands used that could misrepresent the math (c.g., \di cdisplaystyl in inline math)
IMPORTANT: If the original contains multiple equations and the converted docu- ment keeps only one, removes mathematical structure, the score MUST be 1 or lower.
Score 0 only when math equations are stripped, or severely corrupted.
Code Preservation (0-3 or "N/A"): correctness, completeness, syntax, indenta- tion, parsability, and structural fidelity.
Minor whitespace changes equivalent code rewrites are NOT penalized when functionality is preserved.
Penalize only when missing main code content or changes that alter pro- gram behavior or functionality.
Faithfulness (0-3): This dimension assesses the overall integrity and inclusion of the core scientific content.
Penalize Only If:
Main content missing.
Meaning is altered (e.g., hallucinations, fabricated steps, meaning-
changing rewrites).
*
Scientific integrity is compromised.
No Penalty For:
Missing boilerplate, navigation, references, bibliographies, etc. Correcting corrupted math or code fragments to their intended mean- ing.
*
Minor changes in non-essential sections.
Paraphrased headings/content when scientific integrity is preserved.
Readability (0-3): This score evaluates the overall clarity, organization, and co-
herence of the converted document, including:
Logical structure and organization
Clear, descriptive section headings
Proper paragraphing and spacing
Smooth flow and coherence
Absence of jarring formatting or fragmentation
Whether the text reads like clean, human- written explanation
Scoring guide:
3 Highly readable (well-structured, coherent, clearly sectioned; polished and easy to follow)
* 2 2 Mostly readable (minor structural or coherence flaws)
* Hard to read (poor structure, missing context, unclear or disorganized)
0 Very poor readability (fragmented, confusing, incoherent, minimal structure)
Readability does NOT affeet correctness of content, but reflects presenta- tion quality.
Rules:
Use "N/A" ONLY if the ORIGINAL document contains no math or no code, re- spectively.
Output must be STRICT JSON. No commentary before or after.
The "notes" list should contain only meaningful issues, written as short natural language strings. Do NOT include trivial formatting changes or minor spacing differences.
46

---

Published as a conference paper at ICLR 2026
Include meaningful notes. If meaningful issues, output [].
Do NOT escape JSON with backticks.
Here is an example of the EXACT output format you must produce:
"math": 3.
"code": "N/A" "faithfulness":
"readability":.
"notes":
"Missing the 'Related Work' section from the original document",
"Paragraph order slightly altered in Section 2"
Here are the documents to evaluate:
ORIGINAL DOCUMENT:
{original_text]
CONVERTED DOCUMENT:
[text]
A.7 NEMOTRON-CC-MAN CORPUS STATISTICS BEFORE QUALITY FILTERING
To ensure full transparency of data construction pipeline, report the scale of the corpus prior to applying the quality-based filtering and deduplication steps described in $2.2 and $2.3 respectively. Table summarizes the number of documents and total tokens associated with each quality score assigned during the initial Common Crawl extraction stage. Following the practice established by the FineMath dataset, documents assigned scores of and 2-which correspond to the cowest-quality portions of the corpus were removed before subsequent processing.
Score
Documents
Tokens (B)
Score
35,171,234
78.71
Score 2
68,120.438
125.96
Score 3
64,171,676
92.99
Score 4
48,312,962
56.79
Score 5
227.127
0.29
Table 5: Corpus statistics prior quality filtering.
A.8 DATA MIXTURES USED DURING PRE-TRAINING EXPERIMENTS.
To evaluate the value of our data, setup pretraining experiment. We used the same mixture as used IVIDII al. (2025). The data mixture spans eight broad content categories: web crawl, mathematics, Wikipedia, code, academic publications, high quality crawl subset (Crawl++), multi- lingual corpora, and synthetic instruction-style datasets. The Crawl++ category aggregates curated web-derived sources such as OpenWebText, BigScience, and Reddit. The multilingual component covers nine languages: Spanish, German, French, Italian, Portuguese, Chinese, Japanese, Korean, and Russian. To construct the mixtures, NVIDIA al. (2025) applied uniform weighting within datasets of the same quality tier, and they assigned greater weight to datasets of higher quality.
Following NVIDIA et al. (2025). adopt phased pretraining strategy. Phase emphasizes data diversity by leveraging a broad and heterogeneous mixture sources. contrast, Phases 2 pri- marily focus higher-quality datasets, such Wikipedia and academic corpora, refine model
47

---

Published as a conference paper at ICLR 2026
Math
2.9%
Multilingual
5
Crawi++
4. 0%
Academic
8.8%
Web craw!
Code
59.0 0%0%%
20.0%
Phase data mixture.
SFT-Style
Math
14.0%
Wiki
1.7%
Multilingual
Web crawl 40.7%
3.7%
Crawi++
Academic
12.0%
Code
20.0%
(b) Phase 2 data mixture.
Figure Data mixtures for each phase of pretraining experiments presented in Table 2
performance. The data mixtures used in each phase and phase 2 are shown in Figure We begin by pretraining Nemotron-T 8B transformer model using Phase mixture for a total of 9 trillion tokens. To assess the value of each of the math datasets, then conduct series of annealing experiments using the phase mixture as a base. each variant, we substitute the math dataset with target dataset under evaluation, assigning it fixed weight of 30%. The remaining 70% of the mixture is rebalanced proportionally among the other data sources to maintain a consistent total. Table 2 show the results for model trained on an additional 100 and 300 billion token budget.
48


Reviews:
[{"review_id": "Ze4sCUtdKM", "reviewer": "Reviewer_upxT", "summary": "This paper introduces nemotron-cc-math, a math dataset created by utilizing the Lynx text-based browser. By using the browser, the HTML can be rendered as structured format with equation and code layout as human read them. The content is then fed through an LLM (Phi-4 14B) to clean up the boilerplates. The final dataset retains data passing Fineweb classifier (3+), which contains 133B tokens, which is one of the largest set at this quality.", "questions": "How much total tokens are their without the quality filter? Is there an estimate?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- Robust and Proper Pipeline: The lynx + LLM-cleaner pipeline is an effective solution. It addresses the failure mode of previous web math extractors that corrupt math and code. The qualitative examples provided clearly demonstrate its superiority in preserving structure. While this method makes sense, both the lynx rendering and an 14B LLM cleaner are expensive for many practitioners. So the open sharing of this resource will help the community significantly.\n- The paper delivers a dataset that is both larger and higher quality than existing open-source alternatives. The 133B high quality portion is larger than FineMath, though smaller than the 300B MegaMath.\n- Strong experiment results: the methods are tested on an 8B mid-training checkpoint, with 100B and 300B experiments. The scale of the experiment should be sufficient and it shows good results on a range of benchmarks.\n- Once again, the contribution to open science and open source of this work is commendable.", "weaknesses": "This paper can benefit from additional experimental settings. The main experiments are conducted at the mid-train setting. Will there be some confounding factor from the base model itself? Further, would larger amount of unique tokens be helpful and how much repetition can this dataset be used?\n\nThe readers would also benefit from learning about the filtered out portion, i.e., Nemotron-cc-math-1-3. The token size, quality and corresponding model performance may provide valuable information.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TaYZZfThmr", "reviewer": "Reviewer_yQeP", "summary": "This paper introduces Nemotron-CC-Math, a large, high-quality math corpus built from Common Crawl. It uses a domain-agnostic extraction pipeline: layout-aware `lynx` rendering plus structure-preserving LLM cleaning to unify MathJax/KaTeX/MathML into LaTeX, keep equation and code structure, and remove boilerplate.\n\nIt releases two datasets: Nemotron-CC-Math-3+ (133B tokens) and Nemotron-CC-Math-4+ (52B tokens), claimed to surpass MegaMath, FineMath, and OpenWebMath; 4+ has about 5.5× the tokens of FineMath-4+. As for experiments, the authors show that pretraining an 8B model on this data yields 4.8-12.6 on MATH and 4.6-14.3 on MBPP+, with additional gains on benchmarks like MMLU.", "questions": "I do have some questions for the authors:\n\n1. why pre-train on math also boosts code performance? if so, have you compared this with other math&code-related datasets? such as stack-edu, megamath-code?\n\n2. How do you detect and constrain LLM over-editing or hallucinations (e.g., symbol renaming, citation mismatches, skipped derivations)? Did you conduct manual sampling review and an inter-annotator agreement (IAA) evaluation?\n\n3. Could you report quantitative results for LaTeX/code parseability, structural consistency (e.g., AST-based edit distance), and rendering consistency?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. I think overall the data pipelines are very sound. the authors combines layout-aware lynx rendering with structure-preserving LLM cleaning, avoiding information loss from naïve HTML-to-text extraction.\n\n2. The authors also unify MathJax/KaTeX/MathML into LaTeX, preserving equation and code structure while removing boilerplate, which is very important but often under-estimated in previous works.\n\n2. The experiments show very promising results, further demonstrating the quality of the datasets.", "weaknesses": "I don't see any obvious weaknesses.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5GRHQ7gAuC", "reviewer": "Reviewer_AQgW", "summary": "This paper redesigns the text extraction pipeline for web data curation, firstly employing Lynx, a text-based browser, to retain texts, and employing Phi-4 to rewrite text for quality, resulting in Nemotron-CC-Math. It delivers better data quality than existing the-state-of-the-art open-source corpora, such as MegaMath, FineMath.", "questions": "In addition, the potential negative impact introduced during the document refinement process should be clearly discussed. I believe this could be an important point that warrants further analysis.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. Well-written and structured paper, solid experiments;\n2. The lynx’s introduction, which reliably captures equations and maintains code indentation, avoids the heuristics DOM tree operations, such as MegaMath.\n3. The ablation on different refinement models is solid.", "weaknesses": "I believe that the effectiveness of Lynx should be evaluated through an apples-to-apples comparison. For example, the quality of Lynx versus DOM tree optimization (as introduced in MegaMath) on the same mathematical web pages could be compared under a controlled setting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)