Speech-to-LaTeX: New Models and Datasets for Converting Spoken Equations and Sentences
Authors: Dmitrii Korzh
Dmitrii Tarasov
Artyom Iudin
Elvir Karimov
Matvey Skripkin
Nikita Kuzmin
Andrey Kuznetsov
Oleg Rogov
Ivan Oseledets
Venue: ICLR 2026 Poster (2026)
Keywords: ASR
multimodal LLM
speech processing
TTS
datasets
Primary area: datasets and benchmarks

Abstract: Conversion of spoken mathematical expressions is a challenging task that involves transcribing speech into a strictly structured symbolic representation while addressing the ambiguity inherent in the pronunciation of equations. Although significant progress has been achieved in automatic speech recognition (ASR) and language models (LM), the problem of converting spoken mathematics into LaTeX remains underexplored. This task directly applies to educational and research domains, such as lecture transcription or note creation. Based on ASR post-correction, prior work requires 2 transcriptions, focuses only on isolated equations, has a limited test set, and provides neither training data nor multilingual coverage.  To address these issues, we present the first fully open-source large-scale dataset, comprising over 66,000 human-annotated audio samples of mathematical equations and sentences in English and Russian, drawn from diverse scientific domains. In addition to the ASR post-correction models and few-shot prompting, we apply audio language models, demonstrating comparable character error rate (CER) results on the MathSpeech benchmark (28\% vs. 30\%) for the equations conversion. In contrast, on the proposed S2L-equations benchmark, our models outperform the MathSpeech model by a substantial margin of more than 36 percentage points, even after accounting for LaTeX formatting artifacts (27\% vs. 64\%). We establish the first benchmark for mathematical sentence recognition (S2L-sentences) and achieve an equation CER of 40\%. This work lays the groundwork for future advances in multimodal AI, with a particular focus on mathematical content recognition.

Published as a conference paper at ICLR 2026
SPEECH-TO-LATEX: NEW MODELS AND DATASETS FOR CONVERTING SPOKEN EQUATIONS AND SENTENCES
Dmitrii Korzh 1.2 Dmitrii Tarasoy Artyom Iudin
Elvir Karimov 2,6
Matvey Skripkin 3.5
Nikita Kuzmin 2.5 Ivan Oseledets 1.7
Andrey Kuznetsoy
Oleg Y. Rogov1.20, 1.2.5
AXXX, Moscow, Russia
2 MTUCI, Moscow, Russia
3 FusionBrain Lab, AXXX, Moscow, Russia
HSE University, Moscow, Russia
5 Applied AI Institute, Moscow, Russia
6 Innopolis University, Innopolis, Russia
7 Moscow State University, Moscow, Russia
ABSTRACT
Conversion of spoken mathematical expressions a challenging task that involves transcribing speech into strictly structured symbolic representation while ad- dressing the ambiguity inherent in the pronunciation of equations. Although sig- nificant progress has been achieved in automatic speech recognition (ASR) and language models (LM), the problem of converting spoken mathematics into La- TeX remains undercxplored. This task directly applics to educational and research domains, such lecture transcription note creation. Based ASR post- correction, prior work requires transcriptions, focuses only on isolated equa- tions, has limited test and provides neither training data nor multilingual coverage. To address these issues, we present the first fully open-source large- scale dataset, comprising over 66,000 human-annotated audio samples of math- ematical equations and sentences in English and Russian, drawn from diverse scientific domains. In addition the ASR post-correction models and few-shot prompting. we apply audio language models, demonstrating comparable charac- ter error rate (CER) results on the MathSpeech benchmark (28% vs. 30%) for the equations conversion. In contrast, the proposed S2L-equations benchmark, our models outperform the MathSpeech model by substantial margin more than 36 percentage points, even after accounting for LaTeX formatting artifacts (27% vs. 64%). We establish the first benchmark for mathematical sentence recog- nition (S2L-sentences) and achieve equation CER of 40%. This work lays the groundwork for future advances multimodal AI, with a particular focus on mathematical content recognition.
INTRODUCTION
Modern speech recognition models (Baevski et al. 2020: Radford et al.) 2023) demonstrate strong performance on general speech but struggle with domain-specific tasks such as converting spoken mathematical expressions and sentences into formal symbolic representations like LaTeX. While simple symbols (e.g., +, -, π, /) are often correctly recognized, more complex or nested expres- sions remain challenging. This limitation critical in academic and educational contexts, includ- ing automatic lecture transcription, multimodal assistant development, and scientific note-taking. Speech-to-LaTeeX (S2L) models. which can interpret the structure and semantics of mathematical language in speech, essential in these applications. Prior work, such MathBridge (Jung al. 2024]. addresses the Text-to-LaTeX task using language models trained on textual representations of spoken equations.

---

Published as a conference paper at ICLR 2026
Input Waveform
Input Waveform
Whisper
Adapter
Post-ASR LLM
Prediction
D00
BEATs
VWFTT
00 1
ASR Prediction
System messager secoptie spacc connert the cociee lin pothonati
"exp. in the power of 1 nulttplted by pi its equal to ninus one"
ASR Model
Figure 1: S2L methods schematic illustration. (a) Post-correction approach. (b) Multi-mode end- to-end approach (SALMONN). (a), audio transcribed by an ASR model, and the result is passed to an LLM for LaTeX conversion. In (b), raw audio is processed by 2 audio encoders and an adapter, and the resulting audio and textual prompt tokens are fed into LLaMA-baat LLM to generate the LaTeX.
The S2L problem, however, remains largely underexplored. MathSpeech (Hyeon et al. 2025b) proposes an ASR post-correction approach that transcribes spoken equations into text, followed by Text-to-LaTeX generation via language models. Evaluation was performed on 1.1k spoken expres- sions from YouTube. However, this pipeline depends on dual ASR transcriptions, supports only isolated equations (not mathematical sentences), lacks multilingual support, and omits end-to-end multimodal approaches. Moreover, the test set is limited in size and diversity, and the underly- ing training data, voiced-over with TTS from MathBri dge equations and pronunciations, is not publicly released. To address these limitations, it is necessary to develop new S2L datasets that con- tain partial human annotations and employ more robust modeling techniques, including end-to-end systems that integrate ASR, LMs, and audio-based LLMs.
This paper introduces the S2L dataset for spoken mathematical language, which consists of 2 sub- sets: (il-seenennee and S2L-equat innratiinn ions. The dataset contains approximately 12k unique mathematical sentences and 10.7k distinct isolated equations, each annotated by up different speakers (from total of 33 annotators) to capture diverse pronunciations, intonations, and linguis- tic styles. To further expand and augment the dataset, additional artificially-annotii and sentences were added, resulting in 571k generated audio samples.
expressions
We develop several S2L methods combining state-of-the
ASR models (Radford al.) 2023
Chen al. (022) with post-processing via fine-tuned LMs and end-to-end approaches, based on Audio-LLMs (Tang et al., 2024 Chu et al. 2023). These approaches are illustrated in Figure [ Our best models achieve an equation CER between 27.7% and 30.0% English data. Within mathematical sentences, text CER is up to 9.6%, and equation CER is up to 39.7%. These relatively high rates reflect the inherent ambiguity in spoken math. For example, "kappa" may correspond to Nkappa (k) or \varkappa (x); the phrase "one over x plus two" could yield I
x + 2, Ø+2. or 1/x + 2. Despite such ambiguities, our models generate valid LaTeX in most cases, establishing strong performance baseline.
Our contributions might be summarized as follows:
We release the first large-scale, open-source dataset of spoken mathematical expressions and sentences (s2 (s2L=sentences, s21-equatiors) in English and Russian, including 66k human and 571k synthetic audio samples with diverse pronunciations and complexi- ties.
We evaluate multiple S2L methods based on ASR post-correction. few-shot prompting, and audio-LLM integration, demonstrating strong performance across metrics and outperform- ing MathSpeech on several tasks.
We conduct comprehensive evaluation using relevant metrics to establish robust baselines and detailed analysis for future S2L researc
"ode available http://iiiiiiieeeeee//e/oooo.ocn
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
Automatic Speech Recognition. CTC loss (Graves al. 2006; Amodei al. 2016) allows align-
ment between audio and text without a precise labelling. While traditional ASR suffers from context insensitivity, the Conformer model (Gulati et al., 2020) combines convolutions and self-attention to capture both local and global dependencies. Wav2Vec 2.0 (Baevski et al., 2020) uses contrastive self-supervised learning to extract high-quality audio features. Whisper (Radford et al. 2023).
transformer-ban based architecture, is trained in a weakly-supervised manner, demonstrating robust performance across various audio domains.
Language Models, Transformer- based LMs such as BERT (Devlin 2018), T5 (Raffel et al. 2020),
and GPT-3 (Radford et al. 2019) have shown strong performance, including in math-related prob- lems. LMs can process structured and ambiguous data, such as chemistry formulas (Ganeeva et al.) 2024) or code problems (Li et al. (024). Recent models such as Qwen2.5-Math (Yang et al. 2024) and InternLM-Math (Ying et al. 2024) are tuned explicitly for mathematical reasoning, leveraging chain-of-thought prompting and large-scale math corpora. However, they still require fine-tuning
for the text-to-LaTeX conversion.
ASR Post-Correction. Post-correction pipelines are well studied (Ma et al. 2025 2023 Chen ct al. 2023) and effective due to the availability of textual and especially textual math data (e.g., MathBridge dge), compared to available audio data used for fine-tuning ASR models. These ap- proaches use ASR to transcribe audio and then apply LM convert the text into LaTeX. This approach leverages strong LLM priors, which might be pre-trained on relevant mathematical data, without requiring expensive audio annotation. However, performance heavily depends on transcrip- tion quality, and ambiguity in mathematical speech remains challenging.
Audio-LLMs. Multimodal LLMs (M-LLMs) aim to jointly process audio and text by encoding mul- tiple modalities and feeding them into a unified LMM. SALMONN (Tang et al. (0024) combines Whis- decoder, enabling
per and BEATs embeddings via Q-former (Li et al., 2023) into a LLaMA- tasks like ASR and audio QA. Qwen-Audio adapts Whisper encodings to handle instruction-base audio tasks across multiple languages. While promising, these models are not explicitly designed for mathematical LaTeX generation, lack fine control over symbolic precision, and often cannot recognize spoken mathematics completely.
OCR LaTeX Recognition. In contrast to S2L, OCR-based LaTeX recognition has received signifi- cant attention (Genthial 2024 Blecher et al. 2023 OleehyO 2024).
Texteh--Speech (TTS). Modern TTS models (Casanova et al. 2024; Kong al. 2020 can synthe- size natural speech at near-human quality. Recently, MathReader (Hycon et al. 2025a) proposed pipeline for converting La'TeX to speech via LLM-ggnerated pronunciation and standard TTS. Authors of (Roychowdhury et al. 2025) evaluated 5 TTS systems on math expressions, showing that performance varies significantly by expression category and performs substantially worse than human expert readings.
Spoken Mathematics Recognition. Only few works tackle S21-related problems directly. Math- ifier (Batlouni et al. 2011)) targeted fixed-template equation recognition, now largely outdated. The work (Wei al., 2025) introduced the Spoken-Mor benchmark for spoken math reasoning and evaluated several ASR post-correction models and audio-LLMs. While they demonstrated promis- ing results on arithmetic reasoning. LtTeX-style symbolic expressions and advanced expressions were largely absent.
MathSpeech (Hyeon et al. 2025b) introduced a post-correction pipeline using 2 ASR transcripts as input. Equations from Mat WathBridd dge were synthesized via TTS and transcribed with 4 ASR models to produce around 8M samples. 2 TS-small models were trained to correct and convert transeripts into LaTeX. While effective, their approach requires multiple ASRs, lacks sentence-level context, and does not support multilingual or end-to-end modeling.
Datasets. Textual math datasets like Proof PPile (Azerbayev al. 2023 Weber et al. 2024) and OpenWebMathh (Paster et al., 2023) are key for training math-aware LLMs. MathBridog provides 23M LaTeX expressions with artificial text and context, but suffers from low quality and duplicates. OCR-LaTeX datasets like TexITelles (OlechyO) 2024) offer high-quality image- LaTeX pairs and can potentially support S2I. via voice-over. However, large-scale S2L datasets
3

---

Published as a conference paper at ICLR 2026
remain missing: MathSpeech offers only 1.1k test set with training data, and Spoken-MOA contains just 2.3k TTS samples focused on basic arithmetic. No dataset provides large-scale, human- annotated, contextual spoken math data, motivating to begin with dataset collection. A quality benchmark and protocol for LaTeX document generation are introduced (Kale Nadadur 2025
DATASET COLLECTION
In this section, we describe the pipeline of S2L data collection. We combined human- annaotted and artificially generated data to create robust and diverse dataset. We began by collecting mathemat- ical equations and sentences from multiple sources, along with corresponding reference pronuncia- tions. These pronunciations serve both as guidance for non-expert human annotators and as required inputs for artificial annotation via TTS or voice-conversion (VC) models.
Each sample is classified by language (English or Russian), annotation type (human or arti- ficial,, source (e.g., Proof-Pile, MathBridge, TextTeller, or generated), and format (S2L-equat tios for isolated expressions vs. S2L-sentences for in-context mathematical sentences).
3.1 DATA SOURCES PREPARATION
For the s2L 32L-equation ions, we utilized existing sources, MathBridge, and TextTelles and also generated additional equations. For the -2L-sentenees, the primary source was Proof-Pile.
We incorporated subset of MathBridge, which offers large-scale textual pairs of equations and pronunciations with surrounding context. However, MathBricn idge data quality is inconsistent. Common issues include: (i) text instead of formula; (ii) invalid LaTeX; (iii) missing pronun- ciations; (iv) duplicated entries; (v) pronunciation additionally contains LaTeX; (vi) mismatched Aermula-pronnncii
pairs; (vii) lots of nearly duplicated formulas, such as cos(a),... (w). We selected 15,000 candidate equations and filtered them manually, retaining 3,000 high-quality En- glish samples for both human and artificial annotation. Furthermore, we employed heuristic filters and LaTeX compilation checks to automatically clean the whole dataset, reducing it from 23 million to 1.5 million validated samples; of these, 400k were subsequently annotated with TTS in English. The heuristics involved filtering out overly short formulas, text-only entries (which typically con- tained plain text rather than equations), and cases where the equation substantially longer than its spoken form.
TextTeller provides complex equations used in OCR-LaTeX research. We extracted 9,400 unique LaTeX equations and used GPT-4 to generate 4 reference pronunciations (2 English, Rus- sian) per equation, later used for artificial voice synthesis.
To enhance diversity across mathematical domains, we prompted GPT-4 to generate La- pex-pronunciation pairs for common study topics (e.g., Calculus, Mechanics). Examples of gener- ated topics and corresponding equations are provided in Appendix A.I A.I
extracted contextual math sentences from the arxiv subset of Proof-Pile-2. Next preprocessing steps were applied: (i) filtering for inline formulas; (ii) removing LaTeX formatting of text (e.g., \eitep, \textit); and (ii) validating equations KaTeX compilation. Sentences were stratified by equation length (Table 6 and balanced accord- ingly, resulting in 12.4k clean samples with a human-annotation coverage rate of approximately 2. Additionally, we included 1.4k negative examples (no equations) from the LRS3 dataset (Afouras) ctt al. 2018) for artificial annotation.
For S2L- sen-eees,,,e,,
84% equations from S2L equations primarily have length between 3 and 50 symbols, while the rest are primary up 140 symbols and the max length is 230. The majority of samples from s2L-sentences have 1-4 equations per sentence, while the max is 11. The detailed statistics and stratification by equation length and number of equations per sentence are presented in Tables in the Appendix due to the limited space.
4

---

Published as a conference paper at ICLR 2026
3.2 EQUATIONS NORMALIZATION
All LaTeX equations were normalized using a KaTeX (Barabash et al., 2025) fork, with uncompi- lable samples removed. The process standardized notation, eliminated extraneous spaces, inserted required braces, and unified operator names by parsing and reconstructing formulas via Abstract Syntax Tree. This reduced the CER by 1% on S2L 12L-Equat ions. See Table for examples.
Table 1: Examples of LaTeX Equation Normalization.
Original Equation
Normalized Equation
\sum_i
\sum_ (1) (i)"(n]
Afract n n+1 ) H
Viaac (n (n+1) (2)
Nunderset; ; \xi H \max
\max_ ('xi)
Aella 2\siml
Weltal ZAsim) 1
3.3 DATASET COMPOSITIONS AND AUDIO ANNOTATION
Each distinct equation or sentence in our dataset is paired with at least one reference pronunciation, which serves both as a TTS input and human reference. For augmentation, multiple pronunci- ations were collected for several thousand expressions. To reduce annotation cost and augment the data, we explored the viability of training models on artificially generated audio. For this, we used open-source XTTSv2 (Casanova al. (024), and proprietary TTS APIs (e.g., SaluteSpeech). XTTSv2 was selected as the primary annotator due to its public availability, high audio fidelity, and voice conversion capability. For human annotation, MTurk. The process showed speakers a formula or sentence and reference pronunciations. Overall, 33 unique human annotators were involved. Manual verification was performed for each annotator: 10% of their audio was reviewed, and if more than 15% was rejected due to noise or low quality, only the verified subset was retained. To sum up, the resulting statisties are the following:
used crowd- sourcing platform similar to
Human annotation
S2I-equations (Eng): 6,535 unique equations, 27 annotators, total 23,196 audio.
S21 S2l-equations (Rus): 4.274 unique equations, 10 annotators, total 18,134 audio.
S2I-sentences (Eng): 12,395 unique sentences, 20 annotators, total 24,794 au- dio.
Artificial annotation
S2I-equations (Eng): 406,122 unique equations (6,535 as for human annotators, 399,587 new), artificial voices, total 450,874 audio.
S2I-equation ions (Rus): 12,669 unique equations (4,274 as human annotation, 8,395 new), 14,449 reference pronunciations, artificial voices, total 53,109 audio. S2I-sentences (Eng): 12,064 (10,411 as in human annotation, 1,984 new) unique sentences, 4 artificial voices, total 67,069 audio.
3.4 S2L. DATA REPRESENTATIVENESS DISCUSSION
Assessing dataset representativeness in the context of spoken mathematical expressions is inherently challenging due to the breadth of mathematical fields and the diversity of pronunciations. For ex- ample, the spoken phrase "2 squared from x plus 1" can map either 2 is to explicitly include "parentheses" in the pronunciation. Some samples adopt this, but many do not, reflecting real-world variability. Our dataset does not aim to cover the full range of scientific disciplines. It rather prioritizes diversity in structure, notation, and linguistic realization. To this end, we were motivated by the following:
a²41 3 + 1. One strategy
Symbol and syntax coverage: We ensured broad coverage of commonly used LaTeX sym- bols and structures, such as \a 1paa Nomega, \frac (), \sgrt (). Vleft (, etc.
Curricular diversity: We prompted GPT to generate equations and pronunciations across typical undergraduate mathematics and physics topics, removing overly simplistic samples and those dominated by textual content (e.g., \text ( }).
5

---

Published as a conference paper at ICLR 2026
Source variability: The dataset draws from 3 distinct formula sources for s2L-equations, and includes real-world academic content via TextTeller and Proof-Pile-2.
Language and voice variation: Pronunciations were collected in both English and Russian, using multiple TTS voices as well as human annotators.
Pronunciation style diversity: In some cases, included both phonetic (e.g.. "f equals a") and lexical (e.g.. "Newton's second law") variants, reflecting natural pronunciations.
While GPT-based samples may introduce some bias in both equations and pronunciations, this is partially mitigated through the inclusion of real-world academic data and crowd-sourced spoken annotations. Empirically, we observe that models trained on synthetically generated audio gener- alize well to human-annote annotated test cases, with no drastic degradation in performance (see the next section).
Table 2: S2L- EE--uaiioo results. Disjoint split: test equations do not overlap with train equa- tions. "A" artificially (TTS) annotated audio except 400k samples extracted from Mat MathBr idge; "H": human-annotatee audio; "Mix": combination of "A" and "H"; CER is calculated for lower- case. "Q-aB" and "Q-math-aB"" stand for respectively, "Full" implies addition of 400k artificially annotated samples from MathBridee to the "A" set.
and Swen2.5-math-aB-iinee
Model
Traas
Tiain
Test
Test: Mix
TEst H
Test:
Language
Language
CER
TeXBLEU
CER
T-XBLEU
CER
TeXBLEU
MathSpeach
MS-traia
Eng
Eng
64.04
83.71
$9.32
83.66
69.65
83.80
Q-0.56
Eng
33.28
B8.61
33.25
68.54
33.30
88.
Eng+Rus
Fng
88.3
Q40.58
Fng
Fng
35.01
Q-0.5B
If
Enn+RRus
Eng
35.43
88.06
33.94
88.47
37.19
87. 15.56
Q-1.5B
Mix
Fng
$1.41
88.83
31,06
88.87
51.83
88 78
Fng
Q-0.5B
Mis-full
Eng+Rus
Fng
27.21
90.20
27,03
90.14
27.42
90.27
Q-1.5B
kin
$1.24
89.22
31.37
89.15
$1.081
$9.31
Eng
89.
knng
Eng
27.57
Q-1.5B
Eng++Ru
Eng
30.93
89.04
28.85
89.42
33.39
88.
Q-1.58
Mix
Eng
29.76
89.28
28.95
89.44
50.74
89.09
kng
30,08
Q-1.58
Rug+RRu
Fag
24.91
90.66
MMihhT356
Enn++Rus
Eng
25.37
9000
29,
89.80
29.74
90.23
Q-Math- 1.5B
Eng+Rux
Fng
51.45
89.25
30.71
89:43
$2.34
89.02
SALMONN-138
Mix-full
Eng
Eng
93.72
Gemma-Jo 3n-naa-3---e 8td
Mix-foll
kng
kng
$4.24
89.15
33.24
89.23
35.42
89.06
1-88
Mix
Fag
23.25
91.32
23.13
91.31
23.40
91.33
METHODOLOGY AND EXPERIMENTAL SETUPS
Training hyperparameters are described in the Appendix to the limited space. Prior to training, all audio was resampled to 16kHz to ensure consistency across ASR and Audio-LLM pipelines, and signs were additionally excluded from 521 hell-euutiion labels.
4.1 ASR POST- COST-CORRECTION
We first evaluated a Whisper-Large v3 ASR-only baseline for LaTeX transcription, achieving 88% CER on English s2L-equations deemed insufficient. While shallow/deep fusion methods could improve performance, we excluded them due to practical limitations: high memory/lateney costs (shallow fusion) and training complexity (deep fusion). Instead, we adopted an ASR post- correction pipeline, strategy previously shown to be effective for transcription improvement and for math-related speech tasks (Hyeon et al., 2025b Jung et al.) 20044 Chen et al. 2023). Among the ASR models evaluated, Whisper-Large v3 provided the most accurate transcriptions for mathematical symbols, particularly Greek letters and structured expressions. Canary and Qwen-Audio (based on Whisper v2) also performed reasonably well, while WavLM and Wav2Vec2.0 produced frequent symbol errors. Please, refer to Table the Appendix for the transcriptions' comparison. Summing up, used frozen ASR model and fine-tuned LLMss for the post-correction.

---

Published as a conference paper at ICLR 2026
Table 3: SALMONN-13B prediction examples -2T.equati ons, subset.
Prediction
Pronunciation
Pyu An Ap
Piw Au Bv Av
Thee field streagth
eiectromnunctoon
equzls mu
= T
E Mu, 9F)
54.5
An eleetre field equals force over charge.
m(p,o"" 57
57.4
mu. sigma squared over
Vi((X) = =
(ar(X) p²
r(1-p)
13.9
For a negative binomial distribution, the variunce equals times I minus divided by squared.
n(y, 0e)/m 6(0, 02)
a(T, 9x)/n 6(0, 0z)
Gumma, theta divided N equals delta thesa
Bi(x) * 10 cod 4 21)
Al(z) * 10 cod 40 + 21)
Aiyy function the first kind,
cquil one divided
by pi times the insegral from sero to infinity of cosine of cubed divided by three plus x times dr..
1im 210
limg--B 218
10.0 Limit tends negerve of root of4 minus
divided plus
Ei-1 (1)2+1))
El d:d= 12
from equals
equals times
plus one times two plus divided six. is less than or equal to sub 1, sub 2, sub I, sub
1 S ui, 02,
1 S u1.ug,01,
1.3
which are all less than equal to
For S2L-equat i ions, experiments were conducted using Qwen2.5 and Qwen2.5-Matte across En- glish, Russian, and combined splits. All LLMs received the ASR transcription as input and produced LaTeX equations or sentences as output. For $21-sentences, we fine-tuned the Qwen2.5 (0.5B, 1.5B, and 7B) and Qwen2.5-Math-1.. instruct models using 3 training splits: artificial, human- annotated, and mixed. To assess few-shot performance, we tested the same models using 5-shot prompt format, evaluating generalization across parameter sizes.
4.2 MULTIMODAL MODELS
A multimodal S2L pipeline was further explored using Audio-LLMs. This approach bypasses pho- netic transcription by directly converting raw audio into LaTeX expressions or sentences. Audio encoders first extract latent features from the waveform; these are then processed by a modality adapter to align with LLM token embeddings. The resulting audio tokens are concatenated with the textual prompt's i tokens and passed to the L.M for decoding.
We used Qwen-Audio, Gemma-3n (Team et al. 2025), Audio Flamingo-3 (Goel al. 2025 and SALMONN-133 13B for this setting, given their strong benchmark performance. LLMs were fine-tuned using the LoRA technique (Hu et al. 20022,, while freezing audio encoders and the adapter. Since the Qwen-Audio fine-tuning pipeline was not publicly available, we prepared it ourselves.
4.3 EVALUATION
The primary reported metrics are character error rate (CER), and TeXBLEU (Jung al. 2025) metric, recently specifically proposed for LaTeX comparison. There are cases where semantically equivalent LaTeX formulas differ in syntax, which can distort formal metrics based on raw code. For instance, the expressions \int_ \int_(a) (a)"(b) f (x) dx and lint_a"bo (x) dx represent the same formula but have a high CER. Additionally, capitalization (e.g.. \phi vs. \Phi) and font styles (e.g., \mathcal 1 (R) vs. r) introduce further ambiguities. To mitigate these effects, we apply equation normalization as previously described in subsection 3.2 Additionally, all metrics are evaluated on lowercase text except for TeXBLEU.
Table 4: Comparison with MathSpeech on the MathSpeech benchmark and S2L-equat ions (English test). Metric: CER. Qwen: Owenn..500.5--Insttu (multilingual). SALMONN was tuned only in English.
Model
Ma MathSpeech
$21.-equat
MathSpeech 27.7%
64.0%
Qwen
30.0%
27.2%
SALMONN
27.7%
17.5
7

---

Published as a conference paper at ICLR 2026
For S2L-equations, predictions and ground truth compared in LaTeX form, as illustrated in Table 3] For S2L- ences, which contain inline formulas within English text, we sepa- rately evaluate both equation and text components. All formulas extracted from the predicted sequences, concatenated, and compared against the reference formulas using character-level metrics.
4.4 DATASET SPLITS
We explored several dataset splitting strategies to evaluate generalization under different conditions. (i) disjoint formula split: in this setup, the train and test sets contain entirely non-overlapping for- mulas (or sentences), ensuring that no equation seen during training appears in the test split. This setting measures the model's ability to generalize beyond memorization of specific formulas; (īi) source-type split: to assess the utility of synthetic data, we constructed splits where the test set consists solely of human-annotated audio, while the training set comprises either synthetic (TTS), human, or mixed audio. This configuration evaluates whether models trained on inexpensive artifi- cial speech can generalize to real human input; (iii) monolingual effect of cross-lingual data by comparing monolingual and bilingual training setups. This analysis tests whether training both English and Russian subparts improves generalization, whether Lunguage-speciffe models perform better on a test of a fixed language.
bilingual: we examined the
5
RESULTS AND DISCUSSION
5.1 S2L-EQUATIONS RESULTS
Table compares the performance of post-ASR and multimodal S2I. models on the English S2L e ons test subset. Due to the limited space, the complete Table T0 with the Russian test and additional splits moved to the Appendix. The key observations are the following:
Multilingual training is not always beneficial. For Qwen2.5-0.5B, Human English Test scores are worsened 33.94% vs. 35.01 % with Eng vs. Eng+Rus training, while for Qwen2.5-Math they improved from 30.71% % to 28.08%.
For English experiments, adding 400k TTS samples improves metrics, but for the Russian test results worsen, likely due to language imbalance.
SALMONN achieves superior results over all models. Flamingo-3 performs only on par with smaller post-correction LMs. Gemma and Qwen-Audio perform below even small post-correction LMs.
1.5B models outperform 0.5B, but 7B model does not always outperform 1.5B and 0.5B models, likely because they were trained with LoRA and frozen weights, unlike the fully fine-tuned smaller models.
Math-oriented Aieen.5-MMat--1.55 shows no clear advantage over Qwen2.5-1.5B, likely because inputs are given natural language rather than mathematical expressions.
We conducted experiments by adding frequently used LaTeX symbols, such as (. }. ditional tokens not presented in default tokenizer separately. However, this modification did not result in any mcasurable improvement in model performance. The KaTeX compilation success rate of predicted equations varied from 98% to 99.5%, and failure cases mainly included bracket issues. In summary, despite a large nominal CER, the metrics do not reflect the actual situation due to the considerable ambiguity of possible pronunciation and transcriptions, and our models demonstrate satisfactory quality. For instance, one can assess the generation quality of SALMONN in Table
as ad-
Comparison with MathSpeech. We evaluated our approaches on the Ma Speech benchmark and the MathSpeech model on our S2L- ons test. MathSpeech model tends to use oper- ators that do not affect the semantics of the equation, like \displaystyle, Yoperatorname, In contrast, our dataset lacks them, and consequently, our models also lack them. Thus, for a fairer evaluation, we additionally normalized predictions and labels of our and MathSpeech models and datasets. This "improved" MathSpeech' 's metric S2L-equations ions from 92% to 64%, however, is still drastically worse than our models demonstrate (27.2%) while having just slightly better CER (27.7% 30.0%) MathSpeech benchmark, shown in Table 4 We should note that
8

---

Published as a conference paper at ICLR 2026
Table 5: S2I--Seneencss results. Disjoint split. "A": artificially (TTS) annotated audio; "H": human-annotated audio; "Mix": combination of "A" and "H". CER is calculated for lower-case. "Q-aB" and "Q-math-aB" stand for Qwen2.5-aB-instruc and - tively. "Sent." stands for sentence: metric calculated over the whole sentence; "Eq": only for the embedded equations; "Text": only for the text parts of the sentence.
respec-
Model
TeXBLRU
CRR
TEXBLEU
Sent.
Text
Eq.
Sent.
Text
Eq.
Q-0.58
A
33.74
28.25
63.44
82,99
34.34
28.89
67.38
79.58
Q-0-5B
20.1
83.22
Q-0.5B
83,90
79.89
Q-0.3B (25 shots)
IT
30.67
27.30
63.44
73.93
31.47
27.37
72:45
66.36
Q-1:58
84.37
88
Q-1.58
83.79
27.43
78.90
Q-1.58
Mis
33.57
28.27
58.41
84.58
33.05
29.57
58.07 07
80.20
IT
27.97
30.50
63.39
76.76
18.70
23.89
164.74
70.83
Q-1 shots)
24.05
Q-math-1 S8
20.4 20.49
23.56
54.72
85.05
30.35
24.24
58.48
80.18
Q-math-1 S8
H
23.78
18.80
45,48
85.34
27.01
22.25
51.32
79.67
1-7B Lokall
A
20.11
13.52
47.12
35.90
20.55
13.09
05.92
81.31
Q-78 (Loka)
H
20.72
14.67
46,10
84.12
22.55
16,66
50.00
79.99
(Lokal
85.46
shots)
54.19
77.97
73.51
Q-78 (25 shote)
11 20.00
14.23
47.12
80.64
20.73
16.38
48.21
75. 14
SALMONN 13U
23.00
13.05
37,09
17.89
49.92
80.31
SALMONN-138
16.03
79.34
SALMONN-138
Mix
13:43
9.57
39.68
85.76
16.78
10.42
44.96
81.48
MathSpeech has only 120M parameters, but was trained on 6-8 million samples. In contrast, our model has 0.5B parameters but was tuned on =550k samples.
5.2 S2L-SENTENCES RESULTS
The results are presented in Table 5 (full version is in Appendix C performance, in terms of the CER metric, is usually achieved when the model is fine-tuned human-annotated data. This holds true for both the human-annotated test split and the artificial test split. However, the addition synthetic data can also benefit both equation-related and text-related metrics. Compared to equation-only conversion, performance on part noticeably lower. This highlights the added difficulty of transcribing mathematical expres- sions embedded in context. 5-shot and 25-shot few-shot prompted models perform significantly worse than fine-tuned models of any size. Only Qwen2.5-7B (25 prompts) reaches compara- ble result on human-annotated test (equation part) of 47%. Among all models, the SALMONN Audio-LLM achieves the lowest equation CER of 39.7%, likely due to its large parameter count and its end-to-end design, which reduces dependency on intermediate ASR quality. In contrast to S2L-equati fine-tuned with LoRA Qwen2.5-7B noticeably outperforms smaller models.
We observe that the best
equations'
5.3 DISCUSSION AND LIMITATIONS
To bridge the gap with practical applications, mathematical sentences (S2L- and hu- man annotation were incorporated. However, our data does not fully capture real-world lecture conditions, where equations may be paraphrased, incomplete, tied visual content. Addressing this would require costly, fine-graine -reiggraieed annotation of lecture recordings, which remains out of scope.
Our work has several limitations. While S2L results are promising, they remain limited in scope and robustness. Posc-prog LLMs often fail when ASR transcriptions are vague, and similarly, Audio-LLMs struggle with unfamiliar audio domains. More diverse training data is likely needed. Additionally, while synthetic data is helpful for augmentation, it remains less effective than human speech due to its lower complexity and variability.

---

Published as a conference paper at ICLR 2026
6 CONCLUSION
In this paper, we introduced S2L, a novel large-scale open-source dataset for Speech-to-LaTeX con- version, consisting of 66k human-annotated and 571k TTS-generated audio samples of equations and sentences in English and Russian. Our data collection pipeline is openly described and can support future efforts in speech-driven mathematical understanding. We proposed and evaluated multiple approaches. including ASR post-come post-correction and multimodal end-to-end models. Our mod- els achieved competitive results, outperforming prior work and highlighting the feasibility of S2L. conversion when supported by high-quality data. We also demonstrated that handling equations em- bedded within natural language is substantially more challenging than converting isolated equations. Future work might be devoted to enhancing the dataset with more comprehensive human-annee -anannaddtttted real-world data, such as lecture recordings, and improving the conversion quality, with the possible application of audio-visuse methods.
LLM USAGE STATEMENT
LLMs and tools (Grammarly) were used only for grammar correction, text polishing, and shortening.
LCKNOWLEDGMENT
This work was partially supported by the Ministry of Economic Development of the Rus- sian Federation (Agreement No. 139-10-2025-034 dated June 19, 2025, unique identifier IGK 000000C3 13925P4D00022. Dmitrii Korzh and Oleg Y. Rogov acknowledge support by grant pro- vided by the Analytical Center in accordance with the subsidy agreement (ID 25-303-647337-2-0077 000001).
REFERENCES
Triantafyllos Afouras, Joon Son Chung, and Andrew Zisserman. Lrs3-ted: a large-scale dataset for visual speech recognition. arXiv preprint arXiv:1809.00 109.006,, 2018.
Dario Amodei al. Deep speech End-to-end speech recognition english and mandarin. In International conference machine learning, pp. 173-182. PMLR, 2016.
Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen McAleer, Al- bert Q Jiang, Jia Deng, Stella Biderman, and Sean Welleck. Llemma: An open language model for mathematies. arXiv preprint arXiv:2310. 10631, 2023.
Alexei Bacvski, Henry Zhou, Abdel rahman Mohamed, and Michael Auli. wav2vee 2.0: A frame- work for self-supervised learning of speech representations, ArXiv, abs/2006.11177, 2020.
Kevin Barabash, Erik Demaine, Ryan Randall, et al. Katex is fast, easy-to-use javascript library for tex math rendering on the web. fttts:://ithhn htt.:/hhhuu..cc/////aaaaaaaa 2025-06-15.
2025. Accessed:
Salim N Batlouni, Hala S Karaki, Fadi Zaraket, and Fadi N Karameh. Mathifier--peech recog- nition of math equations. In / 18th IEEE International Conference Electronics, Circuits, and Systems, pp. 301-304. IEEE, 2011.
Lukas Blecher, Guillem Cucurull, Thomas Scialom, and Robert Stojnic. Nougat: Neural optical understanding for academic documents. arXiv preprint MXXi:2300.13418 2023.
Edresson Casanova, Kelly Davis, Eren Gölge, Görkem Göknar, Iulian Gulea, Logan Hart, Aya Al- jafari, Joshua Meyer, Reuben Morais, Samuel Olayemi, et al. Xtts: a massively multilingual zero-shot text-to-speech model. arXiv preprint arXiv:2406.00 04900, 2024.
Chen Chen, Yuchen Hu, Chao-Har Huck Yang, Sabato Marco Siniscalchi, Pin- Yu Chen, and Eng- Siong Chng. Hyporadise: An open baseline for generative speech recognition with large language 2023.
models. Advances in Neural Information Procerrrrrere Processing Systems,
10

---

Published as a conference paper at ICLR 2026
Sanyuan Chen, Chengyi Wang. Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, et al. Wavlm: Large-scale self-supervised pre-training for full stack speech processing, IEEE Journal of Selected Topics in Signal Processing, 16(6):1505- 1518, 2022.
Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang, Zhijie Yan, Chang Zhou, and Jingren Zhou. Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models. arXiv preprint arXiv:2311.07911 2023.
Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04550 2018.
Veronika Ganeeva, Andrey Sakhovskiy, Kuzma Khrabrov, Andrey Savchenko, Artur Kadurin, and Elena Tutubalina. Lost in translation: Chemical language models and the misunderstanding of molecule structures, In Findings of the Association for Computational Linguisties: EMNLP 2024,
pp. 12994-13013, 2024.
Guillaume Genthial. im2latex:
Converting images to latex. 2024.
https://github: com/
gui wwwuunggent.iisliiiilie...
Arushi Goel, Sreyan Ghosh, Jachyeon Kim, Sonal Kumar, Zhifeng Kong, Sang-gil Lee, Chao- Han Huck Yang, Ramani Duraiswami, Dinesh Manocha, Rafael Valle, et al. Audio flamingo 3: Advancing audio intelligence with fully open large audio language models. arXiv preprint arXiv:2507.08128, 2025.
Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist tem- poral classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning, pp. 569-376, 2006.
Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang. Zhengdong Zhang, Yonghui Wu, and Ruoming Pang. Conformer: Convolution-augmennte transformer for speech recognition. ArXív, abs/2005.0811 2020.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang. Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022.
Sieun Hyeon, Kyudan Jung, Nam-Joon Kim, Hyun Gon Ryu, and Jaeyoung Do. Mathreader: Text- to-speech for mathematical documents. ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEFF, 2025a.
Sieun Hyeon, Kyudan Jung, Jachee Won, Nam-Joon Kim, Hyun Gon Ryu, Hyuk-Jae Lee, and Jaeyoung Do. Mathspeech: Leveraging small lms for accurate conversion in mathematical speech- to-formula. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp- 2025b.
Kyudan Jung, Sieun Hyeon. Jeong Youn Kwon, Nam-Joon Kim, Hyun Gon Ryu, Hyuk-Jae Lee, and Jaeyoung Do. Mathbridge: A large corpus dataset for translating spoken mathematical expres- sions into later formulas for improved readability. arXiv preprint arXiv:2408.0700 2024.
Kyudan Jung, Nam-Joon Kim, Hyun Gon Ryu, Sieun Hyeon, Seung-jur jun Lee, and Hyuk Jae Lee. Texbleu: Automatic metric for evaluate latex format. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2025.
Sahil Kale and Vijaykant Nadadur. TeXpert: A multi-level benchmark for evaluating latex code generation by LLMs. In Tirthankar Ghosal, Philipp Mayr, Amanpreet Singh, Aakanksha Naik, Georg Rehm, Dayne Freitag, Dan Li, Sonja Schimmler, and Anita De Waard (eds.), Proceedings of the Fifth Workshop on Scholarty Document Processing (SDP 2025), pp. 7-16, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 8799-9-89776-2222 v1 v1/2025.sdp- 1.2. URL https::///// //aps:://cl. acc antho ogy- or..org//2/0 /202. 1. .27
doi: 18.13653/
Jungil Kong, Jachyeon Kim, and Jackyoung Bae. Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis. Advances in neural information processing systems, 33:17022-17033. 2020.
11

---

Published as a conference paper at ICLR 2026
Haoyang Li, Jing Zhang, Hanbing Liu, Ju Fan, Xiaokang Zhang. Jun Zhu, Renjie Wei, Hongyan Pan, Cuiping Li, and Hong Chen. Codes: Towards building open- source language models for text-to-sql. Proceedings of the ACM Management of Data. 2(3):1 2024.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven C. H. Hoi. B. Bootstrapping language- image pre- pre-training with frozen image encoders and large language models. In International Con- ference on Machine Learning, 2023.
Hya Loshchilov, Frank Hutter, al. Fixing weight decay regularization in adam. arXiv preprini arXiv: 1.1111111,11111 5, 2017.
Rao Ma, Mark JF Gales, Kate M Knill, and Mengjie Qian. N-best 15: Robust error correction ing multiple input hypotheses and constrained decoding space. arXiv preprint arXiv:2303 00456, 2023.
Rao Ma, Mengjie Qian, Mark Gales, and Kate Knill. Asr error correction using large language models. IEEE Transactions Audio, Speech and Language Processing, 2025.
OleehyO. Texteller: Convert handwritten text to latex. https://github.. com http:/ttwub.coooleeh......
TexTeller 2024.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu, Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association ffor Computational Linguistics, pp. 311-318, 2002.
Keiran Paster, Marco Dos Santos, Zhangir Azerbayev, and Jimmy Ba. Openwebmath: An open dataset of high-quality mathematical web text. arXiv preprint arXiv: 2310.06786, 2023.
Matt Post. A call for clarity in reporting bleu scores. arXiv preprint arXiv:1804.0877 2018.
Alec Radford, Jeffrey Wu, Rewon Child, David I Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. In International conference on ma- chine learning, pp. 28492-28518. PMLR, 2023.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140): 2020.
Sujoy Roychowdhury, HG Ranjani, Sumit Soman, Nishtha Paul, Subhadip Bandyopadhyay. and Siddhanth Iyengar. Intelligibility of text-to-speech systems for mathematical expressions. arXiv preprint arXiv:2506. /086, 2025.
Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun MA, and Chao Zhang. SALMONN: Towards generic hearing abilities for large language models. In The Twelfth International Conference on Learning Representations, 2024.
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint arXiv:2503. 19786, 2025.
Maurice Weber, Daniel Y. Fu, Quentin Anthony, Yonatan Oren, Shane Adams, Anton Alexandrov, Xiaozhong Lyu, Huu Nguyen, Xiaozhe Yao, Virginia Adams, Ben Athiwaratkun, Rabul Cha- lamala, Kezhen Chen, Max Ryabinin, Tri Dao, Percy Liang, Christopher Ré, Irina Rish, and Ce Zhang. Redpajama: open dataset for training large language models. NeurIPS Datasets and Benchmarks Track, 2024.
Chengwei Wei, Bin Wang, Jung-jae Kim, and Nancy F Chen. Towards spoken mathematical rea- soning: Benchmarking basech-based models over multi- facttee math problems. arXiv preprint arXiv:2505. 15000, 2025.
12

---

Published as a conference paper at ICLR 2026
An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jian- hong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvemment arXiv preprint arXiv:2409.12122, 2024.
Huaiyuan Ying. Shuo Zhang, Linyang Li, Zhejian Zhou, Yunfan Shao, Zhaoye Fei, Yichuan Ma, Jiawei Hong, Kuikun Liu, Ziyi Wang, et al. Internlm-math: Open math large language models toward verifiable reasoning, arXiv preprint 4rXiv:2402.06332, 2024.
13

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 DATASET STATISTICS AND EXAMPLES
Human annotation was performed using the Te data labeling platform. For S2L-equations the mixed (Human+Artiffcial annotation, Eng Rus) test set includes 2.88k samples, while the train set has 143k samples (+400k Eng MathBridge samples for the "full" mix). The train set comprises 18k human and 53k artificial Russian audio samples, and 21.7k human and 50k artificial English audio samples (+400k artificial English samples). The test set has 54% human-annotated audio, with all test equations distinct from the train set. For S2L-sent ences, s, the mixed (Human+Artiffccal annotation, Eng only) test set contains 2.85k samples, and the train set has 89k samples. Both the train and test sets include 27% human-annotated audio samples.
The $2l-sentences dataset has the following stratification by formula length (Table 6 and by the number of equations per sentence (Table
Table 6: Character statistics in S2L-sentencer dataset for the unique human-annotaal annotated expres- sions.
Eq. length 3-10
10-20
20-30
30-50
50+
Counter
2,752
2,751
2,552
2,396
1,941
Table 7: Equations per sentence statistics in $21.-sentences for the unique human-annotated sentences.
# Eq.
4
6-11 11
Counter
4,899
3,726
1,983
1,028
439
321
An overview of the dataset collection pipeline is presented in Figure 2
MathBridge Daleset
15k
LaTeX Formless Latet
Data Cleaning (Eng)
18t (Ba(
[-00 00)
GPT-4
Concateration
power
6K (hus)
Table With Possible
Promounciition LaTex
Topics
Generatted Samples
piss seal 281o frze
1o Human Gista
I Data
G R T5
ecuala n sourred
8
Annotated
GFT-4
Propt:: Corerate batch of -ffok squtters and sourd: like promunctatiors the
Figure 2: S2L-equat collection and annotation pipeline overview.
Let us compare transcriptions of 5 ASR models for one particular numan-annotaterd audio in Table
Let us present several English samples we collected using GPT-4 requests in Table The "Posssible Pronounciation" is necessary for the TTS models to generate speech and is extremely helpful for the
human speech annotators as they can use for reference if they not know how to read the equation properly and simplifies the criteria for the human annotator.
For the 221-sentences, let us illustrate the evaluation challenge. Consider the CER between the predicted sequence "Given fixed graph F, typical problem large graph G on vertices that contains no copy of can have an upper bound on the number of its edges, denoted by X(n, F)" and the ground-truth "Given fixed graph F, typical problem in extremal graph theory asks for the maximum number of edges that a large graph G on n vertices containing no copy of can have, denoted by ex(n, F)." The equation-only CER is 27.27%.
14

---

Published as a conference paper at ICLR 2026
Table 8: Example of transcription of one particular buman-annottted audio of the V.A" Te p equation.
04"
02
Model
Transcription
The covariant derivative a vector a mu equals partial mu with respect to X nu plus gamma upper tho mu nu times a rho.
Whisper-L
the covarient derivative a vector a mou equals partial moo with respect to ex new plus gama upper row
WavLM
times
the covariant derivative of a vector mu equals partial with respect to x-new plus scamma upper row mo new times row
Wav2Vee2
The covariant derivative of vector equals partial mu with respect to nu plus gamma upper row mu nu times a row
Qwen-audio
The covariant derivative a vector amu equals partial amu with respect to x nu plus gamma upper tho moon nu times a rho.
Canary
A.2 METRICS DESCRIPTION
We proceed by examining the primary and additional metrics in detail.
Character Error Rate (CER) which is defined as the ratio of the normalized edit distance (Levenshtein distance) between the predicted sequence and the ground truth, normalized by the total number of characters in the reference:
CER S+D+I
(1)
N
where S is the number of substitutions, D is the number of deletions, is the number of insertions, and N is the total number of characters in the reference.
The Word Error Rate (WER) is defined similarly to the CER but considers words instead of charac- ters. CER and WER are commonly used in ASR tasks.
ROUGE-1 calculates the unigram recall between the predicted output and the reference text.
ROUGE-1 Σ Emiigramree (onoonottuunigannn count (unigram pred))
(2)
Σ Ennigramerer count (unigram)
This metric is widely used for summarization and transcription tasks evaluate the lexical overlap between predicted and reference outputs.
BLEU and sacreBLEU evaluate n-gram precision by comparing the predicted output against the reference. BLEU is computed as:
BLEU = BP exp
Wn, log Pn.
(3)
where BP is the brevity penalty, P., is the precision of n- n-gramms and W,, are weights. SacreBLEU ap- plies different tokenization (Papineni et al., 2002 Post 2018). TeXBLEU is a variant of the BLEU score adapted to evaluate LaTeX string generation tasks, particularly mathematical expressions. It penalizes syntactic mistakes and helps measure the quality of generated LaTeX code.
chrF and chrF++ character-based F-scores metrics that compute balance between precision and recall at the character level:
chrP chrR
chrF, = (1 B²)
(4)
82 chrP chrR
Where chrP and chrR represent the arithmetic mean of character n n-gram precision and recall across all n-grams; chrP is the percentage of character n-grams in the hypothesis that also appear in the
15

---

Published as a conference paper at ICLR 2026
Table 9: Example of the dataset samples for further annotation by speaker and TTS models.
Topic
Possible Pronunciation
Equation
Calculus, Integrals
integral of x cubed dx equals the fourth over plus constant
Integral:
f dr = +
Basic Geometry
the distance between two points (x1, and (x2, y2) is the square root of (x2 minus x1) squared plus (y2 minus y1) squared
d= /=2 - x1)2+ + (8/2 yn)
Basic Functions
f of x is equal to minus divided by squared minus
f(x) = n-3
=-
Partial Derivatives
The partial derivative of with respect to x and then y equals d squared divided by d x y
f Badhy
Linear Algebra
the cross product of vectors a and b is a vector perpendicular to both
x
Differential Equations
the solution to d over d x equals the
# = -ky is y = Ce ki
negative k y is y equals negative k x
Field Theory
the electromagnetie field tensor is given by mu nu equals partial mu A minus partial nu A mu
FE = U,Av AyA,
Quantum Mechanics
the Schrödinger equation for free particle is h bar d psi over d equals minus bar squared over 2 m d squared psi over d x squared
ih
du
12 Le 2m dx
QFT
the Lagrangian density for the gauge field is minus one over four F nu mu
L 1 Fuu puv
Particle Physics
the mass of the Z boson approximately 91.2 GeV/c squared
mz ~ -1.12Ge//e
General Physics
Period of pendulum: two pi times square root of length divided by gravitational acceleration
T 2T 2
Mathematical Physics
Bessel function of the first kind of order zero, j sub zero is equal to the sum from m equals zero to infinity, of minus one to the power m, divided by m factorial squared, times divided by two to the power of 2 m
Jo(x) Σ m=u
(-1)" 1)m (m!)²
2m
Trigonometry
Euler's 's formula, to the power i times pi plus one equals zero
ex 1=
Thermodynamics
Gibbs free energy. equals minus TS
G = H TS
reference, and chrR is the percentage of character n-grams in the reference that are also found the hypothesis. chrF++ is chrF for 2.
16

---

Published as a conference paper at ICLR 2026
TeXBleu is relatively insensitive to the significant enrors.
B TRAINING HYPERPARAMETERES
The default loss function was cross-entropy, and the default optimizer was AdamW (Loshchilov al., (017). Qwen models for S2L- 32L-Equata ons experiments were trained on 1 A100 GPUs for I epoch, and batch size was set to 16 samples per batch. Adam optimizer was used with weight decay of 0.01 with a learning rate 1e and linear learning rate scheduler.
For Sen-nentsscss experiments, Qwen models were trained on single A100 GPU for epoch, and the batch size was set to 16 samples per batch. Adam optimizer was used with weight decay of 0.01 with learning rate le - 4 and linear learning rate scheduler.
For the Qwen2.5-7B experiments, we applied LoRA with rank r 8 and scaling parameter 32. The adapters were integrated solely into the attention projection matrices.
For the Qwen22Audio-77H experiments, we used following LoRA configuration (r 8, a 16) targeting the attention projection matrices of the large language model (LLM) and audio encoder backbone and also LLM LMHead.
SALMONN was trained with the LoRA technique Llama. Target modules were set to attention layers, rank was 8, alpha parameter was 32, and dropout was set 10%. Whisper and Beats models were frozen. The model was trained Nvidia H100-80Gb GPUs for 6 epochs. The learning rate was set to 3e-5 with a warm-up for 3000 steps and cosine decay. Gradient accumulation was set to iterations. The batch size was set to 12 samples per batch. Automated mixed precision with float16 was used.
For the few-shot experiments, we employed the pre-trained Qwen2.5-7B Instruct checkpoint without any further fine-tuning.
C ADDITIONAL RESULTS
C.1 ADDITIONAL RESULTS FOR THE MAIN TEXT TABLES
Full version of the Table 2 from the main text for S2L-equa ions results is Table 10 Additional few-show results for the eel-sentences (T are presented in Table I1
The few-shot experiments were conducted using pretrained models in instruction mode (in other words, with open-source weights that were not fine-tuned on our dataset by us).
One can notice, that although large end-to-end models demonstrate better perforance, smaller ASR post-sorrection models offer practical alternative for the resource-constrained environments.
C.2 ADDITIONAL MODELS
We also evaluated InternLM, ProoIGPT, and FlanT5 on the subsets of S2I-equations addi- tional experiments with different splits. Results are presented in Table [2 ProofGPT-1.3B demon- strated good performance, except for the Russian language. In the setting when train and test data both have mix of genuine and artificial audio, and the test set equations have no overlapping with the equations from the train, SALMONN-13B demonstrates the best metrics except CER on all languages, while Qwen2.5 has a slight edge over SALMONN regarding CER. For instance, on the English subset, SALMONN leads with the highest Rouge-1 (83.88), sBLEU (60.68), and chrF (71.04) scores. However, its CER (42.42) is slightly higher than Qwen2. 5-Math- which has the lowest CER (39.54) and ranks second in Rouge-1 (81.43) and chrF (68.34). The Qwen-Audio performs worse than other methods, probably due to re-implementation nuances. The second part of the table compares Qwen2-0.5B and Qwen2.5-0.5B for English and Russian languages for the ran- dom and disjoint (test equations do not overlap train ones) splits. For both languages, Qwen2.5-0.50 consistently outperforms Qwen2-0.5B in terms of Rouge-1 and sBLEU. Interestingly. in the case of the combined English and Russian datasets, the 2 models exhibit very close performance, with Qwen2.5-0.55 showing marginal improvements in accuracy metrics while having a slightly higher CER.
17

---

Published as a conference paper at ICLR 2026
Table 10: 52I.-equations requations results. Disjoint split: test equations do not overlap with train equa- tions, "A": artificially (TTS) annotated audio except 400k samples extracted from MathBr rige; "H":
audio; "Mix": combination of "A" and "H"; CER is calculated for lower- case. "Q-aB" and "Q-math-aB"" stand for Qwen2.5-oBBinstrue and -wenn.5-math-aB-inntte respectively. "Full" implies addition of 400k artificially annotated samples from MathBridge to the" A" set.
Model
Train
Train
Test:
Language
Lunguage
CER
TeXBLEU
CER
TeXBLEU
CER
T-XBLEU
MathSpoach
Fog
Fng
64.04
83.71
59.32
83.64
69.65
83,80
Q-0.58
Eng
Eng
33.23
Q.0.5
A
Enge
kng
34.78
87.50
34.94
87.57
34,59
88.31
Q-0.5B
H
Eng
Eng
36.91
87.56
35.01
88.25
39.15
87.38
Q-0-58
Eng
33.94
37.19
Q.0.5B
kng
Eng
88.57
31.82
Q10.58
Mix
Fag- Rus
Eng
32.3 02.38
88.60
31.18
83.80
33,69
88.24
Q-0.58
Mix-full
Eng Rus
Eng
27.21
90.20
27.03
90.14
27.42
90.27
Fog
Q-1.5D
A
Eng+Rus
Gog
30.73
88.92
83.73
30.77
89.16
Q-1.5B
TE
Eng
Eng
29.69
89.41
27.57
89
32.22
89.07
0-1.5B
Eng
30.93
28.85
33.39
98.57
Eng
Eng
Q-1.5B
Mix
Eng+ Rus
Eng
31.14
89.37
30.00
32,40
89.28
0-15B
Mix full
Eng+ Bags+Bus
Ene
25.69
90.70
24.91
90.74
26.51
90.66
0-math-1.58
30.00
28.77
89.96
Q-noth-1.5 .B
Eng+ Rus
Feg
90.00
89.80
29.74
90.23
Q-math-1.5 558
H
Fag
Feg
30 16
89.53
97
90.13
31.58
89.46
O 0-math-1.58
H
Eng Rus
Eng
31.45
89.25
30.71
89.43
32,34
89.02
Q-math-1.55 5B
Fog
Feg
28.55
29.05
89,70
Q-math-1.5m .58
Fng- Russ
Eag
23.75
28.01
Q-math-1.55 58
Mix-ffll
Bnn itus
Eng
25.01
90.50
25.00 03
90.50
24.977
90.89
Q-7B
A
28.13
28.01
28.25
90.35
Q:7B
Eng++Rus
Q-7B
H
knng
Eng
27.97
89.59
20.93
90.29
29. 19.20
89.62
Q-7B
H
Eng+BRu
Eng
26.89
90.18
26.43
90.36
27.44
89.95
Q-7B
Mix
Eng
26.10
90.58
25.80
90.68
26.46
90.45
0-7B
Iang 8tus
Eag
27.78
90.37
29.24
0:7B
Mix dull
EngeRRus
Eig
26.17
90.50
25.96
90.51
26.43
90.48
Qwen-Audho SAL.MON
Mix
Eng
Eng
71.67
83.55
101.19
78:45
33,06
39,84
Mis-ffll
Fog
Feg
17.50
93.64
Gonima
Fag
Fag
89.23
89.06
0-0.5B
A
Rus
Rus
32.06
40.95
27.62
90.53
Q-0.58
Fnng Rus
17.78
94.54
0.0.5
H
Rus
Mus
15:24
96.57
15.89
96.76
14,92
97.08
Q-0.58
H
Fag-+ Rus
6.77
97.74
13.97
97.20
3.17
98.01
Q-0.58
Rus
15:34
97.21
15.40
0-0.5B
Isnge itus
13.02
96.77
12:22
97.59
QB0.5B
Mis-full
Fing+ Rus
Rus
8:15
97.03
15.56
96.68
4.44
97.21
Q-1.3B
A
Rus
Rus
10.05
95.17
19.37
95 (5.60
3,40
96.85
Q-1.5B
Eang
Q-1.5D
II
Rus
Rus
4.66
99.38
8.25
99.49
2.85
99.33
Q-1.5B
II
Eng+Rus
Rus
14.30
97.38
14.60
96.67
14,44
97.73
0-1.5B
Rus
14.60
10.79
94,85
Bng+RRu
Q-1.5B
Mix--ful
Eng+ Rus
Rus
4.55
98.89
13.33
96.74
0.16
99.96
Q-math-1555 5B
A
Rix
6.03
98.94
14
97.02
0.48
99.89
0-math-1.58
Fng
11.01
Q-muth-1555
F
Ras
Rus
13:23
96.51
2.86
99.
18.41
08
Q-moth-1.5 5H
H
Fag-+ Rus
Rus
12.49
97.45
11.75
97.78
12.86
97.28
0-math-1.55 58
Mix
Eng Rus
Rus
5.50
98.50
13.65
97.39
1.43
99.66
Q-math-1 5B
Q-nathh11.m 5R
Mis--flll
Fayg Rus
Rus
13.39
97.96
96.72
12.86
98.45
Q-7B
A
Riss
Rus
3.70
99.19
10.79
97.66
0.36
99.36
Q-7B
ius
16.83
99.35
Q:7B
99.33
Q-7B
H
Bng Kus
Mus
21.59
96.73
14.92
97.00
24,92
96.60
Q:7B
Mix
Rus
Rus
1.59
99.57
4.44
98.78
0.16
99.96
Q-7B
Mix
Eng+ Rus
Rus
13.65
97.36
99.96
0-78
Eng Rus
Kus
96.87
98.17
Flamingo
Mis
Rus
Rus
2.01
99.88
0.00
100.00
3.02
09.85
SALMONN
Mix-Full Mis-foll
Rus
Rus
9.38
97.73
6.51
99.55
10.81
96.82
Gemmu
15.30
17.33
C.3 REST OF THE METRICS
We tried to train LLM with pronunciations from all 5 ASR systems from Table to make it an ASR aggossic model, but the model's accuracy was worth more than just with Whisper. For results Table 13
18

---

Published as a conference paper at ICLR 2026
Table 11: sen-ssentecess results for Few-Shot experiments. Disjoint split: test sentences do overlap with train sentences. "A": artificially (TTS) annotated audio: "H": buman-annotated audio; "Mix": combination of "A" and "H". CER calculated for lower-case. "Q-aB" and "Q-math- waw.-----aa-----iistee
and for sentence: metric calculated over the whole sentence; "Eq": only for the embedded equations; "Text": only for the text parts of the sentence.
aB" stand for Qwen2.5-aB-inss
respectively. "Sent." stands
Model
Train
Test: H
Test:
CER
TeXBLEU
CER
TeXBLEU
Sent.
Text
Eq.
Eq.
Sent.
Text
Eq.
Eq.
5-shot
Q-0.5B
A
35.80
35.76
66.22
66.87
34.21
34.20
68.30
60.41
Q-0.5B
H
32.09
29.69
71.95
65.31
31.83
30.16
73.71
58.47
Q-1.SB
26.16
20.62
58.78
76.84
26.64
22.26
60.47
70.64
Q-1.5B
H
27.94
20.50
63.99
76.78
28.70
23.69
64.74
70.83
Q-math-1.5B
A
35.94
35.99
62.73
71.66
41.90
43.85
65.75
64.53
Q-math-1.5B
II
42.52
42.16
73.36
68.23
44.63
45.15
78.05
61.07
Q-7B
23.83
18,44
56.25
77.88
23.63
19.57
56.03
72.64
Q-7B
24.19
17.56
57.91
77.97
23.44
18.76
55.88
73.31
25-shot
Q-0.5B
A
28.74
25.97
61.87
73.77
28.15
25.89
65.14
66.53
Q-0.5B
H
30.67
27.30
63.44
73.93
31.47
27.37
72.45
66.36
Q-1.SB
23.65
17.55
56.84
78.42
24.10
18.82
58.77
72.39
Q-1.5B
24.05
17.26
56.77
78.57
24.49
18.93
57.61
73.43
Q-math-1.51
A
37.65
29.11
88.22
75.14
36.74
27.56
95.89
69.09
Q-math-1.3 1.5B
H
30.58
24.43
67.93
75.67
31.26
27.51
67.43
68.44
Q-7B
21.22
15.85
50.43
79.88
21.65
16.97
51.97
74.65
Q-7B
H
20.00
14.23
47.12
80.64
20.73
16.38
48.21
75.14
We measured case-sensitive performance (for example, ø and t mean different symbols). Results presented in Tables [4 and 15 As we
the performance drop is not severe. This gen- erally means that models were trained well and that data regarding capitalized and non-capitalized symbols labelled well. The rest of the metrics from the Table 12] are represented in Table 16 an addition with the lower-cased metrics for the S2L-eqaut ons part.
C.4 CROSS-LANGUAN LEARNING.
One of the advantages of fine-tuning multilingual language models is the ability to extract in- formation from one language that is not available in another. For example, LaTeX special sym- bols Asimeg and Ahat are not presented in the Russian part of the equations dataset but in En- glish. Qwen2.5, trained in English and Russian, can transeribe "approximately equal" in Russian to Nsimeq (=)) Another observation is that the models are primarily English-oriented, so Qwen2.5- Math-1.5B and Qwen2-0.5B trained in Russian can generate only simple formulas in English. The reverse situation works worse Qwen2.5-0.5B, trained in English, cannot perform post-correction in Russian.
C.5 ADDITIONAL ERROR ANALYSIS
Let us present eитог analysis on S2L-equat tons test. Among 2.8k equations, around 1.8k predic- tions are not exactly identical to the LaTeX reference, but more than half of these mismatches have a character overlap above 0.8, indicating that the majority of errors are local rather than structural.
The main part of errors is based on symbol substitutions or index errors. Some of the issues arise from ASR errors. For example:
preed true (2)
(i, 1),
('phi, \psi, (
19

---

Published as a conference paper at ICLR 2026
Table 12: S2I-equations (subset) results. SALMONN represent end-to-end Audio-LLMss, while all other models use ASR post-correction via fine-tuned LLMs. "A" denotes artificially (TTS) annotated audio. "H" refers cuuman-annotated audio, and "Mix" indicates combination of both. "Rand" indicates random split where equation-prone non-overlapping across train, validation, and test sets. "Disj" specifies disjoint split where test
triplets are
equations do not appear in the training set.
Model
Lang
Train
Test
Split
CER.
Rouge-11
sBLEU
chrFf
Qwen2.5-0.5B
Eng
Mix
Mix
Disj
43.87
77.78
53.33
64.48
Qwen2.5-Math-1155
Eng
Mix
Mix
Disj
39.54
81.43
57.86
68.34
ProofGPT-1.3B
Eng
Mix
Mix
Disj
41.60
78.04
52.31
64.30
InternLM2-1188
Eng
Mix
Mix
Disj
49.23
78.12
61.00
64.24
Flan-TS
Eng
Mix
Mix
Disj
64.92
53.47
11.98
28.78
SALMONN-13B
Eng
Mix
Mix
Disj
42.42
83.88
60.68
71.04
Qwen2.5-0.55
Rus
Mix
Mix
Disj
13.19
89.71
72.78
86.09
Qwen2.5-Math-L..B
Rus
Mix
Mix
Disj
10.49
90.66
74.25
88.11
ProofGPT-LL3B
Rus
Mix
Mix
Disj
16.48
87.82
70.82
84.04
SALMONN-13B
Rus
Mix
Mix
Disj
10.45
93.59
76.63
91.63
Qwen2.5-0.5B
Eng+Rus
Mix
Mix
Disj
22.70
86.22
67.14
79.87
ProofGPT-1.3B
Eng+Rus
Mix
Mix
Disj
23.93
84.85
65.33
78.18
SALMONN-13B
Eng+Rus
Mix
Mix
Disj
24.27
89.93
69.62
84.10
Qwen2-0.57
Eng
A
H
Rand 25.05
86,56
70.39
76.91
Qwen2.5-0050
Eng
A
H
Rand
23.56
86.92
71.37
77.88
Qwen2-0.55 0.5B
Rus
A
H
Rand
7.09
94.44
79.59
92.79
Qwen2.5 Swen2.5-0.5B
Rus
A
H
Rand
7.49
94.58
79.88
92.73
Qwen2-0.35 0.5B Qwen2.5-0.5B
Eng+Rus
A
H
Disj
30.36
83.52
61.72
72.20
Eng+Rus
A
H
Disj
31.13
83.60
61.73
72.22
Table 13: 221.-euuaionss (subset). Metries results (%) for Qwen trained with ASR models.
Model
CERI
Rouge-11
sBLEU1
chrFf
WER
METEOR
BLEU
chrF++1
Qwen2.5-0.5B
43.21
78.49
50.06
60.35
75.33
57.21
47.06
58.88
Table 14: S2L eeuuatinannss (subset). Case-sensitive metrics (%) for different Language Mod-
els. "Mix" means a combination human-annotated and TTS. Lang means the language of the train/validationntest splits.
Model
Lang
Train
Test
Split
CER,
Rouge-1
sBLEU1
chrFt
Qwen2.5-0.5P
Eng
Mix
Mix
Disj
45.79
77.78
50.46
61.06
Owen2.5-Math-1155
Eng
Mix
Mix
Disj
44.39
79.29
$1.02
61.67
SALMONN-13B
Eng
Mix
Mix
Disj
44.47
83.88
56.76
66.70
Flan-T5
Eng
Mix
Mix
Disj
67.52
53.47
10.43
26.01
Qwen-Audio
Eng
Mix
Mix
Disj
54.64
76.63
54.79
57.61
Qwen2,5-0.5B
Rus
Mix
Mix
Disj
13.45
89.71
72.67
85.47
SALMONN-13B
Rus
Mix
Mix
Disj
10.59
93.59
76.52
91.38
Qwen2,5-0.5B
Eng+Rus
Mix
Mix
Disj
23.39
86.22
66.26
78.74
SALMONN-13B
Eng+Rus
Mix
Mix
Disj
24.99
89.93
68.69
82.82
pred Afrac (errac(paatialaaaae \mathcal L) (/Apartiaalphh, true \frac Mtpaaaaaallliaaa (AAAA)((paaaa))a) (partial\phhi \phi).
A significant part of enrors involves missing ambiguous \text blocks.
pred: f (x) = = e (-\abbaa
( x), x\ge 0,
true: (x) = lambda e" - [-Alamada x) (iteet(forr }xlge 0.
20

---

Published as a conference paper at ICLR 2026
Table 15: - ons (subset). Remaining case-sensitive metrics (%) for different Lan- guage Models, "Mix" means combination of Human annotated and TTS. Lang means language of train/validation/eest splits
Model
Lang
Train
Test
Split
WER,
METEOR
BLEU1
chrF++1
Qwen2.5-0.57
Eng
Mix
Mix
Disj
79.60
56.89
47.16
59.44
Qwen2.5-Math-1..5
Eng
Mix
Mix
Disj
76.78
57.52
47.85
60.24
SALMONN-13B
Eng
Mix
Mix
Disj
72.20
61.91
53.08
65.06
Flan-T5
Eng
Mix
Disj
111.83
20.47
Qwen-Audio
Eng
Mix
Mix
Disj
102.91
53.67
42.53
55.89
Qwen2.5-0.58
Rus
Mix
Mix
Disj
28.14
80.78
70.55
83.68
SALMONN-13B
Rus
Mix
Mix
Disj
18.13
84,91
74.95
90.09
Qwen2.5-0.58 Eng+Rus
Mix
Mix
Disj
42.46
73.63
63.80
78.18
SALMONN-13B
Eng+Rus
Mix
Mix
Disj
40.02
77.24
66.77
81.38
Table 16: S2I-equations (subset). Remaining results of lower-case metrics (%) for different models. SALMONN represents Multimodal approach, while the rest of the models represent ASR post-correction. "A" stands for artificially annotated audio (TTS), "H" human annotated audio, "Mix" the combination of both "A" and "H". "Disj" split means that test equations do not intersect with the train equations, and "Rand" split means that train-test split made randomly over generated pairs and equations from train might occur in the test but should be pronounced with different speakers TTS models.
Model
Lang
Train
Test
Split
WER
METEOR
BLEU
chrF++1
Qwen2.5-0.5B
Eng
Mix
Mix
Disj
76.85
56,89
50.42
62.71
Qwen2.5-Math-
Eng
Mix
Mix
Disj
69.16
60.33
55.57
66.77
ProofGPT-1.39
Eng
Mix
Mix
Disj
69.64
55.86
49.73
62.50
SALMONN-133
Eng
Mix
Mix
Disj
68.90
61.91
57.55
69.20
I2tec11.M2-1.88
Eng
Mix
Mix
Disj
81.01
57.30
50.65
62.55
Flan-T5
Eng
Mix
Mix
Disj
109.26
20.47
7.69
27.53
Qwen2.5-0.5B
Rus
Mix
Mix
Disj
27.14
80.78
70.64
84.34
5ren2.5-Math-1.5B
Rus
Mix
Mix
Disj
23,80
81.65
72.03
86.47
ProofGPT-1.3B
Rus
Mix
Mix
Disj
32.14
79.10
68.51
82.22
SALMONN-13B
Rus
Mix
Mix
Disj
17.94
84.91
75.05
90.36
Qwen2.5-0.5B
Eng+Rus
Mix
Mix
Disj
41.47
73.63
64.75
78.18
ProofGPT-1.3B
Eng+Rus
Mix
Mix
Disj
43.26
72.20
62.94
76.37
SALMONN
Eng+Rus
Mix
Disj
38.80
77.24
67.85
82.62
Qwen2-0.5B
Rus
A
H
Rand
14.82
86.74
78.46
91.87
Qwen2.5-0.5B
Rus
Rand
13.91
86.77
78.77
91.92
Qwen2-0.5B
Eng
H
Rand
40.37
73.88
68.60
76.53
Qwen2.5-0.5B
Eng
Rand
38.54
74.59
69.71
76.53
Owen2-0.5B
Eng+Rus
H Disj
57.02
68.83
58.82
70.78
Qwen2.5-0.5B
Eng+Rus
Disj
58.27
68,56
58.60
70.85
pred: (P" (e)) (\theta),
true: (e)) " (Ntext(th))
pred: \sin (x) \ wht.kkiigggarrowwaaaddd y....
true: (sin(x) \text( is } =
Fractions and nested fractions.
pred: (Exacc/della/matthhee { hater//atataaaaaaaaaaaaaaaaaaaar (aatta((yaattaaa__aanhhh \mulphi)) 0.
true: fracc/:parttiaall))))//rrr (d) (d)phi /dx" (Amu))) 0.
21

---

Published as a conference paper at ICLR 2026
Overall, the majority of differences between predictions and labels are small lexical or index-level discrepancies, as well as occasional text-math boundary issues. Structural failures, such as incorrect integrals deeply nested fraction constructions, uncommon.
22


Reviews:
[{"review_id": "f3U4ehCDv8", "reviewer": "Reviewer_1KKx", "summary": "The paper addresses Speech-to-LaTeX for both isolated equations and full sentences, releasing a large bilingual dataset with human and TTS audio. It compares Whisper with LLM post-correction against end-to-end Audio-LLMs like SALMONN-13B, showing strong gains over prior works (MathSpeech).", "questions": "1) Could you quantify how much equation normalization (and lowercasing) changes CER/TeXBLEU per model? Also, any experiments with structure-aware or render-equivalence metrics? Concrete numbers would help judge true semantic progress. \n\n2) Did you try constrained decoding (grammar or AST constraints) in the post-correction step to reduce bracket/scope errors? If so, how did it affect latency and CER?\n\n3) This paper states, ‘We release the first large-scale, open-source dataset of spoken mathematical expressions and sentences.’ Is there an anonymized data pages related to this?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1) This paper targets Speech-to-LaTeX, an underexplored but impactful task for education and research with large-scale S2L dataset.\n\n2) It shows strong empirical results on S2L-equations (English). SALMONN achieves 17.5% CER, outperforming MathSpeech and Qwen; on S2L-sentences, SALMONN attains the best equation CER (39.7%).", "weaknesses": "1) Gap to real lecture conditions. Authors note the dataset does not capture paraphrases, incomplete expressions, or audio-visual coupling typical of classroom settings.\n\n2) It is difficult to verify the reliability of the dataset presented in the paper. Although some sample data are available in the supplementary material, those samples are insufficient to establish whether the constructed dataset adequately covers diverse scenarios of spoken mathematical expressions.\n\n3) Reproducibility risk for one baseline. Qwen-Audio “fails completely,” attributed to re-implementation differences—suggesting pipeline brittleness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nUPwX4GQYh", "reviewer": "Reviewer_rLuj", "summary": "The paper introduces Speech-to-LaTeX (S2L)—the large-scale, open dataset for converting spoken math into LaTeX—addressing limits of prior work that used only synthetic audio, required two ASR passes, and handled only isolated equations. S2L includes 66k human and 571k TTS samples of equations and sentences in English/Russian. Two solutions are evaluated: (1) Whisper + Qwen post-correction, and (2) end-to-end audio-LLMs (e.g., SALMONN). On S2L-equations, Qwen reaches ~25–30% CER, and SALMONN achieves 17.5%, far outperforming MathSpeech (~64% on S2L). The paper also establishes the first benchmark for spoken mathematical sentences, where multimodal models remain strongest but accuracy drops due to contextual ambiguity.", "questions": "**Practicality & generalization**\n\n- What are end-to-end latencies and memory footprints for the strongest systems, and can lighter E2E models approach SALMONN’s accuracy?\n- How robust are models to real-world conditions (noise, disfluencies, accents, mic distance)? Any stress-tests or augmentation plans?\n- Multilingual effects: can you quantify negative transfer vs. language imbalance and test mitigation (balanced sampling, language tags, adapters)?\n- Fairness of comparisons: would training a MathSpeech-style architecture on S2L (or retraining Qwen on MathSpeech’s 6–8M) clarify the role of data vs. model size?\n\n**Dataset & evaluation**\n\n- How were inherently ambiguous utterances annotated (e.g., explicit “parentheses” vs. natural speech)? Any inter-annotator agreement stats and adjudication policies for these cases?\n- Could you add a semantic check (e.g., render-and-compare, AST comparison) alongside CER/TeXBLEU to better reflect user-perceived correctness?\n- Do you have a typology of common error modes (nested fractions, integrals, Greek variants, boundaries between text and math) for equations-in-sentences?\n- Can you release the KaTeX normalization code and exact split scripts to facilitate strict reproducibility?\n\n**Modeling choices & ablations**\n\n- What ablations isolate why 7B (LoRA) < 1.5B (full FT) on equations? Is it LoRA rank, target layers, or optimizer/schedule?\n- Why did Qwen-Audio fail? Can you share a brief post-mortem (feature pipeline, adapter alignment, loss setup)?\n- Did adding 400k MathBridge-derived TTS samples change error composition (e.g., more bracket errors decreased), or only average CER?\n- Given that Qwen-Math didn’t clearly beat Qwen on this task (inputs are NL), did math-specialized token priors help particular symbol families or structures?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- **Large, open S2L resource:** Releases a two-part dataset (S2L-equations, S2L-sentences) with multilingual coverage (English/Russian), mixing 66k human and 571k synthetic clips, collected from diverse sources and 33 annotators. This addresses the data bottleneck and standardizes evaluation.\n- **Clear task framing & thorough splits:** Uses disjoint-formula splits, human vs. TTS source splits, and mono vs. bilingual training setups—plus KaTeX-based equation normalization—to probe generalization beyond memorization.\n- **Strong baselines across paradigms:** Compares post-ASR pipelines (Qwen 0.5B/1.5B/Math/7B) with end-to-end Audio-LLMs (SALMONN, Flamingo-3, Gemma-3n, Qwen-Audio), giving a balanced picture of modular vs. E2E.\n- **Transparent MathSpeech comparison:** After normalization, MathSpeech degrades to 64% CER on S2L-equations, while the proposed models remain strong; also competitive on the original MathSpeech benchmark.\n- **Practical touches in evaluation:** High KaTeX compile rate (≈98–99.5%) suggests many errors are minor formatting differences rather than catastrophic syntax failures.", "weaknesses": "- **Compute/latency not discussed:** The best model (SALMONN-13B) is likely heavy; no throughput/latency/memory reporting limits practical takeaways for real-time use.\n- **Ambiguity handling is under-analyzed:** The work acknowledges inherent ambiguity (“one over x plus two”) but gives limited breakdowns by ambiguity type or guidance on disambiguating conventions during annotation/evaluation.\n- **Model behavior anomalies lack ablations:** 7B (LoRA-tuned, frozen base) underperforms fully fine-tuned 1.5B on equations; Qwen-Audio “fails completely.” Causes are hypothesized but not probed via targeted ablations.\n- **Multilingual training instability:** Adding Russian sometimes harms English performance; analysis is brief (imbalance) without deeper diagnosis of cross-lingual interference.\n- **Real-world robustness is unclear:** Data is largely clean/read speech; the gap to noisy, disfluent lecture audio (and multimodal cues, e.g., slides/pointing) isn’t evaluated.\n- **Tokenization tweak had no effect:** Adding special LaTeX tokens didn’t help; this is noted but not further explored (e.g., tokenizer training, subword strategies).\n- **Metric–semantics gap:** Heavy reliance on CER/TeXBLEU—even with normalization—can penalize semantically equivalent LaTeX or underplay mathematically serious mistakes; no semantic/visual-equivalence metric is reported.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WNpna70c0H", "reviewer": "Reviewer_Sqnu", "summary": "In this paper, the author proposes a new benchmark dataset for the speech-to-LaTeX transcription task. Compared with the existing benchmark, MathSpeech, the proposed dataset includes a broader range of samples that better reflect real-world speech-to-text usage. It introduces more challenging cases, such as mixed text and mathematical formulas, making the benchmark more representative of practical scenarios. In addition, unlike MathSpeech, which relies on synthetic speech, this dataset is collected from real human speakers, further enhancing its realism and applicability. Using the proposed dataset, the author trains models with two different processing approaches, following established speech-to-LaTeX training pipelines, and benchmarks their performance to evaluate the effectiveness of current algorithms in this direction.", "questions": "1. For the few-shot learning experiments, were they conducted on a fine-tuned model or directly on the original pretrained model?\n2. Could the authors provide more details about the data split ratios used in different experimental settings (e.g., training vs. testing)? For instance:\n    - What is the ratio of training to testing equations in the Disjoint setting?\n    - What are the training and testing sample sizes under the source-type split?\n    - In the monolingual and bilingual experiments, do you use all available examples from one language plus a portion of the other language for training? If so, please specify the ratio used.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The benchmark dataset proposed in this paper covers a wider range of diverse, real-world scenarios, making it a valuable contribution to the research community.\n2. The paper provides a comprehensive exploration of the dataset’s potential applications, including post-transcription correction and end-to-end multimodal fine-tuning.\n3. The dataset-splitting strategy is well designed, and the experimental results effectively demonstrate the generalization ability of different models when adapting to various languages and previously unseen formula patterns.", "weaknesses": "1. The technical novelty of the paper is limited. The work mainly reimplements common approaches to fine-tuning models without proposing new model architectures or training algorithms to address the issues identified in the experiments.\n2. The Character Error Rate (CER) remains relatively high in many cases, and the paper lacks an in-depth analysis of the causes behind these failures. A more thorough investigation into the unexpected phenomena observed in the results would strengthen the contribution.\n3. Several parts of the paper are not well organized. For instance, some sections—such as the discussion of OCR methods in the related works—contain redundant content. Additionally, Table 2 is placed far from the results section, making it inconvenient for readers to reference while reviewing the experimental findings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "s13kuzHAjT", "reviewer": "Reviewer_fXZ7", "summary": "The paper introduces the first large-scale open-source dataset of spoken mathematical expressions and sentences in English and Russian, containing about 66,000 human-annotated audio samples from diverse scientific domains. It also provides evaluations of several speech-to-latex methods, including ASR post-correction, few-shot prompting, and audio-LLM integration, showing improvements over existing baselines such as MathSpeech.\n\nOverall, this is a useful dataset paper with clear execution but limited novelty and a somewhat narrow scope for ICLR.", "questions": "Some details need clarification, including which crowdsourcing platform was used for human annotation and the ratio of synthetic to natural speech samples in the dataset created. Providing these is important for the dataset publication.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The work contributes a valuable resource for an understudied task and offers solid baselines that could support future research. The dataset appears well-organized and the experiments are clearly presented.", "weaknesses": "However, the topic is quite niche and seems better aligned with speech or audio processing venues such as Interspeech or ICASSP rather than ICLR. The scientific originality is limited, the main contribution is dataset creation and benchmarking rbut for a very nich domain.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)