Shallow Flow Matching for Coarse-to-Fine Text-to-Speech Synthesis
Authors: Dong Yang
YIYI CAI
Yuki Saito
Lixu Wang
Hiroshi Saruwatari
Venue: NeurIPS 2025 poster ()
Keywords: text-to-speech
speech synthesis
flow matching
coarse-to-fine
generative model
Primary area: deep_learning

Abstract: We propose Shallow Flow Matching (SFM), a novel mechanism that enhances flow matching (FM)-based text-to-speech (TTS) models within a coarse-to-fine generation paradigm. Unlike conventional FM modules, which use the coarse representations from the weak generator as conditions, SFM constructs intermediate states along the FM paths from these representations. During training, we introduce an orthogonal projection method to adaptively determine the temporal position of these states, and apply a principled construction strategy based on a single-segment piecewise flow. The SFM inference starts from the intermediate state rather than pure noise, thereby focusing computation on the latter stages of the FM paths. We integrate SFM into multiple TTS models with a lightweight SFM head. Experiments demonstrate that SFM yields consistent gains in speech naturalness across both objective and subjective evaluations, and significantly accelerates inference when using adaptive-step ODE solvers. Demo and codes are available at https://ydqmkkx.github.io/SFMDemo/.

Shallow Flow Matching for Coarse-to-Fine Text-to-Speech Synthesis
Dong Yang", Yiyi Cai', Yuki Saito', Lixu Wang', Hiroshi Saruwatari'
'The University of Tokyo, IIndependent Researcher, "Nanyang Technological University
yddmlkkxgmail..cmm
Abstract
We propose Shallow Flow Matching (SFM), a novel mechanism that enhances flow matching (FM)-based text-to-speech (TTS) models within a coarse-to-fine generation paradigm. Unlike conventional FM modules, which use the coarse representations from weak generator as conditions, SFM constructs intermediate states along the FM paths from these representations. During training, we introduce an orthogonal projection method to adaptively determine the temporal position of these states, and apply a principled construction strategy based on single- segment piecewise flow. The SFM inference starts from the intermediate state rather than pure noise, thereby focusing computation the latter stages of the FM paths. We integrate SFM into multiple TTS models with a lightweight SFM head. Experiments demonstrate that SFM yields consistent gains in speech naturalness across both objective and subjective evaluations, and significantly accelerates inference when using adaptive-step ODE solvers. Demo and codes available https:://ddqnii biingnnkkx.githbbl github. io/ SFMDemo/
Introduction
Text-to-speech (TTS) synthesis has advanced with generative algorithms in recent years, particularly
autoregressive (AR)
45
25
22
49
38
46),
diffusion 33
12
24. and flow matching 48
2
methods. TTS models typically contain a front-end for processing contextual information and back-end for speech synthesis. Due to the strong temporal alignment between texts and speech, many diffusion- or FM-based TTS models adopt coarse-to-fine generation paradigm: weak generator first produces coarse representations conditioned on the input context, which are then refined by the diffusion or FM module into high-quality mel-spectrograms, and finally converted into audio waveforms by vocoder. There two main approaches to the weak generator. 30 follows traditional TTS designs, where a aan-autorrgressvve encoder and an aligmment module 35 19 jointly generate coarse -pecttrcsss when aligning encoder outputs with target mel-spectrograms. The other 2 g 9
(FM) 23
1S
One 33 13 15
31 51 employs an AR large language model (L.M) context processor and weak generator generate discrete speech tokens, which transformed into continuous mel-spectrograms by a diffusion or FM module.
FM has gained increasing attention due to its efficiency and high-quality generation in TTS. In conventional coarse-to-fine FM-based TTS models, coarse representations are used as conditions for the flow module. However, the generation still starts from pure noise, resulting in suboptimal allocation of modeling capacity. Since the coarse representations already encode a significant portion of the overall semantic and acoustic structure, modeling the early stage from noise becomes redundant and contributes little to the quality of the final output. To deal with this issue, DiffSinger [27] has introduced shallow diffusion mechanism singing voice synthesis (SVS) and TTS. It uses simple mel-spectrogram decoder, with the diffusion module starting generation at a shallow step based on the output of the decoder. Therefore, extend the idea of shallow diffusion mechanism (with the name)
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

on FM and propose shallow flow matching (SFM) for coarse-to to-fine TTS models. During training, we construct intermediate states along the FM paths based on the coarse representations. To achieve this, we employ orthogonal projection method to adaptively determine the corresponding time, define principled construction approach, and formulate a monle-seement piecewise flow. During inference, the generation starts from the intermediate state, skipping the early stages and focusing computation on the latter part of the flow. This leads to more stable generation and enhanced synthesis quality, and accelerates inference when using adaptive-step ordinary differential equation (ODE) solvers.
We validate the proposed SFM method on multiple coarse-to-fine TTS models, covering two main- stream FM architectures, U-Net [36] and DiT [32]. Experimental results show that SFM consistently improves the naturalness of synthesized speech, as measured by both pseudo -MOS metries and subjective evaluations. We also observe significant inference acceleration across various adaptive-step ODE solvers.
2 Preliminaries
2.1 Flow matching
dime-dependent diffeomorphic map Φ. : |0, 1] x R2 + IR describes the smooth and invertible
transformation of data points R over time t [0, 1], then flow is defined via the ODE with
time-dependent vector field (VF) u, [0. 1] x IRdd RR:
d (xn) us (x))
ct (z,),
(I)
dl
VF u, induces a probability density path Pt [0, 1] x Idd - Rzo. which is a -ependent probability density function (PDF), From time 0 to time the PDF of is transported from (0 to (2x) along u4. Chen et al. [6] proposed continuous normalizing flow (CNF) that models the
u, by a neural network 2o v0 1), where are learnable parameters. A CNF reshapes a simple
prior distribution po into complicated distribution P1. Lipman et al. (26 further proposed flow matching (FM), whose objective is LFM = (x.,) [/)(
( t) ur (z:)).. Since appropriate
Pt and u, are unknown, [26] constructed probability paths conditioned on data sample æı
q(x1). Specifically, let po(o))) = (x 00, I). pı (x1) Rs g(x1). the conditional probability path is ((())((1)) N(x (x1), or
The flow and VF are considered with the forms:
(31) (x) (1(21)) + (11))
0, (x1)) = (21)x, (x1)).
(2)
(x1)
where ,4 [0, 1] x d + Rd is the time-dependent mean and o, [0, 1] Rd - R is the time- dependent scalar standard deviation (std). I' denotes the derivative with respect to time, f'= = Corresponding to the optimal transport (OT) displacement interpolant (2)] the mean and std are:
d dt s-
(x1) = tx (z1) = 1 - (1 o )t,
(3)
where min is a sufficiently small value. Substituting Eq. (3) into Eq. 2), the conditional flow and VF take the form:
(2n)) (1 t)ar + f(2; + minn uy (x, an) (æ) + mmin Dn-
Then, we can minimize the conditional flow matching (CFM) loss during training, which proven by (26) to be equivalent to minimizing the FM loss LCFM = ((x))) [e (2, t) un (2 During inference, we use an ODE solver to solve the integral Øpred = xo + fo vo (æx, t) dt.
2.2 Classifier-free guidance
In FM-based generative models, a typical approach for controlling the generation process is to incorporate the input condition e during training and inference. To improve diversity and fidelity, classifier-free guidance (CFG) [16] can be employed. During training, the condition e is randomly dropped, enabling the model to learn from both conditional and unconditional contexts. During inference, a hyperparameter called the CFG strength 3 > 0 is introduced to control the trade-off:
Mo,crG(an (zt, 1, c) vo (o, c) + ((va (z,, I, c) (vo ()).
(5)
Specifically, the FM module runs two forward passes at each time step, once with and once without.

---

2.3 Flow matching- TTS models
Our backbone configurations involve three fully open-source TTS models.
Matcha-TTS: Matcha-TTS [30] is non-AR FM-based TTS model that employs a conventional encoder-dccoder architecture. The Transformer-basee [43] encoder takes phonemes and speaker IDs (for multi-speaker training) input, producing hidden states and predicted phoneme-level durations. The U-Net-based FM decoder receives the encoder's outputs and speaker embeddings as FM conditions and generates mel-spectrograms.
Because the encoder adopts the monotonic alignment search (MAS)-based alignment [8] for mel-spectrograms (via the prior loss
alignment, the encoder outputs are optimized towards the ground paper), resulting in coarse mel-spectrograms.
Cosy Voice g a large zero-shot TTS model that consists four components: the text encoder, the speech tokenizer, the LLM, and the FM module. The speech tokenizer extracts discrete speech tokens from mel-sppettograms of waveforms, while the text encoder processes textual inputs and aligns text encodings with speech tokens. The LLM, Transformer decoder-based model, takes speaker embeddings. text encodings, and prompt speech tokens as input and generates target speech tokens in an AR way. Subsequently, the FM module, conditioned on speaker embeddings, target speech tokens, and masked mel-spectrograms generates the target mel-spectrograms.
CosyVoices Voice:
The FM module adopts an encoder-decoder structure. The Conformer-based [14] encoder encodes the speech tokens generated by the L.LM into hidden states and linearly projects them to the same dimension seeesppctrogsaass. These hidden states then upsampled by length regulator for loken-spectrogram alignment and fed into the U-Net-based decoder, along with other conditions, for FM training and inference. Therefore, the upsampled hidden states can be explicitly supervised to coarse mel-spectrograms.
In Cosy Voice, the speech tokenizer, LLM, and flow modules trained independently. Therefore, in our experiments, we only train the flow module using speech tokens generated by the officially pre-trained speech tokenizer, while employing the officially pre- pre-trained LLM during inference.
an open-source TTS model in which both the encoder and FM de- coder leverage DiT blocks. Its encoder employs MAS-based alignment and outputs coarse mel- spectrograms.
StableTTS: StableTTS [
To assess the effectiveness of our method under different input types (text sequences and speech tokens) within the DiT architecture, further adapt StableTTS as the FM module of CosyVoice. The resulting backbone TTS model is referred to as Cosy Voice-DD throughout our work.
Voice.
3 Method
3.1
Theorems
Proofs of our theorems are provided in Appendix A Theorem 2 can be derived from PeRFlow 47
that divides CondOT paths into several windows and conducts piecewise reflow 28 each window.
Theorem For any random variable xm ~ (tm 2 I), where tm E [0, oo) and om E oo), define transformation that maps Em onto the conditional OT (CondOT) paths. The output distribution varies continuously with respect to tm and om under the Wasserstein-2 metric.
A (1 Tmin Mm oms
(6)
V(1
(1 - (1 - )m 2- σ2 xo + Im
ifA < 1,
(7)
A Bm
if A > 1,
where vn NN I), with corresponding min(!m, Im on the path.
Theorem 2. For arbitrary intermediate states on the CondOT paths:
Et... (1 tm0 + tm (x + xmin tm (0, (0.1), x0 (N,
(8)
3

---

Xn
X
X
t
E
t=
Figure 1: Inference process. Left: standard FM; Right: proposed SFM.
we can divide the paths into two segments at im and represent the flow and VF using piecewise functions:
ift < tm
32
Ii
tut I-tm
(2)
mmin
if lms
(9)
I-tm"
1 fm 20);
isl < Im
U
(10)
(x) + mmi De Fem)
ift 2 tm-
mm
3.2 Proposed methods coarse-to-fine FM-based TTS
We define the mel-spectrogram of an audio waveform as E NxF, where is the number of time frames and F is the number of frequency bins (channels). Specifically, X"" R denotes the n-th mel-spectrogram frame. We refer to the weak generator as 9w (with learnable parameters w), which receives text, speaker features, and other contextual information as the input condition C and outputs coarse mel-spectrogram Xg. X g is supervised to match the target sample X 1, typically using an L2 loss, though other loss functions may also be applied depending on the task:
Leoarse = E X x,11.
(11)
We introduce a lightweight SFM head h.s with learnable parameters vp (see details in Appendix B). which takes the final hidden states H. from gw as input and outputs a scaled mel-spectrogram X
Note that X, 9 is obtained by applying a linear projection to Hg In addition, h& needs to predict time t E (0, 1) and an estimated variance 0% for Xn:
H X2 (C), X o% hw (H)g))
(12)
3.2.1 Orthogonal projection onto CondOT paths
Since the exact location of X, h the CondOT paths and the corresponding time 4, are unknown, we let the model adaptively determine t, during training. To direct X to the CondOT paths, find the orthogonal projection of X h. onto X According to Eq. (3) the projection coefficient is la. which corresponds to an intermediate state on the mean path (x,X1) Then we estimate the time /n and the variance of and minimize the distance between X and t, X, via the loss Ly:
X 1), of = = X
fn max (0,
11? - in 112, (13)
where sg|-| (stop gradient) is used to simplify gradient propagation. of represents the noise scale of X,, and can be interpreted intrinsic noise. Given the large number of mel-spectrogram frames, omit the unbiased correction term in the estimation.
When X R X i. can assume that X (l x I and utilize Theorem to construct intermediate states on the CondOT paths:
A = m1 - Gmin M4, + oh. 1), Xn A Xm, in = A Ln: of = A %
(14)
X1, max (1 - mmin) )2,) 3 0) X
(15)
where X, ~ NO I). During the early stages of training. it is possible that A 2 1. In such cases, X is unable to lie on the CondOT path with the external noise Xo. resulting in deterministic model

---

The scaling factor 1 introduced in Theorem rescales and guides X back onto the CondOT paths until Xo can incorporated. Then we calculate the losses for the two predicted scalars:
behavior.
A
L = (in- Co = ( - -0h)
(16)
3.2.2 Single-segment piecewise flow
During training and inference, the flow starts from in. Therefore, employ Theorem 2 and focus only on the second segment of the paths (t 2 fn):
tu 14, 1], ts S(tu), t1 -i-ntt Er.
(17)
1 ヒー in
X in + ビ- th (X1 + mmin 1-th
X,
(18)
tn
= (1 (1- (s) X in + tx 1 + m Xo),
(19)
U. tn
(X1 + 0 min X Xin);
(20)
where S is arbitrary time scheduler for the randomly sampled l. Finally, combined with the CFM loss, the overall loss for the SFM framework given by:
CCFM = (x.) ug t) U.l?, CSFM Ceoarse + L, Lo Lpp + LCFM- (21)
3.2.3 Inference with SFM strength
During inference, we observe that the adaptively determined l, tends to be small, which limits the amount of prior information. address this issue, we introduce hyperparameter called SFM strength a 2 I, which scales up the in to encourage stronger guidance from Xn.
A = max((((1 mmin)) + ōn). 1), Xn = OF Kn A
In
a In.
0
A o
(22)
Substituting Xn. Tn and o7 into Eq. (55 yields X Xin We can obtain the predicted results prod by solving the integral X pred Xi Jé vo (X,,t) dt with ODE solver. For each model employing the SFM method, we determine the optimal on the validation set. In most cases, the optimal a is relatively small, resulting in A = 1. The scaling factor A has a theoretical upper bound
of TL-owin) )4, +
We provide concise algorithm boxes in Appendix C using minimal notation for clarity and including implementation details. An example of SFM inference is illustrated Fig.
Experimental setup
4.1
Datasets
We use LJ Speech 17 VCTK 44], and LibriTTS [50] in our experiments, where LJ Speech is single-speaker dataset and the others are multi-speaker datasets. For LJ Speech and VCTK, the training, validation, and test sets are divided following the setting of Matcha-TTS, which follows VITS's settings Each validation set contains 100 utterances, and each test contains 500 utterances.
For LibriTTS, the train subsets are used as the training set. We construct the validation and test sets with the dev-clean and lest-clean subsets, respectively. We adopt the cross-sentence evaluation approach in Cosy Voice and F5-TTS: utterances from each subset are paired to form <prompt, target> pairs, where the prompt utterance used the reference to TTS models and the transcript of the target utterance serves as the text input. The prompt utterances are selected such that their durations rounded between 3 and 4 seconds, while the duration of target utterances is between 4 and 10 seconds. We set the validation and test sets to contain 200 and 1000 utterances, respectively.

---

Table 1: Training and inference configurations of TTS models. 8: CFG strength. i: 24,000 maximum frames.
Model
Vocoder
Dataset
Epochs
Batch Size Learning Rate Warmup GPUs f
Solver
Matcha-TTS
Vacos
LJ Speech
800
128
4x10-1
Fuler
Matcha-TTS
Vocos
VCTK
800
128
2x10-1
Euler
StableTTS
Vocos
VCTK
800
128
2x10-4
10%
Dopri(5)
Cosy Voice
HiFi-GAN
LibriTTS
200
dynamici
1x10-1
10%
0.7
Euler
Cosy Vosce-ic-Dii
Vocos
LibriTTS
200
256
1x10-
10%
4
3
Dopri(5)
4.2
Model implementations
Since the SFM head (the architecture is shown in Appendix B receives hidden states as input for richer contextual information, all length regulators in each model upsample the hidden states. Several neural network layers smooth the upsampled hidden states, which are then used to generate coarse mel-spectrograms and input to the SFM head as in Eq. 12). The coarse mel-spectrograms are
supervised using the target mel-spectrograms via Eq.
Matcha-TTS: We utilize Matcha-TTh with the officially pre-trained Vocos [3 as the vocoder. Since the official implementation does not include smoothing layers for the encoder outputs, we designate the final block of the six-block encoder to serve as smoothing layers.
StableTTS: Although STableTTS incorporates reference encoder to extract speaker embeddings and enables zero-shot TTS, we observe that the extracted speaker embeddings are unstable and lead unstable encoder outputs. This instability causes the simple SFM head to struggle with converging and predicting accurate f, and 0% Therefore, initialization instead of the reference encoder. Both the encoder and decoder are configured with six DiT blocks. The Vocos pre-trained in StableTTS is used as the vocoder.
ID-based speaker embeddings with random
we only train the flow module of Cosy Voice. We use officially pre-trained speech tokenizer and LLM generate speech tokens for training and inference, respectively. The pre-trained HiFi-GAN 20 in Cosy Voice is used as the vocoder. In the original FM module, the encoder takes only speech tokens input, while speaker embeddings and masked mel-spectrograms are used as conditions for the FM decoder. However, under this configuration, we find that the encoder fails to produce coarse mel-spectrogramss with sufficient speaker features. To address this, we concatenate the speech tokens, speaker embeddings, and masked mel-spectrogramss, and feed them jointly into the encoder. For ablation studies, evaluate variant (denoted as SFM-t) with the SFM method in which the encoder takes only speech tokens as input.
Cosy Voice: As discussed in Section 2.3
Cosy Voice- DIT: As discussed in Section 2.3 we construct Cosy Voice-Di using the encoder and decoder of STaSle each of which contains six DiT blocks. The speech tokens are fed into the encoder, while the speaker embeddings are incorporated into every DiT block via adaLN-Zero [32 The Vocos pre-trained in StableTTS used as the vocoder.
Baseline models:
We adopt and train the original models as baseline models. Due to necessary architectural modifications, no direct baseline is available for StableTTS and CosyVoice-DDi
For each model, we construct an ablated version to enable a more direct comparison with the SFM model, mainly including the addition of coarse loss and the SFM head. In the ablated models, the SFM head only outputs the X h the condition for the FM module, Therefore, the only difference between the ablated and SFM models is whether the SFM method is applied. This design allows us isolate and assess the effect of the proposed SFM method.
Ablated models:
SFM-c: In our SFM method, the coarse mel-spectrograms are not used as FM conditions. For ablation studies, also evaluate variant (denoted as SFM-c) that not only applies the SFM method but also uses coarse mel-spectrograms as FM conditions.
4.3 Training and inference configurations
All training, inference, and objective evaluations are conducted on 96 GB Nvidia H100 GPUs with half precision (FP16). We follow the official configurations of each baseline as closely as possible, and some key settings are summarized in Table Specifically, a warmup parameter indicates that

---

Table 2: Adaptive-step ODE solvers used in our experiments. All of them are variants of the explicit Runge-Kin Kutta family.
Abbreviation
Full Name
Order
(2)
Adaptive Heun's Method
Fehlberg(2)
Kunge-Kutta-Fehlbery Method
2
Bosh(3)
Method
Dopri(5)
Dormand Prince Method
5
learning rate scheduler is used. Because of large batch sizes, increase the constant peak learning rates based on the linear scaling rule, and then reduce them if gradient explosions are observed. The presence of CFG strength indicates the application of CFG.
4.4 Evaluations
4.4.1 Objective evaluation
We evaluate model performance using three pseudo-MOS prediction models: UTMOS (37, UT- MOSv2 (3, and DistillMOS [41). Their average defined as PMOS and is used for selecting the optimal in SFM methods. In addition, we report word error rate (WER) and speaker similarity (SIM). For WER, we use Whisper-large-ve B4 as the speech recognition model transcribe speech. For SIM, we use WavLM-base-phae
[ 40 to extract speaker embeddings and compute the cosine
similarity between synthesized and ground speech.
4.4.2 Subjective evaluation
We conducted comparative MOS (CMOS) and similarity MOS (SMOS) tests to evaluate naturalness and similarity, respectively. In each single test, recruited 20 native English listeners on Prolifi with L9 per hour.
We our proposed model with SFM as the reference system, and all other systems compared against it. For each system, five utterances are selected and paired with corresponding utterances from the SFM model. Listeners naturalness difference on a 7-point scale ranging from -3 to +3. A score of +-3 indicates that the first utterance sounds much better than the second, while -3 indicates the opposite.
CMOS:
presented with utterance pairs and asked rate the
SMOS: SMOS tests are only conducted for cross-sentence evaluations to evaluate the similarity between the ground-truth prompt utterance and the target utterance from different systems. Each system provides five utterances with corresponding prompts. Listeners are presented with <prompt, target> utterance pairs and asked to rate how similar the target sounds to the prompt 5-point scale ranging from to 5, where indicates "not similar at all" and indicates "extremely similar".
4.5
SFM strength selection
We determine the optimal SFM strength on the corresponding validation sets. Starting from 1.0, we increment a by 1.0 and objectively evaluate the generated speech identify the value that yields the highest PMOS. We then examine its neighbors at a + 0.5 for potential improvements. Finer-grained search avoided prevent overfitting and ensure the robustness of the SFM method.
4.6 ODE solvers and speed analysis
For evaluations, we follow the official ODE solver settings of each model. For speed analysis with adaptive-step ODE solvers, use the odeint from torchdiffeq 4 with various solvers. The default relative and absolute tolerances (1 x 10-7 and x 10-9) are too strict, resulting in significantly longer inference time. Therefore, adopt the settings used in StableTTS: 1x10-5 for both tolerances. As
httpp://huugiingfeeeecccdrereees fttps://www.prooific..

---

Table 3: Partial objective evaluation results on validation sets for a selection. Complete results are provided in Appendix D
The highest value for each metric is highlighted in bold.
9
3g
PMOST UTMOSt UTMOS-21
Distill-MOST
WER(
SIM
Matcha-TTS T7S (SFM)
trained on LJ Speech
1.0
0.099 0.092
4.036
4.194
3.721
4.192
4.641
0.972
2.0
0.198 0.183
4.158
4.305
3.834
4.337
3.496
0.973
2.5 0.248 0.229
4.176
4.276
3.872
4.381
3.556
0.972
3.0 0.297 0.275
4.168
4.260
3.842
4,402
3.496
0.970
3.5 0.347 0.320
4.132
4.190
3.802
4.403
3.496
0.969
4.0 0.397 0.366
4.107
4.137
3.763
4,421
3.556
0.966
5,0 0.496 0.458
4.025
3.977
3.694
4.403
3.376
0.960
6,0 0.520 0.480
3.997
3.958
3.648
4.386
3.315
0.958
7.0 0.520 0.480
3,990
3,960
3.625
4.386
3.315
0.958
8.0 0.520 0.480
3,990
3,959
3.625
4.386
3.315
0.958
9.0 0.520 0.480
3.993
3,956
3.638
4.386
3.315
0.958
10.0 0.520 0.480
3.987
3.955
3.620
4.386
3.315
0.958
shown in Table 2 we employ Huen(2), Fehlberg(2), Bosh(3), and Dopri(5) as adaptive-step solvers, where the number () denotes the order.
We adopt real-time factor (RTF) and number of function evaluations (NFE) as metrics for speed evaluation, where RTF measures the ratio between the ODE solver inference time and the correspond- ing audio duration, and NFE indicates how many times the solver queries the model. To reduce the influence of runtime variance, choose 100 utterances from each model's validation set and run each solver five times, reporting the average RTF. Note that the NFE is constant across runs.
5 Experimental results and analysis
5.1 Evaluation results
Due to page limitations, Table and Table Jonly provide partial results, and complete results are provided in Appendix [nd respectively. Other minor analysis is provided in Appendix E
and the tables in Appendix ig and og are computed according to Eq. 22 and the reported values represent their means over all utterances in the validation sets. From these tables, it is evident that using values of a > 1.0 significantly improves both pseudo-MOS scores and WER. The PMOS, which serves as the selection criterion, tends to increase initially and then decrease as a grows. This observation suggests that the adaptively determined (a during training is generally small. As a result, the intermediate state constructed at inference with a 1.0 corresponds an early position on the CondOT paths. When Gaussian noise Xo is added at this stage, the resulting signal-to-noise ratio is low, making it difficult for the flow decoder to extract sufficient information and weakening the guidance during early sampling steps.
SFM strength selection:
Table and
According to Eq. 13 X is supervised by the linearly down-scaled 4 X,, This allows us to apply the during inference to linearly scale up Xx in Eq. 22 thereby enhancing its guidance effect. However, due estimation errors, increasing a also leads growing distance between X, h and 1,,1, Meanwhile, the incorporated X, decreases and results in more deterministic sampling process. These factors ultimately reduce generation quality.
Objective evaluations: As shown in Table all SFM models outperform their corresponding baseline and ablated models regarding pseudo-MOS scores. However, the results for WER and SIM are more variable, with improvements observed in some cases but not in all. This suggests that there still room to improve the alignment quality of the SFM method.
Subjective evaluations: As shown in Table SFM models achieve better performance in both CMOS and SMOS tests compared to their corresponding baseline, ablated, and SFM variants. This demonstrates that the SEM method efficiently improves the naturalness of synthesized speech.
Cosy Voice (SFM-t): When only speech tokens are input into the encoder of the flow module, Cosy Voice (SFM-1) exhibits lower SIM and significantly worse SMOS performance. This because the Cosy Voice tokenizer is pre-trained with an ASR objective to extract semantic tokens, which

---

Table 4: Evaluation results on test sets. indicates statistically significant differences (p < 0.05)
compared with SFM models in subjective evaluations. The highest value for each metric is bolded.
System
UTMOSS UTMOS=2 DistillMMS//WEER SIM|/CMOST SMOS
Matcha-TTh trained on LJ Speech
Ground truth
4.380
3,964
4.241
3.566
1.000
+0.22
Reconstructed
4.085
3.739
4.208
3.472
0.993
+0.12
Baseline
4.186
3.692
4.282
3.308
0.971
0.48
Ablated
4.217
3.763
4.311
3.355
0.972
0.27
SFM (o=2.5)
4.257
3.848
4.386
3.413
0.972
0.00
Matcha- TTS trained VCTK
Ground truth
3,999
3.562
3.986
1.534
1.000
+0.16
Reconstructed
3.819
3.246
3,977
1.666
0.985
+0.08
Baseline 4.008
2.978
3.870
1.534
0.939
00.1*
Ablated
4.026
2.997
3.872
1.613
0.941
-0.39*
SFM (a=3.5)
4.106
3.105
3.898
0.952
0.937
0.00
StableTTS trained VCTK
Ground truth
3,999
3.562
3.986
1.534
1.000
+0.48*
Reconstructed
3.360
2.908
3.855
1.719
0.972
+0.04
Ablated
3.328
2.958
3.929
1.798
0.932
0.34*
SFM (a=3.0)
3.516
3.020
3.953
1.745
0.933
0.00
SFM-c (a=4.5)
3.507
2.899
3,934
0.931 1.877
-037*
Cosy Voice trained on LibriTTS
Ground truth
4.136
3.262
4.345
3.180
1.000
+0.19
3.40
Reconstructed
3.942
3.126
4.336
3.146
0.993
0.24
2.82*
Baseline
4.191
3.303
4.481
3,513
0.932
-0.21*
3.47
Ablated
4.183
3.369
4.487
3.578
0.932
0..4
3.58
SFM (a=2.0)
4.194
3.480
4.541
3,810
0.931
0.00
3.67
SFM-I (a=2.5)
4.132
3.336
4.547
3.987
0.914
-0.09
2.66*
Vosce-ice-Di7 trained LibriTTS
Ground truth
4.136
3.262
4.345
3.180
1.000
+0.23
3.31
Reconstructed
3.322
2,855
4.211
3.144
0.989
-0.2
2.86*
Ablated
3.499
3.086
4.316
3.614
0.936
0..1*
3.15
SFM (a=2.5)
3.751
3.171
4.502
3.598
0.932
0.00
3.21
SFM-c (a=4.0)
3.752
3.156
4.496
3.634
0.929
0.06
3.10
contain limited speaker-specific information. Although speaker-related features are employed as flow conditions, they fail to guide the generated speech with sufficient speaker characteristics. This further highlights the importance of early-stage flow inference, errors introduced at the beginning difficult to correct later sampling steps.
SFM-c:
When using coarse mel-spectrograms as flow conditions, the adaptively determined f,. tends converge to 0 in models with b-seed architectures, rendering the SFM-c variant inapplicable to these models. Models using DiT blocks, such as StableTTS (SFM-c) and Cosy VosceoiccD-D (SFM-c) c)), perform worse in subjective evaluations than their corresponding SFM models. These results suggest that, for our SFM method, using coarse mel-spectrograms as flow conditions not only fails to improve performance can also degrade synthesis quality or invalidate the method.
5.2 Acceleration of adaptive-step ODE solvers
Table 5 and the tables in Appendix F report only the mean RTF for clarity, as the standard deviations across the five runs are all below 0.013. We also present the speedup rate terms of RTF, measured relative to the ablated model. From these tables, we observe that increasing improves the signal- to-noise ratio of the initial state during flow inference, which stabilizes the ODE solving process. As result, fewer forward steps are required, and the overall inference time is significantly reduced.

---

Table 5: Partial RTF and NFE results for adaptive-step ODE solvers. Complete results are provided Appendix F RTF denotes the mean RTF. Rate (%) denotes the relative speedup in terms of RTF compared to the ablated model.
System
Heun(2)
Fehlberg(2)
Bosh(3)
Dopri(5)
RIF
Ratef
NFE]
RTF,
Rate!
NFEL RTF,
Ratef
NPE,
RIF,
Ratef
NPEL
Matcha-TTe
trained on LJ Speech
Bascline
0.407 -1,496 312.36 0.054
3.571
44.82
0.271
-0.370
223.95
0.142
2.069
120.10
Ablated
0.401 0.000 306.72 0.056
0.000
45.28
0.270
0.000
221.81
0.145
0.000
121.46
SFM (a=1.0)
0.361 9.858
277.56 0.053
4.171
44.08
0.225
16.924
186.23
0.132
9.100
111.14
SFM (o=2.0)
0.299 25.541 229.43
0.048
13.306
39.16
0.187
30,931
153.08
0.115
20.484
96.02
SFM (a=3.0) 0.249 37.931
191.49
0.043
22.749
34.88
0.157
41.895
129.02
0.100
30.753
84.20
SFM (a=4.0)
0.207
48.486
156.78
0.037
34,246
29.74
0.131
51.576
107.90
0.086
40.559
72.56
SFM (a=5.0) 0.157 60.825
120.07
0.027
50.968
22.14
0.110
59.266
90.74
0.076
47.605
63.74
Notably, this acceleration effect is limited adaptive-step solvers, fixed-step solvers perform predefined number of steps, and thus cannot leverage the improved stability of the initial stages to reduce inference cost.
Related works
Our proposed SFM method extends the idea of the shallow diffusion mechanism. To the best of our knowledge, while no prior work exactly matches our approach, several studies adopt similar strategies pursue similar objectives. As discussed in Section 3.1 ReRFlow [47] applies piecewise reflow to divided flow trajectories. PixelFlow also adopts piecewise flow for multi-scale resolution generation. In our case, utilize piecewise flow to divide the CondOT paths into two segments and use only the last segment. addition, shortcut models [[]] employ a self-consistency mechanism to construct shortcuts along the CondOT paths. Modifying flow matching [2] samples from Gaussian distribution centered at a coarse output instead of the standard normal distribution, and further adopts deterministic inference.
7 Limitations
This work applies only minimal modifications to the backbone TTS models and uses a simple SFM head demonstrate the effectiveness of our proposed SFM method. Therefore, there considerable room for improving the SFM framework and corresponding implementation. For example:
1. A more powerful SFM head could be used when the weak generator is unstable.
our implementations, although text or semantic tokens are used to generate the intermediate states, they are not further incorporated as conditions in the FM process. Introducing explicit semantic conditioning may improve the alignment of the final outputs with the input semantic features.
3. When applying SFM on Cosy Voice, we directly concatenate the speech tokens, speaker embed- dings, and masked mel specsterrosss as the encoder input. This naive fusion strategy may negatively affect cross-modal alignment and calls for a more elaborate integration approach.
8 Conclusion
We introduce a novel Shallow Flow Matching (SFM) method for coarse-to-fine TTS models. SFM leverages coarse representations to construct intermediate states the CondOT path, enabling FM module to produce more stable generation, more natural synthetic speech, and faster inference when using adaptive-step ODE solvers.
SFM focuses on cases where the FM module serves as a refiner. Although it validated on TTS tasks, the underlying framework and theoretical foundation are general. It also holds potential for other domains, such as speech denoising and enhancement, as well as image generation tasks like denoising and super-resolution Further exploration of its applications to these tasks remains a promising direction for future work.
10

---

Acknowledgements
This research was supported by JST Moonshot Grant Number JPMJMS2011 and IST SPRING Grant Number JPMJSP2108.
References
[1] StableTTS, htttps://githui htt.://b...oookkaaa///taaaeeeee.
2024.
[2] Anastassiou, J. Chen, J. Chen, Y. Chen, Z. Chen, Z. Chen, Cong, L. Deng, C. Ding, L. Gao, M. Gong, P. Huang. Q. Huang, Z. Huang. Y. Huo, D. Jia, C. Li, F. Li, H. Li, Li, X. Li, X. Li,
L. Liu, S. Liu, Liu, X. Liu, Y. Liu, Z. Liu, L. Lu, J. Pan, Wang, Y. Wang, Wang, Z. Wei, J. Wu, C. Yao, Y. Yang, Y. Yi, J. Zhang, Q. Zhang, S. Zhang, W. Zhang, Y. Zhang, Z. Zhao,
D. Zhong, and X. Zhuang. Seed-Its: family of high-quality versatile speech generation models. arXiv preprint wxXiv:240.0024300 2024.
[3] K. Baba, W. Nakata, Y. Saito, and H. Saruwatari. The t05 system for the VoiceMOS Challenge 2024: Transfer learning from deep image classifier to naturalness MOS prediction of high- quality synthetic speech. In IEEE Spoken Language Technology Workshop (SLT), 2024.
[4] R. Q. Chen. torchdiffeq. http://ggihbbbooom/nngcceeeeerer
2018
[5] S. Chen, C. Wang, Z. Chen, Y. Wu, S. Liu, Z. Chen, Li, N. Kanda, T. Yoshioka, X. Xiao, J. Wu, L.. Zhou, S. Ren, Y. Qian, Y. Qian, J. Wu, M. Zeng. X. Yu, and F. Wei. WavLM: Large-scale self-supervise pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 1605):1505-1518, 2022.
[6] Q. Chen, Y. Rubanova, J. Bettencourt, and D. Duvenaud. Neural ordinary differential equations. In Anmual Conference on Neural Information Processing Systems (NeurIPS), pages 6572-6583, 2018.
[7] Y. Chen, Z. Niu, Z. Ma. K. Deng, C. Wang, J. Zhao, K. Yu, and X. Chen. F5-TTS: A fairytaler that fakes fluent and faithful speech with flow matching. arXiv preprint arX.v.2410006885. 2024.
[8] Z. Du. Q. Chen, S. Zhang, K. Hu, H. Lu, Y. Yang. H. Hu, S. Zheng. Y. Gu, Z. Ma, Z. Gao, and Z. Yan. supervised semantic tokens. arXiv preprint 2rXiv:2407.05400 2024.
A scalable multilingual zero-shot text-to-speech synthesizer based on
[9] Du, Y. Wang, Q. Chen, X. Shi, X. Lv, T. Zhao, Z. Gao, Y. Yang, C. Gao, H. Wang, F. Yu, H. Liu, Z. Sheng, Y. Gu, C. Deng, W. Wang, S. Zhang, Z. Yan, and J. Zhou. CosyVoice 2: Scal- able streaming speech synthesis with large language models. arXiv preprint arXiv:2412.10117 2024.
[10] S. F. Eskimez, X. Wang, M. Thakker, C. Li, C. Tsai, Z. Xiao, H. Yang. 7. Zhu, M. Tang, X. Tan, Y. Liu, S. Zhao, and N. Kanda. E2 TTS: embarrassingly easy fully non-autoregressive zero-shot TTS. In IEEE Spoken Language Technology Workshop (SLT), 2024.
[11] K. Frans, D. Hafner, S. Levine, and P. Abbeel. One step diffusion via shortcut models. In International Conference Learning Representations (ICLR), 2025.
[12] Y. Gao, N. Morioka, Y. Zhang, and N. Chen. E3 TTS: easy end-to-end diffusion-based text to speech. In IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 1-8, 2023.
[13] W. Guan, Q. Su, H. Zhou, S. Miao, X. Xie, L. Li, and Q. Hong. Reflow-tts: A rectified flow model for high-fidelity text-to-speech. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP, pages 10501-10505, 2024.
[14] A. Gulati, J. Qin, C. Chiu, N. Parmar, Y. Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y. Wu, and R. Pang. Conformer: canvolution-uugmentee transformer for speech recognition. In Interspeech, pages 5036-5040, 2020.

---

[15] Y. Guo, C. Du, Z. Ma, X. Chen, and K. Yu. VoiceFlow: Efficient text-to-speede with rectified flow matching, In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 11 11121-11125, 2024.
[16] J. Ho and T. Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598. 2022.
[17] K. Ito and L. Johnson. LJ Sopeech-Dattseri 2017.
The LJ Speech dataset.
https://keithitto.com
[18] J. Kim, S. Kim, J. Kong, and S. Yoon. Glow-tts: A generative flow for text-to-specch via monotonic alignment search. In NeurIPS, 2020.
[19] Kim, J. Kong, and Son. Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech. In International Conference Machine Learning (ICML), volume 139, pages 5530-5540, 2021.
[20] J. Kong, J. Kim, and J. Bae. HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis. In Annual Conference on Neural Information Processing Systems (NeurIPS), 2020.
[21] R. Korostik, R. Nasretdinov, and A. Jukié. Modifying flow matching for generative speech enhancement. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025.
[22] M. Lajszczak, G. Cámbara, Y. Li, F. Beyhan, A. van Korlaar, F. Yang. A. Joly, Á. Martín- Cortinas, A. Abbas, A. Michalski, A. Moinet, S. Karlapati, E. Muszynska, H. Guo, B. Putrycz, S. L. Gambino, K. Yoo, E. Sokolova, and T. Drugman. BASE TTS: lessons from building a billion-parameen text-to-speech model on 100k hours of data. arXiv preprint 3r00:2240..0003, 2024.
[23] M. Le, A. Vyas, B. Shi, B. Karrer, L. Sari, R. Moritz, M. Williamson, V. Manohar, Y. Adi, J. Mahadeokar, and Hsu. Voicebox: Text-guided multilingual universal speech generation at scale. In NeurlPS, 2023.
[24] K. Lee, W. Kim, J. Kim, and J. Cho. DITTO-TTS: Efficient and scalable zero-shot text-to- speech with diffusion transformer. In International Conference on Learning Representations (ICLR), 2025.
[25] S. Liao, Y. Wang, T. Li, Y. Cheng, R. Zhang, R. Zhou, and Xing. Fish-speech: Leveraging large language models for advanced multilingual text-to-speech synthesis. arXiv preprint 1:Xiv:2411.01155. 2024.
[26] Y. Lipman, R. T. Q. Chen, H. Ben- M. Nickel, and M. Le. Flow matching for generative modeling. In International Conference on Learning Representations (ICLR), 2023.
[27] J. Liu, C. Li, Y. Ren, F. Chen, and Z. Zhao. Diffsinger: Singing voice synthesis via shallow diffusion mechanism. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 11020-11022 1028, 2022.
[28] X. Liu, C. Gong, and Q. Liu. Flow straight and fast: Learning generate and transfer data with rectified flow. International Conference on Learning Representations (ICLR), 2023.
[29] R. J. McCann. A convexity principle for interacting gases. Advances in Mathematics, 128(1):153-179 1997.
[30] S. Mehta, R. Tu, J. Beskow, É. Székely, and G. E. Henter. Matcha-TTS, TTS: A fast TTS architecture with conditional flow matching. In IEFE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1131--111 11345, 2024.
[31] L. Meng, L. Zhou, S. Liu, S. Chen, B. Han, S. Hu, Y. Liu, J. Li, S. Zhao, X. Wu, H. Meng, and Wei. Autoregressive speech synthesis without vector quantization. arXiv preprint arXiv:2407.08551, 2024.
12

---

[32] W. Peebles and S. Xie. Scalable diffusion models with transformers. In /EEE/CVF International Conference on Computer Vision (ICCV), 2023.
[33] V. Popov, I. Vovk, V. Gogoryan, T. Sadekova, and M. A. Kudinov. Grad-us: A diffusion probabilistic model for text-to-speech. In International Conference on Machine Learning (ICML), volume 139, pages 8599-8608, 2021.
[34] A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever. Robust speech recognition via large-scale weak supervision. In International Conference Machine Learning (ICML), pages
2023.
[35] Y. Ren, C. Hu, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T. Liu. FastSpeech 2: Fast and high-quality end-to-end text speech. In International Conference Learning Representations (ICLR), 2021.
[36] O. Ronneberger, P. Fischer, and T. Brox. U-Net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2015.
[37] T. Sacki, D. Xin, W. Nakata, T. Koriyama, S. Takamichi, and H. Saruwatari. UTMOS: utokyo- sarulab system for voicemos challenge 2022. In Interspeech, pages 4521-4525, 2022.
[38] S. Shikhar, M. I. Kurpath, S. Mullappilly, J. Lahoud, F. Khan, R. M. Anwer, S. Khan, and H. Cholakkal. LLMVoX: Autoregressive streaming text-to-speech model for any LLM. arXiv preprint arXiv:2503. 04724, 2025.
[39] H. Siuzdak. Vocos: Closing the gap between time-domain and fourier-based neural vocoders for high-quality audio synthesis. In International Conference on Learning Representations (ICLR), 2024.
[40] D. Snyder, D. Garcia-Romero, G. Sell, D. Povey, and S. Khudanpur. X- Vectors: Robust DNN embeddings for speaker recognition. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5329-53333 00399-3333222 2018.
[41] B. Stahl and H. Gamper. Distillation and pruning for scalable self-supervieed representation- based speech quality assessment. arXiv preprint N5KV::2002.03566 2025.
[42] X. Sun, R. Xiao, J. Mo, B. Wu, Q. Yu, and B. Wang. F5R-TTS: Improving mat-hhg based text-to-speech with group relative policy optimization. arXiv preprint arXiv:2504.02400; 2025.
[43] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. In Annual Conference on Neural Information Processing Systems (NeurIPS), pages
2017.
[44] C. Veaux, J. Yamagishi, and K. MacDonald. CSTR VCTK corpus: English multi-speaker corpus for CSTR voice cloning toolkit. University of Edinburgh. The Centre for Speech Technology Research (CSTR), 2017.
[45] C. Wang, S. Chen, Y. Wu, Z. Zhang, L. Zhou, S. Liu, Z. Chen, Y. Liu, H. Wang. J. Li, L. He, S. Zhao, and F. Wei. Neural codec language models are zero-shot text to speech synthesizers. arXiv preprint arXiv:2301.02//I/// 2023.
[46] X. Wang, M. Jiang, 7. Ma, Z. Zhang. S. Liu, I. Li, 7. Liang, Q. Zheng, R. Wang, X. Feng, W. Bian, Z. Ye, S. Cheng, R. Yuan, Z. Zhao, X. Zhu, J. Pan, L. Xue, P. Zhu, Y. Chen, Z. Li, X. Chen, L. Xie, Y. Guo, and W. Xue. Spark-TTS: An efficient Iameb texl-to-speech model with single-stream decoupled speech tokens. arXiv preprint arXiv:2503.01 01710, 2025.
[47] H. Yan, X. Liu, J. Pan, J. H. Liew, Q. Liu, and J. Feng. Perflow: Piecewise rectified flow as universal plug-and-pla accelerator. In Annual Conference on Neural Information Processing Systems (NeurIPS), 2024.
[48] D. Yang. S. Liu, R. Huang, C. Weng, and H. Meng. Instructtts: Modelling expressive TTS in discrete latent space with natural language style prompt. IEFF/ACM Transactions on Audio, Speech, and Language Processing, 32:2913-2925, 2024.
13

---

[49] Z. Ye, X. Zhu, C. Chan, X. Wang, X. Tan, Lei, Y. Peng, H. Liu, Y. Jin, Z. Dai, H. Lin. J. Chen, X. Du, L. Xue, Chen, Z. Li, L. Xie, Q. Kong, Guo, and W. Xue. Llasa: Scaling train-time and inference-time compute for llama-based speech synthesis. arXiv preprint arXiv:2502.04128, 2024.
[50] H. Zen, V. Dang. R. Clark, Y. Zhang, R. Weiss, Jia, Chen, and Y. Wu. LibriTTS: A corpus derived from LibriSpeech for text-to- -peecch. In Proc. Interspeech, pages 1530, 2019.
[51] B. Zhang, C. Guo, G. Yang, H. Yu, H. Zhang, H. Lei, J. Mai. J. Yan, K. Yang, M. Yang, Huang, R. Jin, S. Jiang, W. Cheng, Y. Li, Y. Xiao, Y. Zhou, Y. Zhang, Y. Lu, and Y. He. Minimax-speech: Intrinsic zero-shot text-to-speech with a learnable speaker encoder. arXiv preprint arXiv:2505.07911 2025.
14

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justificcton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The main claims in the abstract and introduction are addressed in Sections and S of the main paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
15

---

Justification:
The limitations are discussed in Appendix ??.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specifiitiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-texi system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't "t acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: The main theoretical insights are discussed in Section and detailed proofs are given in Appendix
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: All the implementation details to reproduce the results are given in Section and Appendix and C
Guidelines:
16

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to hosted model (e.g., s., in the of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: Although we use publicly available datasets and provide our code in supple- mental material, our experiment involves multiple models whose environment settings need to be found on their official GitHub pages.
Guidelines:
The answer NA means that paper does not include experiments requiring code. Please see the NeurlPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/Ctd///oeeeeeeaeeaaaeee ssi onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
17

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
All the experiment details are given in Section
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core paper to a level detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We apply t-tests on subjective evaluation results, and report the std Ap- pendix F
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [No]
Justification: We do not discuss the preliminary or failed experiments that didn't 't make into the paper.
Guidelines:
18

---

The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer:
[Yes]
Justification: Our work adheres all the ethical guidelines outlined by NeurIPS.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification:
The broader impacts are discussed in Appendix ??.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (c.g., disinformation, generating fake profiles, surveillance), fairness considerations deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [No]
19

---

Our work does not pose any explicit risks as we use public datasets, and applying our proposed methods other areas needs dedicated modifications.
Justification:
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All dataset details and original authorship are cited in Sections and
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi twwowethhde...m caree.comdaee datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release any new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mooel as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
20

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [Yes)
Justification: We explain our subjective evaluation settings in Section 4 and provide screen- shots in Appendix ??.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification: This study involves subjective evaluation tests. All participants took part with informed consent, and the study was approved by the IRB of our institution.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
We only use LLM for writing and editing.
Guidelines:
The answer NA means that the core method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy htap::///earrrpsss./Caaeeeerrree for what should or should be described.
21

---

A Theorem Proofs
Theorem For any random variable I.m. (1, m 1). where Im [0, and on 00), define transformation that maps 3m onto the CondOT paths. The output distribution varies continuously with respect to fm and Om under the Wasserstein-2 metric.
Omin Mm
oms
(23)
Tmin mm) 2 σ Lo
Im
if 1,
(24)
if 2 1,
A
where zo NNO,,) I), with corresponding T min(tm: fm) on the path.
Proof.
Since 2m (tmmm1,, ( and zo (0, independent, as linear combination of them, 2, also follows a Gaussian distribution with:
tm:
if A < 1,
(25)
A bms
if 1.
mm = T
if < 1,
p(x))=
(26)
A t T
if A > 1,
(1 (1 0 min 1m )2
0 + 02
Omin)
if A 1,
o(x,)=
(27)
dm = 1 - - o min) 1
1--11 (1- miin)
A > 1,
where uu()) denotes the mean and (-) denotes the std.
Since uxx,) and (xx, satisfy Eq. (3) ær lies on the CondOT paths.
We denote the transformation as T [0, 1] x (0, oo) x Rd x Rd + Rd. Since xp ~ NO and æ sample from the dataset, the distribution of æ, depends only on fm and om. conditioned on æ1. Therefore, we denote the distribution of æ, as:
ar ((((x,) a (xr)) = Tiy (lu Tm).
(28)
To verify that Ta, (tm, Tm) is continuous in the Wasserstein-2 metric with respect to im and om we use formula:
W2 ( E,),Ne(H, Ee)) | 14212 Tr [1 + E2
(29)
our distributions are isotropic, we have:
(Z of Vll,, -,z11 n(o1 -o2)2
(30)
where n is the dimension of the variables. We define 8 for proof.
For bmi
When Im < 1-0m, then A < 1:
I-6min"
lim W2(Tan (fm 6, am); Tay (m)) lim / |2m n(8(1 ominn) )2
(31)
6-2
0,
(32)
lim W2(To, (tm, am), Ta, (tm + 6, ))- lim / || || n((61 - omin)) 6-0
(33)
6-20
0.
(34)
II. When fm 1-0m. I-omin
then A 1:
22

---

lim W2(To, (tm - ò, 0m), Taa (m)) lim v /10001 11 n(0(1 min )
(35)
0,
(36)
Jim W2 (Ims Tm), Top (Im + 8, am))
(37)
lim 8-0
V
11 1 1+ 6(1 mmin)
mm +
xn 1|2 + n(Om
Om
(38)
1 8(1
min
0.
(39)
III. When tm
I- Omiin
then A > 1:
1-
lim W2 (mm 8, om), 6-+
; Tai (bm; 0m))
(40)
-im 8-0
V
A
5(1 mmin)
tm - 8
fm
A
|x1l 112 + nl
A
8(1 mmin)
om )2 A
(41)
= 0,
(42)
lim (2 (tms mm)) Tea (tm om)) 8-+0
(43)
fm
fm + 8
lim
|a 1| + n
)
(44)
8-0
A
A + 5(1 mmiu
A
A + 5(1 min
(45)
For Omi
I. When 0m < 1 - (1 mmin)mm. then A < 1:
lim W2(Ta, (Ims om - -8), Tw, (tms 0m)) 5-+0
lim v 10x 2| + n())
(46)
6-1
0,
(47)
lim W2(Ta, Om); Tan (lm, a. + 8)) lim 6-0
/12, + n()))²
(48)
8-0
(49)
II. When om 1 (1 minu then A =
lim W2 (tm, Fm - 8), Tex (tm, σmm) = lim 8-+0
0z 2| + n())2²
(50)
6-10
0,
(51)
lim W2 (tm, Jm). Tw, (tm, om + 6)) 8-50
(52)
Im
Fm +
= lim
111m
x
1 +
n((am
(53)
8-40
1
1 1 8
=0. 0.
(54)
III. When 0. > 1 (1 mmin \m, then A 1:
23

---

lim W2 (T2,, (Im; Om 8), Tan, (tmm, 0m)) 5-40
(55)
lim
|||
mm
Im
x 112
+
n
mm
0m )²
(56)
8-0
A δ
8
A
0,
(57)
lim (2(Tx, (t mm); Tas, (tm: om 6)) 6-+0
(58)
tm
tm
lim
10111 112
+ 1l
)
(59)
8-0
A
A
A
A + 8
0.
(60)
Thus, Tw, (tm, Tm) is continuous with respect to fm and om in the Wasserstein-2 metric.
24

---

Theorem 2. For arbitrary intermediate states on the conditional OT (CondOT) paths:
Li... = - 1m 2a + tm (21 + Tmin tm E (0, 1), To ~ (),T),
(61)
we cun divide the puths into two segments at bm and represent the flow and VF using piecewise functions:
ID + L. Itm:
ift < tms
tm
(62)
t-tm tu
t-tm (x) + minn
ift 2 tm
I-
(m
x0);
ift tm
t1
8m
(63)
( + ao aimn >
ift 2 Im
In the first segment (t < tm) of the two-segment piecewise flow, the paths start from zp with zt... as the target. In the second segment (t 2 tm), the paths start from Itm. and move towards the target (x) + mo).
Proof. Substituting Eq. 661 into Eq.
662
and Eq.
63
derive the following:
When < tm:
t
t
3t = (1
100
+
(64)
mm
Im
t
t
(1
)xn
+
(1 - fm )0 + t(21 + mmi
(65)
tm
tm
t)300 + t(x1 + omin
(66)
Im (a - xr))
(67)
(mm20 + mm (201 + mmin
(68)
fm
(01 mm) 20-
(69)
II. When 2 bm:
1- tm
t - tm (xx + mm))
= (1 1 tm
)arm
+
(70)
1 tm
(1 t)zo0= + 1 1-t t
- tm (x) + omin
tm (m0 + omin
(71)
tm
tm
= (1 t)ao + t(x1 + wo),
(72)
(x1 + Tmin Ftm)
(73)
tm
1
Im (Ø1 + min min
(x) + mo) - .0 1 I.
(74)
Im
(x) + Tmi
(75)
Therefore, the piecewise flow and VF above satisfy Eq. 44)
25

---

B SFM head
Xn. 8n. logō
Omee + 2
Linear
x dridden
Dropout
LayerNorm
ReLU
4 Chidden
Conv1d
4 dhidden
Dropout
LayerNorm
ReLU
4 x Chiddten
Conv1d
Chidden
Hidden States H,
Figure 2: SFM head architecture.
As shown in Fig. 2 we use simple and lightweight SFM head, whose architecture is derived from the duration predictor of VITS and Matcha-TTS) The kernel size and padding of the two Conv Id layers are set to 3 and respectively.
The output of the SFM head can be split into X in. and log o7. with shapes [batch, mel, sequence]. [batch, 1, sequence], and [batch, 1, sequence], respectively. Since i, > 0. we apply Sigmoid activation to and compute the mean over the frames axis, resulting in an output of shape [batch, 1]. As described in Appendix C the SFM head predicts log 5% instead of o log o3 is also averaged across frames to produce an output of shape [batch, 1].
http://hhhohoussssssss/ss/ssssss models. py#L98 mac.hhicoccas///cppeeeeee/errrrr py#L70
26

---

C Algorithms
In the practical implementation, we apply several detailed modifications and tricks:
1. The SFM-head hy predicts log 07 instead of 0% directly to ensure numerical stability and positive outputs.
Due to the Lraarses the in obtained from Xn, are generally larger than during training. Therefore, we remove the constraint (n 2 0 in Equation 13
We also modify the placement of the C. for implementation convenience. Compared to the formulation in Equation 13 this modification results in scaled version of the original loss by constant factor of 2
Algorithm Training procedure of SFM
Input: the training set C).
1: repeat
2: Sample 1,,, C) from (X, C);
3: Generate coarse representations
Hg. Xg + gw (C)
X In, log oh + h4 H
4: Compute coarse loss
Ccoarse El X X111
S: Project onto CondOT paths
t/2
8g
h
X
of + B|lsg[Xnn ,,,
X1 X1
6: Construct the intermediate state
A t max((1 Omin + thh 1)
1 X th + A th: of + A2
1
Xn +
A
X ~ N(0.1)
X + V mmxx(( (1 m )2,
oh X) X
C, (In - tn)
Lo = (log 0h log 2 C = E||X, X - t, x112
7:
Apply FM
t ~ 4(001] 1], tt Stt(
X, - (1 - X)xx, + +(x,; + Fmin Xo)
U. -- 1
(X + T Xo xx,)
th
t t+ (1 t + tn
LCFM |Bllvo X x, t) U²
8: Apply gradient descent to minimize LSFM
LSFM Lcoarse + Ls + Lo + Lu + LCFM
9: until convergence
27

---

Algorithm 2 Inference procedure of SFM
Input: Condition C.
1: Generate coarse representations
Hg Xg t gw (C)
Xn, in log oh + h (H
of + exp(log o
2: Construct the intermediate state with SFM strength
A f- max(a((1 mmin Mn + oh),
A th: of a² Λz X N(O.I)
X
X th +
A
X max (1 - (1 T ) - 0) 0 + X
3: Use an ODE solver to solve the integral
X pred = Xn + vo(Xi,t) dt
28

---

D Complete results of a selection
We provide complete results in optimal a selection for all models in this section, including Table Table 7 Table
8 and Table
Table Objective evaluation results on validation sets for a selection (Matcha-TTS),
a
PMOS
UTMOS UTMOSv2
Distill-MOST WER( SIM
Matcha-TTH (SFM)
trained on LJ Speech
1.0
0.099 0.092
4.036
4.194
3.721
4.192
4.641
0.972
2.0
0.198 0.183
4.158
4.305
3.834
4.337
3.496
0.973
2.5
0.248 0.229
4.176
4.276
3.872
4.381
3.556
0.972
3.0
0.297 0.275
4.168
4.260
3,842
4,402
3,496
0.970
3.5
0.347 0.320
4.132
4.190
3.802
4,403
3.496
0.969
4.0 0.397 0.366
4.107
4.137
3.763
4,421
3.556
0.966
5.0 0.496 0.458
4.025
3,977
3,694
4,403
3.376
0.960
6.0 0.520 0.480
3.997
3.958
3.648
4.386
3.315
0,958
7.0 0.520 0.480
3,990
3.960
3,625
4.386
3.315
0.958
8.0 0.520 0.480
3,990
3,959
3.625
4.386
3.315
0.958
9.0 0.520 0.480
3.993
3.956
3.638
4,386
3.315
0.958
10.0 0.520 0.480
3.987
3.955
3,620
4.386
3.315
0.958
Matcha-TT'S (SFM)
trained
VСTК
1.0
0.100 0.078
3,462
3.876
2.723
3.787
4.898
0.923
2.0
0.201 0.155
3.647
4.061
2.988
3.891
2.177
0.940
2.5
0.251 0.194
3.652
4.045
3.007
3,904
2.041
0.940
3.0
0.301 0.233
3.678
4.079
3.041
3,914
1.497
0.939
3.5
0.351 0.272
3.679
4.074
3.053
3.910
1.224
0.936
4.0
0.401 0.311
3.660
4.079
3,001
3,900
1.088
0.934
5.0
0.502 0.389
3.615
4.059
2.916
3,869
1.088
0.929
6.0
0.564 0.436
3.555
4.084
2.757
3.824
1.224
0.923
7.0 0.564 0.436
3.554
4.084
2.752
3,824
1.224
0.923
8.0 0.564 0.436
3.563
4.084
2.782
3,824
1.224
0.923
9.0 0.564 0.436
3.559
4.082
2.770
3,825
1.224
0.923
10.0
0.564 0.436
3.546
4.083
2.730
3,824
1.224
0.923
29

---

Table 7: Objective evaluation results on validation sets for selection (StableTTS).
g
og
PMOST UTMOS;
UTMOSv21
Distill-MOST WER(%) SIM
StableTTS (SFM)
trained on VCTK
1.0
0.153 0.098
3.281
3.184
2.782
3,877
8.707
0.910
2.0
0.306 0.197
3.441
3,403
2.981
3,938
2.041
0.931
2.5
0.382 0.246
3,476
3,469
3,013
3.945
1.361
0,933
3.0 0.459 0.295
3.486
3,522
3,001
3,936
1.224
0.933
3.5
0.535 0.345
3.475
3.543
2.959
3.923
1.224
0.933
4.0 0.603 0.388
3.437
3.547
2.857
3,906
1.361
0,933
5.0 0.609 0.391
3.435
3,560
2.829
3,914
1.361
0.933
6.0 0.609 0.391
3.434
3,561
2.825
3,915
1.361
0,933
7.0 0.609 0.391
3.431
3,558
2.819
3,915
1.361
0.933
8.0 0.609 0.391
3.434
3.559
2.828
3,915
1.361
0.933
9.0 0.609 0.391
3.435
3.560
2.832
3,914
1.361
0.933
10.0 0.609 0.391
3.433
3.560
2.824
3,914
1.361
0.933
TTableTTS (SFM-c)
trained on VCTK
1.0 0.111 0.067
3.261
3.165
2.759
3,859
1.224
0.918
2.0
0.222 0.134
3.313
3.260
2.787
3.893
1.633
0.922
3.0
0.333 0.201
3.403
3.375
2.913
3,923
1.497
0.927
3.5 0.388 0.235
3.424
3,419
2.916
3.936
1.361
0,928
4.0 0.444 0.268
3.427
3.462
2.889
3,931
1.088
0.930
4,5 0.499 0.302
3.449
3.495
2.924
3,929
1.088
0.931
5.0 0.555 0.335
3.408
3,510
2.799
3,913
1.361
0.931
6.0 0.624 0.376
3.377
3.553
2.672
3,906
1.224
0.934
7.0 0.624 0.376
3.389
3.554
2.707
3,906
1.224
0,934
8.0 0.624 0.376
3.398
3,554
2.734
3.907
1.224
0.934
9.0 0.624 0.376
3.387
3,553
2.702
3,907
1.224
0.934
10.0 0.624 0.376
3.378
3.554
2.674
3,906
1.224
0.934
Table 8: Objective evaluation results on validation sets for selection (CosyVoice).
9
g
PMOSS UTMOS
UTMOSv21
Distill-MOST WER((%) SIM
CosyVoice (SFM)
trained on LibriTS
1.0
0.152 0.126
3.721
3.729
3.102
4.333
8.902
0.923
1.5
0.228 0.189
4.019
4.113
3.449
4.494
4.810
0.932
2.0
0.303 0.253
4.087
4.180
3,530
4.553
4.499
0,931
2.5
0.379 0.316
4.080
4.165
3.516
4.558
4.475
0.928
3.0 0.455 0.379
4.019
4.119
3,409
4.527
4.523
0.922
4.0 0.546 0.454
3.917
4.075
3.235
4,440
4.427
0.916
5.0 0.546 0.454
3.911
4.081
3.209
4,444
4.475
0.916
6.0 0.546 0.454
3,904
4.070
3.199
4,443
4.523
0,916
7.0 0.546 0.454
3.905
4.070
3.201
4.442
4.475
0.916
8.0 0.546 0.454
3.912
4.075
3.220
4.440
4.427
0.916
9.0 0,546 0.454
3,919
4.077
3,239
4.441
4.475
0.916
10.0 0.546 0.454
3.911
4.081
3.207
4.444
4.475
0.916
CosyVoice (SFM-1)
trained on LibriTTS
1.0 0.088 0.152
3.567
3.501
2.957
4.242
14,932
0.917
1.5
0.132 0.228
3.872
3.932
3.267
4,417
6.102
0.928
2.0 0.176 0.304
4.000
4.089
3.392
4.520
4.882
0.924
2.5 0.219 0.380
4.005
4.078
3.395
4.543
4.834
0,911
3.0 0.263 0.456
3.877
3.940
3.213
4.477
4.355
0.896
4.0 0.349 0.605
3.395
3.498
2.599
4.087
4.355
0.868
5.0 0.364 0.635
3.251
3.375
2.421
3,955
4.379
0.865
6.0 0.364 0.636
3.260
3,376
2.441
3,963
4.331
0.865
7.0 0.364 0.636
3.261
3.380
2.450
3.953
4.355
0.864
8.0 0.364 0.636
3.271
3.379
2.470
3.963
4.403
0.865
9.0 0.364 0.636
3.271
3.377
2.478
3,958
4.427
0.865
10.0 0.364 0.636
3.255
3.375
2.433
3.955
4.379
0.865
30

---

Table 9: Objective evaluation results on validation sets for a selection (Cosy Voice-DiT)
i
og
PMOS
UTMOS1 UTMOSv21 Distill-MOST WER(%)) SIM
Cosy
Voice- DiT (SFM) trained on LibriTTS
1.0
0.143 0.120
3,405
3,077
2.880
4.257
8.423
0.924
2.0
0.286 0.239
3.750
3.569
3.189
4.493
4.499
0.934
2.5
0.358 0.299
3,823
3.694
3.247
4.529
4.331
0.933
3.0
0.429 0.359
3.810
3.720
3.184
4.528
4.212
0.929
3.5
0.501 0.419
3.780
3.758
3.089
4,493
4.068
0.925
4.0
0.545 0.455
3.721
3.750
2.968
4.444
4.092
0.922
5.0
0.545 0.455
3.722
3.750
2.972
4,444
4.092
0.922
6.0
0.545 0.455
3.720
3.749
2.967
4.444
4.092
0.922
7.0 0.545 0.455
3.723
3.751
2.973
4.445
4.092
0,922
8.0
0.545 0.455
3.718
3.750
2.959
4.444
4.092
0.922
9.0
0.545 0.455
3.724
3.750
2.979
4,444
4.092
0.922
10.0
0.545 0.455
3.719
3.750
2.964
4.444
4.092
0.922
Cosy Voice-DiT (SFM-c) trained on LibriTTS
1.0
0.106 0.082
3,419
3.154
2.943
4,161
4.666
0.933
2.0
0.212 0.164
3.655
3.408
3.146
4.409
4.379
0.937
3.0
0.317 0.246
3.766
3.573
3.217
4.509
4.307
0.936
3.5
0.370 0.287
3.799
3.640
3,230
4.526
4.283
0.934
4.0
0.423 0.329
3.815
3.696
3.227
4.522
4.283
0.933
4.5
0.476 0.370
3,814
3.739
3.180
4.521
4.331
0,931
5.0
0.529 0.411
3.794
3.789
3.080
4.513
4.164
0.929
6.0
0.563 0.437
3.780
3.795
3.056
4,488
4.116
0.927
7.0 0.563 0.437
3.767
3.794
3,021
4,488
4.140
0.927
8.0 0.563 0.437
3.770
3.795
3.029
4,488
4.140
0.927
9.0 0.563 0.437
3.767
3.792
3.019
4,488
4.116
0.927
10.0
0.563 0.437
3.776
3.794
3.047
4.488
4.164
0.927
E Minor analysis of evaluation results
Interestingly, the Matcha-TTS (SFM) model trained on VCTK achieves much lower WER than the ground-truth and vocoddr--ecoonstuutee speech. Since VCTK is reading-style corpus, we observe that speakers occasionally exhibit disfluencies such stammering slurring. The synthesized speech, being generated from clean transcripts, avoids such inconsistencies.
For models trained on LibriTTS, vocoder-reconstrucee speech performs significantly worse in SMOS tests. Since we adopt cross-sentence evaluations [8] 77. the prompt and target utterances are from the same speaker but are not necessarily adjacent, often resulting in prosodic mismatches. In contrast, CosyVoice LLM can imitate the prompt's prosody, and the generated speech tokens align well with the prompt. Ground truth still performs competitively, likely due to its superior speech quality.
Reconstructed speech also performs poorly CMOS tests. As LibriTTS is from audiobooks, which often feature expressive and emotional delivery. We observe that listeners tend to perceive such utterances unnatural when heard in isolation. While flow matching-basee models are capable of generating high-quality speech, prosody becomes the dominant factor in listener ratings. Ground-truth speech can outperform synthesized speech in CMOS due to its better sound quality.
31

---

RTF and NFE with using adaptive-step ODE solvers
In this section, we provide not only the RTFs for ODE solver inference in Table 10 but also the RTFs for the full model inference in Table [1 These comparisons reveal that the ODE solver inference dominates the overall inference time. When using adaptive-step ODE solvers, the inference acceleration of our SFM methods is significant.
Table 10: RTF and NFE results for adaptive-step ODE solvers (solver inference). RTF denotes the mean RTF Rate (%) denotes the relative speedup in terms of RTF compared to the ablated model.
System
(2)
Fehlberg(2)
Bosh(3)
Dopri(5)
RTFL
Rate?
NFE, RTF/
Ratet
NFE]
RTF,
Ratef NFE,
RTF,
Rate
NFE
Maicha-TTL trained on LJ Speech
Baseline
0.407
-1.496 312.36 0.054
3.571
44.82
0.271
-0.370
223.95
0.142
2.069
120.10
Ablated
0.401
0.000
306.72
0.056
0.000
45.28
0.270
0.000
221.81
0.145
0.000
121.46
SFM (a=1.0) 0.361 9.858
277.56
0.053
4.171
44.08
0.225
16.924
186.23
0.132
9.100
111.14
SFM (a=2.0) 0.299
25.541 229.43
0.048
13.306
39.16
0.187
30.931
153.08
0.115
20.484
96.02
SFM (a=3.0) 0.249 37.931
191.49
0.043
22.749
34.88
0.157
41.895
129.02
0.100
30.753
84.20
SFM (o=4.0) 0.207 48.486
156.78
0.037
34.246
29.74
0.131
51.576
107.90
0.086
40.559
72.56
SFM (a=5.0) 0.157 60.825 120.07
0.027
50.968
22.14
0.110
59.266
90.74
0.076
47.605
63.74
Matcha- trained VCTK
Baseline
0.972 -2.101 318.42
0.139
-6.107
49.00
0.681
1.304
243.40
0.365
1.617
133.92
Ablated
0,952 0.000 313.18
0.131
0.000
45.52
0,690
0.000
245.69
0.371
0.000
134.76
SFM (o=1.0) 0.950
0.210
312.26
0.123
6.269
42.60
0.690
0.030
244.43
0.371
0.111
134.00
SFM (a=2.0) 0.771 18.998
253.66
0.111
15.463
38.28
0.558
19.071
196.58
0.318
14.356
114.08
SFM (a=3.0) 0.626
34.271
205.49
0.100
23.635
34.50
0.436
36.844
153.86
0.265
28.626
95.48
SFM (a=4.0) 0.519 45.449
170.63
0.093
29.503
31.96
0.350
49.257
123.80
0.231
37.846
83.48
SFM (a=5.0) 0.407 57.242 133.78
0.073
44.413
25.10
0.279
59.547
99.05
0.195
47.454
70.28
StableTTS trained on VCTK
Ablated
2.153 0.000 712.06
0.394
0.000
133.96
1.270
0.000
434.74
0.998
0.000
346.12
SFM (a=1.0) 1.212 43.694 397.26
0.261
33.707
88.36
0.790
37.823
269.32
0.690
30.849
232.60
SFM (a=2.0) 0.982 54.386
316.00
0.232
41.301
76.48
0.697
45.123
23182
0.613 38.593
208.72
SFM (o=3.0) 0.789 63.346
254.74
0.205
48.081
67.44
0.611
51.930
204.64
0.564 43.473
190.60
SFM (a=4.0) 0.592 72.497 192.34
0.169
57.149
53.84
0.542
57.305
181.72
0,489
51.003
166.12
SFM (a=5.0) 0.586 72.792 189.30
0.173
56.096
53.64
0.538
57.674
178.90
0.505
49.377
169.48
Vosevoice Voice trained on LibriTTS
Baseline
2.416 4.581 390.06
0.358
2.981
58.62
1.894
12.920
327.25
1.294
7.373
215.06
Ablated
2.532
0.000 395.25
0.369
0.000
58.91
2.175
0.000
346.90
1.397
0.000
223.37
SFM (a=1.0)
1.745
31.093
275.92
0.283
23.449
45.68
1.454
33.143
232.67
1.061
24.069
170.57
SFM (a=2.0) 1.426 43.667
218.17
0.250
32.196
39.63
1,211
44.326
192.97
0.917
34.395
146.30
SFM (a=3.0)
1.110 56.182 173.28
0.219
40.682
34.85
1.001
53.952
158.57
0.787
43.679
126.53
SFM (o=4.0) 0.963 .67 149.95
0.186
49.642
29.98
0.887
59.217
142.15
0.720
48.469
116.09
SFM (a=5.0) 0.960 62.096 149.90
0.185
49.875
29.96
0.888
59.190
141.79
0.727
47.990
116.15
CosyVoice-DiT trained LibriTTS
Ablated
1.170 0.000 834.46
0,188 0.000
137.56
0.699
0.000
514.30
0.539
0.000
403.36
SFM (a=1.0) 0.621 46.934 421.90
0.125
33.704
86.84
0.396
43.324
277.72
0.337
37.591
234.70
SFM (a=2.0) 0.503 56.980 335.96
0.108
42.475
75.28
0.336
51.894
237.88
0.286
46.910
207.94
SFM (o=3.0) 0.391 66.558 267.38
0.092
51.292
63.82
0.295
57.811
207.58
0.257
52.252
184.24
SFM (a=4.0) 0.300 74.320 205.89
0.073
61.121
51.72
0.259
62.958
182.17
0.229
57.595
163.48
SFM (a=5.0) 0.302 74.212
205.67
0.074
60.825
51.74
0,258
63.061
182.23
0.229
57.523
163.06
32

---

Table 11: RTF results when using adaptive-step ODE solvers (model inference). RTF denotes the mean of RTF. Rate (%) denotes the relative speedup compared to the ablated model.
System
Heun(2)
Fehlberg(2)
Bosh(3)
Dopri(5)
RTF
Ratef RTF,
Ratef
RTF,
Ratet
RTF
Ratef
Matcha- TTS trained on IJ Speech
Baseline
0,409
-1,489 0.055
3.509
0.273 -0.368
0.143
2.055
Ablated
0,403
0.000 0.057
0.000
0.272
0.000
0.146
0.000
SFM (a=1.0) 0.363
9.809 0.055
4.017
0.226
16.814
0.133
8,976
SFM (a=2.0) 0.300
25.430 0,050
12.866
0.188
30.735
0.117
20.230
SFM (a=3.0) 0.251 37.773 0.045
22.063
0.159
41.639
0.102 30.399
SFM (a=4.0) 0,208 48.282 0.038
33.261
0.133
51.268
0.088
40.104
SFM (o=5.0) 0.159 60.578 0.029 49.541
0.112
58.914
0.077
47.076
Matcha- trained on VCTK
Baseline
0.975 -2.094 0.142
-5.185
0.684
1.441
0.369
1,600
Ablated
0,955
0.000 0.135
0.000
0.694
0.000
0.375
0.000
SFM (a=1.0) 0.953
0.211 0.127
6.111
0.693
0.036
0.374
0.116
SFM (a=2.0) 0.775
18,925
0.115
15.046
0.562
18.971
0.321
14.213
SFM (a=3.0) 0.629
34.141
0.104 23.000
0.439
36.651
0.268
28.349
SFM (a=4.0) 0.523
45.274 0.096
28.711
0.354
48.998
0.234
37.481
SFM (a=5.0) 0.411 57.021 0.077 43.209
0.283
59.235
0.199
46.993
StableTTS trained on VCTK
Ablated
2.157 0.000 0.398 0.000
1.274
0.000
1.001
0.000
SFM (a=1.0) 1,216 43.622 0.265 33.396
0.793
37.716
0.694
30.730
SFM (a=2.0) 0.986 54.294 0.235 40.912
0.701
44.992
0.616
38.447
SFM (a=3.0) 0.793 63.242 0.208 47.643
0.614
51.781
0.568
43.315
SFM (a=4.0) 0.596 72.380 0.172 56.640
0.546
57.146
0.492
50.821
SFM (a=5.0) 0.589 72.674 0.177
55.598
0.541
57.513
0.509
49.200
Cosy Voice trained on LibriTTS
Baseline
2.417
4.580 0.359 2.973
1.895
12.914
1.295
7.368
Ablated
2.533
0.000 0.370
0.000
2.176
0.000
1.398
0.000
SFM (a=1.0) 1.746
31.076 0.284 23.353
1.455
33.124
1.062
24.035
SFM (a=2.0) 1.428
43.644 0.252 32.070
1.212
44.301
0.918
34.360
SFM (a=3.0)
1.111
56.155 0.220 40.536
1.003
53.922
0.788
43.638
SFM (a=4.0) 0.964 61.937 0.187 49.471
0.888
59.184
0.721
48.424
SFM (a=5.0) 0.961
62.066 0.186
49.705
0.889
59.157
0.728
47.946
Cosy Voice-DiT trained LibriTTS
Ablated
1.171 0.000 0.190 0.000
0.700
0.000
0.541
0.000
SFM (a=1.0) 0.622 46.870 0.126 33.406
0.397
43.221
0.338
37.472
SFM (a=2.0) 0.505 56.903 0.110 42.113
0.338
51.779
0.288
46.775
SFM (a=3.0) 0.393 66.472 0.093 50.872 0.296 57.683
0.259
52.102
SFM (a=4.0) 0.302 74.225 0.075
60.633
0.260
62.820
0.230
57.432
SFM (a=5.0) 0.303 74.116
0.075
60.336
0.260
62.924
0.230
57.360
33


Reviews:
[{"review_id": "8wuIC3W66J", "reviewer": "Reviewer_d41r", "summary": "This paper proposes a Shallow Flow Matching (SFM) mechanism to enhance flow matching (FM)-based Text-to-Speech (TTS) models that operate within a coarse-to-fine generation paradigm. The core idea is to leverage the coarse representations produced by a weak generator to construct an intermediate starting point for the FM process, rather than starting from pure noise. The authors demonstrate that integrating SFM into several TTS models improves both the sampling quality and inference speed.", "questions": "The main issues I would like to see addressed are already listed in the weaknesses section above.", "limitations": "yes", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 1, "originality": 2, "strengths_and_weaknesses": "Strengths:\n* The paper provides a theoretical validation for the proposed SFM method, with theorems and proofs.\n* The method demonstrates improved performance compared to its baseline counterparts in subjective evaluations of naturalness and similarity.\n* The significant improvement in inference speed when using the proposed method is thoroughly analyzed.\n\nWeaknesses:\n* The experimental results on small datasets like LJSpeech and VCTK do not adequately justify the method's scalability. While LibriTTS is a comparatively larger set, the experiments do not convincingly demonstrate how this approach would perform on the massive, diverse datasets (10k+ hours) used by many state-of-the-art zero-shot TTS models (e.g., VALL-E, Voicebox, MELLE, DiTTo-TTS, E2-TTS, F5-TTS, and CosyVoice 2).\n* The method has limited applicability in its current form. While the authors show its applicability to CosyVoice, a two-stage coarse-to-fine generative model, it is unclear if or how SFM could be applied to other prominent TTS methods, such as single-stage large diffusion/flow-matching models (e.g., DiTTo-TTS, F5-TTS) or discrete-token generative models (e.g., Llasa, MaskGCT).\n* The paper does not compare the proposed acceleration technique to other well-known methods for speeding up generative model inference, such as consistency models. This makes it difficult to assess the significance of the speed improvements in the broader context of fast sampling techniques.\n \n[1] Wang, Yuancheng, et al. \"MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformer.\" The Thirteenth International Conference on Learning Representations.", "was_revised": true, "final_justification": "I have raised my score from 2 to 3 based on the authors' rebuttal.\n\n* The authors' rebuttal clarified the technical novelty of their method compared to other acceleration techniques and justified their experimental scope.\n    * large-scale experiments are not necessary, but it would certainly help assess the proposed method's applicability to recent TTS systems that can follow highly diverse speaker prompts if existed.\n* The primary weakness is the work's limited applicability. Its contribution is confined to a specific class of TTS models.", "rebuttal": "Thank you for your valuable comments. We address your concerns below:\n\n> **Q1:** The experimental results on small datasets like LJSpeech and VCTK do not adequately justify the method's scalability. While LibriTTS is a comparatively larger set, the experiments do not convincingly demonstrate how this approach would perform on the massive, diverse datasets (10k+ hours) used by many state-of-the-art zero-shot TTS models (e.g., VALL-E, Voicebox, MELLE, DiTTo-TTS, E2-TTS, F5-TTS, and CosyVoice 2).\n\n**A1:** We aim to evaluate the general effectiveness of the proposed method on various models rather than develop a new SOTA zero-shot TTS model. To this end, we adopt three widely used datasets that also serve as standard benchmarks in TTS research [12, 13, 15, 17-20, 27, 30, 33, 35, 39]: LJSpeech, VCTK, and LibriTTS, which span both single- and multi-speaker settings. LibriTTS, as you mentioned, is a comparatively larger dataset. \n\nConducting experiments on super massive-scale corpora (10k+ hours) is typically done when the goal is to develop an SOTA zero-shot TTS model, which not only deviates from the intended scope of our work but is also far beyond our available computational resources. We believe our current setup provides a strong and fair validation of the SFM method.\n\n> **Q2:** The method has limited applicability in its current form. While the authors show its applicability to CosyVoice, a two-stage coarse-to-fine generative model, it is unclear if or how SFM could be applied to other prominent TTS methods, such as single-stage large diffusion/flow-matching models (e.g., DiTTo-TTS, F5-TTS) or discrete-token generative models (e.g., Llasa, MaskGCT).\n\n**A2:** \n1. As indicated in the title and throughout the paper, the SFM method is designed for models that combine flow matching (FM) with a coarse-to-fine generation structure. Therefore, our method is not designed for the following situations: \\\n    &nbsp;&nbsp;&nbsp;&nbsp; a. single-stage architectures such as DiTTo-TTS and F5-TTS, which do not produce coarse representations; \\\n    &nbsp;&nbsp;&nbsp;&nbsp; b. discrete-token generative models such as Llasa or MaskGCT, which operate entirely in discrete latent spaces and do not employ FM at all. \\\nSingle-stage models often lack strong contextual modeling and functional versatility, while discrete-token-only models tend to underperform in generation quality compared to those using continuous representations. Due to these limitations, many generative models have begun to use FM within coarse-to-fine multi-stage frameworks. Our proposed SFM method is motivated by this emerging trend, aiming to further enhance this generation paradigm. \n2. As researchers in the TTS field, we develop the SFM method and validate it with TTS models. Since FM has just shown strong potential in TTS, the FM-based **open-source** TTS models are limited, among them: \\\n    &nbsp;&nbsp;&nbsp;&nbsp; a. F5-TTS [7], as mentioned above, adopts DiT blocks as its backbone and is in a single-stage architecture; \\\n    &nbsp;&nbsp;&nbsp;&nbsp; b. VoiceFlow [15] adopts coarse-to-fine generation but has almost the same architecture as Matcha-TTS [30]. \\\nAs of the submission deadline, no other publicly available FM-based TTS models were found. To broaden coverage in terms of input modalities and backbone architectures, we further adapted CosyVoice by replacing its FM module with DiT blocks (**F5-TTS has adopted**), resulting in the CosyVoice-DiT variant introduced in our paper.\nTherefore, within the coarse-to-fine FM-based paradigm, our method is general and applicable to multiple input types, backbone architectures (as mentioned by Reviewer Dwas), and coarse-to-fine paradigms, as summarized (NAR: non-autoregressive, AR: autoregressive):\n\n| Model name | Input | FM backbone | Coarse-to-fine paradigm |\n|------|------|------|------|\n| Matcha-TTS | Phoneme token | U-Net | NAR + FM |\n| StableTTS | Phoneme token | DiT | NAR + FM |\n| CosyVoice | Speech token | U-Net | AR + FM |\n| CosyVoice-DiT | Speech token | DiT | AR + FM |\n\n> **Q3:** The paper does not compare the proposed acceleration technique to other well-known methods for speeding up generative model inference, such as consistency models. This makes it difficult to assess the significance of the speed improvements in the broader context of fast sampling techniques.\n\n**A3:** The acceleration brought by SFM is **fundamentally different** from that of consistency models:\n1. SFM leads to acceleration only when adaptive-step ODE solvers are used. The acceleration arises from skipping the early part of the FM path, which results in smoother inference samples and fewer sampling steps. It is important to note that the generation quality is improved.\n2. Consistency models [a], and their recent evolutions, shortcut models [b] and MeanFlow [c], work by adding regularization terms to the training objective, implicitly performing self-distillation. These methods enable a few or only one sampling step during inference, but typically come at the cost of reduced generation quality.\n\nTherefore, the above acceleration algorithms and SFM are not mutually exclusive and cannot be compared with each other. In contrast, we can directly use them at the same time for further efficiency gains. Since SFM skips the early part of the FM paths, the latter part is the standard FM process and is fully compatible with other acceleration methods on FM. We will further clarify this relationship in the \"Related Works\" section.\n\n[a] Y. Song et al., \"Consistency Models\", ICML, 2023. \\\n[b] K. Frans et al., \"One Step Diffusion via Shortcut Models\", ICLR, 2025. \\\n[c] Z. Geng et al., \"Mean Flows for One-step Generative Modeling\", arXiv:2505.13447, 2025. \n\nWe appreciate your feedback and hope these clarifications help position our method more clearly within the TTS literature."}, {"review_id": "c1TNE3AETg", "reviewer": "Reviewer_i9sS", "summary": "This paper proposes Shallow Flow Matching (SFM), a mechanism designed to improve flow matching (FM)-based text-to-speech (TTS) models within a coarse-to-fine generation paradigm. SFM constructs intermediate states along the FM paths using coarse output representations. During training, it employs an orthogonal projection method to adaptively determine the temporal position of these states and applies a principled construction strategy based on a single-segment piecewise flow. During inference, SFM starts from this intermediate state instead of pure noise, focusing computation on the later stages of the FM paths. The authors integrate SFM into multiple TTS models.", "questions": "NA", "limitations": "Yes", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\nThe motivation is clear. The problem that to speed up the FM-basd TTS models is worth to be explored.\n\nWeaknesses:\n1. As the authors mentioned it, the idea of shallow diffusion is proposed by DiffSinger. The unique challenge of introduce such idea into FM model has not be explained clearly.\n2. The experimental settings is simple. Only three small-scale English dataset (LJSpeech, VCTK, and LibriTTS) are included. The authors could also introduce the single-stage FM model such as E2TTS or F5TTS as the base models.\n3. Some results of Table 3 displayed that using SFM could lead to the negative effect for WER.\n4. The presentation of the efficency benefits after using SFM is not clear. There should be also the RTF results in Table 3 in addition to Table 4.", "was_revised": true, "final_justification": "My concerns regarding Q4 have been addressed, and Q1 (the novelty issue) has been partially addressed. However, I still believe that the other concerns (Q2.1, Q2.2, and Q3) remain unaddressed. Specifically:\n\n1. **About the potential negative effect of the proposed method (Q3), and a lack of larger-scale experiments (Q2.1)**: Given the potential negative impact on WER, as demonstrated in the experimental results (Q3), a larger-scale experiment becomes especially crucial for validation (Q2.1).\n\n2. **The proposed SFM could not be applied to single-stage FM models such as E2TTS and F5-TTS (Q2.2)**: I do not fully understand the reasons why E2-TTS and F5-TTS, two well-established flow-matching-based TTS models, cannot apply the SFM idea. By the way, I do not think the DiT or U-Net is the defining feature of these models (but the author claimed this during the response both to me and to the reviewer d41r). The true uniqueness lies in the single-stage (text to mel-spec directly) approach, rather than the two-stage (text-to-semantic + semantic-to-acoustic) approach. If by \"coarse representations\" the authors refer to the duration-aligned text representations (i.e., the frame-level linguistic representations), then E2-TTS and F5-TTS can also be seen as frame-level linguistic representations, albeit with many padding frames concatenated.", "rebuttal": "Thank you for your helpful comments. We address your concerns below:\n\n> **Q1:** As the authors mentioned it, the idea of shallow diffusion is proposed by DiffSinger. The unique challenge of introduce such idea into FM model has not be explained clearly.\n\n**A1:** While our SFM method is inspired by the idea of shallow diffusion (SD), the mechanisms are fundamentally different due to the differences between diffusion and flow matching (FM) processes:\n1. **Timestep prediction:** Both SD and SFM need to find the timestep $t_\\boldsymbol{h}$ at which the coarse output corresponds to the diffusion or FM trajectory. SD proposes a boundary predictor trained with a cross-entropy loss to classify whether a mel-spectrogram at any timestep $t$ comes from the coarse output or the diffusion process, and then identify the $t_\\boldsymbol{h}$ (called $k$ in [27]) based on the most confident timesteps. In contrast, SFM directly projects the coarse output onto the conditional optimal transport (CondOT) paths using orthogonal projection, yielding $t_\\boldsymbol{h}$ without classification tasks.\n2. **Intermediate state construction:** In SD, once $t_\\boldsymbol{h}$ is identified, the coarse output can be inserted into the diffusion process at $t_\\boldsymbol{h}$ as the intermediate state. In SFM, however, we must consider the intrinsic noise $\\sigma_\\boldsymbol{h}$ of the coarse output and ensure that the constructed intermediate state lies strictly on the CondOT paths. To achieve this, we propose a principled formulation supported by Theorem 1.\n3. **Single-segment training and inference:** SD can directly skip the initial diffusion steps during training and inference. In SFM, to enable this behavior while maintaining theoretical validity under FM, we need to derive a piecewise formulation as shown in Theorem 2.\n4. **SFM strength:** Owing to the linearity of CondOT paths, SFM allows us to scale the coarse output during inference to strengthen its guidance, leading to improved generation quality. Whether a similar strategy can be applied to SD remains an open question.\n\n**In summary**, while the core idea of starting from a shallow timestep is shared, SFM is fundamentally constructed based on the CondOT paths. The challenges are unique to FM mechanisms and are addressed explicitly in our paper.\n\n> **Q2.1:** The experimental settings is simple. Only three small-scale English dataset (LJSpeech, VCTK, and LibriTTS) are included.\n\n**A2.1:** We adopt three widely used datasets that also serve as standard benchmarks in TTS research [12, 13, 15, 17-20, 27, 30, 33, 35, 39]: LJSpeech, VCTK, and LibriTTS, which span both single- and multi-speaker settings. LibriTTS, in particular, is a comparatively larger dataset with over 500 hours of speech. While we agree that super-large datasets would be valuable, we believe our current setup provides a firm and fair validation of the SFM method:\n1. Our goal is to evaluate the generality and efficiency of SFM, not to propose a new zero-shot TTS system;\n2. Conducting experiments on super-large datasets is beyond our available computational resources.\n\n> **Q2.2:** The authors could also introduce the single-stage FM model such as E2TTS or F5TTS as the base models.\n\n**A2.2:** \n\n1. As indicated in the title and throughout the paper, our method is designed for the models that combine FM with a coarse-to-fine generation structure. This modeling paradigm is becoming increasingly popular, as it combines the strength of FM in high-quality generation with the contextual learning ability and functional versatility enabled by the coarse stage.\n\n2. Single-stage FM models, such as E2-TTS or F5-TTS, do not produce coarse representations. Therefore, SFM cannot be applied to them, because SFM needs to use the coarse output to construct an intermediate state along the FM paths.\n\n3. In our experimental settings, we have tried to utilize the limited open-source FM-based TTS models to broaden coverage in terms of input modalities and backbone architectures. We further adapted CosyVoice by replacing its FM module with DiT blocks (**F5-TTS has adopted**), resulting in the CosyVoice-DiT variant introduced in our paper.\nTherefore, within the coarse-to-fine FM-based paradigm, our method is general and applicable to multiple input types, backbone architectures (as mentioned by Reviewer Dwas), and coarse-to-fine paradigms, as summarized (NAR: non-autoregressive, AR: autoregressive):\n\n| Model name | Input | FM backbone | Coarse-to-fine paradigm |\n|------|------|------|------|\n| Matcha-TTS | Phoneme token | U-Net | NAR + FM |\n| StableTTS | Phoneme token | DiT | NAR + FM |\n| CosyVoice | Speech token | U-Net | AR + FM |\n| CosyVoice-DiT | Speech token | DiT | AR + FM |\n\n> **Q3:** Some results of Table 3 displayed that using SFM could lead to the negative effect for WER.\n\n**A3:** As we discussed in Section 5.1, the effect of SFM on WER is sometimes positive, sometimes negative. We believe this is because, in SFM, the coarse output is used to construct intermediate states on FM paths, rather than being used directly as a condition of FM. This shift may introduce some instability in word-speech alignment. This suggests that there is still room for improving alignment quality within the current framework.\n\nHowever, the improvements brought by SFM in both pseudo-MOS and subjective evaluations are consistent and substantial across all models, indicating strong perceptual benefits despite the observed WER variability.\n\n> **Q4:** The presentation of the efficency benefits after using SFM is not clear. There should be also the RTF results in Table 3 in addition to Table 4.\n\n**A4:** We appreciate the suggestion to include RTF results in Table 3. However, we would like to clarify the following points:\n1. The acceleration effect of SFM only works when using adaptive-step ODE solvers. As shown in Table 1, we follow the original configurations of the models, some of which use fixed-step solvers (Euler). Therefore, including RTF values in Table 3 would result in many empty and meaningless entries.\n2. Since acceleration is not the primary focus of this work and due to page limitations, Table 4 presents partial RTF results. As stated in its caption, **the complete results** are provided in Table 10 of Appendix F.\n3. In Table 10, we detail the speed-up achieved by SFM under various adaptive-step solvers and show that the acceleration consistently increases with larger values of $\\alpha$, which the table is specifically designed to highlight.\n\nWe appreciate your comments and hope these clarifications help convey the motivation, novelty, and effectiveness of our work more clearly."}, {"review_id": "TRWhk5jDpU", "reviewer": "Reviewer_esVr", "summary": "The authors propose a Shallow Flow Matching (SFM) algorithm that can inference from the intermediate state rather than pure noise and reduce computation cost and time for the Flow matching generation.  An orthogonal projection method to adaptively determine the temporal position of the intermediate states, and a single-segment piecewise flow is proposed to generate remaining states. Experiments show that SFM consistently improves the naturalness of synthesized speech in both objective and subjective evaluations. It is a solid work.", "questions": "1 In Table 3, the proposed systems are outperform than the baseline systems, which is derived from pure noise. Can we state the generation outputs are better if they were generated the intermediate states instead of noise states? Any explanation?\n\n2. Table 4 and 10 compare the RTFs for different ODE solvers under different $\\alpha$. They only compared with ablated systems, how about the comparison with the baselines mentioned?", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n- An improved flow matching algorithm is proposed to reduce the computation.\n- A proof is provided for the proposed algorithm\n- Good results are achieved compared with multiple baselines\n\nWeaknesses:\n- The computation cost baseline only include the ablated systems it would be helpful if the baselines results are included too.", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's insightful and constructive comments. We provide our responses below.\n\n> **Q1:** In Table 3, the proposed systems are outperform than the baseline systems, which is derived from pure noise. Can we state the generation outputs are better if they were generated the intermediate states instead of noise states? Any explanation?\n\n**A1:** Based on our experiments to date, we believe this statement is valid. Generation from pure noise, as mentioned in our paper, suffers from \"a suboptimal allocation of modeling capacity\". However, in our preliminary experiments, we observed that the following naive approaches failed to improve and even degraded generation quality:\n1. Simply adding noise to the coarse output as the initial or intermediate states;\n2. Applying linear scaling to the coarse output and adding noise to obtain intermediate states.  \n\nThese results led us to realize that constructing effective intermediate states requires a nonlinear transformation of the coarse output, and must strictly follow the CondOT paths, as established by Theorems 1 and 2 in our paper. We hypothesize that this is due to the inherent mismatch between the training objectives of the coarse output and the flow matching process. Further improvements may be achievable by better balancing the associated losses during training, which we leave for future work.\n\n> **Q2:** Table 4 and 10 compare the RTFs for different ODE solvers under different $\\alpha$. They only compared with ablated systems, how about the comparison with the baselines mentioned?\n\n**A2:** Thank you for your suggestion, which indeed improves the completeness of the tables. We have added the results of the three baseline models to Table 10, as shown below. The same as the original Table 10, \"RTF\" denotes the average real-time factor, and \"Rate\" (\\%) indicates the relative speedup compared to the ablated model. We omit the std for clarity, as all std values are below 0.013.\n\nFrom the results, we observe the following:\n1. The RTF differences between the baseline and ablated models are generally small.  \n2. For the CosyVoice trained on LibriTTS setting, the baseline model consistently shows slightly lower RTF values than the ablated model, suggesting that our modifications to this ablated model (also to the SFM model) slightly degrade the smoothness of the outputs during inference. To minimize architectural modifications, we adopted the simplest feature fusion strategy, as discussed in the CosyVoice part of Section 4.2. More sophisticated alternatives could potentially yield further improvements.\n\n### Matcha-TTS trained on LJ Speech\n\n| System | Heun(2) RTF↓ | Heun(2) Rate↑ | Fehlberg(2) RTF↓ | Fehlberg(2) Rate↑ | Bosh(3) RTF↓ | Bosh(3) Rate↑ | Dopri(5) RTF↓ | Dopri(5) Rate↑ |\n|----------|----------|----------|----------|----------|----------|----------|----------|----------|\n| Baseline | 0.407 | -1.496 | 0.054 | 3.571 | 0.271 | -0.370 | 0.142 | 2.069 |\n| Ablated  | 0.401 | 0.000 | 0.056 | 0.000 | 0.270 | 0.000 | 0.145 | 0.000 |\n| SFM ($\\alpha$=1.0) |        0.361 |         9.858 |            0.053 |               4.171 |      0.225 |        16.924 |         0.132 |           9.100   |\n| SFM ($\\alpha$=2.0) |        0.299 |        25.541 |            0.048 |              13.306 |      0.187 |        30.931 |         0.115 |          20.484 |\n| SFM ($\\alpha$=3.0) |        0.249 |        37.931 |            0.043 |              22.749 |      0.157 |        41.895 |         0.100 |          30.753 |\n| SFM ($\\alpha$=4.0) |        0.207 |        48.486 |            0.037 |              34.246 |      0.131 |        51.576 |         0.086 |          40.559 |\n| SFM ($\\alpha$=5.0) |        0.157 |        60.825 |            0.027 |              50.968 |      0.110 |        59.266 |         0.076 |          47.605 |\n\n### Matcha-TTS trained on VCTK\n\n| System | Heun(2) RTF↓ | Heun(2) Rate↑ | Fehlberg(2) RTF↓ | Fehlberg(2) Rate↑ | Bosh(3) RTF↓ | Bosh(3) Rate↑ | Dopri(5) RTF↓ | Dopri(5) Rate↑ |\n|----------|----------|----------|----------|----------|----------|----------|----------|----------|\n| Baseline | 0.972 | -2.101 | 0.139 | -6.107 | 0.681 | 1.304 | 0.365 | 1.617 |\n| Ablated | 0.952 | 0.000 | 0.131 | 0.000 | 0.690 | 0.000 | 0.371 | 0.000 |\n| SFM ($\\alpha$=1.0) | 0.950        | 0.210         | 0.123            | 6.269             | 0.690        | 0.030         | 0.371          | 0.111           |\n| SFM ($\\alpha$=2.0) | 0.771        | 18.998        | 0.111            | 15.463            | 0.558        | 19.071        | 0.318          | 14.356          |\n| SFM ($\\alpha$=3.0) | 0.626        | 34.271        | 0.100            | 23.635            | 0.436        | 36.844        | 0.265          | 28.626          |\n| SFM ($\\alpha$=4.0) | 0.519        | 45.449        | 0.093            | 29.503            | 0.350        | 49.257        | 0.231          | 37.846          |\n| SFM ($\\alpha$=5.0) | 0.407        | 57.242        | 0.073            | 44.413            | 0.279        | 59.547        | 0.195          | 47.454          |\n\n### CosyVoice trained on LibriTTS\n\n| System | Heun(2) RTF↓ | Heun(2) Rate↑ | Fehlberg(2) RTF↓ | Fehlberg(2) Rate↑ | Bosh(3) RTF↓ | Bosh(3) Rate↑ | Dopri(5) RTF↓ | Dopri(5) Rate↑ |\n|----------|----------|----------|----------|----------|----------|----------|----------|----------|\n| Baseline | 2.416 | 4.581 | 0.358 | 2.981 | 1.894 | 12.920 | 1.294 | 7.373 |\n| Ablated | 2.532 | 0.000 | 0.369 | 0.000 | 2.175 | 0.000 | 1.397 | 0.000 |\n| SFM ($\\alpha$=1.0) | 1.745        | 31.093        | 0.283            | 23.449            | 1.454        | 33.143        | 1.061          | 24.069          |\n| SFM ($\\alpha$=2.0) | 1.426        | 43.667        | 0.250            | 32.196            | 1.211        | 44.326        | 0.917          | 34.395          |\n| SFM ($\\alpha$=3.0) | 1.110        | 56.182        | 0.219            | 40.682            | 1.001        | 53.952        | 0.787          | 43.679          |\n| SFM ($\\alpha$=4.0) | 0.963        | 61.967        | 0.186            | 49.642            | 0.887        | 59.217        | 0.720          | 48.469          |\n| SFM ($\\alpha$=5.0) | 0.960        | 62.096        | 0.185            | 49.875            | 0.888        | 59.190        | 0.727          | 47.990          |"}, {"review_id": "SQ3yHXNGVT", "reviewer": "Reviewer_Dwas", "summary": "In this work, the authors present a novel Shallow Flow Matching (SFM) mechanism inspired by DiffSinger's [27] Shallow Diffusion Mechanism. SFM constructs intermediate paths along the Flow Matching's (FM) path, using coarse output representations i.e, it introduces a way of starting the ODE solver from the intermediate state in the flow path instead of Gaussian noise $N(0, I)$ as done by previous SOTA methods. Using the principle of shallow diffusion mechanism, they introduce an adaptive timestep $t_h$ which can be used as a pre-informed point to initiate solving the vector field ODE. During training, authors propose to use an orthogonal projection (by adding an SFM head to the encoder) method to predict the temporal parameter and use it to estimate the segment's starting point (coarse output). During inference, this coarse output is used as the initial point instead of isotropic Gaussian noise. Evaluations confirm that SFM improves both subjective, and objective evaluations and accelerates inference when using an **adaptive ODE solver**.", "questions": "* What are $\\hat{t_g}$ and $\\hat{\\sigma_g}$ in Table 2? it is not concretely defined in the paper; are they already scaled? \n* Practically, most of these architectures (at least all three of Matcha-TTS, Stable-TTS, and CosyVoice {which uses Matcha-TTS's U-Net}), in the architecture of their flow prediction network, concatenate the noise and coarse spectrogram ($\\mu$). This essentially means the initial point is sampling from Gaussian noise centered around  $N(\\mu, I)$. How is this different from such modifications? Is it that SFM also predicts a $\\sigma$ so the noise is scaled at the coarse spectrogram?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths\n- Evaluating the proposed method on three different SOTA architectures, mainly consisting of UNets, and DiT, demonstrates the effectiveness of SFM and decouples the advantage of proposed changes with previous design choices.\n- Novel and principled design improvement, by formalising piecewise-single segment flow and then evaluating it in practice.This grounds the advantage of using SFM especially when using adaptive ODE solver, showing that model can take longer steps confidently when solving the ODE as error is smaller than using just OT-CFM.\n\n### Weakness\n* While in theory method improves the solution of ODE when using an adaptive solver. In practice none of the SOTA models use an adaptive solver. Most use either order 1 or order 2 solvers and perceptually it makes little to no difference.\n* Because of an additional hyperparameter $\\alpha$ which is tuned for each architecture each dataset, the method adds an additional overhead to optimise this parameter.\n* It would have been better if the authors could quantify Number of Function Evaluation (NFE), a commonly used evaluation term in diffusion/flow matching literature.", "was_revised": true, "final_justification": "The authors addressed my concerns; while the paper is grounded well theoretically, it holds less importance in practicality.", "rebuttal": "We appreciate the reviewer's time and valuable feedback. It is our great honor to have the work carefully reviewed and considered. We provide our responses below.\n\n> **Q1:** What are $\\hat{t_\\boldsymbol{g}}$ and $\\hat{\\sigma_\\boldsymbol{g}}$ in Table 2? it is not concretely defined in the paper; are they already scaled?\n\n**A1:** Thank you for pointing this out. This was a notational inconsistency in our submission. The $\\hat{t_\\boldsymbol{g}}$ and $\\hat{\\sigma_\\boldsymbol{g}}$ in Table 2 correspond to $\\tilde{t_{\\boldsymbol{g}}}$ and $\\tilde{\\sigma_{\\boldsymbol{g}}}$ defined in Equation (19). We will correct this notation and clarify their definitions in the final version to avoid confusion. And yes, they are already scaled.\n\n> **Q2:** Practically, most of these architectures (at least all three of Matcha-TTS, Stable-TTS, and CosyVoice {which uses Matcha-TTS's U-Net}), in the architecture of their flow prediction network, concatenate the noise and coarse spectrogram ($\\mu$). This essentially means the initial point is sampling from Gaussian noise centered around $N(\\mu, I)$. How is this different from such modifications? Is it that SFM also predicts a $\\sigma$ so the noise is scaled at the coarse spectrogram?\n\n**A2:** Based on our experimental observations to date, we believe an answer to this question is that directly using the coarse output as an FM condition is less effective than using it to construct intermediate states. When constructing the intermediate states in the SFM method, we found that achieving better generation quality requires:  \n1. applying a nonlinear transformation to the coarse output;\n2. strictly following the CondOT paths, as established by Theorems 1 and 2.\n\nBesides, as discussed on SFM-c in Section 5.1, using coarse output as an FM condition in SFM models either disrupts the SFM process during training or leads to degraded generation quality.\n\nTherefore, we hypothesize that directly conditioning on the coarse output may propagate its inherent errors and negatively affect the final output. In contrast, using it to construct intermediate states mitigates this issue by more effectively incorporating it into the FM process. A thorough validation of this hypothesis requires more dedicated experimental design and is beyond the scope of this paper.\n\n> **Q3 (Weakness 3):** It would have been better if the authors could quantify Number of Function Evaluation (NFE), a commonly used evaluation term in diffusion/flow matching literature.\n\n(We fully agree with the first two concerns raised in the weaknesses. These points are current limitations of our method and will serve as important directions for future improvement.)\n\n**A3:** We sincerely appreciate your recommendation and find the NFE to be a valuable metric, which has not been widely adopted in TTS research. We report the average number of calls to the flow module as NFE, and present the corresponding results in the table below. These values directly align with those in Table 10, which we omit here for clarity.\n\nWe observe that NFE is more stable than RTF across different settings. However, since the actual computation time of each flow module call varies depending on the input sequence length, reporting both NFE and RTF in the paper offers more complete information.\n\n\n### Matcha-TTS trained on LJ Speech\n\n| System | Heun(2) NFE↓ | Fehlberg(2) NFE↓ | Bosh(3) NFE↓ | Dopri(5) NFE↓ |\n|----------|----------|----------|----------|----------|\n| Ablated | 306.72 | 45.28 | 221.81 | 121.46 | \n| SFM ($\\alpha$=1.0) | 277.56 | 44.08 | 186.23 | 111.14 |\n| SFM ($\\alpha$=2.0) | 229.43 | 39.16 | 153.08 | 96.02 |\n| SFM ($\\alpha$=3.0) | 191.49 | 34.88 | 129.02 | 84.20 |\n| SFM ($\\alpha$=4.0) | 156.78 | 29.74 | 107.90 | 72.56 |\n| SFM ($\\alpha$=5.0) | 120.07 | 22.14 | 90.74 | 63.74 |\n\n### Matcha-TTS trained on VCTK\n\n| System | Heun(2) NFE↓ | Fehlberg(2) NFE↓ | Bosh(3) NFE↓ | Dopri(5) NFE↓ |\n|----------|----------|----------|----------|----------|\n| Ablated | 313.18 | 45.52 | 245.69 | 134.76 |\n| SFM ($\\alpha$=1.0) | 312.26 | 42.60 | 244.43 | 134.00 |\n| SFM ($\\alpha$=2.0) | 253.66 | 38.28 | 196.58 | 114.08 |\n| SFM ($\\alpha$=3.0) | 205.49 | 34.50 | 153.86 | 95.48 |\n| SFM ($\\alpha$=4.0) | 170.63 | 31.96 | 123.80 | 83.48 |\n| SFM ($\\alpha$=5.0) | 133.78 | 25.10 | 99.05 | 70.28 |\n\n### StableTTS trained on VCTK\n\n| System | Heun(2) NFE↓ | Fehlberg(2) NFE↓ | Bosh(3) NFE↓ | Dopri(5) NFE↓ |\n|----------|----------|----------|----------|----------|\n| Ablated | 712.06 | 133.96 | 434.74 | 346.12 |\n| SFM ($\\alpha$=1.0) | 397.26 | 88.36 | 269.32 | 232.6 |\n| SFM ($\\alpha$=2.0) | 316.0 | 76.48 | 231.82 | 208.72 |\n| SFM ($\\alpha$=3.0) | 254.74 | 67.44 | 204.64 | 190.6 |\n| SFM ($\\alpha$=4.0) | 192.34 | 53.84 | 181.72 | 166.12 |\n| SFM ($\\alpha$=5.0) | 189.3 | 53.64 | 178.9 | 169.48 |\n\n### CosyVoice trained on LibriTTS\n\n| System | Heun(2) NFE↓ | Fehlberg(2) NFE↓ | Bosh(3) NFE↓ | Dopri(5) NFE↓ |\n|----------|----------|----------|----------|----------|\n| Ablated | 395.25 | 58.91 | 346.90 | 223.37 |\n| SFM ($\\alpha$=1.0) | 275.92 | 45.68 | 232.67 | 170.57 |\n| SFM ($\\alpha$=2.0) | 218.17 | 39.63 | 192.97 | 146.30 |\n| SFM ($\\alpha$=3.0) | 173.28 | 34.85 | 158.57 | 126.53 |\n| SFM ($\\alpha$=4.0) | 149.95 | 29.98 | 142.15 | 116.09 |\n| SFM ($\\alpha$=5.0) | 149.90 | 29.96 | 141.79 | 116.15 |\n\n### CosyVoice-DiT trained on LibriTTS\n\n| System | Heun(2) NFE↓ | Fehlberg(2) NFE↓ | Bosh(3) NFE↓ | Dopri(5) NFE↓ |\n|----------|----------|----------|----------|----------|\n| Ablated | 834.46 | 137.56 | 514.30 | 403.36 |\n| SFM ($\\alpha$=1.0) | 421.90 | 86.84 | 277.72 | 234.70 |\n| SFM ($\\alpha$=2.0) | 335.96 | 75.28 | 237.88 | 207.94 |\n| SFM ($\\alpha$=3.0) | 267.38 | 63.82 | 207.58 | 184.24 |\n| SFM ($\\alpha$=4.0) | 205.89 | 51.72 | 182.17 | 163.48 |\n| SFM ($\\alpha$=5.0) | 205.67 | 51.74 | 182.23 | 163.06 |"}]

Author rebuttal:

Meta-review: This paper introduces Shallow Flow Matching (SFM), an extension of flow matching, where inference begins from intermediate states rather than pure noise. Evaluations across multiple TTS models show consistent gains in generation quality and efficiency, with additional experiments (e.g., NFE results) strengthening the empirical evidence. Two reviewers raised concerns about dataset scale and limited practical significance, but these do not undermine the core contribution. The authors argue that the datasets used are standard in TTS research. The authors have also explained the choice of baselines. These seem to be reasonable. Moreover, while FM is not yet the default in practice, showing the use of shallow FM here is interesting. Given the novelty, and improvements, I conclude that the strengths outweigh these concerns and recommend acceptance of the work.
Decision: Accept (poster)