StuPASE: Studio-Quality PASE

StuPASE is a state-of-the-art generative speech enhancement model trained to remove noise and reverberation while preserving linguistic content and speaker identity, and achieving studio-level perceptual quality. It operates on 16 kHz mono audio.

Model Details

Model Description

StuPASE contains three main components:

DeWavLM-R: Performs low-hallucination phonetic enhancement, fine‑tuned from DeWavLM using dry targets for improved dereverberation.
- Inputs: Noisy speech
- Outputs: Enhanced phonetic representations
CFM: Performs phonetic-guided acoustic enhancement.
- Inputs: Noisy Mel features + Enhanced phonetic representations
- Outputs: Enhanced Mel features
Mel Vocoder: Reconstructs enhanced wavforms.
- Inputs: Enhanced Mel features
- Outputs: Enhanced speech

Developed by: Copyright © 2026 by Cisco Systems, Inc. All rights reserved.
Cisco product group: Collaboration AI: Xiaobin Rong, Mansur Yesilbursa, Kamil Wojcicki
Model type: Generative Speech Enhancement
License: Apache 2.0
Finetuned from: WavLM-Large, DeWavLM

Model Sources

Repository: https://github.com/cisco-open/pase
Paper: https://arxiv.org/abs/2603.09234
Demo: https://xiaobin-rong.github.io/stupase_demo/

Uses

Direct Use

Enhance noisy or reverberant speech recordings
Improve perceptual quality and intelligibility
Preserve speaker identity and linguistic content
Supports 16 kHz mono audio

Out-of-Scope Use

Medical, legal, or safety‑critical decisions
Voice conversion or identity manipulation
Non‑speech audio enhancement

How to Get Started

Refer to the repository for quick-start code and examples:
https://github.com/cisco-open/pase

Training Details

Training Data

We release a StuPASE checkpoint that has been trained on an updated list of datasets. For this release, training used:

Clean speech:
- DNS5 Challenge clean-speech resources derived from the LibriVox public-domain subset
- LibriSpeech
- LibriTTS
- VCTK
Noise:
- DNS5 Challenge noise resources
Room impulse responses:
- OpenSLR26
- OpenSLR28

These source datasets were used to prepare training mixtures and train the released model. The model card and repository do not redistribute the underlying dataset contents; please refer to the original dataset pages and licenses below.

Dataset Attribution

DNS5 Challenge clean speech (LibriVox subset): clean-speech material prepared from LibriVox through the DNS Challenge. The LibriVox recordings used for this portion are public domain and were used as clean-speech training data for the released checkpoint.
LibriSpeech: LibriSpeech by Vassil Panayotov et al., licensed under CC BY 4.0. It was used as clean-speech training data for the released checkpoint.
LibriTTS: LibriTTS by Heiga Zen et al., licensed under CC BY 4.0. It was used as clean-speech training data for the released checkpoint.
VCTK Corpus: the VCTK dataset from the Centre for Speech Technology Research, University of Edinburgh, licensed under CC BY 4.0. It was used as clean-speech training data for the released checkpoint.
DNS5 Challenge noise resources: noise data prepared through the DNS Challenge and used to synthesize noisy training mixtures for the released checkpoint. For this release, the DNS5 noise resources draw on AudioSet material licensed under CC BY 4.0, selected Freesound files licensed under CC0 1.0, and DEMAND environmental recordings licensed under CC BY-SA 3.0.
OpenSLR26 and OpenSLR28: OpenSLR26 and OpenSLR28 room impulse response resources, both licensed under Apache 2.0, were used to add reverberation during training.

All audio was resampled to 16 kHz.

Training Procedure

Preprocessing

Mixtures generated dynamically
SNR sampled from –5 to 15 dB
Reverberation applied with 50% probability

Training Hyperparameters

DeWavLM-R: 50k steps, LR 2e-5, batch size 20
CFM: 100k steps, LR 1e‑4, batch size 60
Mel Vocoder: 200k steps, LR 2e-4, batch size 60
Optimizer: AdamW with warmup + cosine decay
Hardware: 2 × NVIDIA RTX 4090 GPUs

Speeds, Sizes, Times

Total parameters: ~561M
Inference compute: ~104 GMAC/s

Evaluation

Testing Data

Simulated LibriSpeech test set (using test split)
DNS1 test set with/without reverberation

Metrics

DNSMOS, UTMOS
LPS, SpeechBERTScore (SBS)
Speaker Similarity, using WavLM-Large based TCAPA-TDNN (finetuned)
WER, using Whisper-Large-v3

Results

The performance of the retrained version compared to the original one:

Model	DNSMOS	UTMOS	SBS	LPS	SpkSim	WER (%)
DeWavLM-R (orig.)	3.35	3.94	0.84	0.88	0.49	13.22
DeWavLM-R (retrained)	3.38	3.62	0.85	0.89	0.41	12.27
StuPASE (orig.)	3.37	4.08	0.85	0.90	0.68	11.57
StuPASE (retrained)	3.36	4.02	0.85	0.89	0.66	12.06

It can be seen that the retrained version achieves performance very close to that of the original version on our simulated test set.

Overall, StuPASE achieves:

Lowest WER among evaluated generative and discriminative baselines
Highest speaker similarity (SpkSim)
Highest perceptual quality with low hallucination rates
Consistent performance across noisy and reverberant conditions

Bias, Risks, and Limitations

Model trained primarily on English speech; performance may degrade for other languages.
Very strong noise or mismatched reverberation conditions can introduce artifacts.
Speaker characteristics are preserved but not guaranteed perfectly.

Recommendations

Evaluate outputs for your specific use case. Avoid deployments where misunderstanding enhanced speech could have safety or legal consequences.

Citation

If you use StuPASE in your research, please cite:

@misc{StuPASE,
      title={{StuPASE: Towards Low-Hallucination Studio-Quality Generative Speech Enhancement}}, 
      author={Xiaobin Rong and Jun Gao and Zheng Wang and Mansur Yesilbursa and Kamil Wojcicki and Jing Lu},
      year={2026},
      eprint={2603.09234},
      archivePrefix={arXiv},
      primaryClass={eess.AS},
      url={https://arxiv.org/abs/2603.09234}, 
}

Model Card Authorship & Contact

Mansur Yesilbursa: myesilbu@cisco.com

Downloads last month: -; Downloads are not tracked for this model. How to track

Paper for cisco-ai/stupase

StuPASE: Towards Low-Hallucination Studio-Quality Generative Speech Enhancement

Paper • 2603.09234 • Published 25 days ago