johntsi
/

ZeroSwot-Medium_asr-cv_en-to-200

Automatic Speech Recognition

zero_swot_encoder

feature-extraction

speech translation

Model card Files Files and versions Community

johntsi commited on Jun 25

Commit

9133100

•

1 Parent(s): c312f6d

Update README.md

Files changed (1) hide show

README.md +3 -1

README.md CHANGED Viewed

@@ -245,6 +245,8 @@ tags:
 ZeroSwot is a state-of-the-art zero-shot end-to-end Speech Translation system.
 The model is created by adapting a wav2vec2.0-based encoder to the embedding space of NLLB, using a novel subword compression module and Optimal Transport, while only utilizing ASR data. It thus enables **Zero-shot E2E Speech Translation to all the 200 languages supported by NLLB**.
 For more details please refer to our [paper](https://arxiv.org/abs/2402.10422) and the [original repo](https://github.com/mt-upc/ZeroSwot) build on fairseq.
@@ -253,7 +255,7 @@ For more details please refer to our [paper](https://arxiv.org/abs/2402.10422) a
 The compression module is a light-weight transformer that takes as input the hidden state of wav2vec2.0 and the corresponding CTC predictions, and compresses them to subword-like embeddings similar to those expected from NLLB and aligns them using Optimal Transport. For inference we simply pass the output of the speech encoder to NLLB encoder.
-<div align=center><img src="resources/methodology.png" height="100%" width="100%"/></div>
 ## Version

 ZeroSwot is a state-of-the-art zero-shot end-to-end Speech Translation system.
+<div align=center><img src="resources/intro.png" height="75%" width="75%"/></div>
 The model is created by adapting a wav2vec2.0-based encoder to the embedding space of NLLB, using a novel subword compression module and Optimal Transport, while only utilizing ASR data. It thus enables **Zero-shot E2E Speech Translation to all the 200 languages supported by NLLB**.
 For more details please refer to our [paper](https://arxiv.org/abs/2402.10422) and the [original repo](https://github.com/mt-upc/ZeroSwot) build on fairseq.
 The compression module is a light-weight transformer that takes as input the hidden state of wav2vec2.0 and the corresponding CTC predictions, and compresses them to subword-like embeddings similar to those expected from NLLB and aligns them using Optimal Transport. For inference we simply pass the output of the speech encoder to NLLB encoder.
+<div align=center><img src="resources/methodology.png" height="120%" width="120%"/></div>
 ## Version