Mirco commited on
Commit
498c7b4
1 Parent(s): 9fe0bc5

model upload

Browse files
.gitattributes CHANGED
@@ -25,3 +25,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
25
  *.zip filter=lfs diff=lfs merge=lfs -text
26
  *.zstandard filter=lfs diff=lfs merge=lfs -text
27
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
25
  *.zip filter=lfs diff=lfs merge=lfs -text
26
  *.zstandard filter=lfs diff=lfs merge=lfs -text
27
  *tfevents* filter=lfs diff=lfs merge=lfs -text
28
+ asr.ckpt filter=lfs diff=lfs merge=lfs -text
29
+ tokenizer.ckpt filter=lfs diff=lfs merge=lfs -text
30
+ wav2vec2.ckpt filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,132 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: "dar"
3
+ thumbnail:
4
+ pipeline_tag: automatic-speech-recognition
5
+ tags:
6
+ - CTC
7
+ - pytorch
8
+ - speechbrain
9
+ - Transformer
10
+ license: "apache-2.0"
11
+ datasets:
12
+ - Dvoice
13
+ metrics:
14
+ - wer
15
+ - cer
16
+ ---
17
+
18
+ <iframe src="https://ghbtns.com/github-btn.html?user=speechbrain&repo=speechbrain&type=star&count=true&size=large&v=2" frameborder="0" scrolling="0" width="170" height="30" title="GitHub"></iframe>
19
+ <br/><br/>
20
+
21
+
22
+ # wav2vec 2.0 with CTC/Attention trained on DVoice Darija (No LM)
23
+ This repository provides all the necessary tools to perform automatic speech
24
+ recognition from an end-to-end system pretrained on a [DVoice](https://zenodo.org/record/6342622) Darija dataset within
25
+ SpeechBrain. For a better experience, we encourage you to learn more about
26
+ [SpeechBrain](https://speechbrain.github.io).
27
+
28
+ | DVoice Release | Val. CER | Val. WER | Test CER | Test WER |
29
+ |:-------------:|:---------------------------:| -----:| -----:| -----:|
30
+ | v2.0 | 5.51 | 18.46 | 5.85 | 18.28 |
31
+
32
+ # Pipeline description
33
+ This ASR system is composed of 2 different but linked blocks:
34
+ - Tokenizer (unigram) that transforms words into subword units and is trained with the train transcriptions.
35
+ - Acoustic model (wav2vec2.0 + CTC). A pretrained wav2vec 2.0 model ([facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53)) is combined with two DNN layers and finetuned on the Darija dataset.
36
+ The obtained final acoustic representation is given to the CTC greedy decoder.
37
+ The system is trained with recordings sampled at 16kHz (single channel).
38
+ The code will automatically normalize your audio (i.e., resampling + mono channel selection) when calling *transcribe_file* if needed.
39
+
40
+ # Install SpeechBrain
41
+ First of all, please install transformers and SpeechBrain with the following command:
42
+ ```
43
+ pip install speechbrain transformers
44
+ ```
45
+ Please notice that we encourage you to read the SpeechBrain tutorials and learn more about
46
+ [SpeechBrain](https://speechbrain.github.io).
47
+
48
+ # Transcribing your own audio files (in Darija)
49
+ ```python
50
+ from speechbrain.pretrained import EncoderASR
51
+ asr_model = EncoderDecoderASR.from_hparams(source="speechbrain/asr-wav2vec2-dvoice-darija", savedir="pretrained_models/asr-wav2vec2-dvoice-darija")
52
+ asr_model.transcribe_file('speechbrain/asr-wav2vec2-dvoice-darija/example_darija.wav')
53
+ ```
54
+
55
+ # Inference on GPU
56
+ To perform inference on the GPU, add `run_opts={"device":"cuda"}` when calling the `from_hparams` method.
57
+
58
+ ### Training
59
+ The model was trained with SpeechBrain.
60
+ To train it from scratch follow these steps:
61
+ 1. Clone SpeechBrain:
62
+ ```bash
63
+ git clone https://github.com/speechbrain/speechbrain/
64
+ ```
65
+ 2. Install it:
66
+ ```bash
67
+ cd speechbrain
68
+ pip install -r requirements.txt
69
+ pip install -e .
70
+ ```
71
+ 3. Run Training:
72
+ ```bash
73
+ cd recipes/DVoice/ASR/CTC
74
+ python train_with_wav2vec2.py hparams/train_dar_with_wav2vec.yaml --data_folder=/localscratch/darija/
75
+ ```
76
+ You can find our training results (models, logs, etc) [here](https://drive.google.com/drive/folders/1vNT7RjRuELs7pumBHmfYsrOp9m46D0ym?usp=sharing).
77
+
78
+
79
+ # Limitations
80
+ The SpeechBrain team does not provide any warranty on the performance achieved by this model when used on other datasets.
81
+
82
+ # Referencing SpeechBrain
83
+ ```
84
+ @misc{SB2021,
85
+ author = {Ravanelli, Mirco and Parcollet, Titouan and Rouhe, Aku and Plantinga, Peter and Rastorgueva, Elena and Lugosch, Loren and Dawalatabad, Nauman and Ju-Chieh, Chou and Heba, Abdel and Grondin, Francois and Aris, William and Liao, Chien-Feng and Cornell, Samuele and Yeh, Sung-Lin and Na, Hwidong and Gao, Yan and Fu, Szu-Wei and Subakan, Cem and De Mori, Renato and Bengio, Yoshua },
86
+ title = {SpeechBrain},
87
+ year = {2021},
88
+ publisher = {GitHub},
89
+ journal = {GitHub repository},
90
+ howpublished = {\\\\url{https://github.com/speechbrain/speechbrain}},
91
+ }
92
+ ```
93
+
94
+ # About DVoice
95
+ DVoice is a community initiative that aims to provide African low resources languages with data and models to facilitate their use of voice technologies. The lack of data on these languages makes it necessary to collect data using methods that are specific to each one. Two different approaches are currently used: the DVoice platforms ([https://dvoice.ma](https://dvoice.ma) and [https://dvoice.sn](https://dvoice.sn)), which are based on Mozilla Common Voice, for collecting authentic recordings from the community, and transfer learning techniques for automatically labeling recordings that are retrieved from social media. The DVoice platform currently manages 7 languages including Darija (Moroccan Arabic dialect) whose dataset appears on this version, Wolof, Mandingo, Serere, Pular, Diola, and Soninke.
96
+
97
+ For this project, AIOX Labs and the SI2M Laboratory are joining forces to build the future of technologies together.
98
+
99
+ # About AIOX Labs
100
+ Based in Rabat, London, and Paris, AIOX-Labs mobilizes artificial intelligence technologies to meet the business needs and data projects of companies.
101
+
102
+ - He is at the service of the growth of groups, the optimization of processes, or the improvement of the customer experience.
103
+ - AIOX-Labs is multi-sector, from fintech to industry, including retail and consumer goods.
104
+ - Business-ready data products with a solid algorithmic base and adaptability for the specific needs of each client.
105
+ - A complementary team made up of doctors in AI and business experts with a solid scientific base and international publications.
106
+
107
+ Website: [https://www.aiox-labs.com/](https://www.aiox-labs.com/)
108
+
109
+ # SI2M Laboratory
110
+ The Information Systems, Intelligent Systems, and Mathematical Modeling Research Laboratory (SI2M) is an academic research laboratory of the National Institute of Statistics and Applied Economics (INSEA). The research areas of the laboratories are Information Systems, Intelligent Systems, Artificial Intelligence, Decision Support, Network, and System Security, and Mathematical Modelling.
111
+
112
+ Website: [SI2M Laboratory](https://insea.ac.ma/index.php/pole-recherche/equipe-de-recherche/150-laboratoire-de-recherche-en-systemes-d-information-systemes-intelligents-et-modelisation-mathematique)
113
+
114
+ # About SpeechBrain
115
+ SpeechBrain is an open-source and all-in-one speech toolkit. It is designed to be simple, extremely flexible, and user-friendly. Competitive or state-of-the-art performance is obtained in various domains.
116
+ Website: https://speechbrain.github.io/
117
+ GitHub: https://github.com/speechbrain/speechbrain
118
+
119
+
120
+ # Referencing SpeechBrain
121
+ ```
122
+ @misc{SB2021,
123
+ author = {Ravanelli, Mirco and Parcollet, Titouan and Rouhe, Aku and Plantinga, Peter and Rastorgueva, Elena and Lugosch, Loren and Dawalatabad, Nauman and Ju-Chieh, Chou and Heba, Abdel and Grondin, Francois and Aris, William and Liao, Chien-Feng and Cornell, Samuele and Yeh, Sung-Lin and Na, Hwidong and Gao, Yan and Fu, Szu-Wei and Subakan, Cem and De Mori, Renato and Bengio, Yoshua },
124
+ title = {SpeechBrain},
125
+ year = {2021},
126
+ publisher = {GitHub},
127
+ journal = {GitHub repository},
128
+ howpublished = {\\\\url{https://github.com/speechbrain/speechbrain}},
129
+ }
130
+ ```
131
+ # Acknowledgements
132
+ This research was supported through computational resources of HPC-MARWAN (www.marwan.ma/hpc) provided by CNRST, Rabat, Morocco. We deeply thank this institution.
asr.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:572dbf84bdd480550286b1bc5413f0bb7a457d52121e547c646901b4913cae08
3
+ size 12798880
config.json ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "speechbrain_interface": "EncoderASR",
3
+ "activation_dropout": 0.0,
4
+ "apply_spec_augment": true,
5
+ "architectures": [
6
+ "Wav2Vec2Model"
7
+ ],
8
+ "attention_dropout": 0.1,
9
+ "bos_token_id": 1,
10
+ "conv_bias": true,
11
+ "conv_dim": [
12
+ 512,
13
+ 512,
14
+ 512,
15
+ 512,
16
+ 512,
17
+ 512,
18
+ 512
19
+ ],
20
+ "conv_kernel": [
21
+ 10,
22
+ 3,
23
+ 3,
24
+ 3,
25
+ 3,
26
+ 2,
27
+ 2
28
+ ],
29
+ "conv_stride": [
30
+ 5,
31
+ 2,
32
+ 2,
33
+ 2,
34
+ 2,
35
+ 2,
36
+ 2
37
+ ],
38
+ "ctc_loss_reduction": "sum",
39
+ "ctc_zero_infinity": false,
40
+ "do_stable_layer_norm": true,
41
+ "eos_token_id": 2,
42
+ "feat_extract_activation": "gelu",
43
+ "feat_extract_dropout": 0.0,
44
+ "feat_extract_norm": "layer",
45
+ "feat_proj_dropout": 0.1,
46
+ "final_dropout": 0.0,
47
+ "gradient_checkpointing": false,
48
+ "hidden_act": "gelu",
49
+ "hidden_dropout": 0.1,
50
+ "hidden_size": 1024,
51
+ "initializer_range": 0.02,
52
+ "intermediate_size": 4096,
53
+ "layer_norm_eps": 1e-05,
54
+ "layerdrop": 0.1,
55
+ "mask_channel_length": 10,
56
+ "mask_channel_min_space": 1,
57
+ "mask_channel_other": 0.0,
58
+ "mask_channel_prob": 0.0,
59
+ "mask_channel_selection": "static",
60
+ "mask_feature_length": 10,
61
+ "mask_feature_prob": 0.0,
62
+ "mask_time_length": 10,
63
+ "mask_time_min_space": 1,
64
+ "mask_time_other": 0.0,
65
+ "mask_time_prob": 0.075,
66
+ "mask_time_selection": "static",
67
+ "model_type": "wav2vec2",
68
+ "num_attention_heads": 16,
69
+ "num_conv_pos_embedding_groups": 16,
70
+ "num_conv_pos_embeddings": 128,
71
+ "num_feat_extract_layers": 7,
72
+ "num_hidden_layers": 24,
73
+ "pad_token_id": 0,
74
+ "transformers_version": "4.5.1",
75
+ "vocab_size": 32
76
+ }
example_darija.wav ADDED
Binary file (144 kB). View file
 
hyperparams.yaml ADDED
@@ -0,0 +1,96 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ################################
2
+ # Model: wav2vec2 + DNN + CTC/Attention
3
+ # Augmentation: SpecAugment
4
+ # Authors: Titouan Parcollet 2021
5
+ # ################################
6
+
7
+ sample_rate: 16000
8
+ wav2vec2_hub: facebook/wav2vec2-large-xlsr-53
9
+
10
+ # BPE parameters
11
+ token_type: char # ["unigram", "bpe", "char"]
12
+ character_coverage: 1.0
13
+
14
+ # Model parameters
15
+ activation: !name:torch.nn.LeakyReLU
16
+ dnn_layers: 2
17
+ dnn_neurons: 1024
18
+ emb_size: 128
19
+ dec_neurons: 1024
20
+
21
+ # Outputs
22
+ output_neurons: 36 # BPE size, index(blank/eos/bos) = 0
23
+
24
+ # Decoding parameters
25
+ # Be sure that the bos and eos index match with the BPEs ones
26
+ blank_index: 0
27
+ bos_index: 1
28
+ eos_index: 2
29
+ min_decode_ratio: 0.0
30
+ max_decode_ratio: 1.0
31
+ beam_size: 80
32
+ eos_threshold: 1.5
33
+ using_max_attn_shift: True
34
+ max_attn_shift: 140
35
+ ctc_weight_decode: 0.0
36
+ temperature: 1.50
37
+
38
+ enc: !new:speechbrain.nnet.containers.Sequential
39
+ input_shape: [null, null, 1024]
40
+ linear1: !name:speechbrain.nnet.linear.Linear
41
+ n_neurons: 1024
42
+ bias: True
43
+ bn1: !name:speechbrain.nnet.normalization.BatchNorm1d
44
+ activation: !new:torch.nn.LeakyReLU
45
+ drop: !new:torch.nn.Dropout
46
+ p: 0.15
47
+ linear2: !name:speechbrain.nnet.linear.Linear
48
+ n_neurons: 1024
49
+ bias: True
50
+ bn2: !name:speechbrain.nnet.normalization.BatchNorm1d
51
+ activation2: !new:torch.nn.LeakyReLU
52
+ drop2: !new:torch.nn.Dropout
53
+ p: 0.15
54
+ linear3: !name:speechbrain.nnet.linear.Linear
55
+ n_neurons: 1024
56
+ bias: True
57
+ bn3: !name:speechbrain.nnet.normalization.BatchNorm1d
58
+ activation3: !new:torch.nn.LeakyReLU
59
+
60
+ wav2vec2: !new:speechbrain.lobes.models.huggingface_wav2vec.HuggingFaceWav2Vec2
61
+ source: !ref <wav2vec2_hub>
62
+ output_norm: True
63
+ freeze: True
64
+ save_path: model_checkpoints
65
+
66
+ ctc_lin: !new:speechbrain.nnet.linear.Linear
67
+ input_size: !ref <dnn_neurons>
68
+ n_neurons: !ref <output_neurons>
69
+
70
+ log_softmax: !new:speechbrain.nnet.activations.Softmax
71
+ apply_log: True
72
+
73
+ ctc_cost: !name:speechbrain.nnet.losses.ctc_loss
74
+ blank_index: !ref <blank_index>
75
+
76
+ asr_model: !new:torch.nn.ModuleList
77
+ - [!ref <enc>, !ref <ctc_lin>]
78
+
79
+ tokenizer: !new:sentencepiece.SentencePieceProcessor
80
+
81
+ encoder: !new:speechbrain.nnet.containers.LengthsCapableSequential
82
+ wav2vec2: !ref <wav2vec2>
83
+ enc: !ref <enc>
84
+ ctc_lin: !ref <ctc_lin>
85
+
86
+ decoding_function: !name:speechbrain.decoders.ctc_greedy_decode
87
+ blank_id: !ref <blank_index>
88
+
89
+ modules:
90
+ encoder: !ref <encoder>
91
+
92
+ pretrainer: !new:speechbrain.utils.parameter_transfer.Pretrainer
93
+ loadables:
94
+ wav2vec2: !ref <wav2vec2>
95
+ asr: !ref <asr_model>
96
+ tokenizer: !ref <tokenizer>
preprocessor_config.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_normalize": true,
3
+ "feature_size": 1,
4
+ "padding_side": "right",
5
+ "padding_value": 0.0,
6
+ "return_attention_mask": true,
7
+ "sampling_rate": 16000
8
+ }
tokenizer.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:99392f1c87d21e28b1f88989db9af8f04d2fa06b45f6c5edb6b1d2257758a2da
3
+ size 238116
wav2vec2.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:51f502d9cc309400c4b8116b80969470578f6628100e0d45ab046f9e1e990076
3
+ size 1261920693