luomingshuang
commited on
Commit
•
e41fa20
1
Parent(s):
bec81cb
Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,30 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# Pre-trained VisualNet-CTC models for the GRID dataset with icefall.
|
2 |
+
The model was trained on full [GRID](https://zenodo.org/record/3625687#.Ybn7HagzY2w) with the scripts in [icefall](https://github.com/k2-fsa/icefall).
|
3 |
+
See (https://github.com/k2-fsa/icefall/tree/master/egs/grid/AVSR/visualnet_ctc_asr) for more details of this model.
|
4 |
+
## How to use
|
5 |
+
See (https://github.com/k2-fsa/icefall/blob/master/egs/grid/AVSR/visualnet_ctc_asr/Pre-trained.md)
|
6 |
+
## Training procedure
|
7 |
+
The main repositories are list below, we will update the training and decoding scripts with the update of version.
|
8 |
+
k2: https://github.com/k2-fsa/k2
|
9 |
+
icefall: https://github.com/k2-fsa/icefall
|
10 |
+
* Install k2 and lhotse, k2 installation guide refers to https://k2.readthedocs.io/en/latest/installation/index.html, lhotse refers to https://lhotse.readthedocs.io/en/latest/getting-started.html#installation. I think the latest version would be ok. And please also install the requirements listed in icefall.
|
11 |
+
* Clone icefall(https://github.com/k2-fsa/icefall) and check to the commit showed above.
|
12 |
+
```
|
13 |
+
git clone https://github.com/k2-fsa/icefall
|
14 |
+
cd icefall
|
15 |
+
```
|
16 |
+
* Preparing data.
|
17 |
+
```
|
18 |
+
cd egs/grid/AVSR
|
19 |
+
bash ./prepare.sh
|
20 |
+
```
|
21 |
+
* Training
|
22 |
+
```
|
23 |
+
export CUDA_VISIBLE_DEVICES="0"
|
24 |
+
python visualnet_ctc_asr/train.py --world-size 1
|
25 |
+
```
|
26 |
+
## Evaluation results
|
27 |
+
The best decoding results (WER) on GRID TEST are listed below, we got this result by averaging models from epoch 16 to 25, the decoding method is `1best`.
|
28 |
+
||TEST|
|
29 |
+
|--|--|
|
30 |
+
|WER|15.68%|
|