update results

README.md

@@ -36,4 +36,4 @@ The WERs are
 |------------------------------------|------------|------------|------------------------------------------|
 |          greedy search             | 7.27       | 6.69       | --epoch 29, --avg 13, --max-duration 100 |
 |      beam search (beam size 4)     | 6.70       | 6.04       | --epoch 29, --avg 13, --max-duration 100 |
-| modified beam search (beam size 4) | 6.72       | 6.12       | --epoch 29, --avg 13, --max-duration 100 |
+| modified beam search (beam size 4) | 6.77       | 6.14       | --epoch 29, --avg 13, --max-duration 100 |

log/modified_beam_search/log-decode-epoch-29-avg-13-beam-4-2022-04-11-18-44-43

@@ -0,0 +1,112 @@
+2022-04-11 18:44:43,381 INFO [decode.py:454] Decoding started
+2022-04-11 18:44:43,381 INFO [decode.py:460] Device: cuda:0
+2022-04-11 18:44:43,383 INFO [decode.py:470] {'best_train_loss': inf, 'best_valid_loss': inf, 'best_train_epoch': -1, 'best_valid_epoch': -1, 'batch_idx_train': 0, 'log_interval': 50, 'reset_interval': 200, 'valid_interval': 3000, 'feature_dim': 80, 'encoder_out_dim': 512, 'subsampling_factor': 4, 'attention_dim': 512, 'nhead': 8, 'dim_feedforward': 2048, 'num_encoder_layers': 12, 'vgg_frontend': False, 'embedding_dim': 512, 'warm_step': 80000, 'env_info': {'k2-version': '1.14', 'k2-build-type': 'Release', 'k2-with-cuda': True, 'k2-git-sha1': '4fb6b88661cca73e5f66f03df16e5a1d0c4886f8', 'k2-git-date': 'Fri Apr 8 18:29:32 2022', 'lhotse-version': '1.1.0', 'torch-version': '1.11.0', 'torch-cuda-available': True, 'torch-cuda-version': '10.2', 'python-version': '3.8', 'icefall-git-branch': 'tedlium3-pruned-transducer-stateless-new', 'icefall-git-sha1': 'cd6a2d9-clean', 'icefall-git-date': 'Mon Apr 11 16:25:12 2022', 'icefall-path': '/ceph-meixu/luomingshuang/icefall', 'k2-path': '/ceph-ms/luomingshuang/k2/k2/python/k2/__init__.py', 'lhotse-path': '/ceph-meixu/luomingshuang/anaconda3/envs/k2-python/lib/python3.8/site-packages/lhotse-1.1.0-py3.8.egg/lhotse/__init__.py', 'hostname': 'de-74279-k2-train-1-0307195509-54c966b95f-rtpfq', 'IP address': '10.177.22.9'}, 'epoch': 29, 'avg': 13, 'exp_dir': PosixPath('pruned_transducer_stateless/exp'), 'bpe_model': 'data/lang_bpe_500/bpe.model', 'decoding_method': 'modified_beam_search', 'beam_size': 4, 'beam': 4.0, 'max_contexts': 4, 'max_states': 8, 'context_size': 2, 'max_sym_per_frame': 1, 'manifest_dir': PosixPath('data/fbank'), 'max_duration': 100, 'bucketing_sampler': True, 'num_buckets': 30, 'concatenate_cuts': False, 'duration_factor': 1.0, 'gap': 1.0, 'on_the_fly_feats': False, 'shuffle': True, 'return_cuts': True, 'num_workers': 2, 'enable_spec_aug': True, 'spec_aug_time_warp_factor': 80, 'enable_musan': True, 'res_dir': PosixPath('pruned_transducer_stateless/exp/modified_beam_search'), 'suffix': 'epoch-29-avg-13-beam-4', 'blank_id': 0, 'unk_id': 2, 'vocab_size': 500}
+2022-04-11 18:44:43,384 INFO [decode.py:472] About to create model
+2022-04-11 18:44:43,925 INFO [decode.py:483] averaging ['pruned_transducer_stateless/exp/epoch-17.pt', 'pruned_transducer_stateless/exp/epoch-18.pt', 'pruned_transducer_stateless/exp/epoch-19.pt', 'pruned_transducer_stateless/exp/epoch-20.pt', 'pruned_transducer_stateless/exp/epoch-21.pt', 'pruned_transducer_stateless/exp/epoch-22.pt', 'pruned_transducer_stateless/exp/epoch-23.pt', 'pruned_transducer_stateless/exp/epoch-24.pt', 'pruned_transducer_stateless/exp/epoch-25.pt', 'pruned_transducer_stateless/exp/epoch-26.pt', 'pruned_transducer_stateless/exp/epoch-27.pt', 'pruned_transducer_stateless/exp/epoch-28.pt', 'pruned_transducer_stateless/exp/epoch-29.pt']
+2022-04-11 18:44:59,905 INFO [decode.py:497] Number of model parameters: 84514780
+2022-04-11 18:44:59,905 INFO [asr_datamodule.py:357] About to get dev cuts
+2022-04-11 18:44:59,929 INFO [asr_datamodule.py:362] About to get test cuts
+2022-04-11 18:44:59,983 INFO [asr_datamodule.py:300] About to create dev dataset
+2022-04-11 18:44:59,985 INFO [asr_datamodule.py:319] About to create dev dataloader
+2022-04-11 18:45:01,174 INFO [decode.py:374] batch 0/?, cuts processed until now is 10
+2022-04-11 18:45:02,564 INFO [decode.py:374] batch 2/?, cuts processed until now is 33
+2022-04-11 18:45:04,222 INFO [decode.py:374] batch 4/?, cuts processed until now is 45
+2022-04-11 18:45:05,591 INFO [decode.py:374] batch 6/?, cuts processed until now is 67
+2022-04-11 18:45:07,188 INFO [decode.py:374] batch 8/?, cuts processed until now is 77
+2022-04-11 18:45:08,511 INFO [decode.py:374] batch 10/?, cuts processed until now is 96
+2022-04-11 18:45:10,820 INFO [decode.py:374] batch 12/?, cuts processed until now is 101
+2022-04-11 18:45:12,460 INFO [decode.py:374] batch 14/?, cuts processed until now is 111
+2022-04-11 18:45:13,801 INFO [decode.py:374] batch 16/?, cuts processed until now is 125
+2022-04-11 18:45:15,211 INFO [decode.py:374] batch 18/?, cuts processed until now is 140
+2022-04-11 18:45:16,682 INFO [decode.py:374] batch 20/?, cuts processed until now is 158
+2022-04-11 18:45:17,988 INFO [decode.py:374] batch 22/?, cuts processed until now is 184
+2022-04-11 18:45:19,421 INFO [decode.py:374] batch 24/?, cuts processed until now is 198
+2022-04-11 18:45:21,084 INFO [decode.py:374] batch 26/?, cuts processed until now is 209
+2022-04-11 18:45:22,533 INFO [decode.py:374] batch 28/?, cuts processed until now is 223
+2022-04-11 18:45:24,068 INFO [decode.py:374] batch 30/?, cuts processed until now is 237
+2022-04-11 18:45:25,485 INFO [decode.py:374] batch 32/?, cuts processed until now is 256
+2022-04-11 18:45:26,766 INFO [decode.py:374] batch 34/?, cuts processed until now is 278
+2022-04-11 18:45:28,229 INFO [decode.py:374] batch 36/?, cuts processed until now is 294
+2022-04-11 18:45:29,782 INFO [decode.py:374] batch 38/?, cuts processed until now is 306
+2022-04-11 18:45:31,848 INFO [decode.py:374] batch 40/?, cuts processed until now is 314
+2022-04-11 18:45:33,568 INFO [decode.py:374] batch 42/?, cuts processed until now is 345
+2022-04-11 18:45:34,584 INFO [decode.py:374] batch 44/?, cuts processed until now is 378
+2022-04-11 18:45:36,140 INFO [decode.py:374] batch 46/?, cuts processed until now is 388
+2022-04-11 18:45:37,549 INFO [decode.py:374] batch 48/?, cuts processed until now is 404
+2022-04-11 18:45:38,747 INFO [decode.py:374] batch 50/?, cuts processed until now is 412
+2022-04-11 18:45:40,253 INFO [decode.py:374] batch 52/?, cuts processed until now is 425
+2022-04-11 18:45:41,264 INFO [decode.py:374] batch 54/?, cuts processed until now is 435
+2022-04-11 18:45:42,803 INFO [decode.py:374] batch 56/?, cuts processed until now is 446
+2022-04-11 18:45:43,661 INFO [decode.py:374] batch 58/?, cuts processed until now is 458
+2022-04-11 18:45:45,142 INFO [decode.py:374] batch 60/?, cuts processed until now is 474
+2022-04-11 18:45:46,339 INFO [decode.py:374] batch 62/?, cuts processed until now is 483
+2022-04-11 18:45:47,351 INFO [decode.py:374] batch 64/?, cuts processed until now is 493
+2022-04-11 18:45:48,813 INFO [decode.py:374] batch 66/?, cuts processed until now is 507
+2022-04-11 18:45:48,921 INFO [decode.py:391] The transcripts are stored in pruned_transducer_stateless/exp/modified_beam_search/recogs-dev-beam_size_4-epoch-29-avg-13-beam-4.txt
+2022-04-11 18:45:48,946 INFO [utils.py:406] [dev-beam_size_4] %WER 6.77% [1233 / 18226, 181 ins, 398 del, 654 sub ]
+2022-04-11 18:45:49,007 INFO [decode.py:404] Wrote detailed error stats to pruned_transducer_stateless/exp/modified_beam_search/errs-dev-beam_size_4-epoch-29-avg-13-beam-4.txt
+2022-04-11 18:45:49,008 INFO [decode.py:421] 
+For dev, WER of different settings are:
+beam_size_4	6.77	best for dev
+
+2022-04-11 18:45:49,825 INFO [decode.py:374] batch 0/?, cuts processed until now is 14
+2022-04-11 18:45:50,987 INFO [decode.py:374] batch 2/?, cuts processed until now is 51
+2022-04-11 18:45:52,441 INFO [decode.py:374] batch 4/?, cuts processed until now is 67
+2022-04-11 18:45:53,631 INFO [decode.py:374] batch 6/?, cuts processed until now is 104
+2022-04-11 18:45:55,105 INFO [decode.py:374] batch 8/?, cuts processed until now is 118
+2022-04-11 18:45:56,295 INFO [decode.py:374] batch 10/?, cuts processed until now is 145
+2022-04-11 18:45:58,516 INFO [decode.py:374] batch 12/?, cuts processed until now is 152
+2022-04-11 18:45:59,983 INFO [decode.py:374] batch 14/?, cuts processed until now is 165
+2022-04-11 18:46:01,373 INFO [decode.py:374] batch 16/?, cuts processed until now is 185
+2022-04-11 18:46:02,686 INFO [decode.py:374] batch 18/?, cuts processed until now is 205
+2022-04-11 18:46:04,151 INFO [decode.py:374] batch 20/?, cuts processed until now is 219
+2022-04-11 18:46:05,352 INFO [decode.py:374] batch 22/?, cuts processed until now is 262
+2022-04-11 18:46:06,646 INFO [decode.py:374] batch 24/?, cuts processed until now is 281
+2022-04-11 18:46:08,221 INFO [decode.py:374] batch 26/?, cuts processed until now is 297
+2022-04-11 18:46:09,579 INFO [decode.py:374] batch 28/?, cuts processed until now is 316
+2022-04-11 18:46:10,969 INFO [decode.py:374] batch 30/?, cuts processed until now is 334
+2022-04-11 18:46:12,253 INFO [decode.py:374] batch 32/?, cuts processed until now is 358
+2022-04-11 18:46:13,410 INFO [decode.py:374] batch 34/?, cuts processed until now is 389
+2022-04-11 18:46:14,727 INFO [decode.py:374] batch 36/?, cuts processed until now is 408
+2022-04-11 18:46:16,013 INFO [decode.py:374] batch 38/?, cuts processed until now is 428
+2022-04-11 18:46:17,646 INFO [decode.py:374] batch 40/?, cuts processed until now is 441
+2022-04-11 18:46:19,003 INFO [decode.py:374] batch 42/?, cuts processed until now is 489
+2022-04-11 18:46:20,167 INFO [decode.py:374] batch 44/?, cuts processed until now is 560
+2022-04-11 18:46:21,585 INFO [decode.py:374] batch 46/?, cuts processed until now is 573
+2022-04-11 18:46:23,017 INFO [decode.py:374] batch 48/?, cuts processed until now is 589
+2022-04-11 18:46:24,480 INFO [decode.py:374] batch 50/?, cuts processed until now is 605
+2022-04-11 18:46:25,884 INFO [decode.py:374] batch 52/?, cuts processed until now is 622
+2022-04-11 18:46:27,159 INFO [decode.py:374] batch 54/?, cuts processed until now is 645
+2022-04-11 18:46:28,532 INFO [decode.py:374] batch 56/?, cuts processed until now is 672
+2022-04-11 18:46:29,899 INFO [decode.py:374] batch 58/?, cuts processed until now is 692
+2022-04-11 18:46:31,055 INFO [decode.py:374] batch 60/?, cuts processed until now is 729
+2022-04-11 18:46:32,407 INFO [decode.py:374] batch 62/?, cuts processed until now is 749
+2022-04-11 18:46:33,924 INFO [decode.py:374] batch 64/?, cuts processed until now is 761
+2022-04-11 18:46:35,138 INFO [decode.py:374] batch 66/?, cuts processed until now is 784
+2022-04-11 18:46:35,930 INFO [decode.py:374] batch 68/?, cuts processed until now is 807
+2022-04-11 18:46:37,242 INFO [decode.py:374] batch 70/?, cuts processed until now is 829
+2022-04-11 18:46:38,380 INFO [decode.py:374] batch 72/?, cuts processed until now is 858
+2022-04-11 18:46:39,260 INFO [decode.py:374] batch 74/?, cuts processed until now is 883
+2022-04-11 18:46:40,651 INFO [decode.py:374] batch 76/?, cuts processed until now is 898
+2022-04-11 18:46:41,907 INFO [decode.py:374] batch 78/?, cuts processed until now is 928
+2022-04-11 18:46:43,176 INFO [decode.py:374] batch 80/?, cuts processed until now is 949
+2022-04-11 18:46:44,515 INFO [decode.py:374] batch 82/?, cuts processed until now is 964
+2022-04-11 18:46:45,962 INFO [decode.py:374] batch 84/?, cuts processed until now is 979
+2022-04-11 18:46:47,255 INFO [decode.py:374] batch 86/?, cuts processed until now is 998
+2022-04-11 18:46:48,210 INFO [decode.py:374] batch 88/?, cuts processed until now is 1017
+2022-04-11 18:46:49,406 INFO [decode.py:374] batch 90/?, cuts processed until now is 1031
+2022-04-11 18:46:50,676 INFO [decode.py:374] batch 92/?, cuts processed until now is 1055
+2022-04-11 18:46:51,847 INFO [decode.py:374] batch 94/?, cuts processed until now is 1089
+2022-04-11 18:46:52,743 INFO [decode.py:374] batch 96/?, cuts processed until now is 1107
+2022-04-11 18:46:53,528 INFO [decode.py:374] batch 98/?, cuts processed until now is 1117
+2022-04-11 18:46:54,913 INFO [decode.py:374] batch 100/?, cuts processed until now is 1133
+2022-04-11 18:46:55,800 INFO [decode.py:374] batch 102/?, cuts processed until now is 1145
+2022-04-11 18:46:57,014 INFO [decode.py:374] batch 104/?, cuts processed until now is 1155
+2022-04-11 18:46:57,128 INFO [decode.py:391] The transcripts are stored in pruned_transducer_stateless/exp/modified_beam_search/recogs-test-beam_size_4-epoch-29-avg-13-beam-4.txt
+2022-04-11 18:46:57,163 INFO [utils.py:406] [test-beam_size_4] %WER 6.14% [1745 / 28430, 203 ins, 644 del, 898 sub ]
+2022-04-11 18:46:57,256 INFO [decode.py:404] Wrote detailed error stats to pruned_transducer_stateless/exp/modified_beam_search/errs-test-beam_size_4-epoch-29-avg-13-beam-4.txt
+2022-04-11 18:46:57,256 INFO [decode.py:421] 
+For test, WER of different settings are:
+beam_size_4	6.14	best for test
+
+2022-04-11 18:46:57,256 INFO [decode.py:524] Done!

log/modified_beam_search/wer-summary-dev-beam_size_4-epoch-29-avg-13-beam-4.txt

@@ -0,0 +1,2 @@
+settings	WER
+beam_size_4	6.77

log/modified_beam_search/wer-summary-test-beam_size_4-epoch-29-avg-13-beam-4.txt

@@ -0,0 +1,2 @@
+settings	WER
+beam_size_4	6.14