marcoyang's picture
initial commit
a40d894
raw
history blame
No virus
2.66 kB
2022-12-14 11:36:18,616 INFO [compute_perplexity.py:117] Computing perplexity started
2022-12-14 11:36:18,616 INFO [compute_perplexity.py:118] {'max_sent_len': 100, 'sos_id': 1, 'eos_id': 1, 'blank_id': 0, 'lr': 0.001, 'weight_decay': 1e-06, 'best_train_loss': inf, 'best_valid_loss': inf, 'best_train_epoch': -1, 'best_valid_epoch': -1, 'batch_idx_train': 0, 'log_interval': 200, 'reset_interval': 2000, 'valid_interval': 1000, 'nhead': 8, 'embedding_dim': 768, 'encoder_dim': 768, 'dim_feedforward': 2048, 'dropout': 0.1, 'env_info': {'k2-version': '1.22', 'k2-build-type': 'Release', 'k2-with-cuda': True, 'k2-git-sha1': '96c9a2aece2a3a7633da07740e24fa3d96f5498c', 'k2-git-date': 'Thu Nov 10 08:14:02 2022', 'lhotse-version': '1.10.0', 'torch-version': '1.12.1', 'torch-cuda-available': True, 'torch-cuda-version': '11.6', 'python-version': '3.8', 'icefall-git-branch': 'transformer_lm', 'icefall-git-sha1': 'e5ce80d-dirty', 'icefall-git-date': 'Wed Dec 14 10:40:00 2022', 'icefall-path': '/ceph-data4/yangxiaoyu/softwares/icefall_development/icefall_transformer_lm', 'k2-path': '/ceph-data4/yangxiaoyu/softwares/anaconda3/envs/k2_latest/lib/python3.8/site-packages/k2/__init__.py', 'lhotse-path': '/ceph-data4/yangxiaoyu/softwares/anaconda3/envs/k2_latest/lib/python3.8/site-packages/lhotse/__init__.py', 'hostname': 'de-74279-k2-train-1-0307195509-567fcb96d6-kdztg', 'IP address': '10.177.22.10'}, 'epoch': 11, 'avg': 3, 'exp_dir': PosixPath('transformer_lm/exp_full_libri_16layer_maxlen200_8gpu'), 'lm_data': PosixPath('transformer_lm/libri_lm_training_bpe500/sorted-lm-data-libri-test.pt'), 'vocab_size': 500, 'num_layers': 16, 'tie_weights': False, 'batch_size': 50}
2022-12-14 11:36:18,616 INFO [compute_perplexity.py:124] Device: cuda:0
2022-12-14 11:36:18,616 INFO [compute_perplexity.py:126] About to create model
2022-12-14 11:36:19,041 INFO [model.py:68] Not tying weights
2022-12-14 11:36:19,041 INFO [compute_perplexity.py:147] averaging ['transformer_lm/exp_full_libri_16layer_maxlen200_8gpu/epoch-9.pt', 'transformer_lm/exp_full_libri_16layer_maxlen200_8gpu/epoch-10.pt', 'transformer_lm/exp_full_libri_16layer_maxlen200_8gpu/epoch-11.pt']
2022-12-14 11:36:23,796 INFO [compute_perplexity.py:157] Number of model parameters: 98995638
2022-12-14 11:36:23,796 INFO [compute_perplexity.py:158] Number of model parameters (requires_grad): 98995638 (100.0%)
2022-12-14 11:36:23,797 INFO [compute_perplexity.py:164] Loading LM test data from transformer_lm/libri_lm_training_bpe500/sorted-lm-data-libri-test.pt
2022-12-14 11:36:30,579 INFO [compute_perplexity.py:188] total nll: 473318.6812133789, num tokens: 211138, num sentences: 5559, ppl: 9.410