Hugging Face's logo

cantillation
/
Teamim-small_WeightDecay-0.05_Augmented_Old-Data_date-23-07-2024

Automatic Speech Recognition
Transformers
TensorBoard
Safetensors
Hebrew
whisper
hf-asr-leaderboard
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community

he-cantillation

This model is a fine-tuned version of openai/whisper-small on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.3963
  • Wer: 16.4230
  • Avg Precision Exact: 0.8621
  • Avg Recall Exact: 0.8754
  • Avg F1 Exact: 0.8679
  • Avg Precision Letter Shift: 0.8906
  • Avg Recall Letter Shift: 0.9042
  • Avg F1 Letter Shift: 0.8966
  • Avg Precision Word Level: 0.8954
  • Avg Recall Word Level: 0.9066
  • Avg F1 Word Level: 0.9002
  • Avg Precision Word Shift: 0.9532
  • Avg Recall Word Shift: 0.9639
  • Avg F1 Word Shift: 0.9577
  • Precision Median Exact: 0.9231
  • Recall Median Exact: 0.9231
  • F1 Median Exact: 0.9333
  • Precision Max Exact: 1.0
  • Recall Max Exact: 1.0
  • F1 Max Exact: 1.0
  • Precision Min Exact: 0.0
  • Recall Min Exact: 0.0
  • F1 Min Exact: 0.0
  • Precision Min Letter Shift: 0.0
  • Recall Min Letter Shift: 0.0
  • F1 Min Letter Shift: 0.0
  • Precision Min Word Level: 0.0
  • Recall Min Word Level: 0.0
  • F1 Min Word Level: 0.0
  • Precision Min Word Shift: 0.6154
  • Recall Min Word Shift: 0.7
  • F1 Min Word Shift: 0.6667

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 8
  • eval_batch_size: 32
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 500
  • training_steps: 200000
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer Avg Precision Exact Avg Recall Exact Avg F1 Exact Avg Precision Letter Shift Avg Recall Letter Shift Avg F1 Letter Shift Avg Precision Word Level Avg Recall Word Level Avg F1 Word Level Avg Precision Word Shift Avg Recall Word Shift Avg F1 Word Shift Precision Median Exact Recall Median Exact F1 Median Exact Precision Max Exact Recall Max Exact F1 Max Exact Precision Min Exact Recall Min Exact F1 Min Exact Precision Min Letter Shift Recall Min Letter Shift F1 Min Letter Shift Precision Min Word Level Recall Min Word Level F1 Min Word Level Precision Min Word Shift Recall Min Word Shift F1 Min Word Shift
No log 0.0001 1 7.2500 103.0091 0.0 0.0 0.0 0.0002 0.0000 0.0000 0.0021 0.0165 0.0037 0.0285 0.0276 0.0273 0.0 0.0 0.0 0 0 0 0 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0204 1.1457 10000 0.2495 20.8519 0.8158 0.8223 0.8182 0.8489 0.8558 0.8514 0.8532 0.8583 0.8549 0.9474 0.9544 0.9500 0.9091 0.9167 0.9091 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6364 0.6364 0.6364
0.0146 2.2915 20000 0.2638 19.4745 0.8330 0.8436 0.8375 0.8674 0.8788 0.8722 0.8717 0.8825 0.8763 0.9457 0.9559 0.9499 0.9091 0.9167 0.9167 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.4545 0.5 0.4762
0.0073 3.4372 30000 0.2853 18.3725 0.8473 0.8536 0.8497 0.8784 0.8855 0.8812 0.8825 0.8890 0.8850 0.9560 0.9615 0.9580 0.9167 0.9231 0.9231 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5455 0.5455 0.5455
0.0041 4.5830 40000 0.3095 19.1354 0.8449 0.8556 0.8495 0.8788 0.8902 0.8836 0.8822 0.8931 0.8869 0.9487 0.9600 0.9535 0.9167 0.9167 0.9167 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.4 0.4444 0.4211
0.0035 5.7287 50000 0.3130 18.7752 0.8527 0.8654 0.8582 0.8839 0.8979 0.8900 0.8889 0.9018 0.8945 0.9456 0.9593 0.9516 0.9167 0.9231 0.9167 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0031 6.8744 60000 0.3365 18.3513 0.8366 0.8516 0.8432 0.8664 0.8822 0.8734 0.8720 0.8859 0.8781 0.9446 0.9591 0.9509 0.9167 0.9231 0.9167 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0013 8.0202 70000 0.3247 17.4401 0.8558 0.8673 0.8607 0.8863 0.8981 0.8912 0.8912 0.9018 0.8956 0.9551 0.9680 0.9607 0.9167 0.9231 0.9231 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.7 0.75 0.7368
0.0012 9.1659 80000 0.3487 18.4361 0.8485 0.8602 0.8535 0.8847 0.8972 0.8901 0.8890 0.8994 0.8933 0.9528 0.9641 0.9576 0.9167 0.9231 0.9231 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6154 0.7273 0.6667
0.0041 10.3116 90000 0.3509 17.4401 0.8443 0.8585 0.8506 0.8753 0.8900 0.8818 0.8805 0.8931 0.8860 0.9515 0.9659 0.9578 0.9167 0.9231 0.9231 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.5556 0.5263
0.0006 11.4574 100000 0.3491 17.2706 0.8562 0.8671 0.8608 0.8862 0.8985 0.8914 0.8905 0.9018 0.8953 0.9555 0.9678 0.9608 0.9167 0.9231 0.9167 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6923 0.7 0.7200
0.0001 12.6031 110000 0.3638 17.3342 0.8497 0.8605 0.8543 0.8779 0.8891 0.8826 0.8819 0.8914 0.8858 0.9556 0.9667 0.9602 0.9231 0.9231 0.9474 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6667 0.6667 0.6667
0.002 13.7489 120000 0.3561 17.2494 0.8465 0.8556 0.8503 0.8769 0.8865 0.8809 0.8808 0.8893 0.8843 0.9563 0.9659 0.9602 0.9167 0.9231 0.9231 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6923 0.7 0.7273
0.0012 14.8946 130000 0.3741 17.0587 0.8616 0.8756 0.8678 0.8932 0.9080 0.8998 0.8979 0.9111 0.9037 0.9561 0.9684 0.9615 0.9167 0.9231 0.9231 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6923 0.6923 0.6923
0.0015 16.0403 140000 0.3661 17.2070 0.8474 0.8603 0.8530 0.8775 0.8915 0.8837 0.8831 0.8940 0.8878 0.9521 0.9640 0.9572 0.9167 0.9231 0.9231 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6 0.6667 0.6316
0.0001 17.1861 150000 0.3762 17.4825 0.8490 0.8600 0.8538 0.8803 0.8921 0.8854 0.8861 0.8957 0.8901 0.9542 0.9628 0.9577 0.9167 0.9231 0.9167 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6 0.6667 0.6316
0.0 18.3318 160000 0.3845 16.9951 0.8579 0.8683 0.8623 0.8874 0.8981 0.8919 0.8927 0.9013 0.8962 0.9525 0.9614 0.9561 0.9231 0.9231 0.9286 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6667 0.7 0.6957
0.0 19.4775 170000 0.3866 16.7620 0.8581 0.8717 0.8641 0.8889 0.9034 0.8953 0.8935 0.9058 0.8988 0.9551 0.9667 0.9601 0.9231 0.9231 0.9333 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6 0.6667 0.6316
0.0 20.6233 180000 0.3849 16.5713 0.8577 0.8688 0.8624 0.8862 0.8977 0.8911 0.8917 0.9009 0.8955 0.9552 0.9643 0.9589 0.9231 0.9231 0.9412 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6154 0.7 0.6667
0.0 21.7690 190000 0.3930 16.5713 0.8611 0.8749 0.8671 0.8901 0.9044 0.8964 0.8959 0.9073 0.9007 0.9537 0.9644 0.9582 0.9231 0.9231 0.9286 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6154 0.7 0.6667
0.0 22.9148 200000 0.3963 16.4230 0.8621 0.8754 0.8679 0.8906 0.9042 0.8966 0.8954 0.9066 0.9002 0.9532 0.9639 0.9577 0.9231 0.9231 0.9333 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6154 0.7 0.6667

Framework versions

  • Transformers 4.41.2
  • Pytorch 2.2.1
  • Datasets 2.20.0
  • Tokenizers 0.19.1
Downloads last month
10
Safetensors
Model size
242M params
Tensor type
F32
·
Inference Examples
Automatic Speech Recognition
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for cantillation/Teamim-small_WeightDecay-0.05_Augmented_Old-Data_date-23-07-2024

Base model

openai/whisper-small
Finetuned
(2117)
this model

Evaluation results

Metadata error: specify a dataset to view leaderboard