ESPnet2 pretrained model, roshansh-cmu/how2_asr_raw_ft_sum_valid.acc.ave, fs=16k, lang=en
Creators: name: Sharma, Roshan
name: Palaskar, Shruti
name: Black, Alan W
name: Metze, Florian
Publisher: name: Zenodo
Year: 2022
Subjects: ESPnet
deep-learning
python
pytorch
speech-recognition
speech-synthesis
speech-translation
machine-translation
speech-summarization

description: This model was trained by roshansh-cmu using how2 recipe in espnet. <strong>Python API</strong> <pre><code class="language-python">See https://github.com/espnet/espnet_model_zoo</code></pre> <strong>Evaluate in the recipe</strong> <pre><code class="language-bash">git clone https://github.com/espnet/espnet cd espnet git checkout a743d338f8eafc155e77fa70c09b718fb458bbc3 pip install -e . cd egs2/how2/sum1 ./run.sh --skip_data_prep false --skip_train true --download_model roshansh-cmu/how2_asr_raw_ft_sum_valid.acc.ave</code> </pre> <strong>Results</strong> <pre><code> # RESULTS ## Environments - date: `Thu Feb 3 14:00:11 EST 2022` - python version: `3.8.5 (default, Sep 4 2020, 07:30:14) [GCC 7.3.0]` - espnet version: `espnet 0.10.0` - pytorch version: `pytorch 1.7.1` - Git hash: `c2b3f7ecc8d0640c828536abc3b5da038ecd7b50` - Commit date: `Wed Sep 29 22:14:50 2021 -0400` ## asr_raw_ft_sum ### WER |dataset|Snt|Wrd|Corr|Sub|Del|Ins|Err|S.Err| |---|---|---|---|---|---|---|---|---| |decode_sum_asr_model_valid.acc.best/dev5_test_sum|2127|69795|49.5|28.7|21.8|8.1|58.6|98.9| ### CER |dataset|Snt|Wrd|Corr|Sub|Del|Ins|Err|S.Err| |---|---|---|---|---|---|---|---|---| |decode_sum_asr_model_valid.acc.best/dev5_test_sum|2127|376390|61.5|14.7|23.8|8.9|47.4|98.9| ### TER |dataset|Snt|Wrd|Corr|Sub|Del|Ins|Err|S.Err| |---|---|---|---|---|---|---|---|---| |decode_sum_asr_model_valid.acc.best/dev5_test_sum|2127|128250|51.7|25.4|22.9|9.5|57.9|98.9|</code></pre> <strong>ASR config</strong> <pre><code>config: conf/train_asr_conformer_vid_lf.yaml print_config: false log_level: INFO dry_run: false iterator_type: sequence output_dir: exp/asr_raw_ft_sum ngpu: 1 seed: 0 num_workers: 1 num_att_plot: 3 dist_backend: nccl dist_init_method: env:// dist_world_size: 8 dist_rank: 0 local_rank: 0 dist_master_addr: localhost dist_master_port: 45875 dist_launcher: null multiprocessing_distributed: true unused_parameters: true sharded_ddp: false cudnn_enabled: true cudnn_benchmark: false cudnn_deterministic: true collect_stats: false write_collected_feats: false max_epoch: 100 patience: 10 val_scheduler_criterion: - valid - loss early_stopping_criterion: - valid - loss - min best_model_criterion: - - valid - acc - max keep_nbest_models: 10 grad_clip: 5.0 grad_clip_type: 2.0 grad_noise: false accum_grad: 10 no_forward_run: false resume: true train_dtype: float32 use_amp: false log_interval: 5000 use_tensorboard: true use_wandb: false wandb_project: null wandb_id: null wandb_entity: null wandb_name: null wandb_model_log_interval: -1 detect_anomaly: false pretrain_path: null init_param: - exp/asr_raw_utt_conformer/valid.acc.ave_10best.pth:::ctc ignore_init_mismatch: false freeze_param: [] num_iters_per_epoch: null batch_size: 20 valid_batch_size: null batch_bins: 60000000 valid_batch_bins: null train_shape_file: - exp/asr_stats_raw_vid_sum/train/speech_shape - exp/asr_stats_raw_vid_sum/train/text_shape.bpe valid_shape_file: - exp/asr_stats_raw_vid_sum/valid/speech_shape - exp/asr_stats_raw_vid_sum/valid/text_shape.bpe batch_type: length valid_batch_type: null fold_length: - 80000 - 150 sort_in_batch: descending sort_batch: descending multiple_iterator: false chunk_length: 500 chunk_shift_ratio: 0.5 num_cache_chunks: 1024 train_data_path_and_name_and_type: - - dump/raw/tr_2000h_sum_trim/wav.scp - speech - sound - - dump/raw/tr_2000h_sum_trim/text - text - text valid_data_path_and_name_and_type: - - dump/raw/cv05_sum_trim/wav.scp - speech - sound - - dump/raw/cv05_sum_trim/text - text - text allow_variable_data_keys: false max_cache_size: 0.0 max_cache_fd: 32 valid_max_cache_size: null optim: adam optim_conf: lr: 0.001 scheduler: reducelronplateau scheduler_conf: mode: min factor: 0.5 patience: 1 token_list: - - - '[hes]' - S - ▁THE - ▁TO - '''' - ▁AND - ▁YOU - ▁A - ▁IT - T - ▁THAT - ▁OF - ▁I - ▁IS - RE - ▁IN - ING - ▁WE - M - ▁GOING - ▁SO - ▁THIS - ▁YOUR - ▁ON - E - D - ▁BE - ▁CAN - N - Y - O - ER - ▁HAVE - ▁JUST - ▁FOR - ▁WITH - ▁DO - ED - ▁ARE - ▁WANT - ▁UP - R - LL - P - ▁ - L - B - ▁IF - C - ▁ONE - ▁S - ▁OR - A - ▁GO - ▁LIKE - ▁NOW - ▁HERE - VE - LE - U - ▁GET - ▁WHAT - ▁OUT - IN - W - ▁C - ▁LITTLE - ▁THERE - LY - ▁AS - ▁MAKE - I - ▁THEY - ▁MY - K - ▁THEN - ▁BUT - AL - G - ▁ALL - OR - ▁BACK - ▁NOT - ▁ABOUT - ▁RIGHT - ▁OUR - EN - ▁SOME - ▁DOWN - F - ▁WHEN - CH - ▁F - ▁HOW - AR - ▁WILL - ▁RE - CK - ▁G - ES - CE - ▁TAKE - ▁AT - ▁FROM - ▁WAY - TER - ▁SEE - RA - ▁USE - ▁REALLY - RI - TH - ▁TWO - ▁ME - ▁VERY - ▁E - ▁B - AT - ▁THEM - ▁DON - ▁AN - ▁BECAUSE - ▁MORE - RO - H - 'ON' - LI - ▁PUT - ▁ST - IL - ▁BIT - ▁START - ▁NEED - ▁INTO - UR - ▁TIME - ▁OVER - ▁W - ▁DE - ▁LOOK - ▁THESE - ▁LET - ▁GOOD - ▁ALSO - AN - ▁OFF - ▁HE - ▁KIND - ▁SIDE - ▁CO - ▁SURE - ▁AGAIN - ▁MA - ▁KNOW - IT - ▁WOULD - IC - ▁OTHER - LA - ▁P - ▁WHICH - '-' - IR - ▁LA - ▁HAND - EL - ▁LOT - ▁WHERE - ▁THREE - ▁PA - ION - LO - ▁KEEP - ▁SHOW - ▁THING - ▁FIRST - TE - ENT - ATE - ▁COME - AD - ▁GOT - NG - ▁NICE - ▁T - ET - ▁MO - ▁ANY - ▁ACTUALLY - ▁DIFFERENT - ▁SE - GE - ▁WORK - ▁THROUGH - ▁O - KE - V - ▁AROUND - ▁BA - PE - ▁HI - ▁BY - SH - ATION - ▁SU - ▁CA - ▁D - ▁LO - ▁HAS - ▁LI - ▁PLAY - Z - ▁ADD - ▁RO - ▁TA - AS - ▁FOUR - ▁CON - ▁THOSE - MP - NE - ▁SP - UT - ▁GIVE - ▁WELL - ▁BALL - TING - RY - X - ▁HO - INE - IVE - ▁NEXT - ▁PO - ▁STEP - ▁EVEN - TION - ▁MI - MENT - ▁CUT - ▁BO - ▁LINE - ▁MUCH - ▁THINGS - ▁TALK - UN - ▁PART - ▁WAS - ▁FA - ▁SOMETHING - PP - ANCE - ND - DI - ▁RA - AGE - ▁SAME - ▁EXPERT - ▁DOING - ▁LEFT - IST - ▁DI - ▁NO - RU - ME - TA - UL - TI - ▁VILLAGE - DE - ERS - ▁PEOPLE - ▁TURN - VER - ▁FL - ▁LEG - ▁ONCE - ▁COLOR - ▁PULL - ▁USING - VI - ▁WATER - ▁SHE - ▁TOP - ▁OKAY - ▁ANOTHER - ▁THEIR - ▁SAY - URE - ▁HA - ▁IMPORTANT - ▁PIECE - ▁FOOT - ▁TRA - ▁SC - ▁BODY - ▁SET - ▁POINT - ▁HELP - ▁TODAY - ▁BRING - ▁V - ▁END - MA - ▁CH - ▁MOST - ▁K - ▁AHEAD - ▁HER - OL - ▁SA - AM - IES - ▁THINK - ▁NAME - ▁TRY - ▁MOVE - ONE - ▁LE - ▁TOO - TO - UM - ▁PLACE - ▁COULD - ▁FIND - ▁FIVE - ▁ALWAYS - ID - TY - NT - ▁FEEL - ▁HEAD - ▁THAN - NA - ▁EX - ▁EYE - ITY - CI - OP - ▁SHOULD - ▁MIGHT - ▁HOLD - ▁CAR - AND - ▁GREAT - ▁RI - ▁BU - ▁HIGH - ▁OPEN - ▁BEFORE - US - ▁FRONT - ▁LONG - ▁TOGETHER - NI - ▁HAIR - ▁LIGHT - ▁TEN - ▁HIT - EST - OUS - ▁PRETTY - ▁TYPE - IP - CO - ▁FINGER - ▁JO - ▁UN - ▁PRO - ▁STRAIGHT - ▁BEHALF - ▁TI - ▁SIX - ▁CLEAN - ▁DIS - ▁DA - ▁POSITION - IGHT - ACT - ▁CHA - ▁PE - GG - AP - ▁MEAN - ▁COMP - FI - ▁KNEE - ▁CALLED - ▁HANDS - ▁PRE - ▁FORWARD - ▁AREA - ANT - ▁TE - ▁WA - ▁AFTER - ▁SMALL - ▁THROW - ▁EVERY - ▁SHOULDER - NC - PER - ▁MAYBE - ▁ABLE - ▁BASICALLY - ▁AM - ▁READY - ▁BOTTOM - IE - ▁HALF - FF - ▁BIG - ▁EACH - ▁PUSH - ▁EIGHT - ▁NEW - ▁DONE - ▁MAY - ▁GETTING - HO - ▁HIS - ▁HARD - ▁CLOSE - ALLY - ▁SECOND - ▁FEET - ICAL - ▁JA - ▁PAINT - ▁LEARN - ▁SOUND - HE - ▁ROLL - ▁ONLY - ▁DOESN - WA - ▁DRAW - ▁VI - ▁DID - ▁SHA - ▁CENTER - CU - ▁CLIP - ▁PI - ▁CARD - ▁INSIDE - ▁PERSON - ▁STILL - ▁MAKING - 'NO' - ▁EVERYTHING - . - ▁FUN - ARD - ▁REMEMBER - ▁AWAY - ATED - COM - ▁SEVEN - ▁BEEN - ▁MANY - ABLE - ▁DAY - ▁SIT - IZE - ▁REAL - ▁HIP - ▁BASIC - ▁KICK - ▁TU - ATING - ▁STICK - ▁FLAT - ▁WHO - END - HA - ▁EXP - ▁PICK - ▁MIX - ▁TRI - ▁BI - ▁WHOLE - ▁STRETCH - ▁BOTH - ▁PROBABLY - CA - ▁HIM - ▁STRING - ▁EDGE - ▁BASE - ▁COMING - UGH - ▁LIFT - ▁STA - ▁WORKING - ▁MU - ▁QUICK - ▁SOMETIMES - ▁HAPPEN - ▁YOURSELF - ▁TALKING - ▁DR - ▁TELL - ▁ANYTHING - ▁BRA - ▁LOOKING - ▁SLOW - ▁NE - ▁STAND - NER - ▁COMES - ▁GOES - ISE - BE - ▁USED - ▁UNDER - ▁BETWEEN - ▁HU - ▁CREATE - ▁NA - ▁USUALLY - ▁ARM - ▁DRY - ▁RUN - LING - ▁BRUSH - ▁COVER - ▁HEAR - ▁DOES - ▁STAY - ▁EN - ▁FOLD - ▁CHANGE - ▁LAST - ▁EASY - ▁US - ▁PER - ▁FACE - ▁EAR - ▁TIGHT - ▁FE - ▁PIN - ▁MAN - ▁BETTER - ▁CALL - ▁PRI - ▁BEST - ▁KI - ▁COUPLE - ▁WHILE - ▁SHAPE - ▁GAME - IV - ▁SHOT - ▁PAPER - ▁OWN - ▁ALRIGHT - ▁HAD - TIC - ▁BREATH - ▁TOOL - '2' - ▁ENOUGH - ▁COURSE - ▁SKIN - ▁SPIN - ▁VA - ▁ARMS - ▁TEA - ▁BREAK - ▁DOG - ▁1 - QUE - ▁DROP - ▁NUMBER - IG - ▁RED - ▁NOTE - ▁WEIGHT - WARD - ▁PLAYING - ▁FINISH - ▁MINUTE - ▁R - ▁PRESS - ▁EITHER - ▁CHE - ▁PU - BER - ▁FEW - ▁SIZE - ▁MADE - ▁LEAVE - ▁GA - ▁ALREADY - ▁GUY - ▁FAR - ▁HOME - ▁BAR - UP - ▁GRAB - ▁MARK - ▁WHITE - ▁PROPER - ▁CAUSE - ▁OK - ▁ART - HI - ▁SORT - ▁EXERCISE - ▁LOWER - PORT - ▁PLANT - ▁BOARD - ▁CASE - ▁YEAR - CENT - ▁DU - ▁CHECK - ▁WHATEVER - ▁OIL - ▁IDEA - ▁SIMPLE - ▁PRACTICE - ▁FAST - '0' - ▁CONTROL - ▁J - ▁KEY - ▁MIDDLE - ▁FULL - ▁GLASS - ▁OUTSIDE - ▁LOW - ▁REST - ▁STUFF - ▁ACT - ▁UNTIL - ▁BLACK - ▁POP - ▁CLICK - ▁HOLE - ▁Z - ▁COUNT - ▁POT - ▁ALLOW - ▁HAVING - ▁TRYING - ▁MUSCLE - ▁GU - ▁BOX - ▁NOTICE - ▁EXAMPLE - UND - ▁ALONG - FUL - ISH - ▁STORE - ▁LU - ▁FLOOR - ▁MOVING - ▁LARGE - ▁STOP - ▁PH - ▁WALK - '5' - ▁QU - ▁TECHNIQUE - ▁SOFT - ▁GROUND - ▁JUMP - ▁JU - ▁FILL - ▁WHY - ▁BUY - ▁GREEN - ▁WALL - ▁HEEL - NESS - ▁LEVEL - ▁UNDERNEATH - ▁PATTERN - ▁BEHIND - ▁OLD - ▁TIP - ▁COMPLETE - ▁WON - ▁TEACH - ▁FIT - ▁NECK - ▁REMOVE - ▁TRICK - ▁MOVEMENT - ▁TOWARDS - ▁PARTICULAR - ▁CHI - ▁EFFECT - J - ▁FREE - ▁ACROSS - ▁BEND - ▁SAFE - ▁SLIDE - ▁PROBLEM - ▁BLOCK - ▁PAN - ▁NATURAL - ▁TOUCH - ▁CHILD - LINE - ▁CROSS - ▁REASON - '4' - ▁POWER - ▁APPLY - ▁FOLLOW - ▁DESIGN - ▁SPACE - ▁ORDER - ▁WOOD - ▁RID - '3' - ▁COOK - ▁BEGIN - ▁WATCH - ▁STYLE - QUA - ▁PRODUCT - ▁TAKING - ▁PUTTING - ▁EXHALE - ▁THOUGH - ▁DEEP - IAN - ▁REACH - ▁FOOD - ▁ALMOST - ▁COOL - ▁SECTION - ▁SAID - ▁ANGLE - ▁MUSIC - ▁RELAX - ▁CORNER - ▁DARK - ▁CHORD - ▁ESPECIALLY - ▁SCALE - ▁WARM - ▁WITHOUT - ▁WHEEL - ▁SEGMENT - ▁TABLE - ▁BOOK - ▁PASS - ▁ELBOW - ▁ROUND - ▁INHALE - ▁SMOOTH - ▁ROOM - / - ▁NINE - ▁SHORT - ▁MEASURE - ▁LESS - ▁TWIST - ▁BALANCE - ▁PROCESS - ▁SWITCH - ▁GENERAL - ▁CLAY - ▁CERTAIN - ▁NEVER - ▁BLUE - ▁CUP - ▁HOUSE - ▁EXTRA - ▁MOTION - ▁PRESSURE - ▁FIRE - ▁SIMPLY - ▁DOUBLE - ▁TWENTY - ▁CATCH - ▁BECOME - ▁BUILD - ▁SPEED - ▁TRANS - ▁DRUM - ▁CHEST - ▁PICTURE - ▁LENGTH - ▁CONTINUE - ▁COMFORTABLE - ▁FISH - ▁PHOTO - ▁LOOSE - ▁SKI - ▁LIFE - ▁DEGREE - ▁OPTION - ▁WORD - ▁SHARP - ▁SHOOT - ▁FOUND - ▁STRONG - ▁QUITE - ▁THIRD - ▁GLUE - ▁MIND - ▁DEFINITELY - ▁EASIER - GRAPH - ▁HOOK - ▁CLEAR - ▁POSE - ▁BUTTON - ▁CHOOSE - ▁THICK - ▁SYSTEM - ▁PERFECT - ▁BEAUTIFUL - ▁SPOT - ▁GROW - ▁SIGN - ▁ELSE - ▁CONNECT - ▁SELECT - ▁PUNCH - ▁DIRECTION - ▁WRAP - ▁RELEASE - QUI - SIDE - ▁CAREFUL - ▁VIDEO - ▁INSTEAD - ▁CIRCLE - ▁WIRE - ▁NOSE - ▁AMOUNT - ▁FOCUS - ▁NORMAL - ▁MAJOR - ▁WHETHER - ▁SURFACE - ▁THUMB - ▁DRIVE - ▁SCREW - ▁POSSIBLE - ▁OBVIOUSLY - ▁COMMON - ▁REGULAR - ▁ADJUST - ▁WIDE - ▁BLADE - ▁FRET - ▁RECOMMEND - ▁BOWL - BOARD - ▁IMAGE - ▁DEPENDING - ▁PROTECT - ▁CLOTH - ▁HEALTH - ▁WRIST - ▁CLUB - ▁DRINK - ▁SINCE - ▁FRIEND - '00' - ▁RUNNING - ▁ITSELF - ▁RECORD - ▁SWING - ▁DIRECT - ▁MATERIAL - ▁YO - ▁LEAST - ▁EXACTLY - ▁BEGINNING - ▁SLIGHTLY - ▁TREAT - ▁CAMERA - ▁QUARTER - ▁WINDOW - '8' - ▁SOMEBODY - ▁BURN - ▁DEMONSTRATE - ▁DIFFERENCE - ▁COMPUTER - IBLE - ▁SHOE - ▁PERFORM - ▁SQUARE - ▁CONSIDER - ▁DRILL - ▁TEXT - ▁FILE - ▁RUB - ▁FABRIC - ▁HUNDRED - ▁GRIP - ▁CHARACTER - ▁SPECIFIC - ▁KNOT - ▁CURL - ▁STITCH - ▁BLEND - ▁FRAME - ▁THIRTY - '1' - ▁HORSE - ▁ATTACH - ▁GROUP - ▁STROKE - ▁GUITAR - ▁APART - ▁MACHINE - ▁CLASS - ▁COMB - ▁ROOT - ▁HELLO - ▁ENERGY - ▁ATTACK - ▁CORRECT - ▁EXTEND - ▁MINOR - ▁PROFESSIONAL - ▁MONEY - ▁STRIP - ▁FLAVOR - ▁EVERYBODY - ▁RULE - ▁DIFFICULT - ▁PROJECT - ▁DISCUSS - ▁FIGURE - ▁HOWEVER - ▁FINAL - ▁STRENGTH - ▁ENTIRE - ▁FIELD - ▁CONTACT - ▁SUPPORT - ▁PALM - ▁SERIES - ▁ENJOY - '6' - ▁WORLD - ▁DECIDE - ▁SPEAK - ▁SEVERAL - ▁WRITE - ▁PROGRAM - ABILITY - ▁KNIFE - ▁PLASTIC - ▁ORGAN - '7' - ▁UNDERSTAND - ▁FIFTEEN - ▁FLEX - ▁INFORMATION - ▁TWELVE - ▁DETAIL - ▁STRIKE - ▁ACTUAL - ▁SPRAY - ▁LOCAL - ▁MOUTH - ▁NIGHT - ▁VEHICLE - ▁OPPOSITE - ▁SCHOOL - '9' - ▁QUESTION - ▁SPECIAL - ▁BIGGER - ▁DEVELOP - ▁PEPPER - ▁PREFER - Q - '%' - ']' - '[' - '&amp;' - ',' - _ - '#' - '=' - '@' - + - '*' - $ - '~' - init: null input_size: null ctc_conf: ignore_nan_grad: true model_conf: ctc_weight: 0.0 lsm_weight: 0.15 length_normalized_loss: false use_preprocessor: true token_type: bpe bpemodel: data/en_token_list/bpe_unigram1000/bpe.model non_linguistic_symbols: data/nlsyms cleaner: null g2p: null speech_volume_normalize: null rir_scp: null rir_apply_prob: 1.0 noise_scp: null noise_apply_prob: 1.0 noise_db_range: '13_15' frontend: default frontend_conf: n_fft: 512 hop_length: 256 fs: 16k specaug: specaug specaug_conf: apply_time_warp: true time_warp_window: 5 time_warp_mode: bicubic apply_freq_mask: true freq_mask_width_range: - 0 - 30 num_freq_mask: 2 apply_time_mask: true time_mask_width_range: - 0 - 40 num_time_mask: 2 normalize: global_mvn normalize_conf: stats_file: exp/asr_stats_raw_vid_sum/train/feats_stats.npz preencoder: null preencoder_conf: {} encoder: conformer encoder_conf: output_size: 512 attention_heads: 8 linear_units: 2048 num_blocks: 12 dropout_rate: 0.1 positional_dropout_rate: 0.1 attention_dropout_rate: 0.1 input_layer: conv2d normalize_before: true macaron_style: true pos_enc_layer_type: abs_pos selfattention_layer_type: lf_selfattn activation_type: swish use_cnn_module: true cnn_module_kernel: 31 attention_windows: - 40 - 40 - 40 - 40 - 40 - 40 - 40 - 40 - 40 - 40 - 40 - 40 attention_dilation: - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 attention_mode: tvm decoder: transformer decoder_conf: attention_heads: 4 linear_units: 512 num_blocks: 6 dropout_rate: 0.15 positional_dropout_rate: 0.15 self_attention_dropout_rate: 0.15 src_attention_dropout_rate: 0.15 required: - output_dir - token_list version: 0.10.0 distributed: true</code></pre> <strong>LM config</strong> <pre><code>NONE</code></pre>