renwei2024/nemo-github-issues huggingface dataset:
Wav2Vec2 Fine-tuning Tutorial
State: closed | Pull request: no

Hi,

As you currently don't have a tutorial for training wav2vec2 #2342, I tried to create a new yaml file to train wav2vec2.
However, I didn't succeed and I need your guidance and help.

I have created a Colab example that replicate my experiment here:
https://colab.research.google.com/drive/1kJQA0wm7EVLGsDlZmhCv-2siXXJvTpUt?usp=sharing

This is the current error:
```
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
[NeMo I 2021-06-25 14:10:52 exp_manager:216] Experiments will be logged at /content/nemo_experiments/Wav2vec_Pretrain/2021-06-25_14-10-52
[NeMo I 2021-06-25 14:10:52 exp_manager:563] TensorboardLogger has been set up
Traceback (most recent call last):
  File "wav2vec_pretrain.py", line 97, in <module>
    main()  # noqa pylint: disable=no-value-for-parameter
  File "/usr/local/lib/python3.7/dist-packages/nemo/core/config/hydra_runner.py", line 103, in wrapper
    strict=None,
  File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 347, in _run_hydra
    lambda: hydra.run(
  File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 201, in run_and_report
    raise ex
  File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 198, in run_and_report
    return func()
  File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 350, in <lambda>
    overrides=args.overrides,
  File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/hydra.py", line 112, in run
    configure_logging=with_log_configuration,
  File "/usr/local/lib/python3.7/dist-packages/hydra/core/utils.py", line 127, in run_job
    ret.return_value = task_function(task_cfg)
  File "wav2vec_pretrain.py", line 92, in main
    wav2vec_encoder_model = Wav2VecEncoderModel(cfg=cfg.model, trainer=trainer)
  File "/usr/local/lib/python3.7/dist-packages/nemo/collections/asr/models/wav2vec/wav2vec_model.py", line 54, in __init__
    self.global_rank = 0
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1178, in __setattr__
    object.__setattr__(self, name, value)
AttributeError: can't set attribute
```

Your feedback is highly appreciated.
Fine-tune Citrinet config file
State: closed | Pull request: no

I am following tutorial "08_ASR_with_Subword_Tokenization.ipynb" to fine-tune Citrinet. 

My question is: which configuration file should I modify for fine-tuning: 
option 1: the original config file of the model obtained from **asr_model.cfg** 
option 2: download **config_bpe** [here ](https://github.com/NVIDIA/NeMo/blob/main/examples/asr/conf/citrinet/config_bpe.yaml)

Note, I want to use "**stt_en_conformer_ctc_large**"  model. 

Thanks in advance!
No pad_token,eos_token,sep_token,cls_token in SentencePieceTokenizer
State: closed | Pull request: no

https://github.com/NVIDIA/NeMo/blob/70987d1c09776550304d76d3651f1e41ffe90a84/nemo/collections/common/tokenizers/sentencepiece_tokenizer.py#L191.   

Hi I find actually no pad_token,eos_token,sep_token,cls_token in SentencePieceTokenizer's property.
NMT bottleneck
State: closed | Pull request: yes

This PR adds a bottleneck architecture to NMT models, along with support for training VAE and MIM latent variable models.

# Summary

The bottleneck class (MTBottleneckModel) supports two architectures:
  * No bottleneck: model_type=seq2seq, the usual NMT mdoel.
  * Fixed-size bottleneck: model_type in [seq2seq-br, seq2seq-mim, seq2seq-vae], where the output of the encoder is projected to a fixed number of steps.

The projection to a fixed number of steps is based on the paper <https://arxiv.org/pdf/1703.03130.pdf>.
The idea is to use K attention heads to compute K weighted-average hidden states, projecting a variable number of steps into K steps.

The bottleneck variants offer different losses:
  * The loss of seq2seq-br is reconstruction only (like seq2seq).
  * The loss of seq2seq-mim is reconstruction + latent entropy minimization  (See MIM, <https://arxiv.org/pdf/2003.02645.pdf>).
  * The loss of seq2seq-vae is reconstruction + latent entropy regularization (See <https://arxiv.org/pdf/1312.6114.pdf>) correspondingly.

# YAML Configuration

The following configurations were added to the YAML config:

```
model:
  model_type: 'seq2seq-br' # supports seq2seq, seq2seq-br, seq2seq-mim, seq2seq-vae (see description above)
  min_logv: -8 # minimal allowed logv for seq2seq-mim
  ortho_loss_coef: 0.0 # orthogonality coefficient for attention bridge
  att_bridge_size: 512 # dimension of a step in attention bridge
  att_bridge_k: 16 # fixed number of steps in attention bridge
  att_bridge_inner_size: 1024 # feedforward size in attention bridge
  non_recon_warmup_batches: 200000 # warm-up steps for seq2seq-mim, seq2seq-vae
  recon_per_token: true # when false reconstruction is computed per sample, not per token
```

* ortho_loss_coef - encou
* non_recon_warmup_batches - anneals the KL divergence term (for VAE) or latent entropy term (for MIM)
* recon_per_token - if false, loss is computed per sample (i.e., summed over all tokens), otherwise loss is averaged per token (the default behaviour).

# Usage

See usage example below, training a seq2seq-br with 32 steps.

NOTE: max_generation_delta must be big enough to allow the generation of the longest sequence given the chosen bottleneck size.

```
python -- examples/nlp/machine_translation/enc_dec_nmt-bottleneck.py \
      --config-name=aayn_bottleneck \
      ...
      model.max_generation_delta=256 \
      ...
      model.model_type=seq2seq-br \
      model.att_bridge_size=1024 \
      model.ortho_loss_coef=0.0 \
      model.att_bridge_k=32 \
      model.att_bridge_inner_size=1024 \
      model.recon_per_token=true \
      model.non_recon_warmup_batches=150000 \
      ...
```

# Additional Info

* The attention bridge class (AttentionBridge) was added to ```nemo/collections/nlp/modules/common/transformer/transformer_modules.py```.
* The bottleneck class (MTBottleneckModel) supports logging of various loss terms.
Gradient overflow
State: closed | Pull request: no

**Describe the bug**

I'm receiving these warnings and errors when I'm training Quartznet using spanish dataset at A100:

***Warnings:***
- Could not import torchaudio. Some features might not work.  warnings.warn('Could not import torchaudio. Some features might not work.')libibverbs: 
- Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs0

***Error***:
- Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 1.0

**Environment overview (please complete the following information)**

 - Docker image: https://ngc.nvidia.com/catalog/containers/nvidia:nemo
 - Using Enroot to start the instance
 - Following the steps of: Spanish__Cross_Language.ipynb
 - NeMo Parameters:
     - nproc_per_node=8
     - batch_size=32
     - amp_opt_level = 'O1'
 - Dataset: Mozilla Common Voice - Spanish Version 6.1
 - Running at a DGX A100 80G
Citrinet finetuning ? 
State: closed | Pull request: no

I want to finetune pre-trained citrinet-1024 for my **8Khz** data set. What steps should I follow? 
I did these steps
1) tokenizer traininig with my train data
But then 
2) asr_model = nemo_asr.models.EncDecCTCModelBPE.from_pretrained(model_name="stt_en_citrinet_1024") does not see my config file and gives error
3) then i created the citrinet model from scratch using config_bpe.yaml file,it gives very high WERs.

What should i do for this task ?
Erratic learning rate regarding FastSpeech2 (NoamAnnealing) and HiFi-GAN
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

Given the default configuration of fastspeech2 and HifiGAN, their implemented learning rates deviate severely from what's reported in their respective original papers.

**Steps/Code to reproduce bug**

Preprocess LJSpeech as specified by the official NeMo example, 
python fastspeech2.py (or hifigan.py)

**Expected behavior**
![expected](https://user-images.githubusercontent.com/37045193/122874113-95059880-d36d-11eb-8f87-7f35e06fb232.png)


**What is observed**
FastSpeech2
![observed](https://user-images.githubusercontent.com/37045193/122874437-f4fc3f00-d36d-11eb-8f53-5a76ca1e46f1.png)

HiFiGAN
![observed_hifigan](https://user-images.githubusercontent.com/37045193/122874450-f88fc600-d36d-11eb-85ab-a4e94f24462f.png)

We believe the error comes from the missing scaler, which needs to be 1000. given the default model dimension 256.
mult = self._normalize * min(step ** (-0.5), step * (self.warmup_steps ** (-1.5)))
mult = 1000. * self._normalize * min(step ** (-0.5), step * (self.warmup_steps ** (-1.5)))
Update ASR scripts for tokenizer building and tarred dataset building
State: closed | Pull request: yes

# Changelog
- Update docker container to nemo:1.0.1 
- Add Citrinet 1024 Gamma 0.25 model card for Mandarin to CTC char models
- Update tokenizer scripts to support adding bos, eos and pad tokens to SentencePiece tokenizers via `--spe_bos`, `--spe_eos` and `--spe_pad` flags.
- Update dataset building script to always provide a max length when building tarred datasets.

Signed-off-by: smajumdar <titu1994@gmail.com>
BERTLMModel not serializable?
State: closed | Pull request: no
Labels: name: feature

Hello, I tried to run a BERT pre-training in a distributed fashion with [Ray](https://github.com/ray-project), more specifically using [ray_lightning](https://github.com/ray-project/ray_lightning) as a plugin. Apparently, the BERTLMModel cannot be serialized (stack trace below)
```
Traceback (most recent call last):
  File "models/bert-nvidia/bert_pretraining.py", line 34, in main
    trainer.fit(bert_model)
  File "/home/mallonus/tunbert-pip/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 499, in fit
    self.dispatch()
  File "/home/mallonus/tunbert-pip/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 546, in dispatch
    self.accelerator.start_training(self)
  File "/home/mallonus/tunbert-pip/lib/python3.8/site-packages/pytorch_lightning/accelerators/accelerator.py", line 73, in start_training
    self.training_type_plugin.start_training(trainer)
  File "/home/mallonus/tunbert-pip/lib/python3.8/site-packages/ray_lightning/ray_ddp.py", line 213, in start_training
    results = self.execution_loop(trainer, tune_enabled=False)
  File "/home/mallonus/tunbert-pip/lib/python3.8/site-packages/ray_lightning/ray_ddp.py", line 180, in execution_loop
    model_ref = ray.put(model)
  File "/home/mallonus/tunbert-pip/lib/python3.8/site-packages/ray/_private/client_mode_hook.py", line 62, in wrapper
    return func(*args, **kwargs)
  File "/home/mallonus/tunbert-pip/lib/python3.8/site-packages/ray/worker.py", line 1527, in put
    object_ref = worker.put_object(value)
  File "/home/mallonus/tunbert-pip/lib/python3.8/site-packages/ray/worker.py", line 280, in put_object
    serialized_value = self.get_serialization_context().serialize(value)
  File "/home/mallonus/tunbert-pip/lib/python3.8/site-packages/ray/serialization.py", line 326, in serialize
    return self._serialize_to_msgpack(value)
  File "/home/mallonus/tunbert-pip/lib/python3.8/site-packages/ray/serialization.py", line 306, in _serialize_to_msgpack
    self._serialize_to_pickle5(metadata, python_objects)
  File "/home/mallonus/tunbert-pip/lib/python3.8/site-packages/ray/serialization.py", line 266, in _serialize_to_pickle5
    raise e
  File "/home/mallonus/tunbert-pip/lib/python3.8/site-packages/ray/serialization.py", line 262, in _serialize_to_pickle5
    inband = pickle.dumps(
  File "/home/mallonus/tunbert-pip/lib/python3.8/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 73, in dumps
    cp.dump(obj)
  File "/home/mallonus/tunbert-pip/lib/python3.8/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 580, in dump
    return Pickler.dump(self, obj)
NotImplementedError: object proxy must define __reduce_ex__()
Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
```
I was wondering if there was a quick work-around about this and if it's possible for me to fix it and still use the [BERT pre-training script](https://github.com/NVIDIA/NeMo/blob/main/examples/nlp/language_modeling/bert_pretraining.py). Any help will be much appreciated, thanks in advance!
Fine-tuning QuartzNet 15*5 with multiple speaker audio dataset
State: closed | Pull request: no

Hi,
Can I use Persian Multiple speaker audio datasets for fine-tuning QuartzNet15*5? Will be the WER low as I fine-tune it with a single speaker dataset?
Thanks


**Environment overview**

- Environment location: Google Colab
- Method of NeMo install: !pip install nemo_toolkit[asr]
- NeMo version: 1.0.0
- Learning Rate: 1e-3

**Environment details**

- OS version : "Ubuntu20.04.3 LTS"
- PyTorch version : "1.7.1"
Add FastEmit support for RNNT Losses
State: closed | Pull request: yes

# Changelog
- Adds Numpy + CPU + Numba CUDA support for FastEmit regularization from the paper [FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularization](https://arxiv.org/abs/2010.11148)

# Usage

```yaml
model:  # RNNT Model config
  ...
  loss:
    loss_name: "warprnnt_numba"
    warprnnt_numba_kwargs:
      fastemit_lambda: 0.0
```
Script for ASR inference on long files
State: closed | Pull request: yes

* speech_to_text_buffered_infer.py  supports inference on long audio files by running inference on smaller chunks of audio and them merging the tokens for final transcription.
* This version only supports EncDecCTCModelBPE with AudioMelSpectrogramProcessor as the preprocessor
* Feature normalization is done on each buffer using "per_feature" normalization, future versions will have other methods for experimentation
Speaker Biometrics
State: closed | Pull request: no
Labels: name: feature

**Hi, how are you? will you be able to help me with this question...**
About the "Speaker Recognition Verification", after all the training process you give us the option to extract the Embeddings from the speakers, but we would like to know if we can extract the speaker ID, we also look at your other scripts the ones from "Speaker Diarization Inference" and "ASR with Speaker Diarization" which also don't reach the point we want, would you have an example of how I can get a speaker's biometrics where I can calculate the probability rate of that speaker who is speaking in the audio really be himself?
How to export Torch checkpoint in .NeMo
State: closed | Pull request: no
Labels: name: feature

Hi,

I'm using the container https://ngc.nvidia.com/catalog/containers/nvidia:nemo_asr_app_img and running the step by step from the notebook 'English to Spanish Cross Language Finetuning'. I did the training and everything works perfectly.
At the end of training as a result we have checkpoints that can be used for decoding in Jasper.
Is there a possibility to export these checkpoints in model .NeMo?

![image](https://user-images.githubusercontent.com/49459897/122222165-6298fe80-ce88-11eb-8014-3b7d0427ab5f.png)

Since most of the tutorials I find in the documentation are for .NeMo models, I was left with this question
byt5 unicode implementation
State: closed | Pull request: yes

This PR adds:

* a straightforward implementation of unicode encoding & decoding from [ByT5](https://arxiv.org/abs/2105.13626)
* a very bare-bones text processor for use with this tokenizer that can support any language.

Usage:

To enable this, add the following lines to your nemo launch script:

```
model.encoder_tokenizer.library='byte-level'
model.decoder_tokenizer.library='byte-level'
```


Things work as-is with tarred datasets, but you will need to re-create your datasets using the `byte-level` tokenizer class.

Currently, the following model, which is 260M params, gets within 1 BLEU (18.9) of my transformer-large run on wmt20 ja-en (19.8):

```
    model.encoder_tokenizer.library='byte-level' \
    model.decoder_tokenizer.library='byte-level' \
    model.encoder.max_sequence_length=1024 \
    model.encoder.hidden_size=1536 \
    model.encoder.inner_size=4096 \
    model.encoder.num_attention_heads=16 \
    model.encoder.attn_layer_dropout=0.1 \
    model.encoder.ffn_dropout=0.1 \
    model.encoder.num_layers=10 \
    model.encoder.pre_ln=true \
    model.decoder.max_sequence_length=1024 \
    model.decoder.hidden_size=1536 \
    model.decoder.inner_size=4096 \
    model.decoder.num_attention_heads=16 \
    model.decoder.attn_layer_dropout=0.1 \
    model.decoder.ffn_dropout=0.1 \
    model.decoder.num_layers=2 \
    model.decoder.pre_ln=true \
    model.src_language=ja \
    model.tgt_language=en \
    model.beam_size=3 \
    model.max_generation_delta=3 \
    model.label_smoothing=0.1 \
    model.optim.lr=4e-4 \
    +model.optim.sched.warmup_steps=35000 \
    ~model.optim.sched.warmup_ratio \
```
New release of Torch
State: closed | Pull request: no
Labels: name: bug

A few hours ago the release of Torch 1.9.0 and Torchaudio 0.9.0 was released and since then the notebooks are not working on Colab. As the scripts don't specify the exact version number, pip is installing the latest one.

An example of this error is in the notebook "Speaker Diarization Inference":
![MicrosoftTeams-image](https://user-images.githubusercontent.com/49459897/122088598-43449780-cddc-11eb-86bb-d5878ad222c8.png)

I used Torch versions 1.8.0 and torchaudio versions 0.8.0 and the problems were fixed.
![MicrosoftTeams-image (1)](https://user-images.githubusercontent.com/49459897/122088730-6a02ce00-cddc-11eb-9a07-cdb00fbc6302.png)

I found it important to report this bug to help new users.
Audios length - Training
State: closed | Pull request: no
Labels: name: feature

I want to know if it is necessary that the audios should be less than 25 seconds in the training set? I have created the manifest and in "duration" if the number is bigger than 25 it doesn't work and gives an error while setting up the training data. I am not sure what's going wrong. I need help!

I get this error when the duration number is set to a bigger number: ValueError: num_samples should be a positive integer value, but got num_samples=0
[Question] Low accuracy in VAD when inference?
State: closed | Pull request: no

I trained VAD model using my own dataset follow instructions in [tutorial 6](https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/06_Voice_Activiy_Detection.ipynb)
My dataset is Vietnamese telephony conversation, 15 hours with 8 hours of speech and 7 hours of noise. I got accuracy 95% on my test set.
The format of dataset quite similar with google speech dataset. Each sample of speech is one statement and have duration 2s to 5s.

When i follow instruction in [tutorial 7](https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/07_Online_Offline_Microphone_VAD_Demo.ipynb) to inference sample in test set what contain only one statement, the prediction is good. The problem is when i inference long audio has duration 20s of speech and noise, the prediction is really bad.

This is predict sample of speech has only one statement
![foo](https://user-images.githubusercontent.com/83386486/121889898-f44b2380-cd43-11eb-9a6f-6b5abf18b1e9.png)

And long audio, 0 to 5s of audio is noise.
![bar](https://user-images.githubusercontent.com/83386486/121889893-f2816000-cd43-11eb-8dc0-929b4f2af029.png)

1. Any idea to deal with this problem
2. I want to tune threshold but it required RTTM ground truth, what is the format of RTTM file and how to generate this file from dataset like google speech.
3. How to tune best window size with my own dataset
Speech Synthesis (TTS) training in another language
State: closed | Pull request: no

I'd like to train my own TTS model in Italian, using the portion of the MAILABS dataset (~18 hours). If I train the Tacotron 2 model with this dataset (16 kHz) can I fine tune from a pre-existing English model? I think that would increase performance but I didn't find any tutorial or example that performs it. Another question is whether vocoder training would be necessary or whether the English/Chinese pre-trained vocoders give acceptable performance even in a different language such as Italian. Last question is if the two end-to-end models, FastPitch_HifiGan_E2E and FastSpeech2_HifiGan_E2E, can be trained or fine-tuned from pre-existing checkpoints avoiding the need to train both Spectrogram Generator and Vocoder.
Thanks in advance for your answers
Incorrect Citrinet transcription
State: closed | Pull request: no

**Describe the bug**
I am trying to transcribe an audio file with following ground truth:
**in the late 1980s switzerland witnessed a huge increase**
and getting the following prediction by Citrinet:
**in the late one thousand nine hundred eighty seconds switzerland witnessed a huge increase**

**Steps/Code to reproduce bug**
NeMo version : 1.0.0rc1
Enviroment: Nvidia-docker
Inference script: https://github.com/NVIDIA/NeMo/blob/v1.0.0rc1/examples/asr/transcribe_speech.py

**Expected behavior**
Seems like a bug in text-normalization that wrongly normalizes 1980s to one thousand nine hundred eighty seconds

**Environment overview (please complete the following information)**

 - Environment location:  Nvidia-Docker
 
 - Docker run command: docker run -it  nvcr.io/nvidia/nemo:1.0.0b1, installed nemo-1.0.0rc1 using reinstall.sh

**Additional context**
GPU model: RTX 2080
How to load local pretrain .ckpt in config file
State: closed | Pull request: no

 I changed the diarizer.vad.model_path in [Config file ](https://github.com/NVIDIA/NeMo/blob/main/examples/speaker_recognition/conf/speaker_diarization.yaml) to use my own trained models. But it required .nemo

Is posible to use .ckpt model in Config file.
Many thanks.
Fixed Hydra-core configuration
State: closed | Pull request: yes

Hydra-core version 1.1.0  was released on 11th June 2021, and it requires OmegaConf == 2.1.\*, while Nemo requires OmegaConf >=2.0.5 & <2.1.*. Therefore, the existing tutorial notebooks( can be found [here](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/starthere/tutorials.html), on importing the models, reported a configuration error. This pull request changed the specified version of hydra-core, specified in the requirements, from >=1.0.4 to ==1.0.6, which supports OmegaConf <2.1.0, and the tutorial notebooks run correctly, without any error
Error while import Models
State: closed | Pull request: no
Labels: name: bug

If I run following code:
`punctuation_model = PunctuationCapitalizationModel.from_pretrained("punctuation_en_bert")`
 I get this error:

```
ConfigAttributeError                      Traceback (most recent call last)
<ipython-input-2-e68f34283879> in <module>()
     15 import time
     16 translate_model = MTEncDecModel.from_pretrained("nmt_de_en_transformer12x2")
---> 17 punctuation_model = PunctuationCapitalizationModel.from_pretrained("punctuation_en_bert")
     18 qa_model = QAModel.from_pretrained("qa_squadv1.1_bertbase")
     19 #nmt_model = nemo_nlp.models.MTEncDecModel.from_pretrained(model_name='nmt_ru_en_transformer6x6').cuda()

14 frames
/usr/local/lib/python3.7/dist-packages/omegaconf/dictconfig.py in _get_node(self, key, validate_access, throw_on_missing_value, throw_on_missing_key)
    468         if value is None:
    469             if throw_on_missing_key:
--> 470                 raise ConfigKeyError(f"Missing key {key}")
    471         elif throw_on_missing_value and value._is_missing():
    472             raise MissingMandatoryValue("Missing mandatory value")

ConfigAttributeError: Missing key update_node
    full_key: update_node
    object_type=dict
```
It happens since today. Yesterday it worked fine that way
A clear and concise description of what the bug is.

Normally it should just import the models so I am able to use them. Here is my insall code

```
BRANCH = 'v1.0.0'
!python -m pip install git+https://github.com/NVIDIA/NeMo.git@$BRANCH#egg=nemo_toolkit[all]
```
and my imports:

```
import nemo
import nemo.collections.nlp as nemo_nlp
import nemo.collections.tts as nemo_tts
from nemo.collections.nlp.models import MTEncDecModel
import numpy as np
import IPython.display as ipd
from IPython.display import Audio 
from IPython.core.display import display
from nemo.collections.nlp.models import PunctuationCapitalizationModel
from nemo.collections.nlp.models import QAModel
from nemo.collections.nlp.models import TokenClassificationModel
```
It is executed on a Google Collab with 14 GB of VRAM
Wav2Vec2 Fine-tuning Tutorial
State: closed | Pull request: no

I tried to find an example for fine-tuning wav2vec2 but I couldn't.
I was able only to find the pre-training script.

Could you please share an example for fine-tuning wav2vec2? 
https://github.com/NVIDIA/NeMo/blob/ee480c1a7905a6488ede29c9d892b6be3038759f/examples/asr/experimental/wav2vec/wav2vec_pretrain.py
FineTune ASR with freeze encoder
State: closed | Pull request: no
Labels: name: feature

Hello!

I try FineTune ASR with freeze encoder on Russian language.

I am taking the parameters of the optimizer, learning rate from the this paper
https://arxiv.org/pdf/2005.04290.pdf and i set warmup_steps the value to 2000, but when training the model, the loss value is "buried", and WER is in one. 
What actions to take to solve this problem?

model_weights.ckpt is unpack stt_en_jasper10x5dr.nemo model
code for training:
```
@hydra_runner(config_path="./", config_name="jasper_10x5dr.yaml")
def main(cfg):
    logging.info(f'Hydra config: {OmegaConf.to_yaml(cfg)}')

    trainer = pl.Trainer(**cfg.trainer, amp_level='O1')
    exp_manager(trainer, cfg.get("exp_manager", None))
    asr_model = EncDecCTCModel(cfg=cfg.model, trainer=trainer)
    # Initialize the weights of the model from another model, if provided via config
    
    state_dict = torch.load('model_weights.ckpt' , map_location='cpu')
    encoder_state_dict = {key[8:] : value for key,value in state_dict.items() if key.split('.')[0] == 'encoder'}
    asr_model.encoder.load_state_dict(encoder_state_dict)
    del state_dict
    del encoder_state_dict
    torch.cuda.empty_cache()
    asr_model.encoder.freeze()

    trainer.fit(asr_model)

    if hasattr(cfg.model, 'test_ds') and cfg.model.test_ds.manifest_filepath is not None:
        gpu = 1 if cfg.trainer.gpus != 0 else 0
        test_trainer = pl.Trainer(
            gpus=gpu,
            precision=trainer.precision,
            amp_level=trainer.accelerator_connector.amp_level,
            amp_backend=cfg.trainer.get("amp_backend", "native"),
        )
        if asr_model.prepare_test(test_trainer):
            test_trainer.test(asr_model)
if __name__ == '__main__':
    main()

```
speech_to_text.py doesn't work
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

I am trying to run the "speech_to_text.py " but it fails and gives the following error:
```
  File "speech_to_text.py", line 88, in main
    asr_model.maybe_init_from_pretrained_checkpoint(cfg)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 948, in __getattr__
    type(self).__name__, name))
AttributeError: 'EncDecCTCModel' object has no attribute 'maybe_init_from_pretrained_checkpoint'
```

**Steps/Code to reproduce bug**

The whole code is accessible through this Colab notebook:
https://colab.research.google.com/drive/1Z5d7KL_LOWQthozKAPnBpubN8Ub3ZuZ3?usp=sharing


**Expected behavior**

I am expecting the model to work and load a pretrained model.

 - Environment location: Google Colab
 - Method of NeMo install: pip install nemo_toolkit['all']

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version: Ubuntu
- PyTorch version:  1.8
- Python version:  3.7

**Additional context**
I am trying to finetune a pretrained QuartzNet15x5.