renwei2024/nemo-github-issues huggingface dataset:
Added check for microbatch calculator
State: closed | Pull request: yes

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Checks if micro batch calculator has already been initialized before calling apex  micro batch setup function. 

**Collection**: BigNLP

# Changelog 
- Changed megation_init.py

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Drop support for kaldi-io
State: closed | Pull request: yes

Signed-off-by: smajumdar <titu1994@gmail.com>

# What does this PR do ?

Remove all support for kaldi-io.
Reason - very old library that has poor logging format and is very rarely used.

**Collection**: [ASR]

# Changelog 
- Remove any support for kaldi-io
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
[TTS] Restore_buffer bug fix and update NeMo checkpoint URL
State: closed | Pull request: yes

Signed-off-by: Subhankar Ghosh <subhankar2321@gmail.com>

# What does this PR do ?

Removes the requirement of 'max_token_duration', 'min_token_duration' and 'zero_emb' to be restored from the state_dict. This PR sets persistent=False in register_buffer for the above variables.
This PR also sets the correct URL to download the latest Fastpitch checkpoint.

**Collection**: [Note which collection this PR will affect]

1) nemo.collection.tts.models
2) nemo.collection.tts.modules

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [x] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Patch commons.py
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: Core

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Force import test on PR
State: closed | Pull request: yes

Signed-off-by: smajumdar <titu1994@gmail.com>

# What does this PR do ?

Tries to force the CI to perform import tests

**Collection**: [Core]

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.
Merge r1.8.0 main
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Conformer CTC occasionally returns transcripts with multiple consecutive spaces
State: closed | Pull request: no
Labels: name: bug

While analysing datasets via [SpeechDataExplorer](https://github.com/NVIDIA/NeMo/tree/main/tools/speech_data_explorer) I discovered that transcripts (in our case generated with Conformer CTC BPE) occasionally contain multiple consecutive spaces. This influences the computation of CER, but is also very confusing because the `text diff` section in SDE does not show any visual differences. The `text diff` also fails to show differences for UTF-8 symbols that appear in different versions in text and pred_text (composed and decomposed form), but the difference is correctly taken into account when computing WER and CER, hence this becomes confusing because there is no visual difference between the two transcripts. 

Tracking down the source for multiple consecutive spaces I see that this can occur if a `blank_id` token separates consecutive space characters: https://github.com/NVIDIA/NeMo/blob/91e6c2552a9ca8d00bf8ce758adea68d9c49f803/nemo/collections/asr/metrics/wer.py#L162-L165

Perhaps following Huggingface `tokenizer.decode` a flag `clean_up_tokenization_spaces` with the default value of `True` could be added to `decode_tokens_to_str` and `decode_ids_to_tokens`. The sole purpose of this flag would be to clean up the resulting data of multiple consecutive space symbols. If you agree I can make a PR.
T5 finetuning for generic small text-to-text datasets
State: closed | Pull request: yes

# What does this PR do ?

Adds the ability to fine-tune to T5 models on generic and small text-to-text datasets provided a two separate text files (one for source and one for target).

**Collection**: NLP

# Changelog 
- Create a new MegatronT5FinetuneModel class that inherits from MegatonT5Model. MegatronGLUEModel now inherits from MegatronT5FinetuneModel and only overrides `build_train_valid_test_datasets()`. 

# Usage
```python 
      python megatron_t5_seq2seq_finetune.py \
      -cn megatron_t5_finetuning \ 
      model.data.train_ds.src_file_name=/raid/Data/KLUE/klue-nli-v1.1/train_knli_src.txt \ 
      model.data.train_ds.tgt_file_name=/raid/Data/KLUE/klue-nli-v1.1/train_knli_tgt.txt \ 
      model.data.validation_ds.src_file_name=/raid/Data/KLUE/klue-nli-v1.1/validation_knli_src.txt \ 
      model.data.validation_ds.tgt_file_name=/raid/Data/KLUE/klue-nli-v1.1/validation_knli_tgt.txt \ 
      model.restore_from_path=/path/to/your.nemo \
      trainer.devices=2 \
      trainer.val_check_interval=100 \
      trainer.max_epochs=10
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Prompt learning Notebook
State: closed | Pull request: yes

# What does this PR do ?
This adds a tutorial notebook on how to use p-tuning and prompt-tuning within NeMo. It also updates the NeMo Documentation page and adds some example data preprocessing scripts. 

**Collection**: BigNLP

# Changelog 
- Add Multitask prompt and p-tuning notebook
- Add squad, intent and slot, and financial sentiment analysis dataset preprocessing example scripts
- Updates documentation site 

# Usage

```python

python squad_preprocessing.py 
python financial_phrase_bank_preprocessing.py
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add support for Huggingface Hub to NeMo `from_pretrained()`
State: closed | Pull request: yes

Signed-off-by: smajumdar <titu1994@gmail.com>

# What does this PR do ?

Adds the ability to use Huggingface Hub to load pretrained models. **This includes publically published models from any user that uploads their model to the hub - so it enables public contributions to NeMo from the community**.

**Collection**: [Core, Documentation]

# Changelog 
- Refactor NGC + HF Hub handling

# Note
As of right now, there are no uploaded checkpoints on the HF Hub, so the below call will fail, but we hope to have some checkpoints there soon.

# Usage

## Uploading a new checkpoint

When uploading a new checkpoint to HF Hub, make sure the name of the .nemo model matches the name of the model card itself

Example : If user ABC has model card `stt_en_citrinet_1024.nemo` then the model card should have the name `ABC/stt_en_citrinet_1024` when loading it in nemo.

## Use after uploading

```python
# Assuming you have nemo_toolkit[asr] installed ...
import nemo.collections.asr as nemo_asr

# replace `nvidia` with any username / public checkpoint repo on the hub for community checkpoints
model = nemo_asr.models.ASRModel.from_pretrained("nvidia/stt_en_conformer_ctc_medium")
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

## Who can review?
@okuchaiev  @ericharper
refactor tn data folder, and update of measure
State: closed | Pull request: yes

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Number of Speakers Speaker_Diarization_Inference
State: closed | Pull request: no
Labels: name: feature

Hello everyone, in the diarization process I need to receive the exact number of speakers within an audio without having to pass this number in the manifest.

Exemplifying better:
My problem is that I can only guarantee the maximum number of speakers within these audios, I have no way of knowing if the audio will come with 2, 3, 5 speakers. When I pass the 10 speakers manifest it tries to "find" 10 speakers within an audio that only has 3 speakers.

Is there any alternative in which I receive the exact number of speakers in the rttm output without having to go through the manifest first?
Exp manager log rank 0 only arguments
State: closed | Pull request: yes

# What does this PR do ?

Adds log_local_rank_0_only and log_global_rank_0_only args to exp_manager.

**Collection**: All

# Changelog 
- Makes creation of log files based on world size configurable instead of hard-coded.

# Usage
* exp_manager.log_rank_0_only: True
* exp_manager.log_global_rank_0_only: False

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add known issues to notebook
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Bugfix and variable name change for clustering code
State: closed | Pull request: yes

Signed-off-by: Taejin Park <tango4j@gmail.com>

# What does this PR do ?

This PR fixes the type casting issue in nmesc_clustering.py. With torch.jit.script decorator, torch.tensor should always strictly match the type when assigining/changing a variable. Without this fix, in some cases torch.jit compiler returns an error at nmesc_clustering.py.


**Collection**: [Note which collection this PR will affect]
ASR

# Changelog 
- changed variable name from random_trial to n_random_trials
- changed variable assignment with a proper type conversion

# Usage
Please refer to [speaker diarization part of the NeMo docs](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/asr/speaker_diarization/datasets.html).

```python
python examples/speaker_tasks/diarization/offline_diarization.py \
    diarizer.manifest_filepath="/path/to/ch109_input_manifest.json" \
    diarizer.oracle_num_speakers=2  \ # Since there are exactly 2 speakers per each CH109 session
    diarizer.oracle_vad=True \ # Use oracle VAD extracted from RTTM files.
    diarizer.collar=0.25 \
    diarizer.ignore_overlap=True \
    diarizer.speaker_embeddings.model_path="titanet_large" \
    diarizer.speaker_embeddings.window_length_in_sec=[1.5,1.0,0.5] # Multiscale setting \
    diarizer.speaker_embeddings.shift_length_in_sec=[0.75,0.5,0.25] # Multiscale setting \
    diarizer.speaker_embeddings.parameters.multiscale_weights=[0.33,0.33,0.33] # Equal weights 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [  ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [] New Feature
- [x] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Fixing pretrained name
State: closed | Pull request: yes

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update Vietnamese ITN
State: closed | Pull request: yes

# What does this PR do ?

Update grammar and fix errors for Vietnamese ITN

**Collection**: Vietnamese ITN

# Changelog 
- Update Vietnamese grammar
- Fix errors in Cardinal FST, Decimal FST, and Time FST
- Change Date and Time verbalizers
- Update test cases

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? Pynini
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
EncDecCTCModel from_pretrained() is crashing
State: closed | Pull request: no
Labels: name: bug

Hello!

After pull new git changes something is not working properly.

The way to reproduce the error is the next one:

from nemo.collections.asr.models import EncDecCTCModel
asr_model = EncDecCTCModel.from_pretrained(model_name="asr_talknet_aligner").cpu().eval()

After executing this code it will crash with the next error: (dataset_durations.py line 29 is the above code)

[NeMo I 2022-04-16 10:40:01 cloud:56] Found existing object /home/pc/.cache/torch/NeMo/NeMo_1.9.0rc0/qn5x5_libri_tts_phonemes/656c7439dd3a0d614978529371be498b/qn5x5_libri_tts_phonemes.nemo.
[NeMo I 2022-04-16 10:40:01 cloud:62] Re-using file from: /home/pc/.cache/torch/NeMo/NeMo_1.9.0rc0/qn5x5_libri_tts_phonemes/656c7439dd3a0d614978529371be498b/qn5x5_libri_tts_phonemes.nemo
[NeMo I 2022-04-16 10:40:01 common:705] Instantiating model from pre-trained checkpoint
[NeMo W 2022-04-16 10:40:01 features:233] Using torch_stft is deprecated and has been removed. The values have been forcibly set to False for FilterbankFeatures and AudioToMelSpectrogramPreprocessor. Please set exact_pad to True as needed.
[NeMo I 2022-04-16 10:40:01 features:259] PADDING: 1
[NeMo I 2022-04-16 10:40:01 features:276] STFT using torch
Traceback (most recent call last):
  File "dataset_durations.py", line 29, in <module>
    asr_model = EncDecCTCModel.from_pretrained(model_name="asr_talknet_aligner").cpu().eval()
  File "/home/pc/Downloads/NeMo/nemo/core/classes/common.py", line 708, in from_pretrained
    instance = class_.restore_from(
  File "/home/pc/Downloads/NeMo/nemo/core/classes/modelPT.py", line 305, in restore_from
    instance = cls._save_restore_connector.restore_from(
  File "/home/pc/Downloads/NeMo/nemo/core/connectors/save_restore_connector.py", line 156, in restore_from
    instance.load_state_dict(state_dict, strict=strict)
  File "/home/pc/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1497, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for EncDecCTCModel:
	Missing key(s) in state_dict: "preprocessor.featurizer.window". 
	Unexpected key(s) in state_dict: "preprocessor.featurizer.stft.forward_basis", "preprocessor.featurizer.stft.inverse_basis". 

I checkout to tag v1.7.2 and it is working well. But I would prefer to use the latest code if possible.

Could you help me to solve it?

- OS version: Ubuntu 21.10
- Python3.8
- GPU: RTX 3080
pytorch-lightning               1.6.0
torch                           1.11.0+cu113
torch-stft                      0.1.4
torchaudio                      0.11.0+cu113
torchmetrics                    0.7.3
torchvision                     0.12.0+cu113


Thank you!

David Martin Rius
feat: Added the QAT finetuning script
State: closed | Pull request: yes

Signed-off-by: Anurag Dixit <a.dixit91@gmail.com>

# What does this PR do ?
Adds a QAT finetuning workflow for ASR models.

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]
ASR 
# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
TRAIN_MANIFEST=<path to manifest file for training dataset>
EVAL_MANIFEST=<path to manifest file for evaluation dataset>
CONF_PATH=<path to model configuration>
NUM_GPU=<number of gpus>
NUM_NODES=<number of nodes>
BATCH_SIZE=<batch size>
LEARNING_RATE=0.00015
NUM_WORKERS=32
QUANT_MODEL=<Path to quantized model>
SAVE_MODEL=<Path to save model>
TOKENIZER_PATH=<Path to tokenizer>

python speech_to_text_qat.py \
--config-path=${CONF_PATH} \
--config-name=<config file name for model> \
model.train_ds.manifest_filepath=${TRAIN_MANIFEST} \
model.train_ds.batch_size=${BATCH_SIZE} \
model.validation_ds.manifest_filepath=${EVAL_MANIFEST} \
+model.train_ds.normalize=false \
++model.train_ds.num_workers=${NUM_WORKERS} \
++model.validation_ds.num_workers=${NUM_WORKERS} \
++model.train_ds.pin_memory=True \
++model.validation_ds.pin_memory=True \
+use_cer=false \
+trainer.devices=${NUM_GPU} \
trainer.num_nodes=${NUM_NODES} \
hydra.run.dir=${PWD} \
trainer.log_every_n_steps=50 \
+save_model=${SAVE_MODEL} \
+quantized_model=${QUANT_MODEL} \
++model.optim.lr=${LEARNING_RATE} \
++model.tokenizer.dir=${TOKENIZER_PATH} \
++model.tokenizer.type='bpe'
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
nemo2riva of tts_en_fastpitch:1.4.0 fails with CUDA out of memory
State: closed | Pull request: no
Labels: name: bug

@borisfom As per comment https://github.com/NVIDIA/NeMo/issues/3619#issuecomment-1089551439, I'm re-opening the bug as the issue persists with `nemo:22.01` (1.7.1), `nemo:1.7.2` and `nemo:1.9.0rc0` (e4ee26b601a8e6f907f926019367d3d545c62f11) combined with `Riva:2.0.0`. I tried both by starting off with `nemo:22.01` and upgrading to more recent versions of nemo, as well as building new containers from the specific GitHub repo tag. I tested with T4 (16GB) and A100 (40GB) GPUs. On all occasions the conversion of [tts_en_fastpitch_align.nemo](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/tts_en_fastpitch) fails with `RuntimeError: CUDA out of memory`.
ThutmoseTaggerModel, a new model for inverse text normalization
State: closed | Pull request: yes

Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com>

# What does this PR do ?

A new tagger-based model for inverse text normalization

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ X] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix single GPU training issue + change deprecated Lightning args
State: closed | Pull request: yes

# What does this PR do ?

Critical: Fix single GPU training issue
Also, change deprecated Lightning args
**Collection**: Core + most others

# Changelog 
The PR changes:
`num_process` --> `devices`/`num_devices`
`num_gpus` --> `num_devices`

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Add NMT method to translate with TN/ITN pre/post-processing
State: closed | Pull request: yes

# What does this PR do ?

Adds a method to NMT to apply ITN/TN before/after translating text.

**Collection**: NLP

# Changelog 
- Adds an itn_translate_tn method to `mt_enc_dec_model`.

# Usage

```python
inverse_normalizer = InverseNormalizer(lang="en")
normalizer = Normalizer(input_case="cased", lang="de")
translation_model.itn_translate_tn(["this is a sentence"], "en", "de", inverse_normalizer=inverse_normalizer, normalizer=normalizer)
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[Draft] [TTS] Remove AudioToCharWithPriorAndPitchDataset dependency from fastpitch
State: closed | Pull request: yes

# What does this PR do ?

AudioToCharWithDursF0Dataset class will be deprecated so this PR adds code and config to use old FastPitch checkpoint (checkpoint that was trained using AudioToCharWithDursF0Dataset class which is now deprecated) with new code (TTSDataset) to support inference.

**Collection**: tts.models.fastpitch

# Changelog 
- Adds Warning when using old checkpoint
- Adds config file to be used with old checkpoint

# Usage
* Fastpitch will only support inference using the old fastpitch checkpoint


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix changed function name in offline vad asr notebeook
State: closed | Pull request: yes

Signed-off-by: fayejf <fayejf07@gmail.com>

# What does this PR do ?

update changed function name in notebook

**Collection**: ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
normalization as tagging (Thutmose tagger)
State: closed | Pull request: yes

# What does this PR do ?

Add a new model for inverse text normalization.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Online streaming vad asr
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix megatron notebook
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.
Fix broken links and maintain backward compatibility for MegatronBert pertained models from NGC
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Finetuning changes for BART
State: closed | Pull request: yes

# What does this PR do ?

Adds support for fine-tuning BART models.

**Collection**: NLP

# Changelog 
- Checkpoint to nemo converter addition for BART.

# Usage

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Pretrained conformer CTC with character encoding for ASR
State: closed | Pull request: no

Is there a pretrained model with character encoding using conformer-CTC for English? I could only find the onw with BPE encoding.

As per https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/asr/models.html#conformer-ctc

> This model supports both the sub-word level and character level encodings. You can find more details on the config files for the Conformer-CTC models at Conformer-CTC <./configs.html#conformer-ctc>. The variant with sub-word encoding is a BPE-based model which can be instantiated using the EncDecCTCModelBPE class, while the character-based variant is based on EncDecCTCModel.

Does this mean, that it is there to be trained, but there is no pretrained version?
improvements to geet_commonvoice_data script
State: closed | Pull request: yes

# What does this PR do ?

compatibility with MCV versions 2 through8

**Collection**: ASR

# Changelog 
- If tsv file does not include .mp3 extension for files, it is added
- If an mp3 file is empty, it is correctly ignored
- if the archive file is not in the {language}.tar.gz format, it is downloaded in that format anyways for correct downstream processing
- if the archive does not have layered directories (e.g. /version/language/files.tsv), it is correctly processed anyways

# Usage


```python
python get_commonvoice_data.py --data_root /data/temp --log --num_workers=16 --language es -- version cv-corpus-6.1-2020-12-11
python get_commonvoice_data.py --data_root /data/temp --log --num_workers=16 --language es --version cv-corpus-4-2019-12-10
python get_commonvoice_data.py --data_root /data/temp --log --num_workers=16 --language es --version cv-corpus-3
python get_commonvoice_data.py --data_root /data/temp --log --num_workers=16 --language es --version cv-corpus-2
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Megatron work-arounds
State: closed | Pull request: yes

# What does this PR do ?



**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix nemo megatron restore with artifacts
State: closed | Pull request: yes

# What does this PR do ?

Nemo megatron models were not properly registering artifacts for the tokenizer.

**Collection**: NLP

# Changelog 
- Update register_artifact calls
- 
# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[question] computation of val_wer when validation_ds contains multiple manifest files
State: closed | Pull request: no
Labels: name: bug

It seems that if one uses multiple manifest files for validation_ds the `val_wer` used in `checkpoint_callback` is based on the first manifest file, or computed as the average over all manifest files. This has an obvious influence on which checkpoints are retained `save_top_k`. Is this intended by design? 

If not; having two manifests, one small and one big will give an unfairly large influence to the small manifest in case of averaging, or exclude one if based on manifest with index 0. I believe the correct approach would be to do a division between the sum of numerators and sum of denominators over individual manifests (hence the same result if the two manifests were merged into one) as mentioned in https://github.com/NVIDIA/NeMo/blob/2aa68822bbfb8e7e8c221817b66de28928d6443b/nemo/collections/asr/metrics/wer.py#L75, but I do not see this happening. 

Browsing through code I can see https://github.com/NVIDIA/NeMo/blob/2aa68822bbfb8e7e8c221817b66de28928d6443b/nemo/collections/asr/models/asr_model.py#L40-L45, but my Tensorboard apart from `val_wer` contains also `<manifestfile>_val_wer` and `val_wer==<manifestfile#0>_val_wer`. This value is also used for checkpoints. I am unable to find where in the code the individual `<manifestfile>_val_wer` are appended.
Fix compat override for TalkNet Aligner
State: closed | Pull request: yes

# What does this PR do ?

Moves the override to `load_state_dict()` from TalkNet (which doesn't need it) to EncDecCTCModel for compatibility with the TalkNet Aligner checkpoint. Related ISSUE: https://github.com/NVIDIA/NeMo/issues/4015

This should be removed in 1.9.0 when the TalkNet Aligner model is deprecated.

**Collection**: TTS, ASR

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation