renwei2024/nemo-github-issues huggingface dataset:
NormalizerWithAudio error for written digits
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

See below, the normalization is incorrect for specific regex cases.

**Steps/Code to reproduce bug**

```python
import nemo; nemo.__version__ # '1.9.0rc0'

from nemo_text_processing.text_normalization.normalize_with_audio import (
    NormalizerWithAudio,
)

normalizer = NormalizerWithAudio(input_case="cased")

# SUCCESS
written_text = "01"
spoken_texts = normalizer.normalize(written_text, n_tagged=-1)
print(spoken_texts)
"""
{'oh one', 'zero one'}
"""

# SUCCESS
written_text = "010"
spoken_texts = normalizer.normalize(written_text, n_tagged=-1)
print(spoken_texts)
"""
{'oh one oh', 'oh ten', 'zero one zero', 'zero ten'}
"""

# FAIL
written_text = "0101"
spoken_texts = normalizer.normalize(written_text, n_tagged=-1)
print(spoken_texts)
"""
{'oh one hundred and one',
 'oh one hundred one',
 'oh one oh one', 
 'oh one one', ########### WRONG ###########
 'zero one hundred and one',
 'zero one hundred one',
 'zero one one', ########### WRONG ###########
 'zero one zero one'}
"""

# FAIL
written_text = "101"
spoken_texts = normalizer.normalize(written_text, n_tagged=-1)
print(spoken_texts)
"""
{'one hundred and one',
 'one hundred one',
 'one oh one',
 'one one', ########### WRONG ###########
 'one zero one'}
"""


# FAIL
written_text = "101010"
spoken_texts = normalizer.normalize(written_text, n_tagged=-1)
print(spoken_texts)
"""
{'one hundred one thousand and ten',
 'one hundred one thousand ten',
 'one oh one oh one oh',
 'one one thousand and ten', ########### WRONG ###########
 'one one thousand ten', ########### WRONG ###########
 'one zero one zero one zero'}
"""

# FAIL
written_text = "507"
spoken_texts = normalizer.normalize(written_text, n_tagged=-1)
print(spoken_texts)
"""
{'five hundred and seven',
 'five hundred seven',
 'five oh seven',
 'five seven', ########### WRONG ###########
 'five zero seven'}
"""

# etc.

```

**Expected behavior**

A clear and concise description of what you expected to happen.

**Environment overview (please complete the following information)**

 - Environment location: [Bare-metal, Docker, Cloud(specify cloud provider - AWS, Azure, GCP, Collab)]
 - Method of NeMo install: [pip install or from source]. Please specify exact commands you used to install.
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version
- PyTorch version
- Python version

**Additional context**

Add any other context about the problem here.
Example: GPU model
[NLP] [question/bug?] `train_loss` falling, whereas `val_loss`, `punct_f1`, `capit_f1` all increasing
State: closed | Pull request: no
Labels: name: bug

name: stale

@PeganovAnton Training on multiple GPUs I'm noticing that `train_loss` is decreasing, and `f1` scores are increasing, but so is `val_loss`. Is `val_loss` the right metric to be monitored? Would mean `f1` score be better suited?

What does `logits_ndim` in https://github.com/NVIDIA/NeMo/blob/fea3775c00adfacfe0a414dea15544abc96db8dc/nemo/collections/nlp/models/token_classification/punctuation_capitalization_model.py#L126
stand for? Is this initialisation still correct if one changes the number of classes, i.e. includes support for additional punctuation marks? What if the numbers of classes for punctuation and capitalisation differ?  

<img width="348" alt="image" src="https://user-images.githubusercontent.com/4784070/180222056-3f2d6dbc-6123-43a1-a4d1-c49b1651f979.png">
<img width="348" alt="image" src="https://user-images.githubusercontent.com/4784070/180222242-6fde5196-fc70-454e-86c7-fbe3c800f5a0.png">
<img width="348" alt="image" src="https://user-images.githubusercontent.com/4784070/180222345-ee0f65a2-7153-4d8e-b652-2b549e937be0.png">
<img width="348" alt="image" src="https://user-images.githubusercontent.com/4784070/180222409-27f760c0-4d4e-41ef-a58f-0a905d73862f.png">

**Environment details**

`pytorch:22.06-py3` + `nemo:1.10.0` + PR #4553 patch
parser not found during inference of Tacotron2 TTS on non-english language
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**
I am facing potentially the following two issue:
First
I am training a non-english language TTS model using Tacotron2. During inference I try to pre-process the input text using the parse() function but I get the error `AttributeError: 'Tacotron2Model' object has no attribute 'parser'`. 

Second
On looking deeper into parser and it's initialisation. The property parser function return the parser created in dataset loader class _("AudioToCharDataset")_ but the parser object created inside the class is not instance variable. It is local to the class and not available with dataset object. 
Moreover, in the `nemo.collections.tts.models.Tacotron2Model` there is not placeholder for defining the name of the parser. Ideally it should come from config file.

**Steps/Code to reproduce bug**
Train the TTS on any non-english language dataset.
During inference use _parse_ function to pre-process the text to get tokens.

**Expected behavior**

An error saying `AttributeError: 'Tacotron2Model' object has no attribute 'parser'`

**Environment overview (please complete the following information)**

 - Environment location: [Bare-metal, Docker, Cloud(specify cloud provider - AWS, Azure, GCP, Collab)]
 - Method of NeMo install: pip install nemo-toolkit\['all'\]
 - commands used `docker pull nvcr.io/nvidia/pytorch:22.05-py3` & `docker run --gpus all -it --rm -v /home/prashant/code/NeMo:/NeMo --shm-size=8g -p 8888:8888 -p 6006:6006 --ulimit memlock=-1 --ulimit stack=67108864 --device=/dev/snd nvcr.io/nvidia/pytorch:22.05-py3` 

**Environment details**
Nvidia docker image used mentioned in the README
Input features for MarbleNet-VAD
State: closed | Pull request: no

I am performing VAD using examples/asr/vad_infer.py . I see that, from the code,  MarbleNet directly operates on raw audio data using convolution filters.  There is no separate feature extraction from the audio. 

But , when I see the published MarbleNet paper from Nvidia, looks like first MFCC is extracted from audio and then model operates on those MFCCs. 

Am I correct with this observation ?
Speaker Identification in Speaker dilation
State: closed | Pull request: no
Labels: name: bug

Hello,
I am using speaker dilation for finding out number of speaker and time when which speaker has spoken. when i am setting up meta data like following where i am providing num_spearker = num_of_speaker then it is working perfectly and showing correct result.
T1 T2 Speaker_1
T3 T4 Speaker_0  #Correct result
meta = {
    'audio_filepath': an4_audio, 
    'offset': 0, 
    'duration':None, 
    'label': 'infer', 
    'text': '-', 
    'num_speakers': 2, 
    'rttm_filepath': None, 
    'uem_filepath' : None
}
But, when i am initializing num_spearker = None, Its just showing number of speaker=1 and it's initializing all the spoken time speaker to speaker_0.
T1 T2 Speaker_0
T3 T4 Speaker_0 #Wrong result

But, I have worked with speaker dilation few days back and it was working fine for both num_spearker = num_of_speaker, num_spearker = None. But, now it's not showing correct result for num_spearker = None. 
Did this is happening because of new changes in repository? Or do i need to made some changes in tutorial to make it work??
Tutorial link:- https://colab.research.google.com/github/NVIDIA/NeMo/blob/stable/tutorials/speaker_tasks/Speaker_Diarization_Inference.ipynb#scrollTo=CbKTDFr8SZ9F
Fp16 support for Conformer
State: closed | Pull request: yes

# What does this PR do ?
enables support for fp16 training

Add a one line overview of what this PR aims to accomplish.
enables support for fp16 training by forcing MHSA to float32 or bfloat16 if available.

**Collection**: [Note which collection this PR will affect]
ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below
trainer.precision=16

```python
# Add a code snippet demonstrating how to use this 
```
n/a

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[Fix] Relative audio path in speech data explorer
State: closed | Pull request: yes

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

# What does this PR do ?

Enables audio file loading for manifests which use audio_filepath relative to the location of the manifest.

**Collection**: Tools

# Changelog 
- Added `absolute_audio_filepath`
- Use `absolute_audio_filepath` instead of `item['audio_filepath']`

Note:
Another option may be to include the above path conversion when data is loaded (cf. `data.append(...)`).
However, that would mean each audio file is checked even if it's not loaded.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Quantize conformer model that is trained before
State: closed | Pull request: no
Labels: name: bug

Hi, I trained a conformer model before. Now I wanna quantize that model and convert it to TRT. But when I run the **speech_to_text_quant_infer_trt.py** an error occur. The error is:

`TypeError: Error instantiating 'nemo.collections.asr.modules.conformer_encoder.ConformerEncoder' : __init__() got an unexpected keyword argument 'quantize'
`

Could you please help me to solve it?
Asymmetric Encoder and Decoder Configuration for Megatron Models
State: closed | Pull request: yes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Adds the ability to configure encoder and decoder asymmetrically for encoder and decoder.

**Collection**: NLP

# Changelog 
- Make YAML 1-step hierarchical
- Function in lm_encoder_decoder to maintain backward compatibility.

# Usage
* You can potentially add a usage example below

```python
model.encoder.num_layers=12
model.decoder.num_layers=2
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update diarization data loader to train meeting data
State: closed | Pull request: yes

Signed-off-by: Taejin Park <tango4j@gmail.com>

# What does this PR do ?

This PR adds a new feature to `audio_to_diar_label.py`.
So far, we have only trained with 2 speaker audio where one of the speakers always exists.
This PR makes dataloader to take 2 speaker subset from more-than-2-speaker audio clips with corresponding groundtruth.
Thus, there are cases where grround-truth-cluster label is "-1" where speakers are not included in either of two targeted speakers.
ex) Two speaker ground-truth label ([0,0,1,1,-1,-1,-1,1,1,1,0]

Also removed unnecessary variables.

**Collection**: [Note which collection this PR will affect]

ASR

# Changelog 

-  `audio_to_diar_label.py`

-- Removed unused variables in the functions.  
-- base_clus_label now can have -1 label. 
-- Few cosmetic changes

-  `collections.py`

A few changes include:

-- Added get_rttm_speaker_index() function since sess_spk_dict could have more than 2 speakers.

-  `speaker_utils.py`

-- Added get_rttm_speaker_index that takes speakers from RTTM files.

# Usage

The usage will be provided when model and module codes are published.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Prompt Learning Inference Improvements
State: closed | Pull request: yes

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Separates the gpt prompt learning eval script from the gpt model eval script to simplify inference script files and match t5 prompt learning inference workflow. 

**Collection**: NLP

# Changelog 
- Adds megatron_gpt_prompt_learning_eval.py script
- Adds prompt learning inference config
- Updated megatron_gpt_eval.py script
- Updates gpt inference config
- Updated gpt prompt learning dataset class
- Updated gpt prompt learning model class generate and predict step methods


# Usage
```
python megatron_gpt_prompt_learning_eval.py \
	virtual_prompt_model_file=models/squad_p_tuning_126m_tokens100_epochs100_layers4_1e-4.nemo \
        gpt_model_file=models/gpt_126m.nemo \
        inference.greedy=True \
        inference.add_BOS=False \
        trainer.devices=1 \
        tensor_model_parallel_size=1 \
        data_paths=["data/squad_short_test.jsonl"]

python megatron_gpt_prompt_learning_eval.py \
	virtual_prompt_model_file=models/squad_prompt_tuning_5b_tokens100_steps35000_2e-4.nemo \
        gpt_model_file=models/gpt_5b.nemo \
        inference.greedy=True \
        inference.add_BOS=False \
        trainer.devices=2 \
        tensor_model_parallel_size=2 \
        data_paths=["data/squad_short_test.jsonl"]

```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[Fix] schedulers with no max_steps param
State: closed | Pull request: yes
Labels: name: stale

All LR schedulers in PyTorch do not have the `max_steps` parameter, so we should not add `max_steps` to their `scheduler_args`. Previous code tackle the problem in case-by-case manner, while here we try to solve the problem in a more general way. 

To do so, we define a set of schedulers that don't have the `max_steps` parameter, and set the `add_max_args_flag` accordingly. 

Similarly, we define a set of epoch-based schedulers to set the `interval`, while previous code only checks for `ReduceLROnPlateau` scheduler.

Besides, the previous code also has bugs, since it doesn't set the correct `add_max_args_flag` and `interval` when `if 'args' in scheduler_config` is `True`.
Zero Shot Slot Filling Model
State: closed | Pull request: yes
Labels: name: stale

name: NLP

name: CI

Signed-off-by: Lily Lee <lilee@nvidia.com>

# What does this PR do ?

Zero Shot Slot Filling Model

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix seeding in neural machine translation examples
State: closed | Pull request: yes
Labels: name: stale

Signed-off-by: sarthmit <sarthmit@gmail.com>

# What does this PR do ?

Fixes the seed for everything (training/initialization/etc.) during training of nlp machine translation systems to aid reproducibility.

**Collection**: examples/nlp/machine_translation

# Changelog 
- Using pytorch_lightning's seed_everything function to fix the seed within each training file for machine translation in nlp.
- Update the config files to start with default seed as 0

# Usage
* Usage is as per earlier, where the seed can be changed by just adding a seed=[seed] to override the default 0 in the configs.

**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests? 
- [x] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

## Who can review? 
@MaximumEntropy @okuchaiev
Add Tokenization and Normalization pre-proecssing script for NMT
State: closed | Pull request: yes

Signed-off-by: Abhinav Khattar <aklife97@gmail.com>

# What does this PR do ?

Adds Tokenization and Normalization script 

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below
python preprocess_tokenization_normalization.py --input-src train.en \
    --input-tgt train.zh \
    --output-src train.tok.norm.en \
    --output-tgt train.tok.norm.zh \
    --source-lang en \
    --target-lang zh 


# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
prefix tuning for Megatron gpt models
State: closed | Pull request: yes
Labels: name: stale

# What does this PR do ?

implements prefix tuning for Megatron GPT models.

**Collection**: `examples/nlp/languge_modeling`

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
```
/NeMo/examples/nlp/language_modeling$python megatron_gpt_prefix_tuning.py
```
The prefix tuning `.yaml` file is here: `/NeMo/examples/nlp/language_modeling/conf/megatron_gpt_prefix_tuning_config.yaml`, the settings in this config are used by default when `NeMo/examples/nlp/language_modeling/megatron_gpt_prefix_tuning.py` is launched.

Once trained, you can evaluate using:
```
python megatron_gpt_eval.py \
    prefix_tuned_model_file=<PATH_.nemo_FILE_GENERATED_AT_END_OF_TRAINING> \
    gpt_model_file=<PATH_TO_.nemo_FILE_WITH_PRETRAINED_GPT_MODEL> \
    inference.greedy=True \
    inference.add_BOS=False \
    trainer.devices=1 \
    trainer.num_nodes=1 \
    tensor_model_parallel_size=1 \
    pipeline_model_parallel_size=1 \
    prompts=["<PATH_TO_.jsonl_FILE_TO_EVALUATE>"]
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Removed NLPDDPPlugin Import check
State: closed | Pull request: yes

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Fixes `max_steps` calculation bug by setting the number of data loading workers to equal data parallel size. 

**Collection**: NeMo Core

# Changelog 
- Updated `ModelPT` class

# Usage
N/A same as before

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Typo
State: closed | Pull request: no

In: https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/nlp/prompt_learning.html

cartographic -> catastrophic 

Instead of: "Because the original model parameters are frozen and never altered by either method, p-tuning/prompt-tuning also avoid cartographic forgetting issues often encountered when fine-tuning models."

It should be: "Because the original model parameters are frozen and never altered by either method, p-tuning/prompt-tuning also avoid catastrophic forgetting issues often encountered when fine-tuning models."
fix tarred dataset len when num shards is not divisible by workers
State: closed | Pull request: yes
Labels: name: bug

Signed-off-by: Iztok Lebar Bajec <ilb@fri.uni-lj.si>

# What does this PR do ?

Fixes #4522. When using tarred datasets and the number of shards is not divisible by `world_size`, nemo logs a warning, but training continues. However, in this case the computation of `DataLoader.len()` becomes incorrect. This leads to invalid `lr_scheduler.max_steps`, and other issues down the road. One example being `Validation check` not running at end of epoch. 

**Collection**: [nlp]
- language_modelling
- machine_translation
- text_normalization
- token_classification

# Changelog 
In case of `shard_strategy='scatter'` the `len()` is computed based on the amount of taken shards and assuming total `num_baches` are equally distributed over all shards, i.e. that each shard contains the same number of batches. In addition, following other implementations this PR adds `shard_strategy` support to NLP/PC tarred datasets.

There are two additional occasions, where tarred datasets are used  
https://github.com/NVIDIA/NeMo/blob/fa2e55e607f787d129141523c724b29dab7a411c/nemo/collections/asr/data/audio_to_text.py#L186
which based on the implementation does not seem to be affected by this issue, and
https://github.com/NVIDIA/NeMo/blob/fa2e55e607f787d129141523c724b29dab7a411c/nemo/collections/nlp/data/language_modeling/l2r_lm_dataset.py#L169-L184
which might, but I chose to leave it as is, as the computation of `len()` 
https://github.com/NVIDIA/NeMo/blob/fa2e55e607f787d129141523c724b29dab7a411c/nemo/collections/nlp/data/language_modeling/l2r_lm_dataset.py#L239-L240
greatly differs from others
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [x] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
note to @PeganovAnton for NLP/C

# Additional Information
* Related to #4522
FastSpeech2 training example
State: closed | Pull request: no
Labels: name: bug

Hi all!

I couldn't find any examples of FastSpeech2 training. The only thing that I've found was the data preprocessing [tutorial](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/tts/datasets.html)

Can anyone tell me if there is a py script such as those for tacotron2 and other models? And, is there an example of a config file?
Add length ratio filtering script
State: closed | Pull request: yes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Adds a script to filter a parallel corpus based on min, max and length ratios.

**Collection**: NLP

# Changelog 
- Adds a script to filter a parallel corpus based on min, max and length ratios.

# Usage
* You can potentially add a usage example below

```python length_ratio_filter.py --input-src train.en \
    --input-tgt train.de \
    --output-src train_lang_filtered.en \
    --output-tgt train_lang_filtered.de \
    --removed-src train_removed.en \
    --removed-tgt train_removed.de \
    --min_length 1 \
    --max_length 512 \
    --ratio 1.3
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Adding multispeaker fastpitch and hifigan en model links to available…
State: closed | Pull request: yes

… models

Signed-off-by: subhankar-ghosh <subhankar2321@gmail.com>

# What does this PR do ?

This PR adds links to fastpitch and hifigan multispeaker en models trained on HiFiTTS to list of available models in Fastpitch and HifiGAN classes respectively.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Facing Error while Speaker Diarization Inference
State: closed | Pull request: no
Labels: name: bug

Facing error while running tutorial code for speaker Diarization from NVIDIA tutorial site

after setting up config for ORACLE-VAD DIARIZATION in code when I am trying to compute.
oracle_model.diarize()
it's showing following error:-
RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/nemo/collections/asr/parts/utils/nmesc_clustering.py", line 450, in getCosAffinityMatrix
    """
    emb = _emb.half()
    sim_d = cos_similarity(emb, emb)
            ~~~~~~~~~~~~~~ <--- HERE
    sim_d = ScalerMinMax(sim_d)
    return sim_d
  File "/usr/local/lib/python3.7/dist-packages/nemo/collections/asr/parts/utils/nmesc_clustering.py", line 55, in cos_similarity
    a_norm = a / (torch.norm(a, dim=1).unsqueeze(1) + eps)
    b_norm = b / (torch.norm(a, dim=1).unsqueeze(1) + eps)
    res = torch.mm(a_norm, b_norm.transpose(0, 1))
          ~~~~~~~~ <--- HERE
    res.fill_diagonal_(1)
    return res
RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'

Few days back when i tried to run this same tutorial it was running successfully and it was giving correct out put after doing diarize(). But from 2-3 dyas i am facing this issue with doing diarize() with model.
Kindly help me with this.
remove the variable that is not used in the context.
State: closed | Pull request: yes

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
ImportError: Could not compile megatron dataset C++ helper functions in Synthetic tabular data turorial
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**
In NeMo/tutorials/nlp/Megatron_Synthetic_Tabular_Data_Generation.ipynb notebook, when I wanted to run it in Google Colab, the ImportError occurs. And I am not sure what package should I install to resolve this error.

**Steps/Code to reproduce bug**

It occurs in this step: ( I got the main branch of the NeMo, and follow the steps in notebook)
```
!python megatron_gpt_pretraining.py \
        trainer.devices={NUM_GPUS} \
        trainer.accelerator=gpu \
        trainer.log_every_n_steps=100 \
        trainer.val_check_interval=500 \
        trainer.accumulate_grad_batches=1 \
        trainer.max_steps=10000 \
        trainer.precision=16 \
        trainer.gradient_clip_val=1.0 \
        exp_manager.exp_dir=gpt_creditcard_results \
        model.tensor_model_parallel_size={TENSOR_MP_SIZE} \
        model.pipeline_model_parallel_size={PIPELINE_MP_SIZE} \
        model.optim.name=fused_adam \
        model.optim.lr=2e-4 \
        model.optim.sched.warmup_steps=2 \
        model.optim.sched.constant_steps=2 \
        model.optim.sched.min_lr=8e-5 \
        model.max_position_embeddings={SEQ_LENGTH} \
        model.encoder_seq_length={SEQ_LENGTH} \
        model.data.seq_length={SEQ_LENGTH} \
        model.tokenizer.type=Tabular \
        model.tokenizer.library=tabular \
        model.tokenizer.vocab_file={CC_OUTPUT_P} \
        model.tokenizer.delimiter=\',\' \
        model.data.eod_mask_loss=True \
        model.data.splits_string=\'3800,198,2\' \
        model.num_layers={NUM_LAYERS} \
        model.hidden_size={HIDDEN_SIZE} \
        model.num_attention_heads={NUM_ATTENTION_HEADS} \
        model.activations_checkpoint_method=\'block\' \
        model.activations_checkpoint_num_layers=1 \
        model.data.data_prefix=[tabular_data_text_document]
```
Update megatron prompt learning interface to dialogue 
State: closed | Pull request: yes

# What does this PR do ?

Update megatron prompt learning interface to dialogue 

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
wait for global_rank_zero() to move old files
State: closed | Pull request: yes

Signed-off-by: Iztok Lebar Bajec <ilb@fri.uni-lj.si>

# What does this PR do ?

In multi-gpu settings there is no guarantee that `global_rank_zero()` will be the first process to start. Assuming a shared storage this means that when resuming training with `log_global_rank_0_only=False`, this may lead to previous run `nemo_log_globalrank-*_localrank-*.log` files being overwritten and current run files being moved to the `run_*` subfolder.  

**Collection**: [nemo/utils]

# Changelog 
Ensure all files from previous run are moved before continuing execution of non `global_rank_zero()` processes.

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[text processing] Chinese text normalization PR
State: closed | Pull request: no
Labels: name: feature

name: TN/ITN

This is a preliminary inquiry if NeMo team is open to accept PR for Chinese text normalization, or you guys have internal development plans for it ?

[SpeechColab](https://github.com/SpeechColab) is assigning 1 intern student to work on Chinese text normalization(based on Pynini), we'd like to contribute this work to NeMo, hopefully in the next couple of weeks.  Basically what we will have is a set of WFST rewrite rules just like existing [NeMo en TN](https://github.com/NVIDIA/NeMo/tree/main/nemo_text_processing/text_normalization/en), reproducing TN behaviors in [this project](https://github.com/speechio/chinese_text_normalization), which handles many typical TN scenarios for Chinese. 

**Known Issue**
The intern student will no longer be maintaining the grammars after the internship, but the resulting PR is guaranteed to establish a foundation, enabling Chinese TN in NeMo.

Would like to hear your thoughts on this. @yzhang123  @ekmb
Fine-tuning Voice Activity Detection (VAD) models
State: closed | Pull request: no
Labels: name: feature

**Is your feature request related to a problem? Please describe.**

After running diarization baselines, I wanted to fine-tune the component models (VAD + embedding) on my own labeled diarization dataset. For the embedding model this has gone well, and for VAD I have been able to set up training using Google Speech Recognition V2 dataset, as in the examples. 

Now that I try to create/adapt the Nemo manifests for VAD training on my dataset, I realise that the manifest format does not suffice for this: 

`{"audio_filepath": "<audio_path>", "duration": <full_file_duration>, "label": "speech", "text": "_", "offset": <start_of_speech_segment}`

If I understand correctly, this only allows for a single speech segment per audio file. This works for the Google dataset where each file is short and has a single utterance, but my (and I believe most) diarization dataset with RTTM labels defines many segments of speech per file. 

A workaround is technically possible where each file is cut into smaller segments, each with a single utterance, but this seems excessive. The requirement is also surprising, as other tools (e.g. PyAnnote) allow for training VAD models using the RTTM/diarization annotation format.

I'd love to know if I understand the situation correctly, or whether perhaps I've missed something.

Otherwise, thanks for a great framework! Especially the neural types is a feature that would enhance many other DL repositories.

**Describe the solution you'd like**

The ability to fine-tune VAD models using files with more than one utterance.

**Describe alternatives you've considered**

See workaround above

**Additional context**
-
Tensorboard step on resume training restarts from 0
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

I have noticed that in previous nemo versions, e.g. 83444044327d8f0b697bbf0729bd96b7e16e3a56, Tensorboard showed consecutive steps when training was continued from the last checkpoint of an interrupted run; recent versions e.g. 77be76349901d0dd99ad6778509ef5c0663a65c3, as well as 1.10.0 do not; they always start from 0. Is this by intention or bug?

with 83444044327d8f0b697bbf0729bd96b7e16e3a56
<img width="365" alt="image" src="https://user-images.githubusercontent.com/4784070/178539639-a0e1cd2a-daf0-4460-96d1-20524bf4c969.png">

with 77be76349901d0dd99ad6778509ef5c0663a65c3
<img width="365" alt="image" src="https://user-images.githubusercontent.com/4784070/178539957-30a5d0cf-064d-4cbb-b850-e0b9d91aa4a9.png">
Add MTEncDec Finetune support
State: closed | Pull request: yes

Signed-off-by: Abhinav Khattar <aklife97@gmail.com>

# What does this PR do ?

Add finetune support for NMT

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add nsys profiling
State: closed | Pull request: yes

# What does this PR do ?

Enable nsys profiling for any NeMo model from CLI or YAML.

Note this PR was originally: #4531 , authored by @erhoo82 

**Collection**: [Core]

# Changelog 
- Add setup nsys method to ModelPt
- Add on_train_batch_start/end hooks to enable nsys profiling

# Usage
* Add the following to your NeMo Model config:

```
# Nsys profiling options
nsys_profile:
  enabled: False
  start_step: 10  # Global batch to start profiling
  end_step: 10 # Global batch to end profiling
  ranks: [0] # Global rank IDs to profile
  gen_shape: False # Generate model and kernel details including input shapes
```
Then wrap your training command with:
```
nsys profile -s none -o <path_to_output> -t cuda,nvtx --force-overwrite true --capture-range=cudaProfilerApi --capture-range-end=stop python ...
```

See https://docs.nvidia.com/nsight-systems/UserGuide/index.html#cli-profiling for more details.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
RADTTS ADLR-NEMO porting
State: closed | Pull request: yes

# What does this PR do ?

This PR adds RADTTS code to NeMo

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage

```python
# Add a code snippet demonstrating how to use this 

python examples/tts/radtts_train.py trainer.max_epochs=1000 export_dir="directory to export checkpoints and logs" model.optim.lr=0.0001 model.train_ds.dataloader_params.batch_size=32 model.validation_ds.dataloader_params.batch_size=32 model.modelConfig.decoder_use_partial_padding=True
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [X] Did you write any new necessary tests?
- [X] Did you add or update any necessary documentation?
- [X] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [X] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [X] New Feature
- [ ] Bugfix
- [ ] Documentation

This code base uses pytorch implemenation of RAdam which hasn't been tested fully when this is released we are waiting for SWIPAT approval to use https://github.com/LiyuanLucasLiu/RAdam/blob/master/LICENSE. Until that you can integrate this and train. we will switch code to this implementation once we get SWIPAT approval which could take a couple of days.
the NMT infer OOM 
State: closed | Pull request: no
Labels: name: feature

**Is your feature request related to a problem? Please describe.**

1. Inference does not support automatic batch processing according to length, which leads to the OOM of excessively long sentences and makes it difficult to enlarge the beam.
2. And I found that as the inference goes on, the occupation of CUDA memory is gradually increasing.


A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

**Describe the solution you'd like**
1. support automatic batch processing according to length
2. I have try to solve it by `torch.cuda.empty_cache()`