renwei2024/nemo-github-issues huggingface dataset:
Training FastPitch Using More Than 1 GPU Fails
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**
When training FastPitch on more than 1 GPU the following error occurs:
`AttributeError: Can't pickle local object 'FilterbankFeatures.__init__.<locals>.<lambda>'`

docker run command:
```
python fastpitch.py --config-name=fastpitch_align_v1.05.yaml train_dataset=/path/to/manifest.json validation_datasets=/path/to/manifest.json  sup_data_path=/path/to/wav exp_manager.exp_dir=path/to/dir trainer.max_epochs=10 trainer.check_val_every_n_epoch=1 model.train_ds.dataloader_params.batch_size=70 model.validation_ds.dataloader_params.batch_size=30 model.n_speakers=1 model.pitch_mean=152.3 model.pitch_std=64.0 model.pitch_fmin=30 model.pitch_fmax=512 model.optim.lr=1e-1 trainer.devices=6 trainer.strategy=null
```

**Environment details**
Docker image: nvcr.io/nvidia/nemo:23.01
GPU model: NVIDIA A100-SXM
Typos and cosmetic fixes
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

Correct a few typographic errors and make a few cosmetic suggestions to markup.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Corrected typographical errors in some of the NLP documentation.

# Usage
Not applicable.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
Not opened.
[TTS] fixed broken path.
State: closed | Pull request: yes
Labels: name: TTS

name: cherry-pick

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix custom `forward_torch_softmax`
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

forward_torch_softmax gives error difference in apex only when mask is set. 
default to normal behavior when mask is None

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Facing issue when i run model with my own audio file
State: closed | Pull request: no
Labels: name: stale

Specifically, it would be helpful to know the audio format, bitrate, size and sampling rate of the audio file. These details will help me to identify any potential issues that might be causing the problem.
![issue1](https://user-images.githubusercontent.com/105053572/235134513-93dcd5f7-8144-4c40-9551-293493ce3378.png)
[TTS] fixed broken path.
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
False error message: 'last epoch number of samples exceeded max value' when training model
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

This bug occurs on a specific dataset where the data loading process mistakenly checks for `<` instead of `<=` in an assert statement, causing the "last epoch number of samples exceeded max value" error message to be falsely displayed.


**Steps/Code to reproduce bug**

1. Prepare a dataset and configure the GPT training. The size of the model is not important.
2. Start the training process.
3. During the data loading phase, the issue arises.

This issue does not occur on all datasets. It is difficult to specify which datasets it occurs on.

**Expected behavior**

If the training process excludes the dataset where the issue occurs, it proceeds normally.

**Environment overview (please complete the following information)**

 - Environment location: Bare-metal with slurm and enroot
 - Method of NeMo install: `docker pull nvcr.io/ea-bignlp/bignlp-training:23.03-py3`
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Additional context**

It appears that https://github.com/NVIDIA/NeMo/blob/65db9359bcb1cee96ffe13f36325f66bb8f17fd5/nemo/collections/nlp/data/language_modeling/megatron/gpt_dataset.py#L588-L590 is causing the issue. It should work properly if `<=` is used instead.
Fix custom `forward_torch_softmax`
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

forward_torch_softmax gives error difference in apex only when mask is set. 
default to normal behavior when mask is None

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Fix FastPitch energy code
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Update the energy prediction logic in FastPitch to match the new conditioning logic, similar to pitch. Current code throws a syntax error because `prosody_input `is no longer defined.

**Collection**: [TTS]

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Support dynamic length batches with GPT SFT
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add support for using dynamic length batches for GPT SFT

Significantly improves perf
~2 it/s to ~4 it/s on a local Quadro RTX 6000 GPUs with the same test as on CI

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Add script for mapping speaker names to indices
State: closed | Pull request: yes

# What does this PR do ?

The existing TTS workflow requires the user to provide speaker as an integer index in the training manifest. This is awkward because

1. User can provide raw ID from datasets like LibriTTS as an integer, but then we end up creating 10,000+ element speaker array for only hundreds of speakers.
2. User can manually create a map from speaker ID string to sequential integer index and then overwrite their training manifest.
3. Training with multiple manifests is awkward because you need to rewrite the speaker field for all manifests.

So the idea is for the new data loading logic, user will provide speaker as a string name/ID in the manifest and then provide a separate JSON file which maps the speaker IDs to integer IDs. The below script is an example of how to create that map.

**Collection**: [TTS]

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
[TTS] Add script for computing feature stats
State: closed | Pull request: yes

# What does this PR do ?

This script is similar to `compute_speaker_stats.py` but uses the new experimental featurizer class for saving and loading the features. We can consider deprecating `compute_speaker_stats.py` in the future if we end up deprecating existing TTS data loader and `extract_sup_data.py`.

**Collection**: [TTS]

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
[TTS] Create initial TTS dataset feature processors
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Add an interface and initial implementations to process features at training time, as part of TTS data loading refactoring.

The intention is for the data loader to load and merge feature dictionaries (produced by tts.parts.preprocessing.feature.py) and to use these processors to apply on necessary on the fly calculations, such as feature normalization.

I can provide a diff for the data loader itself if it helps provide context. Otherwise I will post PR for the data loader after this one, as it depends on this change.

**Collection**: [TTS]

# Changelog 
- Define interface to processing dataset features
- Provide implementation for feature scaling, log compression, and mean-variance normalization.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
hydra package while running the G2P in 00_NeMo_Primer is not loading, please allow the access for testings.
State: closed | Pull request: no
Labels: name: stale

**Is your feature request related to a problem? Please describe.**

A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

**Describe the solution you'd like**

A clear and concise description of what you want to happen.
Provide a code snippet on how new APIs/changes would be used by others.

**Describe alternatives you've considered**

A clear and concise description of any alternative solutions or features you've considered.

**Additional context**

Add any other context or screenshots about the feature request here.
fix broken links r1.18.0
State: closed | Pull request: yes
Labels: name: TTS

name: cherry-pick

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Broken Notebook
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

The notebook in https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/Offline_ASR.ipynb cannot be visualized as it says "The Notebook Does Not Appear to Be Valid JSON"

**Steps/Code to reproduce bug**

Open https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/Offline_ASR.ipynb
Fix error appearing when using tar datasets
State: closed | Pull request: yes
Labels: name: ASR

When using tar dataset this error appears due to undefined reference:

datastore_path_to_webdataset_url(p) if is_datastore_path(p) and is_tarred_path(p) else p NameError: name 'is_tarred_path' is not defined

# What does this PR do ?
fixes error mentioned above

Add a one line overview of what this PR aims to accomplish.

**Collection**: [ASR]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix broken links r1.18.0
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[BugFix] Force _get_batch_preds() to keep logits in decoder timestamp…
State: closed | Pull request: yes
Labels: name: ASR

…stamps generators

# What does this PR do ?

The argument of `_get_batch_preds` in `FrameBatchASRLogits` class has not been updated when `FrameBatchASR` is updated.
Now, this PR force `_get_batch_preds()` to keep logits in decoder timestamps generators.
This is for r1.18.0 of PR #6499.

**Collection**: [Note which collection this PR will affect]
ASR


# Changelog 

- Added `force_keep_logits` for `_get_batch_preds` function in class `FrameBatchASRLogits` in `decoder_timestamp_utils.py`
- Changed class name `FrameBatchASR_Logits` to `FrameBatchASRLogits`

# Usage
* You can potentially add a usage example below

```python
python offline_diar_with_asr_infer.py \
    diarizer.manifest_filepath=<path to manifest file> \
    diarizer.out_dir='demo_asr_output' \
    diarizer.speaker_embeddings.model_path=<pretrained modelname or path to .nemo> \
    diarizer.asr.model_path=<pretrained modelname or path to .nemo> \
    diarizer.asr.parameters.asr_based_vad=True \
    diarizer.speaker_embeddings.parameters.save_embeddings=False
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

ASR contributors

# Additional Information

This is for main, and there will be another PR for r1.18
[BugFix] Force _get_batch_preds() to keep logits in decoder timestamps generator
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?
The argument of `_get_batch_preds` in `FrameBatchASRLogits` class has not been updated when `FrameBatchASR` is updated.
Now, this PR force `_get_batch_preds()` to keep logits in decoder timestamps generators.

**Collection**: [Note which collection this PR will affect]
ASR


# Changelog 

- Added `force_keep_logits` for `_get_batch_preds` function in class `FrameBatchASRLogits` in `decoder_timestamp_utils.py`
- Changed class name `FrameBatchASR_Logits` to `FrameBatchASRLogits`

# Usage
* You can potentially add a usage example below

```python
python offline_diar_with_asr_infer.py \
    diarizer.manifest_filepath=<path to manifest file> \
    diarizer.out_dir='demo_asr_output' \
    diarizer.speaker_embeddings.model_path=<pretrained modelname or path to .nemo> \
    diarizer.asr.model_path=<pretrained modelname or path to .nemo> \
    diarizer.asr.parameters.asr_based_vad=True \
    diarizer.speaker_embeddings.parameters.save_embeddings=False
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

ASR contributors

# Additional Information

This is for main, and there will be another PR for r1.18
Add interleaved pp support
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add support for virtual pipeline parallelism conversion of Megatron GPT models

**Collection**: [NLP]

# Changelog 
- 

# Usage

```python

python megatron_change_num_partitions.py \
    --model_extracted_dir="<Directory of Pytorch Lightning ckpt folders>" \
    --target_file="<Name of the target NeMo file>" \
    --ckpt_name="<Name of a single Pytorch Lightning ckpt file inside the extracted dir>" \
    --tensor_model_parallel_size=<TP SIZE> \
    --target_tensor_model_parallel_size=<TARGET TP SIZE> \
    --pipeline_model_parallel_size=<PP SIZE> \
    --target_pipeline_model_parallel_size=<TARGET PP SIZE> \
    --virtual_pipeline_model_parallel_size=<VP SIZE> \
    --hparams_file="<Path to HPARAMS.yaml file>" \
    --precision=bf16

```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
How to create pickle files, which include out, output_sizes, conv, rnn_0, rnn_1, rnn_2, rnn_3, rnn_4 using the STT De Conformer-Transducer Large for an german audio files smaples?
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

How to create pickle files, which include out, output_sizes, conv, rnn_0, rnn_1, rnn_2, rnn_3, rnn_4 layers using the STT De Conformer-Transducer Large for an german audio files samples?
Add disclaimer about dataset for ASR
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

**Collection**: [ASR]

# Changelog 
- Add disclaimer to all ASR tutorials regarding dataset usage.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation
A quickstart speech enhancement tutorial
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

This PR adds a quickstart tutorial for training a simple speech enhancement model with NeMo.

**Collection**: ASR, Tutorials

# Changelog 
- added a simple tutorial for speech enhancement
- moved examples from `asr` to `audio_tasks`
- added an option to save noise in `add_noise.py`
- minor fixes in `audio_models`

# Usage
Check `Speech_Enhancement_with_NeMo.ipynb`.

Tested locally and on Google Colab with GPU runtime.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Change to spawn mp and persistent_wokers for T5
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Fix BigNLP mT5 pre-training mp bug

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Dummy PTL 2.0 PR
State: closed | Pull request: yes
Labels: name: stale

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update README to add core installation
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

Update README to add core installation

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update README to add core installation
State: closed | Pull request: yes

# What does this PR do ?

Update README to add core installation

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
WFST for inverse text normalization (ITN)
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

when i declare some states with pynini.string_map  it cant find the shortest pass of its inverse for no reason

**Steps/Code to reproduce bug**

```
import pynini
from pynini.lib import pynutil


def apply_fst(text, fst):
    """ Given a string input, returns the output string
    produced by traversing the path with lowest weight.
    If no valid path accepts input string, returns an
    error.
    """
    try:

        print(pynini.shortestpath(text @ fst).string())
        return pynini.shortestpath(text @ fst).string()
    except pynini.FstOpError:
        print(f"Error: No valid output with given input: '{text}'")


teens = pynini.string_map([
    ("ده", "10"),
    ("یازده", "11"),
    ("دوازده", "12"),
    ("سیزده", "13"),
    ("چهارده", "14"),
    ("چاهارده", "14"),
    ("پونزده", "15"),
    ("1پا نزده", "5"),
    ("1پانزده", "5"),
    ("شانزده", "16"),
    ("1شونزده", "6"),
    ("1هفده", "7"),
    ("1هیفده", "7"),
    ("هجده", "18"),
    ("هیجده", "18"),
    ("19", "نوزده")
])

graph_tens = teens

apply_fst("یازده", graph_tens)
apply_fst("دوازده", graph_tens)
apply_fst("سیزده", graph_tens)
apply_fst("چهارده", graph_tens)
apply_fst("چاهارده", graph_tens)
apply_fst("پونزده", graph_tens)
apply_fst("پانزده", graph_tens)
apply_fst("شانزده", graph_tens)
apply_fst("شونزده", graph_tens)
apply_fst("هفده", graph_tens)
apply_fst("هیفده", graph_tens)
apply_fst("هیجده", graph_tens)
apply_fst("هجده", graph_tens)
apply_fst("نوزده", graph_tens)

```

**Expected behavior**

11
12
13
14
14
15
15
16
16
17
17
18
18
19

**produced output**

11
12
13
14
14
15
Error: No valid output with given input: 'پانزده'
16
Error: No valid output with given input: 'شونزده'
Error: No valid output with given input: 'هفده'
Error: No valid output with given input: 'هیفده'
18
18
Error: No valid output with given input: 'نوزده'
Error: No valid output with given input: 'یک'
ERROR: StringFstToOutputLabels: Invalid start state
ERROR: StringFstToOutputLabels: Invalid start state
ERROR: StringFstToOutputLabels: Invalid start state
ERROR: StringFstToOutputLabels: Invalid start state
ERROR: StringFstToOutputLabels: Invalid start state
ERROR: StringFstToOutputLabels: Invalid start state

Process finished with exit code 0



**Environment overview (please complete the following information)**

 - Environment location: my pc
 - Method of NeMo install: conda install pynini
 - 
**Environment details**

- OS version : ubuntu 20
- PyTorch version 
- Python version : 3.8
Update SDP docs
State: closed | Pull request: yes

# What does this PR do ?

Add more detail to SDP section in NeMo docs, and add link to that page in the NeMo README.rst.

**Collection**: docs

# Changelog 
- update SDP section in NeMo docs
- update NeMo README

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [N/A] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ N/A ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix cache aware hybrid bugs
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Fixes some bugs to support Hybrid and FastConformer variants of cache-aware streaming models.

# Changelog 
- Fixed a bug which prevented switching between decoders for Hybrid models
- Added support of Hybrid models to the cache-aware streaming inference scripts
- Added the feature to switch between decoders to the cache-aware streaming simulation script
- Added support of causal downsamling for dw_striding approach.

# Usage
Users may use the --set_decoder "rnnt" or "ctc" to change the decoder with speech_to_text_cache_aware_streaming_infer.py.
They may use asr_model.change_decoding_strategy(decoder_type='ctc' or 'rnnt') to change the decoding for a Hybrid model in general.

**PR Type**:
- [X] New Feature
- [X] Bugfix
- [ ] Documentation
Support in external sample mapping for Megatron datasets
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

This PR adds support in external sample mapping (to be able to compute on the fly).

# What does this PR do ?

The added support does not add any new functionality, but allows providing the sample mapping externally to be able to avoid loading dataset samples.

**Collection**: Megatron `dataset_utils`.

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Manual garbage collection with an interval
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Set garbage collection interval manually. This helps align garbage collection execution across GPUs, which mitigates CPU overhead dramatically.
Using an integer value for `gc_interval` set the garbage collection interval in global-batch steps.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
removing unnecessary avoid_bfloat16_autocast_context
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Removes the no longer unnecessary avoid_bfloat16_autocast_context in conv2d subsampling 

**Collection**: ASR

# Changelog 
In `nemo/collections/asr/parts/submodules/subsampling.py`, no longer wrapping conv2d calls in the avoid... context

# Usage
Use bf16 precision as usual, performance should improve a few % because the conv2d subsampling is no longer wrapped in fp32.
Performance in fp16 and fp32 is not affected.

```python
n/a
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Megatron KERPLE positional embeddings
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

name: CI

# What does this PR do ?

This PR adds support to KERPLE (https://arxiv.org/abs/2205.09921) positional embeddings for Megatron encoder-decoder classes.

**Collection**: nlp megatron

# Changelog 

- Added implementation of KERPLE
- Added Jenkins test
- Added comment to document feature in YAML file (`examples/nlp/language_modeling/conf/megatron_model_base_config.yaml`)

# Usage

Set config
```
model.encoder.position_embedding_type: 'kerple'
model.decoder.position_embedding_type: 'kerple'
```


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add conformer post attn and non pos_emb
State: closed | Pull request: yes
Labels: name: stale

name: ASR

# What does this PR do ?

Add post attention for conformer and off pos_emb computation

**Collection**: ASR

# Changelog 
- Using prefix `post_` for all `self_attention_model` options will move attention after conv
- New `non_pos` option for `self_attention_model` for off computation of positional emb

# Usage
* You can potentially add a usage example below

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Megatron KERPLE positional embeddings
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

This PR adds support to KERPLE (https://arxiv.org/abs/2205.09921) positional embeddings for Megatron encoder-decoder classes.

**Collection**: nlp megatron

# Changelog 

- Added implementation of KERPLE
- Added Jenkins test
- Added comment to document feature in YAML file (`examples/nlp/language_modeling/conf/megatron_model_base_config.yaml`)

# Usage

Set config
```
model.encoder.position_embedding_type: 'kerple'
model.decoder.position_embedding_type: 'kerple'
```


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)