renwei2024/nemo-github-issues huggingface dataset:
T5 Prompt Learning Fixes for Pipeline Parallel
State: closed | Pull request: yes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Fixes hanging with T5 prompt learning for PP > 1

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
`AutoTokenizer` produces non-invertible tokens
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

The wrapper for HF tokenizers misses some key functionality.

**Steps/Code to reproduce bug**

```python
from nemo.collections.nlp.modules.common.tokenizer_utils import get_tokenizer


tokenizer = get_tokenizer(
    tokenizer_name="bert-base-multilingual-cased",
    tokenizer_model="huggingface"
)
# The problem
print(f"{tokenizer.tokenizer.do_basic_tokenize=}")

text = "Knock, knock. Who's there? Interrupting cow. Interrupting cow wMOOOOOOOOOOOOOOO"

print(tokenizer.ids_to_text(tokenizer.text_to_ids(text)))
```

Output:

```
Knock , knock . Who ' s there ? Interrupting cow . Interrupting cow wMOOOOOOOOOOOOOOO
```

**Expected behavior**
Expected that for some tokenizer, `ids_to_text(text_to_ids) == text`.

We can accomplish this if we reach under NeMo's wrapper:

```python
print(tokenizer.tokenizer.decode(tokenizer.text_to_ids(text)))
```

Output:

```
Knock, knock. Who's there? Interrupting cow. Interrupting cow wMOOOOOOOOOOOOOOO
```
Fixes for docs/typos + remove max_utts parameter from tarred datasets as it causes hang in training
State: closed | Pull request: yes

# What does this PR do ?

Fixes for docs/typos + remove max_utts parameter from tarred datasets as it causes hang in training

**Collection**: asr mainly

# Changelog 

The changes should be straightforward in most cases. The only thing to double check is that I fully removed `max_utts` parameter from tarred datasets (confirmed that it's not usable for ASR). But I'm not sure if there are some additional use-cases of this parameter besides ASR (e.g., TTS?) that might still be valid and working - but at least it does not seem to be present in any of the checked-in configs.

Another thing to mention is about changes in tutorials/00_NeMo_Primer.ipynb - I've only removed ";" at the end of some lines, since it seems like it was there by mistake (e.g., in cases of `model.summarize()` - we'd want to display the output, right?). But please let me know if it's there on purpose and I can put it back.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
pp2 support for T5 IA3 learning and T5 Adapters learning
State: closed | Pull request: yes

Signed-off-by: arendu <adithya.r@gmail.com>

# What does this PR do ?

Adds pipeline parallel support for T5 based IA3 and Adapter learning and inference.

**Collection**: [NLP]

# Changelog 
- all changes are in the `megatron_t5_adapters_model.py` file.
- added CI tests 

# Usage
* You can potentially add a usage example below

```python
python examples/nlp/language_modeling/tuning/megatron_t5_adapter_tuning.py
python examples/nlp/language_modeling/tuning/megatron_t5_adapter_eval.py
python examples/nlp/language_modeling/tuning/megatron_t5_ia3_tuning.py
python examples/nlp/language_modeling/tuning/megatron_t5_ia3_eval.py
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
bugfix: pybtex.database.InvalidNameString: Too many commas in author …
State: closed | Pull request: yes
Labels: name: cherry-pick

…field.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix the self-attention export bug for cache-aware streaming Conformer
State: closed | Pull request: yes

# What does this PR do ?

Fixed the self-attention export bug for cache-aware streaming Conformer

# Changelog 
- There was an issue with ONNX conversion of cache-aware streaming Conformer in the self-attention module which raised error for empty caches. Fixed it with wrapping the code by torch script.


**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
bugfix: pybtex.database.InvalidNameString: Too many commas in author …
State: closed | Pull request: yes

…field.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Apex Not Found. Provided solution does not solve. 
State: closed | Pull request: no
Labels: name: bug

**The bug**

Megatron GPT training requires NVIDIA Apex to be installed. Token_Classification-BioMegatron.ipynb does not import module and provided solution found through (Error(s): Apex was not found. Please see the NeMo README for installation instructions: https://github.com/NVIDIA/NeMo#megatron-gpt.) does not solve import error. 

**To reproduce bug**

Run script through google colab as presented on Nvidia nemo tutorials or through provided github. Notebook will error out at line 25 when model is initialized. Follow steps listed in error messaging to attempt to correct error. 

**Expected behavior**

Notebook will error out at line 25 when model is initialized and return "Error(s): Apex was not found. Please see the NeMo README for installation instructions: https://github.com/NVIDIA/NeMo#megatron-gpt."

**Environment overview (please complete the following information)**

 - Environment location: Colab
 - Method of NeMo install: 
 !python -m pip install git+https://github.com/NVIDIA/NeMo.git@$BRANCH#egg=nemo_toolkit[nlp]
from nemo.collections import nlp as nemo_nlp
from nemo.utils.exp_manager import exp_manager
import os
import wget 
import torch
import pytorch_lightning as pl
from omegaconf import OmegaConf

**Environment details**

- OS version: posix
- PyTorch version: 1.12.1+cu113
- Python version: Python 3.7.14

**Additional context**

Standard GPU selection on Colab

**Screen capture after implementing official solution**
![BioMeg_ModuleNotFound_10_2022](https://user-images.githubusercontent.com/107068406/194599521-9837f3f4-3aaa-44c5-ba34-a216ce94c2db.png)
[Bug fix] PC lexical + audio
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[Bug fix] PC lexical + audio
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Asr concat dataloader
State: closed | Pull request: yes

# What does this PR do ?

Adds support for concat dataloader into ASR models

**Collection**: ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
Set the following dataset variables:
train_ds["is_concat"] = True
train_ds["concat_sampling"] = "random"
train_ds["concat_probabilities"] = [0.3, 0.6, 0.1]
train_ds["manifest_filepath"] = ["path1", "path2", "path3"]
In this case, there are three manfests and they will be sampled from with the probabilities provided.

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ X] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Multiprocessing fix
State: closed | Pull request: yes
Labels: name: cherry-pick

Signed-off-by: Matvei Novikov <mattyson.so@gmail.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Multiprocessing fix
State: closed | Pull request: yes

Signed-off-by: Matvei Novikov <mattyson.so@gmail.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Uppdate container version to 22.09
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Transformer Engine Integration
State: closed | Pull request: yes

# What does this PR do ?

This PR integrates NeMo Megatron with Transformer Engine: https://github.com/NVIDIA/TransformerEngine

Transformer Engine enables FP8 training. Currently this will only be enabled for GPT but is planned for all NeMo Megatron models.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
T5 Decoding with PP > 2 fix
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

Fixes a bug that would cause decoding with an enc-dec model with PP > 2 to hang.

**Collection**: NLP

# Changelog 
- Set sequence lengths for the pipeline properly while decoding. 

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] fixed wrong val loss for epoch 0 and inconsistent metrics names
State: closed | Pull request: yes
Labels: name: cherry-pick

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?
This PR fixed two issues below,
1. initial checkpoint got wrong `val_loss` so the name was not correct. The root-cause is that `trainer.max_epochs=1` is less than `train.check_val_every_n_epoch=5`. So specify `train.check_val_every_n_epoch=1` would fix the issue for FastPitch. Similar fix for HiFiGAN. See results comparison below,
```
# before fix:
$ ls resultGermanTTS/FastPitch/2022-10-05_12-35-50/checkpoints/
 FastPitch.nemo  'FastPitch--val_loss=0.0000-epoch=1-last.ckpt 
$ ls resultGermanTTS/HifiGan/2022-10-05_19-29-39/checkpoints/
 HifiGan.nemo  'HifiGan--val_loss=0.0000-epoch=1-last.ckpt'

# after fix:
$ ls resultGermanTTS/FastPitch/2022-10-05_14-18-25/checkpoints/
 FastPitch.nemo  'FastPitch--val_loss=4.0512-epoch=0.ckpt'  'FastPitch--val_loss=4.0512-epoch=1-last.ckpt'
$ ls resultGermanTTS/HifiGan/2022-10-05_20-10-50/checkpoints/
 HifiGan.nemo  'HifiGan--val_loss=0.5253-epoch=0.ckpt'  'HifiGan--val_loss=0.5253-epoch=0-last.ckpt'
```
2. TTS loss metrics used `v_loss` rather than `val_loss` as the keyword for validation loss. So this check (https://github.com/NVIDIA/NeMo/blob/r1.12.0/nemo/utils/exp_manager.py#L906) was never succeeded. Rename to `val_loss` helps to make the check valid.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
T5 prompt learning fixes missing from r.11.0 merge
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

Adds fixes that were missing from r1.11.0 merge to main.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
add autoflake to remove unused imports
State: closed | Pull request: yes

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?
This hook does following process. I also manually add `# noqa` to avoid changes for some lines of codes for safety purpose. Domain experts please help to review. 
```
# after process, we fixed,
22 for Unused import
1 for 'import *' may pollute namespace
```
```
  --expand-star-imports
                        expand wildcard star imports with undefined names; this only triggers if there is only one star import in the file; this is skipped if there are any uses of `__all__` or `del` in the file
  --remove-all-unused-imports
                        remove all unused imports (not just those from the standard library)
  --ignore-init-module-imports
                        exclude __init__.py when removing unused imports
  --ignore-pass-after-docstring
                        keep `pass` statements after a new line ending on """
```

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix numba
State: closed | Pull request: yes

Signed-off-by: smajumdar <titu1994@gmail.com>

# What does this PR do ?

Manually apply patches for numba checks.

**Collection**: [ASR]

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Fix Unhashable type list for Numba Cuda spec augment kernel
State: closed | Pull request: yes
Labels: name: cherry-pick

Signed-off-by: smajumdar <smajumdar@nvidia.com>

# What does this PR do ?

Fixes an issue where Numba Spec Augment kernel can no longer execute due to passing a list of values for `blocks_per_grid` on Numba > 0.56.0

**Collection**: [ASR]

# Changelog 
- Convert the list to tuple before passing to numba kernel launch

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

Closes #4863
Fix Unhashable type list for Numba Cuda spec augment kernel
State: closed | Pull request: yes

Signed-off-by: smajumdar <smajumdar@nvidia.com>

# What does this PR do ?

Fixes an issue where Numba Spec Augment kernel can no longer execute due to passing a list of values for `blocks_per_grid` on Numba > 0.56.0

**Collection**: [ASR]

# Changelog 
- Convert the list to tuple before passing to numba kernel launch

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

Closes #4863
Make it possible to specify output_filename in normalize_with_audio.py
State: closed | Pull request: yes

Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com>

# What does this PR do ?

Makes it possible to specify output_filename in normalize_with_audio.py

**Collection**: [Note which collection this PR will affect]
nemo_text_processing

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
python <nemo>/nemo_text_processing/text_normalization/normalize_with_audio.py \
    --audio_data <audio_data> \
    --language <lang> \
    --output_filename "outputs.json"
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
T5 Decoding with PP > 2 fix
State: closed | Pull request: yes

# What does this PR do ?

Fixes a bug that would cause decoding with an enc-dec model with PP > 2 to hang.

**Collection**: NLP

# Changelog 
- Set sequence lengths for the pipeline properly while decoding. 

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] remove phonemizer.py
State: closed | Pull request: yes

Signed-off-by: Xuesong Yang [1646669+XuesongYang@users.noreply.github.com](mailto:1646669+XuesongYang@users.noreply.github.com)
# What does this PR do ?

Manually cherry pick https://github.com/NVIDIA/NeMo/pull/5071 to main because automatically cherry-pick was ignored.

we decided to remove phonemizer python script. We decided to add an instruction about how to transliterate graphemes into phonemes using phonemizer + espeak in the tutorial.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
update strategy in notebook from ddp_fork to dp
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

Fix qa notebook bug when device has >1 gpu

**Collection**: nlp

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
update strategy in notebook from ddp_fork to dp
State: closed | Pull request: yes

# What does this PR do ?

Fix qa notebook bug when device has >1 gpu

**Collection**: nlp

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] fixed wrong val loss for epoch 0 and inconsistent metrics names
State: closed | Pull request: yes

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?
This PR fixed two issues below,
1. initial checkpoint got wrong `val_loss` so the name was not correct. The root-cause is that `trainer.max_epochs=1` is less than `train.check_val_every_n_epoch=5`. So specify `train.check_val_every_n_epoch=1` would fix the issue for FastPitch. Similar fix for HiFiGAN. See results comparison below,
```
# before fix:
$ ls resultGermanTTS/FastPitch/2022-10-05_12-35-50/checkpoints/
 FastPitch.nemo  'FastPitch--val_loss=0.0000-epoch=1-last.ckpt 
$ ls resultGermanTTS/HifiGan/2022-10-05_19-29-39/checkpoints/
 HifiGan.nemo  'HifiGan--val_loss=0.0000-epoch=1-last.ckpt'

# after fix:
$ ls resultGermanTTS/FastPitch/2022-10-05_14-18-25/checkpoints/
 FastPitch.nemo  'FastPitch--val_loss=4.0512-epoch=0.ckpt'  'FastPitch--val_loss=4.0512-epoch=1-last.ckpt'
$ ls resultGermanTTS/HifiGan/2022-10-05_20-10-50/checkpoints/
 HifiGan.nemo  'HifiGan--val_loss=0.5253-epoch=0.ckpt'  'HifiGan--val_loss=0.5253-epoch=0-last.ckpt'
```
2. TTS loss metrics used `v_loss` rather than `val_loss` as the keyword for validation loss. So this check (https://github.com/NVIDIA/NeMo/blob/r1.12.0/nemo/utils/exp_manager.py#L906) was never succeeded. Rename to `val_loss` helps to make the check valid.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
only set adapter params in the optimizer_param_groups
State: closed | Pull request: yes

Signed-off-by: arendu <adithya.r@gmail.com>

# What does this PR do ?
- GPT and T5 adapter/ia3 training depends on requires_grad=False to be obeyed by all Apex modules, because all the params of the frozen model are supplied into the optimizer_param_groups.
- This PR only supplies the optimizer_param_groups the parameters that only belong to the adapters/ia3 and does not supply params from the base model.

Add a one line overview of what this PR aims to accomplish.

**Collection**: [NLP]

# Changelog 
- Modified the `setup_optimizer_param_groups` method in GPTBaseAdapter and T5BaseAdapter classes.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Release SOTA Lang ID model 
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

Release SOTA lang id model AmberNet which achieves 5.22% error rate on the official evaluation set of [voxlingua107](http://bark.phon.ioc.ee/voxlingua107/) and update doc.

**Collection**: ASR

# Changelog 
# Usage

```python
langidmodel = nemo_asr.models.EncDecSpeakerLabelModel.from_pretrained(model_name="langid_ambernet")

```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Notebook bug fixes
State: closed | Pull request: yes

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Fixes Entity Linking and Prompt Learning Notebook Bugs:
https://nvbugs/3802936 [ML-NeMo][A40] Entity_Linking_Medical.ipynb failed with TypeError: __init__() got an unexpected keyword argument 'checkpoint_callback'
https://nvbugs/3801562 [ML-NeMo][A40] Multitask_Prompt_and_PTuning.ipynb contains broken links
https://nvbugs/3803232 [ML-NeMo][A40] Multitask_Prompt_and_PTuning.ipynb failed with FileNotFoundError: [Errno 2] No such file or directory: 'data/SQuAD/train-v1.1.json'

**Collection**: NLP

# Changelog 
- Updates entity linking and prompt learning notebooks

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Integrate megatron core
State: closed | Pull request: yes
Labels: name: stale

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

NeMo Megatron will and Megatron-lm will use a shared core so that more model parallelism code can be shared between the two repositories. 

Apex dependency will become optional and only used for performance improvements.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
How to train/Finetune for Custom AD(Activity Detection), like laughing, etc
State: closed | Pull request: no
Labels: name: stale

Hi all!

I am working on developing pipelines for Laughing, Crying etc. I want to achieve them, like VAD in pyannote/Nemo/silero pipeline. How can I use Nemofor training purposes? Moreover, I want to use the pipeline on device: Android/IoS, so kindly also guide me in that direction as well..

Thanks