renwei2024/nemo-github-issues huggingface dataset:
Added ASR model comparison to SDE
State: closed | Pull request: yes

Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add `speaker_tasks` datasets folder, add diarization datasets voxconverse/aishell
State: closed | Pull request: yes

# What does this PR do ?

Adds a folder to store the `speaker_tasks` datasets. Few things to consider:

* I could create a nested `diarization` folder to be more specific, but I thought just one folder would be fine
* I might've missed some scripts that would be better suited to exist in this new folder

# Changelog 
- Add `speaker_tasks` datasets folder, add diarization datasets voxconverse/aishell.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
[CI] Enable previous build abort when new commit pushed
State: closed | Pull request: yes

Signed-off-by: SeanNaren <snarenthiran@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix bugs
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

Fix bugs relating to dialogue and question answering due to pytorch lightning and transformers version upgrade

**Collection**: nlp

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Updating Megatron LM conversion according to PTL 1.7
State: closed | Pull request: yes

# What does this PR do ?

Updating Megatron LM conversion according to PTL 1.7

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Updated megatron_lm_ckpt_to_nemo.py

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix bugs
State: closed | Pull request: yes

# What does this PR do ?

Fix bugs relating to dialogue and question answering due to pytorch lightning and transformers version upgrade

**Collection**: nlp

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
text_memmap dataset index range testing fix
State: closed | Pull request: yes

# What does this PR do ?

1. Improved range testing of text-memmap dataset when __len__ method  is overriden.

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix decoding bug for megatron enc-dec models with O2
State: closed | Pull request: yes
Labels: name: cherry-pick

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Fixes a bug that causes decoding for enc-dec models with O2 to crash

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Change the module being inspected with O2 to determine forward args.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Punctuation capitalization model docs
State: closed | Pull request: yes

# What does this PR do ?

Adds docs for p&c lexical audio model

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [X] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Punctuation Capitalization Lexical Audio model docs
State: closed | Pull request: yes

# What does this PR do ?

It adds docs for p&c lexical audio model.

**Collection**: nlp

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [X] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
patch to fix tao error for bug 3809486
State: closed | Pull request: yes
Labels: name: stale

# What does this PR do ?

Added patch for tao realted bug 3809486. Included functionality to restore the value of is_model_being_restore to true if user has sent restore_attr_is_model_being_restore=True

**Collection**: [Note which collection this PR will affect]

# Changelog 
- modified nlp_model.py to restore is_model_being_restore=True as at line 81( bert_model = get_lm_model(..) ) while intializing the model , the states is_model_being_restore is set to false . This  needs to be true to initialize the model (at line super().__init__(cfg, trainer) ). 

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add black to pre-commit
State: closed | Pull request: yes

# What does this PR do ?

Figured out how to do this with keeping click the correct version to allow this older black to run.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Cross-Lingual objectives (XLM) and multilingual (many-many) support for Megatron-NMT
State: closed | Pull request: yes
Labels: name: NLP

name: common

# What does this PR do ?

Adds the following

1. XLM + UL2-based datasets for BERT and SeqSeq
2. Multilingual support for MegatronNMT (x -> En, En -> x and x <-> y).

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add interleaved pipeline schedule to GPT
State: closed | Pull request: yes

# What does this PR do ?

Adds interleaved pipeline schedule to GPT.

The interleaved pipeline schedule helps to improve the efficiency of training pipeline parallel models. With the interleaved schedule, we split our model into chunks where the number of chunks is equal to the virtual pipeline model parallel size. By chunking our  model and using the interleaved pipeline schedule we can reduce the "pipeline bubble" and increase throughput.  See https://arxiv.org/pdf/2104.04473.pdf for more details.

Note: in order to run interleaved, Apex needs to have the virtual pipeline parallel setter

**Collection**: [NLP]

# Changelog 
- Make virtual pipeline size configurable
- Add interleaved pipeline functions
- Properly sync embeddings when using interleaved
- Properly skip loading inputs when using interleaved

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix bug in Squeezeformer Conv block
State: closed | Pull request: yes
Labels: name: cherry-pick

Signed-off-by: smajumdar <smajumdar@nvidia.com>

# What does this PR do ?

Fixes a bug in squeezeformer preventing loading of checkpoints.

**Collection**: [ASR]

# Changelog 
- Fix a calculation of groups that prevents loading of checkpoints

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Bug fix - Limit val batches set to 1.0 
State: closed | Pull request: yes

Signed-off-by: shanmugamr1992 <shanmugamr1992@gmail.com>

# What does this PR do ?
* Limit val batches 1.0 does not work as expected when num validation iterations * global batch size > num samples

**Collection**: [Note which collection this PR will affect]
Large language model

# Changelog 
- Add specific line by line info of high level changes in this PR.
* If limit val batches is set to 1.0 it automatically uses only one epoch of validation data for every validation iteration
* Bug fix when adding blendable dataset as values for train / test / validation keys . 

# Usage
* You can potentially add a usage example below
limit_val_batches: 1.0

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Remove guard, allow transformers online
State: closed | Pull request: yes

# What does this PR do ?

The transformers library should be fixed for offline, however we'll try to allow HF API calls based on the latest release to see if this is feasible.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix diarization ASR inference link in notebook
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

Fixes invalid link in notebook.

**Collection**: ASR

# Changelog 
- Fix invalid ASR inference link.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Sde transcribe
State: closed | Pull request: yes
Labels: name: stale

# What does this PR do ?

Adds new feature to SDE, and transcribe speech

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [+] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
need fastspeech2 customization
State: closed | Pull request: no

### Discussed in https://github.com/NVIDIA/NeMo/discussions/5018

<div type='discussions-op-text'>

<sup>Originally posted by **rkrohit14092** September 27, 2022</sup>
How to add additional words in the dictionary of fastspeech2. I want to customize the dictionary as per my use case. 
![image](https://user-images.githubusercontent.com/52836074/192561638-b3c7c51e-3c29-4a86-a926-9e36cc6067c5.png)
</div>
AttributeError: type object 'LightningModule' has no attribute 'trainer'
State: closed | Pull request: no
Labels: name: bug

/opt/miniconda3/envs/nemo/bin/python /home/henry/Projects/NeMo/nemo_text_processing/text_normalization/normalize.py --lang zh --text 最高气温38°C
[NeMo W 2022-09-27 17:45:38 optimizers:55] Apex was not found. Using the lamb or fused_adam optimizer will error out.
Traceback (most recent call last):
  File "/home/henry/Projects/NeMo/nemo_text_processing/text_normalization/normalize.py", line 38, in <module>
    from nemo.collections.common.tokenizers.moses_tokenizers import MosesProcessor
  File "/home/henry/Projects/NeMo/nemo/collections/common/__init__.py", line 16, in <module>
    from nemo.collections.common import data, losses, parts, tokenizers
  File "/home/henry/Projects/NeMo/nemo/collections/common/losses/__init__.py", line 15, in <module>
    from nemo.collections.common.losses.aggregator import AggregatorLoss
  File "/home/henry/Projects/NeMo/nemo/collections/common/losses/aggregator.py", line 19, in <module>
    from nemo.core.classes import Loss, typecheck
  File "/home/henry/Projects/NeMo/nemo/core/__init__.py", line 16, in <module>
    from nemo.core.classes import *
  File "/home/henry/Projects/NeMo/nemo/core/classes/__init__.py", line 33, in <module>
    from nemo.core.classes.modelPT import ModelPT
  File "/home/henry/Projects/NeMo/nemo/core/classes/modelPT.py", line 43, in <module>
    class ModelPT(LightningModule, Model):
  File "/home/henry/Projects/NeMo/nemo/core/classes/modelPT.py", line 1374, in ModelPT
    @LightningModule.trainer.getter
AttributeError: type object 'LightningModule' has no attribute 'trainer'
Fix diarization ASR inference link in notebook
State: closed | Pull request: yes

# What does this PR do ?

Fixes invalid link in notebook.

**Collection**: ASR

# Changelog 
- Fix invalid ASR inference link.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
add close inactive issues and PRs github action.
State: closed | Pull request: yes

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

Add a GitHub action to automatically stale and close issues and PRs. The grace periods and actions are shown below,
* Add a label "Stale" on issues and pull requests after 60 days of inactivity and comment on them.
* Close the stale issues after 14 days of inactivity.
* Don't close stale pull requests.
 
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix bug in Squeezeformer Conv block
State: closed | Pull request: yes

Signed-off-by: smajumdar <smajumdar@nvidia.com>

# What does this PR do ?

Fixes a bug in squeezeformer preventing loading of checkpoints.

**Collection**: [ASR]

# Changelog 
- Fix a calculation of groups that prevents loading of checkpoints

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Speech Data Explorer installation instructions are incomplete
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

Following the instructions here, speech data explorer doesn't work out of the box because some dependencies aren't installed: https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/tools/speech_data_explorer.html#getting-started

**Steps/Code to reproduce bug**

Start a new conda environment (I used python 3.8),  then:
```
git clone --recursive https://github.com/NVIDIA/NeMo.git
cd NeMo
pip install -r tools/speech_data_explorer/requirements.txt
python tools/speech_data_explorer/data_explorer.py
```

Gives:

```
Traceback (most recent call last):
  File "tools/speech_data_explorer/data_explorer.py", line 42, in <module>
    from plotly import express as px
  File "/home/dgalvez/scratch/miniconda3/envs/nemo/lib/python3.8/site-packages/plotly/express/__init__.py", line 10, in <module>
    raise ImportError(
ImportError: Plotly express requires pandas to be installed.
```

A  helpful guide on on how to craft a minimal bug report  http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports. 


**Expected behavior**

Clearly some of the requirements of speech data explorer are missing. It works after I do `pip install pandas` manually.

**Environment overview (please complete the following information)**

 - Environment location: [Bare-metal, Docker, Cloud(specify cloud provider - AWS, Azure, GCP, Collab)]
 - Method of NeMo install: [pip install or from source]. Please specify exact commands you used to install.
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
Ubuntu 18.04
python 3.8
What is the key to the better ASR performance of the 1.6.0 version of the Conformer-Transducer-Large model?
State: closed | Pull request: no

Hello, thanks for the great and helpful toolkit! 

As said in [page](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/stt_en_conformer_transducer_large), the 1.6.0 version of the Conformer-Transducer-Large model has much better word error rate performance on Librispeech than the version 1.4.0 (3.6/1.7 vs 4.3/1.9). 

Could you please tell me the key to the better word error rate? Is it the training dataset, the model config or the training recipe?

Many Thanks!
Question about speaker interpolation in Fastpitch
State: closed | Pull request: no
Labels: name: TTS

name: stale

Hi, thanks for the great work!
I saw the recently added speaker interpolation tutorial and decided to have a try.

Sometime ago I followed the `FastPitch_Finetuning.ipynb` tutorial and mixed a small dataset with a big dataset and did the finetuning with `n_speakers=2`. This model can be used to do speaker interpolation, is that right?

The speaker interpolation tutorial says `Choose new_speaker ID such that it is not already assigned to an existing speaker` and in [fastpitch.py](https://github.com/NVIDIA/NeMo/blob/a74f262dbd39aff9d7ec3c12c9322c3a4036a97c/nemo/collections/tts/models/fastpitch.py#L687-691) it says `the new_speaker_id should be less than the total number of speakers`.

So in my case, speaker 0, 1 have been assigned, and `n_speakers=2`. How should I set the `new_speaker` ID?

Thank you!
Add test for nested ASR model
State: closed | Pull request: yes

Signed-off-by: smajumdar <smajumdar@nvidia.com>

# What does this PR do ?

Adds a test for Nested ASR model with detached state dict management.

**Collection**: [ASR]

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [x] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
[TTS] bugfix 'EnglishPhonemesTokenizer' object has no attribute 'encode_from_g2p'
State: closed | Pull request: yes
Labels: name: cherry-pick


Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

fix the bug of tokenizer below,
```
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In [16], line 13
     10 print(pron2_g2p)
     12 # Tokenize!
---> 13 pron1_tokens = aligner.tokenizer.encode_from_g2p(pron1_g2p)
     14 pron2_tokens = aligner.tokenizer.encode_from_g2p(pron2_g2p)
     15 print("\n=== Tokens===")

AttributeError: 'EnglishPhonemesTokenizer' object has no attribute 'encode_from_g2p'
```

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix filename extraction in vad_utils.py
State: closed | Pull request: yes

Signed-off-by: Ondřej Vacek <ondrej.vacek@entry-cz.com>

# What does this PR do ?
- Fix filename extraction in vad_utils.py
- Old behavior makes vad_infer script unfunctional on windows since it assumed path separator is always "/"
Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]
- asr.parts.utils.vad_utils.py 
---> load_tensor_from_file(filepath: str) -> Tuple[torch.Tensor, str]:

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Makes filename extraction os independent in vad_utils.py
State: closed | Pull request: yes

# What does this PR do ?

- Makes filename extraction os independent in vad_utils.py
- Old behavior makes vad_infer script unfunctional on windows since it assumed path separator is always "/"

**Collection**: [Note which collection this PR will affect]
- nemo.collections.asr.parts.utils.vad_utils 


# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [X] Bugfix


If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] FastPitch speaker interpolation
State: closed | Pull request: yes

# What does this PR do ?

Adds support for FastPitch speaker interpolation

**Collection**: [Note which collection this PR will affect]
TTS

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] FastPitch speaker interpolation
State: closed | Pull request: yes

# What does this PR do ?

Adds speaker interpolation support to FastPitch

**Collection**: [Note which collection this PR will affect]
TTS

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
megatron_lm_ckpt_to_nemo.py  and the hparams.yaml
State: closed | Pull request: no
Labels: name: feature

The example at the top of megatron_lm_ckpt_to_nemo.py  sugests that you can convert the megatron-lm checkpoint without hparams.yaml file. I did not spend huge amount of time trying to understand this code but its seems not possible and the only reason why hparams is not a manadatory argument seems to be because one can optionally pass cnf via kwargs.

Could we add literally one sentence describing that in the top of the file preferably also providing an example of a yaml file - happy to send out working 5B example for GPT.
[TTS] bugfix 'EnglishPhonemesTokenizer' object has no attribute 'encode_from_g2p'
State: closed | Pull request: yes


Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

fix the bug of tokenizer below,
```
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In [16], line 13
     10 print(pron2_g2p)
     12 # Tokenize!
---> 13 pron1_tokens = aligner.tokenizer.encode_from_g2p(pron1_g2p)
     14 pron2_tokens = aligner.tokenizer.encode_from_g2p(pron2_g2p)
     15 print("\n=== Tokens===")

AttributeError: 'EnglishPhonemesTokenizer' object has no attribute 'encode_from_g2p'
```

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix ASR issues
State: closed | Pull request: yes
Labels: name: cherry-pick

Signed-off-by: smajumdar <smajumdar@nvidia.com>

# What does this PR do ?
Bug fixes

**Collection**: [ASR]

# Changelog 
- Add specific line by line info of high level changes in this PR.

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
[TTS] Remove PhonemizerTokenizer
State: closed | Pull request: yes

Signed-off-by: Ryan <rlangman@nvidia.com>

# What does this PR do ?

Removing Phonemizer/eSpeak dependency.

**Collection**: [TTS]

# Changelog 
- Remove PhonemizerTokenizer and associated code

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation
Fix decoding bug for megatron enc-dec models with O2
State: closed | Pull request: yes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Fixes a bug that causes decoding for enc-dec models with O2 to crash

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Change the module being inspected with O2 to determine forward args.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)