renwei2024/nemo-github-issues huggingface dataset:
NeMo not compatible with latest numpy
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

Numpy has deprecated several type alias' in 1.24 (https://numpy.org/devdocs/release/1.24.0-notes.html)

NeMo requires `numpy>=1.22`, meaning a clean installation results in 1.24.3 at the time of writing

running `NeMo/examples/nlp/dialogue/dialogue.py` results in an exception due to the use of deprecated numpy type alias i.e.

```
File "[/home/dave/dev/nemo-dialogue/NeMo/nemo/collections/nlp/data/dialogue/dataset/dialogue_bert_dataset.py](https://file+.vscode-resource.vscode-cdn.net/home/dave/dev/nemo-dialogue/NeMo/nemo/collections/nlp/data/dialogue/dataset/dialogue_bert_dataset.py)", line 114, in __getitem__
    np.array(self.all_input_mask[idx], dtype=np.long),
  File "[/home/dave/dev/nemo-dialogue/.venv/lib/python3.10/site-packages/numpy/__init__.py](https://file+.vscode-resource.vscode-cdn.net/home/dave/dev/nemo-dialogue/.venv/lib/python3.10/site-packages/numpy/__init__.py)", line 320, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'long'
```

**Steps/Code to reproduce bug**

Downloaded this tutorial to my local machine and ran it in a clean `venv` (python310 on ubuntu 22.04)

https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/nlp/Dialogue.ipynb

**Expected behavior**

NeMo requirements should be `numpy>=1.22,<1.24` until numpy >=1.24 is supported (I also downgraded to numba==0.56.4)

**Environment overview (please complete the following information)**

 - Environment location: Bare-metal
 - Method of NeMo install: [pip install or from source]. Source

```
import os 
BRANCH = 'main'
!apt-get update && apt-get install -y libsndfile1 ffmpeg
!git clone https://github.com/NVIDIA/NeMo --branch $BRANCH
os.chdir('NeMo')
!./reinstall.sh
os.chdir('..')
```

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version  Ubuntu 22.04
- PyTorch version 2.0.1
- Python version 3.10

**Additional context**
Error finetune hifigan：stack expects each tensor to be equal size
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**
When using Nemo 1.18.1 to fine tune hifigan, I can't continue at this step, and the number in the error message is different every time. After adjusting according to the fine-tuning of fastpitch, some iussues mentioned the sampling rate issue and also adjusted the sampling rate, but the result still did not work
![image](https://github.com/NVIDIA/NeMo/assets/19503929/c9e8de04-69db-4e3e-a834-f5defe2a57c4)


A clear and concise description of what the bug is.

**Steps/Code to reproduce bug**
Command I use to run the finetuning:
![image](https://github.com/NVIDIA/NeMo/assets/19503929/a05ee461-ecac-441b-976e-ac0a45a8c63b)


Please list *minimal* steps or code snippet for us to be able to reproduce the bug.

A  helpful guide on on how to craft a minimal bug report  http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports. 


**Expected behavior**

A clear and concise description of what you expected to happen.

**Environment overview (please complete the following information)**

 - Environment location: [Bare-metal, Docker, Cloud(specify cloud provider - AWS, Azure, GCP, Collab)]
 - Method of NeMo install: [pip install or from source]. Please specify exact commands you used to install.
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version
- PyTorch version
- Python version

**Additional context**

Add any other context about the problem here.
Example: GPU model
Add trainer.validate example for GPT
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Enables an example of calling trainer.validate with GPT.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
add reference to our paper
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

add reference to our SpellMapper paper on arxiv

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update core version in Jenkinsfile
State: closed | Pull request: yes
Labels: name: cherry-pick

name: CI

# What does this PR do ?

Update core version in Jenkinsfile

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
lora pp2
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update core version in Jenkinsfile
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Update core version in Jenkinsfile

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
cherry pick 6788
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Attention matrix optional, cherry picked from https://github.com/NVIDIA/NeMo/pull/6788/files

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add ub communicator initialization to validation step
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Fix TP-comm overlap with userbuffer, where validation step precedes a training step.

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
ASR Confidence update and tutorial
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Minor fixes, renames, tests and tutorial.

**Collection**: ASR

# Changelog 
- Remames: `renui` -> `renyi`, `method` -> `measure`.
- Some classes and methods are moved to separate files.
- Tests added.
- Tutorial added.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

@Kipok @titu1994
`intent_slot_classification_model` tutorial wont converge
State: closed | Pull request: no
Labels: name: bug

name: stale

I'm trying to run the `intent_slot_classification_model` and I'm not getting anywhere near the accuracy that the workbook implies I should. I'm running the tutorial from `Joint_Intent_and_Slot_Classification.ipynb`

The workbook contains the following markedown

```
After training for 5 epochs, which should take no more than few minutes, you can expect training precision for this data set to be around these numbers (the accuracy will gradually continue to improve for this dataset up to about 50 epochs of training): 

Intents:

    micro avg                                               85.04      85.04      85.04       1076
    macro avg                                               81.13      80.81      79.36       1076
    weighted avg                                            84.10      85.04      83.54       1076
    
Slots:

    -------------------
    micro avg                                               89.10      89.10      89.10       7199
    macro avg                                               21.86      18.56      18.18       7199
    weighted avg                                            84.42      89.10      86.01       7199
```

But I'm not able to reproduce this, the training / eval loss is stuck around 4.12 and the precision/recall/f1 is really low (less than 2%) - I don't actually get any improvement after the first epoch.

I've tried on bare-metal and aws sagemaker with the exact same results.

**Expected behavior**

Expect much better precision/recall when running tutorial workbook

**Environment overview (please complete the following information)**

 - Environment location: AWS Sagemaker (Pytorch 2.0.0 Python 3.10 GPU Optimised)
 - Method of NeMo install: [pip install or from source]. Ran the install command from the workbook

```
BRANCH = 'stable'
!python -m pip install git+https://github.com/NVIDIA/NeMo.git@$BRANCH#egg=nemo_toolkit[nlp]
```

 - If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Environment details**

AWS Sagemaker (Pytorch 2.0.0 Python 3.10 GPU Optimised)

(same results running on bare metal with torch 2.0.1 with 3090 RTX GPU)

Note I don't have Apex installed, I suspect perhaps I'm using a newer version of lightning that the one that was used when the notebook was created but not really sure?
Add ub communicator initialization to validation step
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Fix TP-comm overlap with userbuffer, where validation step precedes a training step.

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Add cosine distance option to TTS aligner
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Add configuration to TTS aligner which uses scaled cosine distance instead of euclidean distance.

**Collection**: [TTS]

# Changelog 
- Add code for cosine distance based alignment

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

# Additional Information

The current aligner used for FastPitch training has a few problem:

1. It aligns slowly for datasets with lots of speakers.
2. If binarization loss is enabled/warmup too soon then alignment learning stops before accurate alignments are learned.
3. When fine-tuning, the model does not learn alignment of target speaker well.

Using cosine distance instead of L2 distance can fix these problems. Cosine distance in general a lot more accurate and stable when it comes to similarity problems. It also avoids possible issues with the binarization loss because the scale of the cosine distance is fixed regardless of the scale or content of the alignment embeddings.

To test this I trained FastPitch on the VCTK dataset for 300k steps and provide the alignments and loss graphs below.

As seen, using cosine distance aligns faster, produces more accurate and sharper alignments, has a lot lower alignment loss (especially binarization loss), and as a result converges to a model with significantly better accuracy all around (mel, duration, pitch, energy). Though the gap between the two would likely shrink if we train it for several million steps.

**Alignments**

_Example 1_

`L2`
![align_l2_1](https://github.com/NVIDIA/NeMo/assets/9942053/3a14f1b4-7c09-453b-906a-74ce01cb0c9a)
`Cosine`
![align_cos_1](https://github.com/NVIDIA/NeMo/assets/9942053/9d7397ff-1944-4443-bcbd-4025b0b95189)

_Example 2_

`L2`
![align_l2_2](https://github.com/NVIDIA/NeMo/assets/9942053/e92098a7-4b51-4422-ae62-0a80eee050a5)
`Cosine`
![align_cos_2](https://github.com/NVIDIA/NeMo/assets/9942053/a22917ee-f8b0-4b43-a567-f3eda43b8d88)

**Alignment Losses**

![ctc_loss](https://github.com/NVIDIA/NeMo/assets/9942053/b3fed69c-686e-470b-b44e-7222d3594d99)
![bin_loss](https://github.com/NVIDIA/NeMo/assets/9942053/1b802736-db78-43a7-9fed-5eb3d2c7b790)

**Reconstruction Losses**

![mel_loss](https://github.com/NVIDIA/NeMo/assets/9942053/12346ca2-9c5f-48aa-a0cc-4eb3bef281de)
![dur_loss](https://github.com/NVIDIA/NeMo/assets/9942053/437090d7-083a-48f9-9aa8-574e9d439df1)
![pitch_loss](https://github.com/NVIDIA/NeMo/assets/9942053/3dd307a9-5569-4847-a0b1-a23e0ad2b986)
![energy_loss](https://github.com/NVIDIA/NeMo/assets/9942053/0b15fefe-c20a-4ec4-9df5-225446082927)
Bug fix for reset_sequence_parallel_args
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Megatron MPT-7B Support
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Adds changes to Megatron GPT which supports the MPT-7B architecture

**Collection**: NLP

# Changelog 
- nemo/collections/nlp/models/language_modeling/megatron/gpt_model.py
   + Adds support for the megatron_legacy argument to flow down to CoreAttention
   + Adds a bypass to load_state_dict which lets you load directly into the language_model

- nemo/collections/nlp/models/language_modeling/megatron_base_model.py
   + Fixes an error caused by register_artificat demanding cfg.tokenizer.model when one isn't needed, for HF tokenisers

- nemo/collections/nlp/models/language_modeling/megatron_gpt_model.py
  + Adds support for the megatron_legacy flag
  + Adds an optional override_vocab_size flag (necessary for hard import of HF weights). If this param isn't given, it loads padded_vocab_size as normal

- nemo/collections/nlp/modules/common/megatron/layer_norm_1p.py
  + Added support for the MPT-7B lower precision fp16 layer norm which works without a bias. Unfortunately, the existing lower precision layer norm from apex absolutely requires a bias term

- nemo/collections/nlp/modules/common/megatron/transformer.py
  + Added logic to remove bias terms on layer norm layers

# Usage
* You can restore an MPT-7B model using the MegatronGPTModel class with `restore_from`

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [X] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix spellmapper tutorial, change branch to main
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?
fix spellmapper tutorial, change branch to main, small fix in markup syntax

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Bug fix for reset_sequence_parallel_args
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Bug fix for reset_sequence_parallel_args
State: closed | Pull request: yes
Labels: name: core

name: TTS

name: ASR

name: NLP

name: Speaker Tasks

name: CI

# What does this PR do ?
Bug fix

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix GPTDataset Assert
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Fixes an assert in the GPTDataset

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
added changes to ramp up bs 
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

1. Fixed consumed samples calculation for changeable batch size.
2. Changed ramp up bs feature in order to use it with changeable number of nodes duriong training:
      - Deleted unnecessary assertions for ramp up batch size.
      - Added trainer stop flag in order to do validation and save the checkpoint when global batch size is changed.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix GPTDataset Assert
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Fixes an assert in the GPTDataset

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[question] Pad time-dimension to a multiple
State: closed | Pull request: no
Labels: name: stale

For Conformer setups, does NeMo support / recommend padding batches after frontend to a certain multiple, e.g. so that after Conformer input batch time dimension is e.g. a multiple of 32 (which may suit best cudnn / gemm kernels for fp16/TensorCores)

Having batches of a few fixed shapes also benefits pytorch allocator performance and cudnn benchmarking / cache
[Bug] A bug in filtering checkpoints
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

https://github.com/NVIDIA/NeMo/blob/cfbe0924db02a1a557aeeea55cc1a01b83903e71/nemo/utils/callbacks/nemo_model_checkpoint.py#L98-L99

I think Line 99 should be
```python
 if index != len(self.monitor): 
```
Add trainer.validate example for GPT
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Enables an example of calling trainer.validate with GPT.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Tensor-parallel communication overlap with userbuffer backend
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Add (1) interfaces to TE and initialized (2) process group setting to support tensor-parallel communication overlap with userbuffer backend.

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
Set `ub_tp_comm_overlap` to `True`

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [*] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Debug Transformer Engine FP8 support with Megatron-core infrastructure
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

name: CI

# What does this PR do ?

Debugs Transformer Engine FP8 support with Megatron-core infrastructure

**Collection**: NLP

# Changelog
- Construct FP8 amax reduction group 

# Usage

Set `transformer_engine: True` and `fp8: True` in the config.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

Pinging @aklife97 @yen-shi @erhoo82 @ksivaman 

# Additional Information
* Requires https://github.com/NVIDIA/Megatron-LM/commit/f267e6186eae1d6e2055b412b00e2e545a8e896a
* Fixes bug from https://github.com/NVIDIA/NeMo/pull/6393
* Subset of https://github.com/NVIDIA/NeMo/pull/6627
* Closes https://github.com/NVIDIA/NeMo/issues/6625
Lddl bert
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

This adds the ability to use LDDL dataloaders in Nemo FW

**Collection**: [NLP, NemoFW

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage

add LDDL as dataloader in config. Instructions will be added to the Megatron Launcher for data prep and use.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
-  New Feature



## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update asr_normalize with changes in main
State: closed | Pull request: yes
Labels: name: core

name: TTS

name: ASR

name: NLP

name: CI

name: common

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Gpt alibi fa
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

* merged main
* make computing full logprobs matrix during inference optional
* make computing attention matrix during inference optional (to save memory if used with FA)

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix get_parameters when using main params optimizer
State: closed | Pull request: yes
Labels: name: core

name: cherry-pick

name: NLP

# What does this PR do ?

When using fused_adam, self._optimizer.get_parameters() was returning an empty list.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
add call to p2p overlap
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

name: CI

# What does this PR do ?

add call to p2p overlap 

**Collection**: NLP
# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
peft eval directly from ckpt
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)