renwei2024/nemo-github-issues huggingface dataset:
Expose ClusteringDiarizer device
State: closed | Pull request: yes
Labels: name: bug

name: ASR

name: Speaker Tasks

# What does this PR do ?

Closes #5660 

Expose the device so users can set this via the config.

**Collection**: ASR

# Changelog 
- Expose clustering diarizer device so user can set device.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS][ZH] fix broken link for the script.
State: closed | Pull request: yes
Labels: name: TTS

name: cherry-pick

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
CSP Tools
State: closed | Pull request: yes

# What does this PR do ?

Add cluster validation tests for cloud service providers to NeMo Megatron directory.

**Collection**: nemo_megatron_launcher

# Changelog 
- For all CSPs
    - DCGM diagnostics test job
    - NCCL tests build job
    - NCCL `all_reduce_perf` test job
    - NCCL topology file
    - All-in-one run script (cluster_validation.sh)
- AWS, additionally
    - Dockerfile on top of NeMo Megatron container with AWS EFA drivers

# Usage
See NeMo Megatron README (Cluster Validation section)

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Adding TPMLP to T5-based models
State: closed | Pull request: yes
Labels: name: stale

name: NLP

Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com>

# What does this PR do ?

Adds tpmlp to T5-based models

**Collection**: [Note which collection this PR will affect]

# Changelog 
- modifying "megatron_base_prompt_learning_model.py"

# Usage


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[WIP] remove nemo_text_processing
State: closed | Pull request: yes
Labels: name: TTS

name: stale

name: common

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?

TN/ITN moved to a new repo https://github.com/NVIDIA/NeMo-text-processing


**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
NeMo - replace named tuples with dataclasses in dataloader returns.
State: closed | Pull request: no

        No action item below, but need to consider long term.

I wonder if we should start moving away from named tuples, switching to data classes instead. NamedTuple somehow interfere with pickling which might become necessary soon with PT 2.0 (maybe)

_Originally posted by @titu1994 in https://github.com/NVIDIA/NeMo/pull/5356#discussion_r1049069660_
unify AudioToMelSpectrogramPreprocessor and AudiotoSpectrogram
State: closed | Pull request: no

        this is very close to 
https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/asr/modules/audio_preprocessing.py#L95. it would just be adding one or two args to return intermediate STFT from AudioToMelSpectrogramPreprocessor or sub-classing it.

_Originally posted by @krishnacpuvvada in https://github.com/NVIDIA/NeMo/pull/5356#discussion_r1018418772_
speech enhancement - return dataclass in `process` method
State: closed | Pull request: no

        I think prepare a futureproof solution and return (by option) the equivalent of a Hypothesis dataclass object. Ofc this will be some other dataclass name and content but will allow flexibility in the future. By default just return str paths, and if user requests for this dataclass, fill it with contents as needed and return.

_Originally posted by @titu1994 in https://github.com/NVIDIA/NeMo/pull/5356#discussion_r1049079041_
[NLP] Supervised Fine-tuning for GPT
State: closed | Pull request: yes
Labels: name: TTS

name: stale

name: NLP

# What does this PR do ?

The goal of this PR is to add a supervised fine-tuning functionality for GPT.

**Collection**:  NLP - language modeling 

# Changelog 
- Created a new class similar to GPT pretraining: MegatronGPTSFTModel
- Create a new dataset class for SFT, allowing answer-only loss and the inclusion of EOS and SEP tokens
- Create a new usage example

**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [X] Did you write any new necessary tests?
- [X] Did you add or update any necessary documentation?
- [X] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [X] New Feature
- [ ] Bugfix
- [ ] Documentation


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Propagate attention_dropout flag for GPT-3
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Allow changing `attention_dropout` from a config file level in the GPT model

**Collection**: language_model

# Changelog 
- Allow changing `attention_dropout` from a config file level in GPT

# Usage
Set `model.attention_dropout` in GPT config file

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix EMA restart by allowing device to be set by the class init
State: closed | Pull request: yes
Labels: name: common

# What does this PR do ?

Fixes a bug when restarting from a save checkpoint with multiple GPUs. We shouldn't override the device (or even store the device at all), relying on the LightningModule to provide the device.

**Collection**: Common

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Adding non-tokens to transcription
State: closed | Pull request: no

Hi All, 
I would like to add some special symbols in to transcription like lang. id of each utterance "\<english\>" and retrain the model. Is there a way to do that? Can I do it by passing some parameters to process_asr_text_tokenizer.py.
Currently, I have tried adding some unused symbols like "<" for English, and ">" for Hindi etc. Pls let me know if this is correct? Or let me know if there is any better/efficient way to do that. Thanks
[TTS][ZH] fix broken link for the script.
State: closed | Pull request: yes
Labels: name: TTS

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
common\losses\cross_entropy.py doesn't work on some gpus if mask tensor has integer type
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

common\losses\cross_entropy.py doesn't work on some gpus if mask tensor has integer type.
Given that mask is a tensor of all 1 with dtype=torch.int64
in lines 71-72
       if loss_mask.dtype is not torch.bool:
              loss_mask = loss_mask > 0.5
loss_mask should become all True, if run on cpu or on some (newer?) gpus.
But I see two cases when loss_mask becomes all False (GTX780,  Tesla K40m )
It seems that comparison between int and float works differently.

This leads to labels_flatten be empty and torch original cross entropy function is called with not-flattened tensors resulting in frustrating error like: RuntimeError: Expected target size [4, 11], got [4, 256]

**Steps/Code to reproduce bug**

import torch
import numpy as np
from nemo.collections.common.losses import CrossEntropyLoss

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
loss_fn = CrossEntropyLoss(logits_ndim=3)
logits = torch.tensor(np.ones((4, 256, 11), dtype=np.float), device=device)
labels = torch.tensor(np.ones((4, 256), dtype=np.int64), device=device)
labels_mask = torch.tensor(np.ones((4, 256), dtype=np.int64), device=device)
val_loss = loss_fn(logits=logits, labels=labels, loss_mask=labels_mask)
print("val_loss=", val_loss)

**Expected behavior**

This works on cpu and in colab's gpu, but gives error in GTX780 and Tesla K40m

**Environment overview (please complete the following information)**

 - Environment location: [Bare-metal, Docker, Cloud(specify cloud provider - AWS, Azure, GCP, Collab)]
 - Method of NeMo install: [pip install or from source]. Please specify exact commands you used to install.
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version
- PyTorch version
- Python version

**Additional context**

Add any other context about the problem here.
Example: GPU model
[TTS][ZH] bugfix for ngc cli installation.
State: closed | Pull request: yes
Labels: name: TTS

name: cherry-pick

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

add ngc cli installation following what TAO toolkit used.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix P-Tuning Truncation
State: closed | Pull request: yes
Labels: name: NLP

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Dataset sample truncation sometimes errors in NeMo. This is because the text segments from a users dataset can have different tokenization depending on the prompt template text that is present around that text segment. This makes it difficult to find the truncated field's subsequence of token ids and thus truncate just that portion of the training example. This PR is a different approach to truncation where we: 

1. After getting the truncated field's text id's, I convert these token ids back into a string using the tokenizer
2. re-build the prompt using this truncated text string instead of the full text string
3. re-tokenize the entire input at the same time.

**Collection**: NLP

# Changelog 
- GPT Prompt Learning Dataset Class 

# Usage
 Same as before

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to https://nvidia.slack.com/archives/C02A7LYGHK8/p1670592730656789 and https://nvidia.slack.com/archives/C03AQCUKQ5T/p1671029551818229
[TTS] add tts dict cust notebook
State: closed | Pull request: yes
Labels: name: TTS

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add AutoConfigurator (former HP Tool)
State: closed | Pull request: yes

# What does this PR do ?
Adds AutoConfigurator to NeMo Megatron directory.

**Collection**: nemo_megatron_launcher

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [X] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?
@okuchaiev @yaoyu-33 @ericharper @Davood-M 

# Additional Information
* Related to # (issue)
Enable gpu device selection in ClusteringDiarizer()
State: closed | Pull request: no
Labels: name: bug

Can you add the ability to select the GPU device when using [ClusteringDiarizer](https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/asr/models/clustering_diarizer.py)? Currently, it always gets mapped to GPU 0 but it seems like that should be configurable unless I'm missing something.
Hybrid ASR-TTS models
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

This PR adds support for hybrid ASR-TTS models.

**Features:**
- [x] hybrid ASR-TTS model (`ASRWithTTSModel`) from CTC-BPE and RNNT-BPE for finetuning
- [x] hybrid ASR-TTS model (`ASRWithTTSModel`) from ASR model config for training from scratch
- [x] fusing BatchNorm to improve the quality of the finetuned models
- [x] text-to-text datasets
- [x] mixing ASR and text-to-text datasets
- [x] enhancer support: will be added after https://github.com/NVIDIA/NeMo/pull/5565

Dependent on:
- https://github.com/NVIDIA/NeMo/pull/5671
- https://github.com/NVIDIA/NeMo/pull/5774

**Collection**: [ASR]

# Changelog 
- **new hybrid ASR-TTS model** `ASRWithTTSModel` in `nemo/collections/asr/models/hybrid_asr_tts_models.py`
- **text-to-text datasets**: `TextToTextDataset` (map-style) + `TextToTextIterableDataset` (iterable) in `nemo/collections/asr/data/text_to_text.py`
- **fused BatchNorm**: `nemo/collections/asr/parts/submodules/batchnorm.py` + support for fused BatchNorm in ConformerEncoder in `nemo/collections/asr/parts/submodules/conformer_modules.py`
- **default config** for hybrid model: `examples/asr/conf/asr_tts/hybrid_asr_tts.yaml`
- **example scripts** for finetuning existing models and training from scratch: `examples/asr/asr_with_tts/speech_to_text_bpe_with_text_finetune.py` + `examples/asr/asr_with_tts/speech_to_text_bpe_with_text.py`

# Usage
- `examples/asr/asr_with_tts/speech_to_text_bpe_with_text_finetune.py` for finetuning pretrained ASR model with TTS
- `examples/asr/asr_with_tts/speech_to_text_bpe_with_text.py` for training ASR with TTS from scratch

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
Add DLLogger support to exp_manager
State: closed | Pull request: yes

# What does this PR do ?

Adds an option to log with DLLogger in the experiment manager.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Changed exp_manager.py to add a new logger
- Updated the exp_manager documentation

# Usage
In the exp_manager config:

```yaml
enable_dllogger_logger: True
dllogger_logger_kwargs:
    stdout: False
    verbose: False
    json_file: "./dllogger.json"
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Nemo megatron launcher
State: closed | Pull request: yes

# What does this PR do ?

In this PR, we OSS internal nemo megatron launching scripts to NeMo github repo. It includes the launch scripts to train large language models, nemo megatron Dockerfile and nemo megatron readme. 

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add nemo megatron launching scripts in side `nemo_megatron_launcher/`
- Add nemo megatron Dockerfile in side `nemo_megatron_launcher/`
- Add nemo megatron README in side `nemo_megatron_launcher/`

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Draft: Fused Adapters with Retro
State: closed | Pull request: yes
Labels: name: core

name: TTS

name: ASR

name: NLP

name: Speaker Tasks

name: CI

name: common

# What does this PR do ?

Combines adapters with retrieval augmented language model retro with different configurations.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
UL2 sampling is Poisson instead of uniform
State: closed | Pull request: no
Labels: name: bug

name: stale

UL2 objectives should be sampled with uniform probability, including subtasks. Instead, currently, first the overarching denoiser is uniformly sampled and then the denoising subtask is again sampled uniformly. This is only the case [here for X-denoisers](https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/nlp/data/language_modeling/megatron/ul2_dataset.py#L174-L178) currently, because one R- and one X-denoising objective from the paper are missing.

Compare:

```python
import itertools
import random

# The integers indicate the different task types in the paper.
tasks = {'R': [0, 1], 'S': [2], 'X': [3, 4, 5, 6]}

# How it's currently done in NeMo:
denoiser_subtasks = random.choice(tasks)
subtask = random.choice(denoiser_subtasks)
# Each of 0, 1 has 1/3 * 1/2 chance.
# 2 has 1/3 chance.
# Each of 3, 4, 5, 6 has 1/3 * 1/4 chance.

# How it's done in the code snippet from the UL2 paper:
subtask = random.choice(list(itertools.chain.from_iterable(tasks.values())))
# Each of 0, 1, 2, 3, 4, 5, 6 has 1/7 chance.
```
No-script TS export, prepared for ONNX export 
State: closed | Pull request: yes
Labels: name: core

name: TTS

name: CI

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>

# What does this PR do ?
Changed unfold to reshape for inference to be able to export in ONNX.
Also using extra hx parameter to LSTM call to work around Pytorch issues with tracing the hx=None.
Now Torchscript export works without any scripting - so I was also able to get rid of unnecessary autocast=False block.
ONNX export still fails due to internal bugs, even though each part of the network is now ONNX-exportable.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS][ZH] bugfix for ngc cli installation.
State: closed | Pull request: yes
Labels: name: TTS

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

add ngc cli installation following what TAO toolkit used.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add duration padding support for RADTTS inference
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?
During inference, radtts++ duration predictions tend to be close to zero when given random inputs. This can be an issue as a sequence with length less than the grouping size in the radtts flow operation will result in a crash. 

The solution is to set the last input token to correspond to a blank space, and stretch out the duration for said token until the the total duration of the sequence reaches the next multiple of group_size.

**Collection**: [Note which collection this PR will affect]
TTS

# Changelog 
- sets the last element of the generated text token sequences in RadTTSModule.input_example() to zero (represents blank space). Note that this still needs to be done separately during inference with raw text, otherwise a non-space token may be replicated.
- helpers.regulate_len is extended to support last-valid-token replication until the next nearest multiple of group_size
- RadTTSModule.infer() now calls regulate_len with additional parameters for the replication behavior

# Usage
* Radtts should now pass the tests in:

```python
pytest tests/collections/tts/test_tts_exportables.py
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add prompt learning tests
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
some tokenizers do not have additional_special_tokens_ids attribute
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

Signed-off-by: arendu <adithya.r@gmail.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS][ZH] bugfix for the tutorial and add NGC CLI installation guide.
State: closed | Pull request: yes
Labels: name: bug

name: TTS

name: cherry-pick

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

This tutorial requires users to download datasets from NGC, but we did not provide any guidance about how to install NGC registry CLI. Verified to running through without errors.

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[workflow] add exclude labels option to ignore cherry-picks in releas…
State: closed | Pull request: yes
Labels: name: CI

…e changelog.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add duration padding support to next largest multiple of group size in radtts inference
State: closed | Pull request: yes
Labels: name: core

name: TTS

name: ASR

name: NLP

name: Speaker Tasks

name: CI

name: common

# What does this PR do ?
During inference, radtts++ duration predictions tend to be close to zero when given random inputs. This can be an issue as a sequence with length less than the grouping size in the radtts flow operation will result in a crash. 

The solution is to set the last input token to correspond to a blank space, and stretch out the duration for said token until the the total duration of the sequence reaches the next multiple of group_size.

**Collection**: [Note which collection this PR will affect]
TTS

# Changelog 
- sets the last element of the generated text token sequences in RadTTSModule.input_example() to zero (represents blank space). Note that this still needs to be done separately during inference with raw text, otherwise a non-space token may be replicated.
- helpers.regulate_len is extended to support last-valid-token replication until the next nearest multiple of group_size
- RadTTSModule.infer() now calls regulate_len with additional parameters for the replication behavior

# Usage
* Radtts should now pass the tests in:

```python
pytest tests/collections/tts/test_tts_exportables.py
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS][ZH] bugfix for the tutorial and add NGC CLI installation guide.
State: closed | Pull request: yes
Labels: name: bug

name: TTS

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

This tutorial requires users to download datasets from NGC, but we did not provide any guidance about how to install NGC registry CLI. Verified to running through without errors.

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
some tokenizers do not have additional_special_tokens_ids attribute
State: closed | Pull request: yes
Labels: name: NLP

Signed-off-by: arendu <adithya.r@gmail.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)