renwei2024/nemo-github-issues huggingface dataset:
[TTS] porting VITS implementation
State: closed | Pull request: yes
Labels: name: TTS

name: ASR

name: NLP

name: CI

name: common

# What does this PR do ?

Implementation of VITS model

**Collection**: [TTS]

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Crash eval_beamsearch_ngram.py code for large data sets
State: closed | Pull request: no
Labels: name: bug

@VahidooX 
Hi,
When we use [`eval_beamsearch_ngram.py`](https://github.com/NVIDIA/NeMo/blob/main/scripts/asr_language_modeling/ngram_lm/eval_beamsearch_ngram.py) code for large data sets (e.g., with 500,000 utterances), the code crashes after some hours of computation (i.e., the OS kill automatically the process!!). According to my observation, the code runs correctly when the ASR acoustic model runs and the raw transcriptions of data sets extracted correctly, but when the LM wants to apply in a huge amount of raw transcriptions then it crashes. I googled this issue, and I found below link (which I think the issue related to the shortage of memory of the server):
- [Unable to allocate array with shape and data type](https://stackoverflow.com/questions/57507832/unable-to-allocate-array-with-shape-and-data-type)   
According to the above-mentioned link, I have changed the `overcommit_memory` mode of my server. however, unfortunately the issue didn't resolved. Would you please help me to address the issue?
Thanks a lot
Update minimum version requirements for Pytorch Lightning, for tensorboard/tensorflow doesn't support for now protobuf > 3.20
State: closed | Pull request: no
Labels: name: bug

name: stale

PL soon to be released will work around this problem: https://github.com/Lightning-AI/lightning/issues/9900#issuecomment-1345390290

Might be useful to update the minimum version requirements of PL for Nemo
[TTS][ZH] add fastpitch and hifigan model NGC urls and update NeMo docs.
State: closed | Pull request: yes
Labels: name: TTS

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix race condition bug with hydra multirun
State: closed | Pull request: yes
Labels: name: core

name: cherry-pick

Signed-off-by: smajumdar <titu1994@gmail.com>

# What does this PR do ?

Fixes an issue with delayed process finalization, and race condition with the data thread

**Collection**: [Core]


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.
Fix race condition bug with hydra multirun
State: closed | Pull request: yes
Labels: name: core

Signed-off-by: smajumdar <titu1994@gmail.com>

# What does this PR do ?

Fixes an issue with delayed process finalization, and race condition with the data thread

**Collection**: [Core]


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.
TTS time data extraction
State: closed | Pull request: no
Labels: name: TTS

I'm using fastpich & univnet from Nemo to generate audio from text (TTS). I want to have the time information of each word in such a way that it shows in what period of time each word was uttered. There are some ways to predict such information, but all of them are applied after running the encoder and vocoder models. Is there any way to get such information from TTS models (any of them) without having to post-process our data?
If there is not, I would be grateful if you tell me what is the most efficient way to get such information. 

@XuesongYang
Better patch hydra
State: closed | Pull request: yes
Labels: name: core

name: cherry-pick

# What does this PR do ?

Re-add threading of the stderror and drain threads

**Collection**: [Core]

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.
Better patch hydra
State: closed | Pull request: yes
Labels: name: core

# What does this PR do ?

Re-add threading of the stderror and drain threads

**Collection**: [Core]

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.
Patch hydra launch
State: closed | Pull request: yes
Labels: name: core

name: cherry-pick

# What does this PR do ?

Removed unnecessary busy waiting in loop

**Collection**: [Core]

# Changelog 
-  Remove busy waiting in tight loop


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?
Patch hydra launch
State: closed | Pull request: yes
Labels: name: core

# What does this PR do ?

Removed unnecessary busy waiting in loop

**Collection**: [Core]

# Changelog 
-  Remove busy waiting in tight loop


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?
Patch hydra launcher
State: closed | Pull request: yes
Labels: name: core

name: cherry-pick

name: ASR

name: NLP

name: common

# What does this PR do ?

Removed unnecessary busy waiting in loop

**Collection**: [Core]

# Changelog 
- Remove busy waiting in tight loop

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?
Patch hydra launcher
State: closed | Pull request: yes
Labels: name: core

name: ASR

name: NLP

name: common

# What does this PR do ?

Removed unnecessary busy waiting in loop

**Collection**: [Core]

# Changelog 
- Remove busy waiting in tight loop

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?
Small fix to prevent unit test from intermittent failures
State: closed | Pull request: yes
Labels: name: stale

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[Fix] setup_multiple validation/test data
State: closed | Pull request: yes
Labels: name: core

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

# What does this PR do ?

We missed a c/p mistake in just-merged PR #5569.
This bug does not affect default functionality, just the new param added in #5569.

Fix:
When setting up validation and test data, we need to use `setup_multiple_*`.

**Collection**: Core

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage

n/a

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #5437 5569
Port stateless timer to exp manager
State: closed | Pull request: yes
Labels: name: NLP

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Allows configuring a stateless timer via `exp_manager.max_time=03:55:00`.

**Collection**: All

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Implementing SFT for GPT 
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

The goal of this PR is to add a supervised fine-tuning functionality for GPT.

**Collection**:  NLP - language modeling 

# Changelog 
- Created a new class similar to GPT pretraining: MegatronGPTSFT
- Create a new dataset class for SFT, allowing answer-only loss and the inclusion of EOS and SEP tokens
- Create a new usage example

**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [X] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Config settings for Building a custom decoder using Pretrained conformer CTC Hindi model as Encoder
State: closed | Pull request: no
Labels: name: stale

Hi, I am trying to use pretrained conformer-ctc-medium Hindi model as encoder, and I would like to add conformer-transducer as decoder initialized with random weights.

I tried following things, but none of them is working:
1. I have used **conformer_transducer_bpe.yaml** thinking that the decoder config parameters would be loaded from the config & "RNNTDecoder" would be added. 
2. I have tried initializing in the python code, but that is also not working:
conf_ctc_model = nemo_asr.models.EncDecCTCModelBPE.from_pretrained(model_name="stt_hi_conformer_ctc_medium")
conf_ctc_model.cfg.decoder._target_='nemo.collections.asr.modules.RNNTDecoder' 
3. I have tried adding decoder in python code as 
RNN_decoder  = dict(
_target_ = 'nemo.collections.asr.modules.RNNTDecoder',
normalization_mode = 'null', 
random_state_sampling = 'false', 
blank_as_pad = 'true', 

prednet = dict(
    pred_hidden = 640, 
    pred_rnn_layers = 1,
    t_max= 'null',
    dropout= 0.2
)
)
params['model']['decoder'] = RNN_decoder 

4. In addition to RNN_decoder, tried initializing joint, decoding, loss, variational_noise config parameters. But did not help.
5. Tried change_vocabulary before & after config parameters loading

But, In all cases  decoder was being displayed as ConvASRDecoder, loss = CTCloss, encoder= ConformerEncoder, both during training & in hparams.yaml file.

Please suggest how to use pretrained conformer-ctc with a custom decoder. This decoder can be a conformer/transformer.
or
Can I also try using encoder decoder based "Quartznet" hindi pretrained model to accomplish the same.

Or Can I use pyCTCdecoder for decoding? 
Thanks
remove useless files.
State: closed | Pull request: yes
Labels: name: common

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

remove `vocab.py` because nowhere else uses it.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
AttributeError: 'EncDecSpeakerLabelModel' object has no attribute 'get_label'
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

I cannot get the language of a file with AmberNet

**Steps/Code to reproduce bug**

Code:

```python
import nemo 
from nemo.collections.asr.models import EncDecSpeakerLabelModel

model = EncDecSpeakerLabelModel.from_pretrained('langid_ambernet')

print('NeMO:', nemo.__version__)

print('AmberNet method:', model.get_label)
```

Result:

```
...

NeMO: 1.13.0

Traceback (most recent call last):
  File "/Users/yehorsmoliakov/Work/egorsmkv/asr-corpus-creator/rep.py", line 8, in <module>
    print('AmberNet method:', model.get_label)
  File "/Users/yehorsmoliakov/.local/share/virtualenvs/asr-corpus-creator-Sp_hMA78/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1265, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'EncDecSpeakerLabelModel' object has no attribute 'get_label'
```

**Expected behavior**

Info about the method `get_label`
MegatronNMT: updated decode function to support beam search
State: closed | Pull request: yes
Labels: name: stale

name: NLP

Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com>

# What does this PR do ?

Adds simple beam search to the lm_encoder_decoder's decode function

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Updated decode function

# Usage
* You can specify beam search size and the penalty function parameter

```bash
python NeMo/examples/nlp/machine_translation/nmt_transformer_infer_megatron.py --config-path=conf --config-name="nmt_megatron_infer"  model_file=./megatron_nmt.nemo srctext=./valid.de-en.en tgtout=./output_cache.txt source_lang=en target_lang=de batch_size=32 beam_size=6
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[Add] ASR+VAD Inference Pipeline
State: closed | Pull request: yes
Labels: name: feature

name: ASR

name: common

# What does this PR do ?

This PR aims to provide a pipeline for integrating Voice Activity Detection (VAD) results in ASR inference. 

**Collection**: [ASR]

# Changelog 
- Added example folder `examples/asr/asr_vad`
- Added `feature_to_label.py` and `feature_to_label_dataset.py`
- Added `feature_to_text.py` and `feature_to_text_dataset.py`
- Added classes to `collections.py`
- Updated `manifest.py`

# Usage
To run the code with ASR+VAD default settings:

```bash
python speech_to_text_with_vad.py \
    manifest_filepath=/PATH/TO/MANIFEST.json \
    vad_model=vad_multilingual_marblenet \
    asr_model=stt_en_conformer_ctc_large \
    vad_config=../conf/vad/vad_inference_postprocess.yaml
```

To use only ASR and disable VAD, set `vad_model=None` and `use_rttm=False`.

To use only VAD, set `asr_model=None` and specify both `vad_model` and `vad_config`.

To enable profiling, set `profiling=True`, but this will significantly slow down the program.

To use or disable feature masking, set `use_rttm` to `True` or `False`.

To normalize feature before masking, set `normalize=pre_norm`, 
and set `normalize=post_norm` for masking before normalization.

To use a specific value for feature masking, set `feat_mask_val` to the desired value. 
Default is `feat_mask_val=None`, where -16.530 (zero mel-spectralgram value) will be used for `post_norm` and 0 (same as SpectralAugmentation) will be used for `pre_norm`.

See more options in the `InferenceConfig` class.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Remove SDP (moved to separate repo)
State: closed | Pull request: yes

# What does this PR do ?

Remove SDP code from tools/ folder and add docs page briefly describing SDP and its new location

**Collection**: tools

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ N/A ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ N/A ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
optimized loop and bugfix
State: closed | Pull request: yes
Labels: name: bug

name: ASR

Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com>

# What does this PR do ?
Bugfix and code optimization
Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]
ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fixed RadTTS unit test
State: closed | Pull request: yes
Labels: name: TTS

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>

# What does this PR do ?

It looks like (due to updates to internals?) RadTTS now requires a bit higher minimum length of the input.
I will investigate some more but that fix would unblock the test.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
NeMo Forced Aligner
State: closed | Pull request: yes

Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com>

# What does this PR do ?

Add NFA - a tool to do forced alignment using NeMo CTC-based models. 

**Collection**: [Note which collection this PR will affect]
Tools 

# Changelog 
- Add NeMo/tools/nemo_forced_aligner/

# Usage
* You can potentially add a usage example below

```bash
python $NEMO_PATH/tools/nemo_forced_aligner/align.py \
    pretrained_name="stt_en_citrinet_1024_gamma_0_25" \
    model_downsample_factor=8 \
    manifest_filepath="manifest.json" \
    output_dir="nfa_ctms/" \
    batch_size=1 \
    additional_ctm_grouping_separator="|" \
    audio_filepath_parts_in_utt_id=1

```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ N/A ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ N/A ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Merge r1.13.0 main
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add an option to defer data setup from ``__init__`` to ``setup``
State: closed | Pull request: yes
Labels: name: core

name: ASR

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

# What does this PR do ?

This PR adds an option to defer dataloader setup in `ModelPT` from `__init__` to `setup`.
Default behavior remains unchanged.

**Collection**: Core

# Changelog 
- In `ModelPT::__init__()`
  - Nothing is changed if `self._cfg.train_ds.defer_setup` is not set or is set to `False` (default)
  - If `self._cfg.train_ds.defer_setup` is set to `True`, `self.setup_training_data` will be skipped
  - Analogous behavior for validation and test data

- In `ModelPT::setup()`
  - If dataloader is not initialized and `defer_setup==True`, run data setup

# Usage

To use deferred setup, set the following

```python
  cfg.model.train_ds.defer_setup = True
  cfg.model.validation_ds.defer_setup = True
  cfg.model.test_ds.defer_setup = True
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add support for Time Stamp calculation using transcribe_speech.py
State: closed | Pull request: yes
Labels: name: ASR

Signed-off-by: smajumdar <titu1994@gmail.com>

# What does this PR do ?

Adds flag to compute word and char level timestamp using the transcribe speech script.

**Collection**: [ASR]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage

```python

python transcribe_speech.py \
    model_path=null \
    pretrained_name=null \
    audio_dir="<remove or path to folder of audio files>" \
    dataset_manifest="<remove or path to manifest>" \
    output_filename="<remove or specify output filename>" \
    batch_size=32 \
    compute_timestamps=True \
    compute_langs=False \
    cuda=0 \
    amp=True

(Optionally)
    ctc_decoding.ctc_timestamp_type="all"  # (default all, can be [all, char, word])
    rnnt_decoding.rnnt_timestamp_type="all"  # (default all, can be [all, char, word])

```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?
Vits exp
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?
Implementation of VITS model

**Collection**: [tts]


# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update torchmetrics 
State: closed | Pull request: yes
Labels: name: ASR

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

# What does this PR do ?

Update torch metrics version 

**Collection**: Speaker Tasks

# Change Log

Torch metrics updated their API to use `final_state` instead of `get_final_stat`. 

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
[TTS] GAN-based spectrogram enhancer
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Add a GAN-based spectrogram enhancer to NeMo.

The intended purpose is to add details to blurry spectrograms from TTS models like FastPitch.

**Collection**: TTS

# Changelog 
- Add the spectrogram enhancer NeMo model
- Add example config and training script

# Usage
Given a trained model,
```python
model = SpectrogramEnhancer.restore_from("checkpoint.nemo")
with torch.no_grad():
    # spectrograms :: B x 80 x L
    # lengths  :: B - the usual LongTensor with lengths or every spectrogram in the batch
    enhanced_spectrograms = model.forward(condition=spectrograms, lengths=lengths)
    # enhanced_spectrograms = B x 80 x L
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Based on https://github.com/lucidrains/stylegan2-pytorch (MIT License)
Fix all gather while writing to a file during T5 finetuning
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Fixes the all-gather logic to only gather across data parallel ranks to write to a file. Previously, examples would be duplicated in the file since all model parallel ranks have the same data.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fixed Docker build
State: closed | Pull request: yes
Labels: name: CI

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix all gather while writing to a file during T5 finetuning
State: closed | Pull request: yes

# What does this PR do ?

Fixes the all-gather logic to only gather across data parallel ranks to write to a file. Previously, examples would be duplicated in the file since all model parallel ranks have the same data.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Optimizing distributed Adam when running with one work queue
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

When running GPT with `CUDA_DEVICE_MAX_CONNECTIONS=1`, @erhoo82 observed poor overlapping between the backward compute and the gradient reduce-scatters in the distributed Adam optimizer. This PR integrates performance optimizations from https://github.com/NVIDIA/apex/pull/1551.

I haven't fully tested, but this also seems to fix a hang that occurs when running with 32+ Selene nodes.

**Collection**: NLP

# Changelog 

- Manually configure dist Adam for GPT, T5, and BERT so each Transformer layer occupies one effective param bucket.
- Make sure dist Adam reduce-scatters are synchronized before all-reducing embedding gradients.

# Usage

Set the optimizer to `distributed_fused_adam` in the config file: https://github.com/NVIDIA/NeMo/blob/f1c8714f2fe0bc18a338fb02f0e608144981b291/examples/nlp/language_modeling/conf/megatron_gpt_config.yaml#L192

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

Pinging @erhoo82 and @markelsanz14.

# Additional Information
* Depends on https://github.com/NVIDIA/apex/pull/1551
Remove broadcast from T5 prompt learning inference
State: closed | Pull request: yes
Labels: name: cherry-pick

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Removes a broadcast call that causes hanging during T5 prompt learning inference.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)