renwei2024/nemo-github-issues huggingface dataset:
Add interface for p2p communication overlap and send receive overlap
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?
- Overlap P2P communication between pipeline stages
- Overlap individual send and receive ops

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update README.md
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Hybrid Autoregressive Transducer (HAT)
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Add HAT model as a new joint network type (HATJoint) for RNNT model. The difference is only in decoding time -- HAT.joint.joint returns two outputs: hat_logprobs and internal_lm_logprobs (for internal lm subtraction in case of Shallow Fusion with external n-gram LM).

**Collection**: [ASR]

# Usage
* For HAT model training you need replace `_target_: nemo.collections.asr.modules.RNNTJoint` with `_target_: nemo.collections.asr.modules.HATJoint` in `joint` part of standard transducer config.

* For Shallow Fusion with external n-gram LM use RNNT `maes` decoding algorithm which is able to work with HATJoint model.  

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
NER task failing with CUDA errors with text blocks > 300 tokens
State: closed | Pull request: no
Labels: name: bug

name: stale

We are using NeMo to do named entity recognition on blocks of text. The NER task works correctly when the text size we apply is under around 250 or 300 tokens. Once the text size goes above that, the task will start to fail. Between 250 tokens and around 600 tokens, the task will randomly either succeed or fail. Above 500-600 tokens or so, the task fails consistently.

The errors given are somewhat varied, depending on circumstance. They are one of the following: 

```
RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling `cublasCreate(handle)`

RuntimeError: CUDA error: CUBLAS_STATUS_INTERNAL_ERROR when calling `cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc)`

RuntimeError: CUDA error: device-side assert triggered\nCUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.\nFor debugging consider passing CUDA_LAUNCH_BLOCKING=1.
```

Following the advice in that last message, the error changes to:
```
RuntimeError: CUDA error: device-side assert triggered
```

The model we are using is a dataset trained off of the medmentions data set, and the .nemo file is 1.2 GB and is being run on a 20GB GPU RAM slice of an A100 card which is shared on a kubernetes cluster. Based on our reading of other error reports, we suspected that running out of GPU RAM might be the ultimate issue. However, with 20 GB of GPU RAM we were surprised that only 300 tokens would be too much for it.

We have written code to chop the text into smaller windows and process it successively. However, both performance and accuracy suffer the smaller the window is. We would very much like to be able to run this on much larger blocks of text if possible.
Torch Multi Head Attention
State: closed | Pull request: no
Labels: name: stale

**Is your feature request related to a problem? Please describe.**

I've seen that you are using custom implementation of MHA block - https://github.com/NVIDIA/NeMo/blob/da6d825b8c2fec0a3b8e0d2c33a7600f242db4c1/nemo/collections/asr/parts/submodules/multi_head_attention.py#L108

I think it would be better to use `torch.nn.MultiheadAttention` which is meant to improve the performance of this block (there is FlashAttention now) and torch continues to maintain and update this block while you're doing basic implementation of it.

**Describe the solution you'd like**

I'd like to see an update of models with torch implementation of MHA

https://pytorch.org/docs/stable/generated/torch.nn.MultiheadAttention.html

What do you think about it?
UL2 with GPT-style Megatron models
State: closed | Pull request: yes
Labels: name: stale

name: NLP

name: common

# What does this PR do ?

Adds the following:

1. `MegatronUGPTModel` which subclasses `MegatronGPTModel` and overrides dataset building, attention mask changes etc.
2. `UGPTDataset` which subclasses UL2Dataset and implements GPT-specific data processing.
3. Megatron U-GPT pre-training/fine-tuning script and YAML config.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix buckeing seeding
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

When the global seeding is set and controlled by the user, the fully_randomized bucketing uses the same order of buckets on all workers. This PR fixes this seeding bug of the bucketing dataset by adding the rank of data loader to the seed.

# Changelog 
- Add the rank of the data worker to the bucketing dataset to keep the randomness for fully_randomzied mode of buceking.

  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation
Fix buckeing seeding
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

When the global seeding is set and controlled by the user, the fully_randomized bucketing uses the same order of buckets on all workers. This PR fixes this seeding bug of the bucketing dataset by adding the rank of data loader to the seed.

# Changelog 
- Add the rank of the data worker to the bucketing dataset to keep the randomness for fully_randomzied mode of buceking.

  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation
[ASR] Conformer global tokens in local attention
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Adds Longformer-style attention for Conformer encoder (limited context + few tokens with full context attention). Unlike with just limited context attention, when using global tokens model needs to be fine-tuned with this approach to get good results. After fine-tuning results are improved particularly on long audio (even if it's much longer than training data). When using the default Fast Conformer configs with this attention, inference on audio above 1 hour is supported.

**Collection**: ASR

# Changelog 
- Add option to use global tokens
- Adds corresponding test
- Configs for training Conformer with this attention and mention in docs

# Usage

Add the following parameters to the Conformer encoder in config:



```  
    self_attention_model: rel_pos_local_attn # longformer-style attention (sliding window + global tokens)
    global_tokens: 1 # number of tokens that attend and are attended to by all tokens
    global_attn_separate: false # whether global tokens should use separate q,k,v layers
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [X] Did you write any new necessary tests?
- [X] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [X] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
added retrieval of preprocessor n_fft and hop_length
State: closed | Pull request: yes
Labels: name: TTS

name: stale

# What does this PR do ?

Correct a bug in waveglow model where you used predefined values for the n_fft and hop_length. The commit aims at retrieving the correct values from the preprocessor

**Collection**: tts

# Changelog 
- Retrieve the correct n_fft and hop_length values for the waveglow model

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Further modifications in other modules called used by waveglow may be needed to properly allow to change the preprocessor feature banks values changed and the waveglow model accordingly
Regarding the training details of the ASR model on limited resources
State: closed | Pull request: no

Hi, @titu1994.
I am trying to train a conformer_ctc_bpe (small) model with librispeech960 using four NVIDIA 2080 GPUs (11GB memory).
But, when training model from scratch, I can't reproduce the reported performance with my experimental settings.


Looking at your training details, it seems that a batch size of 2K (maybe 2,048?) and a learning rate of 5.0 are used.

However, with my GPUs of 11GB, I can't use such a large batch size; the maximum batch size I can use is 12.

Could you please share some guidelines and tips about appropriate training details for my experimental condition..?
(i.e., **batch size**, **learning rate**, **warmup step**, **weight decay**, **accumulate_grad_batches**, etc. or another optimizer / scheduler ..?)

Some configurations I set are as follows
```
model:
  sample_rate: 16000
  log_prediction: true
  ctc_reduction: 'mean_batch'
  skip_nan_grad: false
  
  train_ds:
    manifest_filepath: "/home/shmun/NeMo/data/train_all_960.json"
    sample_rate: 16000
    batch_size: 12 # you may increase batch_size if your memory allows
    shuffle: true
    num_workers: 8
    pin_memory: true
    use_start_end_token: false
    trim_silence: false
    max_duration: 16.7
    min_duration: 0.1
    is_tarred: false
    tarred_audio_filepaths: null
    shuffle_n: 2048
    bucketing_strategy: "synced_randomized"
    bucketing_batch_size: null

  tokenizer:
    dir: "/home/shmun/NeMo/data/tokenizer_ctc_conformer_large_128/"
    type: "bpe"

  spec_augment:
    _target_: nemo.collections.asr.modules.SpectrogramAugmentation
    freq_masks: 2 
    time_masks: 5  #10
    freq_width: 27
    time_width: 0.05

  optim:
    name: adamw
    lr: 5.0  #2.0
    betas: [0.9, 0.98]
    weight_decay: 1e-3

    sched:
      name: NoamAnnealing
      d_model: 176
      warmup_steps: 10000
      warmup_ratio: null
      min_lr: 1e-6
```
ASR Beam search documentation
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Update docstrings for asr beam search computation

**Collection**: [ASR]

# Changelog 
- Update train kenlm arg
- Update args for beam search eval

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation
Bug fixing for bucketing dataset
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Fixes a bug in the ASR bucketing which made the training to hang at the end of first epoch. 

**Collection**: ASR

# Changelog 
- Fixed the bucketing bug by making the bucketing dataloaders to loop over the samples indefinitely.

**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation
RadTTS: added support for multi-resolution extension and using transformers
State: closed | Pull request: yes
Labels: name: TTS

name: stale

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
ASR Beam search documentation
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Update docstrings for asr beam search computation

**Collection**: [ASR]

# Changelog 
- Update train kenlm arg
- Update args for beam search eval

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation
Flashlight improvements and fixes, also fixed potential VRAM leak in ctc_models.transcribe
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Flashlight improvements and fixes, also fixed potential VRAM leak in ctc_models.transcribe

**Collection**: ASR

# Changelog 
- Set logits to CPU in ctc_models.transcribe
- Changed all `self.decoder.vocabulary` calls in ctc_models to be `OmegaConf.to_container(self.decoder.vocabulary)`, this is necessary for performance reasons, because downstream decoders like Flashlight access the vocabulary a lot, and if it's passed as a ListConfig, it's incredibly expensive to access repeatedly, hence pass as Python list instead
- Made changes to silence/unknown token handling in flashlight decoder
- Removed the unit_lm boolean requirement in flashlight_cfg. It's not necessary, the decoder knows that if you pass a lexicon it should use it, otherwise do lexicon-free mode. There's no need for a boolean too, it's superfluous, the presence of the lexicon file is the only indicator the class needs

# Usage
No changes to how anything is used, except don't pass `unit_lm` anymore to `flashlight_cfg`

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix typos
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

Fix typos in notebooks

**Collection**: [ASR]

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Fix typos
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fix typos in notebooks

**Collection**: [ASR]

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
fix small issue when running alsd beam-search with stateless decoders
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

This PR fixes a small issue when running ALSD beam-search with stateless decoder transducers.

**Collection**: ASR

# Changelog 
Add additional checks in alsd beam-search code to handle different decoders differently.

# Usage
N/A


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
small bugfix and add asr evaluator to doc
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

small bugfixes for rng in augmentor and eval_metrics for asr evaluator
add asr evaluator to doc.

[TODO] add asr evaluator to main page once we merge 1.17.0 to main so we had link to doc.

**Collection**: ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix bugs with interctc mixin
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Bug fixes from https://github.com/NVIDIA/NeMo/pull/6215/

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[BugFix] Fix diarization result path errors in tutorial notebook for r1.17.0
State: closed | Pull request: yes
Labels: name: Speaker Tasks

# What does this PR do ?

Fix diarization result path errors in tutorial notebook for `r1.17.0`
Merging the change happened for main in https://github.com/NVIDIA/NeMo/pull/6217 to release branch


**Collection**: 
ASR 

# Changelog 

- Fixed diarization result path errors and tested notebook again.


# Usage
- Run [Speaker_Diarization_Inference.ipynb](https://github.com/NVIDIA/NeMo/blob/main/tutorials/speaker_tasks/Speaker_Diarization_Inference.ipynb) tutorial notebook

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?
ASR contributors
[ASR] Re-organizing data simulator with faster audio loading speed for data-simulator
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Added a feature to data-simulator that loads audio files faster by limiting the target audio duration.

**Collection**: ASR

# Changelog 
- Audio reading functions in `build_sentence` are updated
- Added fully reproducible seeding mechanism for all the randomness.
- Changed all the wav reading functions to `AudioSegment.from_file()`. 
- Moved util functions to `data_simulation_utils.py`
- Added unit tests in `test_data_simul_utils.py`

# Usage
For relatively longer audio file sources (e.g. Fisher, Swbd etc) set the following 
`data_simulator.session_params.max_audio_read_sec`
to 2.5~3.5 seconds   (min is 2.5 sec). 
The shorter the faster the simulation is, but it will concatenate lots of words to form a sentence (speaker turn)

For datasets that are already segmented (e.g VoxCeleb, LibriSpeech etc), `max_audio_read_sec` does not need to be short (no speed gains). For already-segmented datasets, I recommend 10~20 seconds.

When we choose 2.5 secs for `data_simulator.session_params.max_audio_read_sec` in case of unsegmented datasets (Fisher, SWBD, TEDtalks, AMI, etc), we can reach 80~90% speed of the segmented datasets (Libri, Vox, etc)

```python
  python <NEMO_ROOT>/tools/speech_data_simulator/multispeaker_simulator.py  \
    num_workers=10 \
    data_simulator.random_seed=42 \
    data_simulator.manifest_filepath=manifest_with_alignment_file.json \
    data_simulator.session_params.max_audio_read_sec=3.0 \
    data_simulator.outputs.output_dir=./simulated_data \
    data_simulator.outputs.output_filename=sim_spk2_sess20 \
    data_simulator.session_config.num_sessions=1000 \
    data_simulator.session_config.num_speakers=2 \
    data_simulator.session_config.session_length=20 \
    data_simulator.background_noise.add_bg=False \
    data_simulator.background_noise.background_manifest=background_noise.json \
    data_simulator.background_noise.snr=40 \

```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

## Who can review?

Anyone in ASR
Micro batch loader for bert model
State: closed | Pull request: yes
Labels: name: cherry-pick

name: stale

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
RNNT patch
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fix issue with latest container interaction with Numba RNNT loss

**Collection**: [ASR]

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
rnnt loss contiguous labels fix
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

fixes contiguous labels exception in RNNT model training

**Collection**: ASR

# Changelog 
in losses/rnnt.py:
if not targets.is_contiguous():
            targets = targets.contiguous()

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
small bugfix and add asr evaluator to doc
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

small bugfixes for rng in augmentor and eval_metrics for asr evaluator
add asr evaluator to doc.

[TODO] add asr evaluator to main page once we merge 1.17.0 to main so we had link to doc.

**Collection**: ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix bugs with interctc mixin
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Bug fixes from https://github.com/NVIDIA/NeMo/pull/6215/

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Link to nm launcher
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Link to nm launcher
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Gpt integrate core
State: closed | Pull request: yes
Labels: name: core

name: NLP

# What does this PR do ?

Allow pre-commit to run.

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[NLP] Support T5 with Megatron Core
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Adds megatron core support for our T5 model. This only works for pre-training, due to dynamic max lengths in the GLUE/XLNI datasets fine-tuning seems to be broken. 

The fix going forward I think will be to pad to the maximum length to ensure the size of sequences are always the same. This is a requirement of the iterator object now used in the `training_step`.

I also need to confirm that the weights of the model are the same between main and this branch.

- [ ] Fix support for GLUE/XLNI Fine-tuning
- [x] Ensure convergence is the same
- [x] Fix support for encode/decode functions

**Collection**: NLP

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Download from cloud failed.
State: closed | Pull request: no
Labels: name: stale

Trying to load AmberNet with the next command:
vad_model = nemo_asr.models.EncDecSpeakerLabelModel.from_pretrained(model_name="langid_ambernet")
and get download error messages:

[NeMo I 2023-03-16 11:50:19 cloud:81] Download from cloud failed. Attempt 1 of 3
[NeMo I 2023-03-16 11:50:20 cloud:81] Download from cloud failed. Attempt 2 of 3
[NeMo I 2023-03-16 11:50:20 cloud:81] Download from cloud failed. Attempt 3 of 3
[TTS] fixed code alert 871 and typos
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

fixed code alert https://github.com/NVIDIA/NeMo/security/code-scanning/871

    * fixed alert
    * removed unused p_voiced argument.
    * remove unsued out_lens argument in func get_first-order-features
    * remove unused sigma values for infer function


**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue) https://github.com/NVIDIA/NeMo/issues/5938
[TTS] fix bugs for chinese and german tutorials.
State: closed | Pull request: yes
Labels: name: TTS

name: cherry-pick

# What does this PR do ?

remove whitelist path.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[BugFix] Fix diarization result path errors in tutorial notebook
State: closed | Pull request: yes
Labels: name: Speaker Tasks

# What does this PR do ?

Fix diarization result path errors in tutorial notebook


**Collection**: 
ASR 

# Changelog 

- Fixed diarization result path errors and tested notebook again.


# Usage
- Run [Speaker_Diarization_Inference.ipynb](https://github.com/NVIDIA/NeMo/blob/main/tutorials/speaker_tasks/Speaker_Diarization_Inference.ipynb) tutorial notebook

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?
ASR contributors