renwei2024/nemo-github-issues huggingface dataset:
Add more glob patterns for labeler
State: closed | Pull request: yes

# What does this PR do ?

Larger coverage for the auto-labeler!

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
See if auto labeler labels
State: closed | Pull request: yes
Labels: name: ASR

name: NLP

Signed-off-by: SeanNaren <snarenthiran@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Add adapter for new speaker adaptation
State: closed | Pull request: yes
Labels: name: TTS

name: stale

# What does this PR do ?

Adds adapter support for new speaker adaptation. The following are the contributions of this PR:
1) Adds adapter support to TTS (FastPitch) pipeline for new speaker adaptation.
2) Adds Global Style Token based speaker representation to multi-speaker TTS pipeline.
3) Adds Speaker Verification based speaker representation to multi-speaker TTS pipeline.


**Collection**: [tts]

# Changelog 
- `examples/tts/conf/fastpitch_speaker_adaptation.yaml` - A new config file that is used for adapter training or using GST speaker representation.
- `examples/tts/fastpitch_finetune_adapters.py` - Script to Finetune adapters for new speaker adaptation. This script edits the config to add adapters, freeze the pretrained model and trains just the adapters.
- `nemo/collections/tts/parts/mixins/fastpitch_adapter_mixins.py` - This is a new file that contains all adapter related methods for FastPitch model. `parts/mixins` follows ASR folder structure.
- `nemo/collections/tts/modules/aligner.py, fastpitch.py, transformer.py` - the changes in these files are corresponding to inserting adapters to each of these modules.
- `nemo/collections/tts/modules/submodules.py` - GST based speaker representation module's implementation is in this file.
- `nemo/collections/tts/torch/tts_data_types.py, data.py` - GST related changes made to dataloader. GST needs a reference audio for a particular speaker. So a new datatype `GSTRefAudio` is introduced, this datatype refers to the reference audio.
- `tutorials/tts/FastPitch_Speaker_Adaptation.ipynb` - Tutorial explaining step by step how to use adapters to adapt a new speaker in FastPitch.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add auto-labeler
State: closed | Pull request: yes

# What does this PR do ?

Test to see if adding this action will allow us to auto-label based on the files changed.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Draft: Add activations_checkpointing_granularity to Bert models.
State: closed | Pull request: yes

# What does this PR do ?

Adds activations_checkpoint_granularity to BERT models, which is now required to run any sort of activations checkpointing.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Bert sequence parallel support
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Bug fix/gpt
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Suggest users to add GitHub labels to PRs
State: closed | Pull request: yes

# What does this PR do ?

Suggests users to add labels to their PR. This helps classify for maintainers.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fixed missing fields
State: closed | Pull request: yes
Labels: name: bug

# What does this PR do ?
bugfix

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]
ASR
# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [+] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Sde transcribe 22_11
State: closed | Pull request: yes

# What does this PR do ?
bugfix

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]
ASR
# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [+] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add missing import in transcribe_utils.py
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: ASR

# Changelog 
- Add missing import `model_utils` in `nemo/collections/asr/parts/utils/transcribe_utils.py`
  - When I use `examples/asr/transcribe_speech.py`, the following error has been occurred:
```
Traceback (most recent call last):
  File "/workspace/NeMo/examples/asr/transcribe_speech.py", line 146, in main
    asr_model, model_name = setup_model(cfg, map_location)
  File "/workspace/NeMo/nemo/collections/asr/parts/utils/transcribe_utils.py", line 184, in setup_model
    imported_class = model_utils.import_class_by_path(classpath)  # type: ASRModel
NameError: name 'model_utils' is not defined
```

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add core classes and functions for online clustering diarizer
State: closed | Pull request: yes
Labels: name: feature

name: ASR

Signed-off-by: Taejin Park <tango4j@gmail.com>

# What does this PR do ?

Add core classes and functions for online clustering diarizer

**Collection**: [Note which collection this PR will affect]

# Changelog 

#### `OnlineSegmentor` class in speaker_utils.py: 
online version of speech segmentations

#### `OnlineSpeakerClustering` in nmesc_clustering.py
online version of `SpeakerClustering` class (online speaker counting and permutation matching)

#### `OnlineClusteringDiarizer` in online_diarizer.py (inherits ClusteringDiarizer)
online version of `ClusteringDiarizer` class in clustering_diarizer.py

#### `test_diar_utils.py` in NeMo/tests/collections/asr/ 
Tests all clustering and online diarization related functions

<img width="966" alt="image" src="https://user-images.githubusercontent.com/13211483/204661137-78d0272b-1594-4509-a486-a8720e835bc4.png">



Checkout this slide for better understanding on why online diarizer needs
(1) linear sum assignment (Hungarian alg)
(2) Interval merging
(3) merge_vector function
[![image](https://user-images.githubusercontent.com/13211483/203663864-2966ad2a-0268-4d5c-9819-cf2ca4d4e8ad.png)
Slides link
](https://docs.google.com/presentation/d/1LDUTIGRNVuLJsy81V3GaHZlFDUpxZ9rux-fNPfgcLmw/edit?usp=sharing)

If you are curious about why online diarization needs FIFO-queue, take a look at the most recent paper on online neural diarization
[Online Neural Diarization of Unlimited Numbers of Speakers](https://arxiv.org/pdf/2206.02432.pdf)

# Usage

This is core classes and functions update.
Example script will be pushed in the second round of PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.
[Online Neural Diarization of Unlimited Numbers of Speakers](https://arxiv.org/pdf/2206.02432.pdf)

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Cleanup docs on r1.13.0
State: closed | Pull request: yes
Labels: name: stale

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>

# What does this PR do ?

Some fixes and minor improvements to README.rst and RTD docs,

# Changelog 

- [x] Moves Requirements and Installation into single source of truth (on RTD)
- [x] Cleans up README.rst to make it less verbose and removing redundant and obsolete links
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation
Inference Size Error Speaker_ Diarization
State: closed | Pull request: no
Labels: name: bug

name: stale

name: Speaker Tasks

On the speaker_diarization ipynb file. If I switch the link to the audio to a custom audio then I get a shape error on inference. It shows that I have a  tensor but they required in some other format.

**Steps/Code to reproduce bug**
[Link of Notebook
](https://github.com/NVIDIA/NeMo/blob/main/tutorials/speaker_tasks/Speaker_Diarization_Inference.ipynb)
On this notebook, just switch the link of audio or upload a audio and replace it with the an4 one.

**Expected behavior**
Diarization perfectly finished.

**Environment overview (please complete the following information)**
Colab.

**Environment details**
Colab.
export_utils bugfix
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

Updates MatchedScaleMaskSoftmax replacement

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Removes redundant class in export_utils

# Usage
* N/A

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[BugFix] Removing <unk> tokens from decoding timestamp
State: closed | Pull request: yes

Signed-off-by: Taejin Park <tango4j@gmail.com>

# What does this PR do ?

Removing `<unk>` tokens from decoding timestamp.
This solves length mismatching error in audio files contain lots of silence.

**Collection**: [Note which collection this PR will affect]
ASR (speaker tasks)

# Changelog 

in decoder_timestamp_utils.py, this PR is making change to exclude
`<unk>`  tokens for both word and word-timestamp output.

`<unk>` tokens are outputted as " " (blank space) in text hypothesis output, 
and this created mismatch of length since split() command removes all kinds of space 
at the start/end

- Also fixed ASR with diar notebook bugs
- Also changed hard-coded variable to a function argument

# Usage

```python
python offline_diar_with_asr_infer.py \
    diarizer.manifest_filepath=<path to manifest file> \
    diarizer.out_dir='demo_asr_output' \
    diarizer.speaker_embeddings.model_path=<pretrained modelname or path to .nemo> \
    diarizer.asr.model_path=<pretrained modelname or path to .nemo> \
    diarizer.asr.parameters.asr_based_vad=True \
    diarizer.speaker_embeddings.parameters.save_embeddings=False
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in ASR
export_utils bugfix
State: closed | Pull request: yes

# What does this PR do ?

Updates MatchedScaleMaskSoftmax replacement

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Removes redundant class in export_utils

# Usage
* N/A

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[ASR] Transcribe for multi-channel signals
State: closed | Pull request: yes
Labels: name: feature

name: ASR


# What does this PR do ?

This PR adds the following optional parameters to ASR transcription:
- select a single channel from multi-channel input
- override the default audio key (`"audio_filepath"`) when loading paths from a manifest

**Collection**: ASR

# Changelog 
- Added optional `channel_selector` parameter to `TranscriptionConfig`
- Added optional `audio_key` parameter to `TranscriptionConfig`
- Propagated the two parameters to `transcribe` and `transcribe_utils`

# Usage

Transcription config can be updated, for example as

```python
cfg.channel_selector = 0 # Evaluate on the first channel
cfg.audio_key = 'alternative_audio_filepath' # Load audio from an alternative filepath provided in the manifest
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix a bug with positional vs key-word based argument passing in the transformer layer
State: closed | Pull request: yes
Labels: name: bug

name: cherry-pick

name: NLP

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Fixes a bug that affects T5-RPE due to positional vs key-word based argument passing.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add Silence Augmentation
State: closed | Pull request: yes
Labels: name: feature

name: ASR

name: Speaker Tasks

Signed-off-by: fayejf <fayejf07@gmail.com>

# What does this PR do ?

Add support for online silence perturbation for data augmentation  and robustness test. 

**Collection**: ASR

# Changelog 
- Add silence at the beginning and/or end of an utterance. 

# Usage
* You can potentially add a usage example below

```python
train_ds:
    .....
    augmentor:
      silence:
        prob: 0.8
        min_start_silence_secs: 0
        max_start_silence_secs: 5
        min_end_silence_secs: 0
        max_end_silence_secs: 5
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
       Will add this to [Online_Noise_Augmentation tutorial](https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/Online_Noise_Augmentation.ipynb) later if necessary
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix a bug with positional vs key-word based argument passing in the transformer layer
State: closed | Pull request: yes
Labels: name: bug

name: NLP

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Fixes a bug that affects T5-RPE due to positional vs key-word based argument passing.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Bug Fix for bert to run on entire validation dataset
State: closed | Pull request: yes
Labels: name: bug

name: NLP

Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add num layers check for full activation checkpointing
State: closed | Pull request: yes
Labels: name: cherry-pick

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Adds a check that performs full activation checkpointing only if activation_checkpoint_num_layers > 0

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Radtts 1.13 plus
State: closed | Pull request: yes
Labels: name: TTS

name: cherry-pick

# What does this PR do ?
I finally managed to properly script only necessary part, and added a guard that works around the issue with auto-casted TorchScript not workin in non-Python environment (Triton)
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add num layers check for full activation checkpointing
State: closed | Pull request: yes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Adds a check that performs full activation checkpointing only if activation_checkpoint_num_layers > 0

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
ALiBi Positional Embeddings
State: closed | Pull request: yes
Labels: name: feature

name: NLP

name: CI

Adding support of ALiBi positional embeddings to Megatron 
https://arxiv.org/abs/2108.12409

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] DE Multi-Speaker FastPitch HiFiGAN Tokenization Error
State: closed | Pull request: no
Labels: name: TTS

name: stale

When following the basic instructions [here](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/tts_de_fastpitch_multispeaker_5): 
just replace the text promp with: "hallo wie geht es ihnen?" - German for "Hello, how are you doing?"

This occours: 

[hallo wie geht es ihnen?] contains unknown char: [g]. Symbol will be skipped.

**Expected behavior**

The letter "g" is commonly used in the German alphabet and is essential to pronunciation of certain words. It seems that this issue is related to the trained models that are provided under that link.
Question Answering Colab Notebook Error
State: closed | Pull request: no
Labels: name: bug

name: stale

name: NLP

**Describe the bug**

I get an error when trying to run inference on the provided question answering tutorial notebook.

**Steps/Code to reproduce bug**

Run this notebook to reproduce the error.
https://colab.research.google.com/github/NVIDIA/NeMo/blob/stable/tutorials/nlp/Question_Answering.ipynb

**Expected behavior**

It should run successfully

**Environment overview (please complete the following information)**

 - Environment location: Collab

**Additional context**

Error: `ReferenceError: weakly-referenced object no longer exists`
Code: `exp_dir = exp_manager(model.trainer, config.exp_manager)`
Radtts 1.13
State: closed | Pull request: yes
Labels: name: TTS

name: cherry-pick

# What does this PR do ?

Cherry pick of #5358 from main.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Vits exp
State: closed | Pull request: yes
Labels: name: TTS

name: stale

name: ASR

name: NLP

name: Speaker Tasks

name: CI

name: common

# What does this PR do ?

Adding VITS model to NeMo

**Collection**: [Note which collection this PR will affect]```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
NeMo's BERT based intent slot classification model (nemo_nlp.models.IntentSlotClassificationModel) fails to train
State: closed | Pull request: no
Labels: name: bug

name: stale

name: NLP

NeMo's BERT based intent slot classification model (nemo_nlp.models.IntentSlotClassificationModel) fails to train (training loss remains the same after every epoch) when we use NeMo's stable versions (eg. v1.12.0, 'main' branch). However it trained perfectly well when we used the `r1.11.0` branch instead.

Please run this [Joint Intent Slot Classification](https://github.com/NVIDIA/NeMo/blob/main/tutorials/nlp/Joint_Intent_and_Slot_Classification.ipynb) notebook as it is to reproduce the bug.

 - Environment location: GCP Compute Engine (NVIDIA T4 Tensor Core GPU)
 - Method of NeMo install: 
       `BRANCH = 'main'`
       `!python -m pip install git+https://github.com/NVIDIA/NeMo.git@$BRANCH#egg=nemo_toolkit[nlp]`
Decoding the words using tokens of same language on an ASR trained using a code-switched bilingual dataset
State: closed | Pull request: no
Labels: name: stale

name: ASR

Hi All,
I have trained a Hindi-English code-switched ASR system using the dataset having the transcription written in English (for English words) &  Hindi (for Hindi words). 
Example:
 क्या सौ रुपए loan लेना चाहती है आप
The tokenization script resulted in all the letters from Hindi + English for vocabulary creation. Nemo pretrained Hindi model followed by transfer learning using this dataset is done. When I decode on test set using this model I get some words having tokens from both languages.
Example: "doubल"  --> "doub" (English) + "ल" (Hindi) --> ideally it should have decoded as "double".. 

I want to know, is there any way (any settings/options) to restrict all the tokens in a word to be from a single language (and not a mix of more than one language). Please suggest. 
Thanks
Radtts 1.13 plus
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?
I finally managed to properly script only necessary part, and added a guard that works around the issue with auto-casted TorchScript not workin in non-Python environment (Triton)
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Support speaker level pitch normalization
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Add script for computing speaker specific pitch mean & standard deviation, and logic in data loader to use it at training time.

Global pitch normalization can be done using the previous configs, or by providing a JSON with only a "default" key.

There is a bit of awkward typing logic due to our speaker IDs being integers but JSON only supporting string keys.

**Collection**: [TTS]

# Changelog
- Add script for computing the mean and standard deviation of pitch for each speaker in a dataset and storing it in a JSON dictionary
- Add data loading logic to support parsing these pitch statistics from the JSON.
- Register stats file in FastPitch model
- Add pitch stats for Spanish openSLR data and modify .yaml config example.
- Added map from speaker name to speaker ID (integer) used as the keys in the pitch statistics, and as the possible inputs to the NGC model (https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/tts_es_multispeaker_fastpitchhifigan). The script for creating this speaker mapping will not be merged until I add the script/tutorial for downloading and processing the openSLR data.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation