renwei2024/nemo-github-issues huggingface dataset:
Fix bug into Dialogue tutorial
State: closed | Pull request: yes

# What does this PR do ?

set find_unused_parameters=True to overcome RuntimeError of unused pooler layer for BERT model.

**Collection**: nlp

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] fixed type of filepath and rename openslr.
State: closed | Pull request: yes

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?
* rename openslr directory to openslr_95 for disambiguation.
* fixed wrong type of filepath.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add SDP documentation
State: closed | Pull request: yes

Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com>

# What does this PR do ?

Adds documentation to Speech Dataset Processor

**Collection**: [Note which collection this PR will affect]
tools/speech_dataset_processor

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Pcla tutorial fixes
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Pcla tutorial fixes
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add Gradio App to ASR Docs
State: closed | Pull request: yes

Signed-off-by: smajumdar <titu1994@gmail.com>

# What does this PR do ?

Adds a Gradio demo app embedded in the ASR Docs page

**Collection**: [ASR]

# Usage
Visit https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/asr/intro.html after PR merge on main branch

Example - 
![image](https://user-images.githubusercontent.com/3048602/198553699-417ae89c-3b42-4649-9fbb-8b8ecab4eb86.png)


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
created
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.
Adds the volume loss to CTC models

**Collection**: [Note which collection this PR will affect]
ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.
Added the new reduction to the ctc loss class

# Usage
In the config file, set
ctc_reduction: 'mean_volume'

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ X] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix failing speaker counting for short audio samples
State: closed | Pull request: yes

# What does this PR do ?

From the previous PR (https://github.com/NVIDIA/NeMo/pull/5191/files), 
Speaker clustering code is now fully jit-scriptable, but speaker counting algorithm for 
short audio sample (less than 40s) was dropped.

In this PR, we are adding `getEnhancedSpeakerCount` function back and now tutorial notebooks are working correctly.

**Collection**: [Note which collection this PR will affect]

# Changelog 

- added `getEnhancedSpeakerCount` function back that handles very short audio samples
- checked the python and torch.jit.script modes

# Usage
* You can potentially add a usage example below

```python
  python neural_diarizer/multiscale_diar_decoder_infer.py \
    diarizer.manifest_filepath=<path to manifest file> \
    diarizer.out_dir='demo_output' \
    diarizer.speaker_embeddings.parameters.save_embeddings=False \
    diarizer.vad.model_path=<pretrained model name or path to .nemo> \
    diarizer.speaker_embeddings.model_path=<pretrained speaker embedding model name or path to .nemo> \
    diarizer.msdd_model.model_path=<pretrained MSDD model name or path .nemo> \
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Text Memmap Parsing Improvements
State: closed | Pull request: yes

# What does this PR do ?

1. Fixed text-memmap issue when boundary (e.g., new-line) is missing from end of file.
2. Added support in custom index building method (determines how to split int data into samples) using `build_index_fn(fn, newline_int)` argument. The method can override default indexing of the file `fn`.
3. Added support in partial sample loading and alternative decoding by overriding `_fetch_sample_from_memmap(self, mdata, i, j)`.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Specifying audio signal dropout separately for the Conformer Encoder
State: closed | Pull request: yes

# What does this PR do ?

Adds the option to specify the audio signal dropout separately for a conformed encoder.

**Collection**: asr

# Changelog 
- Updated the configs using conformed encoder
- Add new dropout_audio parameter in conformer_encoder.py

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation


## Who can review?

Anyone in the NeMo ASR team.
[TTS] Create script for processing TTS training audio
State: closed | Pull request: yes

Signed-off-by: Ryan <rlangman@nvidia.com>

# What does this PR do ?

Add a script for doing audio preprocessing for TTS training.

**Collection**: [TTS]

# Changelog 
- Created a "data" directory under nemo.collections.tts following the same convention as the ASR & NLP code.
- Added data_util class for generic operations like processing training manifest. Not sure of volume normalization should be there, but currently have no better place to put it.
- Added silence trimming implementations using energy and VAD based approaches.
- Added audio_preprocessing script for running the data processing logic over a training manifest, with example config files.

# Usage

```
HYDRA_FULL_ERROR=1 python /home/rlangman/Code/NeMo/scripts/dataset_processing/tts/audio_processing/preprocess_audio.py \
    --config-path=/home/rlangman/Code/NeMo/scripts/dataset_processing/tts/audio_processing/config \
	--config-name=preprocessing.yaml \
	data_base_dir="/home/rlangman/Data/nemo_processing" \
	config.num_workers=2 \
	config.max_entries=100 \
	trim=vad \
	trim.device=cuda
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

# Additional Information

I manually tested the silence trimming using the openSLR Spanish data. For the 2 modes the dataset size decreases from:

Energy: 35 hours -> 28.8 hours
VAD: 35 hours -> 22.6 hours

This is because lots of the audio has noise in the long silences at the start and end of files. The VAD can differentiate the noise from speech while energy cannot.

Doing trimming on my workstation with the ~24k utterance dataset takes about 5 minutes with energy and 30 minutes with VAD.

More automated testing might be possible if we add CI tests that use actual audio, but I would leave that for a future PR as we do not yet have guidelines for TTS CI tests.
Fix links to speaker identification notebook
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

Fixes #5259 

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix links to speaker identification notebook
State: closed | Pull request: yes

# What does this PR do ?

Fixes #5259 

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix links to speaker identification notebook
State: closed | Pull request: yes

# What does this PR do ?

Fixes #5258

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Extend English ITN rules for cardinal to support hundreds
State: closed | Pull request: yes

Signed-off-by: Vasyl Spachynskyi <vspachyn@gmail.com>

# What does this PR do ?

Extend ITN rules for English (cardinals) to support hundreds, like: 
- eleven hundred -> 1100
- twenty one hundred -> 2100
- ninety nine hundred and ninety nine -> 9999

**Collection**: [Text Processing]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Issue with english WFST inverse text normalizer and "per cent"
State: closed | Pull request: no
Labels: name: bug

name: stale

name: TN/ITN

**Describe the bug**

The english WFST inverse text normalizer fails to correctly tag "per cent". 

**Steps/Code to reproduce bug**

```
from nemo_text_processing.inverse_text_normalization.inverse_normalize import InverseNormalizer
inverse_normalizer = InverseNormalizer(lang='en')

spoken = "The share price of BHP increased by eleven per cent"
un_normalized = inverse_normalizer.inverse_normalize(spoken, verbose=False)
print(un_normalized)
```

**Expected behavior**

I expect "The share price of BHP increased by 11 %". 

The actual output behavior is an unchanged input - "The share price of BHP increased by eleven per cent"

NB: If I change the input to "... per cents", then I do get the expected output. There's an issue with how the plural / singular forms are handled in this case
Update ASR Scores and Results
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

Updates new models, results and configs

**Collection**: [ASR]


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation
Update ASR Scores and Results
State: closed | Pull request: yes

# What does this PR do ?

Updates new models, results and configs

**Collection**: [ASR]


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation
fix hypotheses return
State: closed | Pull request: yes

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

# What does this PR do ?

Fix issue with transcriptions type

**Collection**: ASR

# Changelog 
- Add return_hypothesis to stay consistent with rest of asr models for partial audio transcribe


# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Fix link to inference notebook
State: closed | Pull request: yes
Labels: name: cherry-pick

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

# What does this PR do ?

Fixes the Inference_ModelSelect notebook link in the NeMo TTS Primer notebook.

**Collection**: TTS

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Floating point exception
State: closed | Pull request: no
Labels: name: bug

Hello,

I installed all necessary dependencies and Nemo, but I got an issue when I tried to transcribe or diarize using Nemo
The exception is: **Floating point exception**, without any stacktrace, just this message
This exception occurred when the Nemo model was initialized

Dependencies:
> Cuda version: 11.2
> Python version: 3.8.10
> Torch version: 1.12.0+cu113
> Nemo version: 1.12.0 (I followed the steps for PIP in the readme)
> OS - Ubuntu 20.04.5 LTS
> Graphic card - Nvidia A40

I tried to use lower versions of PyTorch, it didn't help though.
Usage of PyTorch 1.9.0+cu111 and lower caused another exception: **OSError: libtorch_hip.so: cannot open shared object file**
I found a few topics where developers faced this issue and I even tried to fix it, but without success

If anyone knows how to handle this exception, I will appreciate any help in advance.
Modernize RNNT ONNX export and add TS export
State: closed | Pull request: yes

# What does this PR do ?

Improve the script for onnx rnnt export and add support for torchscript export of rnnt

**Collection**: [ASR]

# Changelog 
- Add support for pretrained checkpoint in onnx export
- Add GPU inference support with `onnxruntime-gpu` installed for RNNT models
- Add support for Torchscript exported RNNT model
- Add support for eval of torchscripted RNNT model on GPU

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [x] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
O2 style amp for gpt3 ptuning
State: closed | Pull request: yes
Labels: name: core

name: NLP

1. Fixes a precision bug for ptuning which improves perf by  ~25% 
- If model is in bf16, Pytorch autocasting currently unnecessarily casts operations to be in fp16 from bf16, and casts the output back again to bf16 
- This fix keeps the precision in bf16, removing many unnecessary casts

2. Implements amp O2 which improves convergence times

- O2 accumulates prompt encoder gradients in fp32 master weights, rather than in bf16 or fp16 currently


![Capture](https://user-images.githubusercontent.com/67203904/202251706-4ff787fb-c1ee-4fab-8d16-201c26801a0b.PNG)
(validation loss of GPT3 530b on 7 nodes, O2 (orange) vs baseline (pink))
Option to pad the last validation input sequence if its smaller than the encoder sequence length for MegatronGPT
State: closed | Pull request: yes

Option to pad the last validation input sequence if its smaller than the encoder sequence length for MegatronGPT

# What does this PR do ?

If validation_drop_last is set to False, the last input sample from the validation dataset will not be dropped if its smaller than the encode sequence length for MegatronGPT

**Collection**: [Note which collection this PR will affect]
nlp/language_modeling/megatron

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] HiFi-TTS Download Script
State: closed | Pull request: yes

# What does this PR do ?

Download hifitts dataset and reformat into ljspeech format.

**Collection**: [Note which collection this PR will affect]

# Usage

```python
python scripts/dataset_processing/tts/hifitts/get_data.py --data-root /home/ovolkovskyi/NeMo/
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
update tutorials to use meeting config as default and VAD
State: closed | Pull request: yes

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

# What does this PR do ?

* Update existing scripts to use meeting config as default 
* Add VAD as default in tutorials


**Collection**: Speaker Diarization

# Changelog 
- yaml file update
- tutorial update


# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation
Debug support for interleaved pipeline parallelism with the distributed Adam optimizer
State: closed | Pull request: yes

Signed-off-by: Tim Moon <tmoon@nvidia.com>

# What does this PR do ?

Debug support for interleaved pipeline parallelism with the distributed Adam optimizer.

@erhoo82 has observed poor overlapping of gradient synchronizations and backward compute when running with interleaved pipeline parallelism. This is mostly fixed in Apex with https://github.com/NVIDIA/apex/pull/1514, which makes some changes to the API. This is mostly because the synchronization behavior is different for pipeline parallelism if the interleaved schedule is enabled or disabled (without interleaving we only want overlapped grad syncs in the first pipeline stage, while with interleaving we want overlapped grad syncs in all pipeline stages).

**Collection**: NLP

# Changelog 
- Use updated Apex API for pipeline parallelism, namely for overlapped grad syncs

# Usage

Set the optimizer to `distributed_fused_adam` in the config file:
https://github.com/NVIDIA/NeMo/blob/232db1513259a5c3fcd5689b0f6b89c212f83d03/examples/nlp/language_modeling/conf/megatron_gpt_config.yaml#L189

Enable interleaved pipeline parallelism by setting `virtual_pipeline_model_parallel_size` to a value greater than 1 in the config file:
https://github.com/NVIDIA/NeMo/blob/232db1513259a5c3fcd5689b0f6b89c212f83d03/examples/nlp/language_modeling/conf/megatron_gpt_config.yaml#L51

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

Pinging @ericharper and @erhoo82.

# Additional Information
* Depends on https://github.com/NVIDIA/apex/pull/1514
* Follow-up to https://github.com/NVIDIA/NeMo/pull/4900 and https://github.com/NVIDIA/NeMo/pull/5025
Add NGC links for Aligner and FastPitch
State: closed | Pull request: yes

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

# What does this PR do ?

Add option to load IPA-based checkpoints for FastPitch and the TTS Aligner.

**Collection**: TTS

# Changelog 
- Added one entry each to the Aligner model and FastPitch model NGC model loading.
- Did not add entries to the checkpoints table in the TTS documentation since they are new versions of existing model cards, and each entry in each checkpoint table is a single model card.

# Usage

```python
from nemo.collections.tts.models import AlignerModel, FastPitchModel

# IPA Aligner
aligner = AlignerModel.from_pretrained("tts_en_radtts_aligner_ipa")
# IPA FastPitch
fastpitch = FastPitchMode.from_pretrained("tts_en_fastpitch_ipa")
```
Check for A100 in Jenkins test
State: closed | Pull request: yes

# What does this PR do ?

Updates test to specifically check for A100

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
remove CV requirements.
State: closed | Pull request: yes

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] unify requirements_tts.txt and requirements_torch_tts.txt
State: closed | Pull request: yes

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update perturb.py
State: closed | Pull request: yes

Add checking for channels mismatch for audio and noise data, throw an exception if they have different number of channels. Also fixed `perturb_with_foreground_noise` as done in `perturb_with_input_noise` 

Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com>
CUDA out of memory after 44 epochs
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

Hi
I am trying to run QuartzNet 10x5 model on my dataset of almost 200 hours audio. But the strange thing is I got this error:

```
RuntimeError: CUDA out of memory. Tried to allocate 130.00 MiB (GPU 0; 11.91 GiB total capacity; 10.99 GiB already allocated; 50.38 MiB free; 11.19 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
```

I can't understand why I got out of memory error after 44 epochs!

**Steps/Code to reproduce bug**

TRAIN_DS  config:
```
  batch_size: 4
  trim_silence: true
  max_duration: 62.0
  normalize_transcripts: false
  shuffle: true
  num_workers: 8
  pin_memory: true
  is_tarred: false
  tarred_audio_filepaths: null
  shuffle_n: 2048
  bucketing_strategy: synced_randomized
  bucketing_batch_size: null
  augmentor:
    white_noise:
      prob: 0.4
      min_level: -90
      max_level: -46
    gain:
      prob: 0.4
      min_gain_dbfs: 0
      max_gain_dbfs: 50
    speed:
      prob: 0.3
      sr: 16000
      resample_type: kaiser_best
      min_speed_rate: 0.5
      max_speed_rate: 2.0
      num_rates: -1
    time_stretch:
      prob: 0.3
      min_speed_rate: 0.5
      max_speed_rate: 2.0
      num_rates: -1
    noise:
      prob: 0.45
      manifest_path: ./noise_manifest.json
      min_snr_db: -4
      max_snr_db: 10
      max_gain_db: 300.0
```
VALIDATION_DS config:

```
  batch_size: 4
  normalize_transcripts: false
  shuffle: false
  num_workers: 8
  pin_memory: true
  trim_silence: true
```

TRAINER config:
```
trainer = ptl.Trainer(devices=-1,
                      accelerator='gpu',
                      max_epochs=EPOCHS,
                      accumulate_grad_batches=1,
                      enable_checkpointing=False,
                      logger=False,
                      log_every_n_steps=5,
                      check_val_every_n_epoch=1,
                      strategy="ddp")
```

**Environment overview (please complete the following information)**

 - Environment location: Bare-metal
 - Method of NeMo install: pip install nemo_toolkit['all']
 

**Environment details**

- OS version : Ubuntu 20.04 LTS
- PyTorch version:   1.12.1
- Python version: 3.8.0

**Additional context**

I ran my model on 2 GPU's : 
TITAN X 12288MiB and
GTX 1080 Ti 11264MiB
Spellchecking asr customization
State: closed | Pull request: yes

# What does this PR do ?

This is a model for spellchecking ASR customization, scripts for data preparation and description of the pipeline

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Thutmose tagger for tn
State: closed | Pull request: yes

# What does this PR do ?

1. Some corrections to preprocessing. Should lead to improved scores compared to current thutmose ITN for English.
2. Add experimental TN direction. 

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Thutmose tagger for tn
State: closed | Pull request: yes

# What does this PR do ?

1. Some corrections to preprocessing. Should lead to improved scores compared to current Thutmose ITN for English.
2. Add experimental TN direction. 

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Support for inserting additional subsampling in conformer encoder
State: closed | Pull request: yes

# What does this PR do?

Added support for specifying an additional subsampling layer in conformer encoder. The user can specify the type and the position of the subsampling layer.

**Collection**: asr

# Changelog 
- New `SubsamplingReductionModule` class in `nemo/collections/asr/parts/submodules/subsampling.py`
- Updated the `ConformerEncoder` in `nemo/collections/asr/modules/conformer_encoder.py` to accept the new reduction parameters for adding a new subsampling layer in the desired position.
- Updated the _transducer-related configs_ with reduction parameters.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

## Who can review?

Everyone in the NeMo ASR team.