renwei2024/nemo-github-issues huggingface dataset:
add suffix [add]?[digit] to thousand graph
State: closed | Pull request: yes
Labels: name: stale

Signed-off-by: Cuong Nguyen Dang <nndcuong@gmail.com>

# What does this PR do ?

Fix issue \(https://github.com/NVIDIA/NeMo/issues/5367)

**Collection**: ITN

# Changelog 
- add suffix ```[and]?[digit]``` to thousand graph

# Usage
```python
from nemo_text_processing.inverse_text_normalization.en.taggers.cardinal import CardinalFst
from nemo_text_processing.inverse_text_normalization.en.taggers.ordinal import OrdinalFst
from nemo_text_processing.inverse_text_normalization.en.taggers.date import DateFst

cardinal = CardinalFst()
ordinal = OrdinalFst(cardinal)
date_fst = DateFst(ordinal).fst

text = "two thousand and nine"
print(pynini.shortestpath(text @ date_fst).string())
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (5367)
Clarifications on Noise augmentation experiments with Nemo
State: closed | Pull request: no
Labels: name: ASR

name: question

Hi 
I followed the instructions @ https://github.com/NVIDIA/NeMo/issues/1660 and could successfully train a musan noise augmented nemo ASR model. However, I have few queries:

1. Where does the noise augmented files are stored? Or Are these noise augmented files are simply produced on the fly, as and when needed, and discarded once the job is done (without retaining them)?

2. When I use augmentors during training, does it include original clean files also for training along-with noise augmented files? Or it only uses noise augmented files for training?

3. For the kind of experiments & datasets that we used, we had expected lower WERs with noise augmentation compared to WERs obtained on clean datasets. However the obtained results are other way (noise augmented datasets have 1% higher WER than clean data). This made us to suspect the implementation part of nemo asr training. I have used exactly the same config of audio_augmentations mentioned by @jbalam-nv. There were no errors during training as well. Do you still feel there might be some implementation gaps (which has led to lower WER). Any suggestions on this would be very helpful.

4. I have used musan free sound noise and RIR. Is it fine? or any suggestions on this? 

Thanks a lot,
Manjunath
error when trying inverse "two thousand and nine"
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

```pynini``` raises error *ERROR: StringFstToOutputLabels: Invalid start state* when I am trying to accept "two thousand and nine" with DateFst, which means that DateFst hasn't well-define to accept this text
![image](https://user-images.githubusercontent.com/57069030/200786099-62590f4b-4e44-4e52-829c-24ff96c7c995.png)

**Steps/Code to reproduce bug**

First, initialize the DateFst object
```
from nemo_text_processing.inverse_text_normalization.en.taggers.cardinal import CardinalFst
from nemo_text_processing.inverse_text_normalization.en.taggers.ordinal import OrdinalFst
from nemo_text_processing.inverse_text_normalization.en.taggers.date import DateFst

cardinal = CardinalFst()
ordinal = OrdinalFst(cardinal)
date_fst = DateFst(ordinal).fst
```
Then, compose the text ```"two thousand and nine"``` with the FST above
```
text = "two thousand and nine"
print(pynini.shortestpath(text @ date_fst).string())
```

**Expected behavior**

DateFST can accept ```"two thousand and nine"``` and return the result that looks like ```date { year: "2009" preserve_order: true }```
update doc in terms of get_label for lang id model
State: closed | Pull request: yes

Signed-off-by: fayejf <fayejf07@gmail.com>

# What does this PR do ?

add get_label for lang id model in doc  and reflect PR#[5278](https://github.com/NVIDIA/NeMo/pull/5278)

**Collection**: ASR


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
training stuck with multi gpu and dali training loader
State: closed | Pull request: no
Labels: name: bug

name: stale

name: ASR

**Describe the bug**

Training get stuck when using ddp and dali loader together.


![image](https://user-images.githubusercontent.com/1567200/200734230-11aff089-a2e8-402f-9396-007c74159b93.png)

the usage of second gpu is zero. 

it works if I change `use_dali` to False

**Steps/Code to reproduce bug**

I follow the asr finetuing tutorial (https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/ASR_CTC_Language_Finetuning.ipynb), and change `devices=2`, `strategy='ddp'`.

```
trainer = ptl.Trainer(devices=2, 
                      accelerator=accelerator, 
                      strategy='ddp',
                      max_epochs=EPOCHS, 
                      accumulate_grad_batches=1,
                      enable_checkpointing=False,
                      logger=False,
                      log_every_n_steps=5,
                      check_val_every_n_epoch=10)

```

also, I change  `cfg.train_ds.use_dali=True`

Any idea?
Adding Hybrid RNNT-CTC model
State: closed | Pull request: yes
Labels: name: ASR

name: CI

# What does this PR do ?

This PR is a refactored version of the following PR created by https://github.com/iankur:
https://github.com/NVIDIA/NeMo/pull/4854

It adds the Hybrid RNNT/CTC model which has two decoders of CTC and RNNT(Transducer) over the encoder.
It enables to train a single model instead of two which works with both CTC and RNNT decoding. It also speeds up the convergence for CTC models.

Added the following examples:
./examples/asr/asr_hybrid_transducer_ctc/speech_to_text_hybrid_rnnt_ctc_char.py
./examples/asr/asr_hybrid_transducer_ctc/speech_to_text_hybrid_rnnt_ctc_bpe.py

Along with these sample configs for hybrid Conformer model:
./examples/asr/conf/conformer/hybrid_transducer_ctc/conformer_hybrid_transducer_ctc_char.yaml
./examples/asr/conf/conformer/hybrid_transducer_ctc/conformer_hybrid_transducer_ctc_bpe.yaml

**Collection**: 
ASR

# Changelog 
- Added support for Hybrid RNNT/CTC ASR model.

**PR Type**:
- [X] New Feature
- [ ] Bugfix
- [X] Documentation
support to disable sequence length + 1 input tokens for each sample in MegatronGPT
State: closed | Pull request: yes

# What does this PR do ?

On setting the data config option: _no_seqlen_plus_one_input_tokens_ to True (default: False) in Megatron-GPT, the number of input tokens for each sample will be equal to the sequence length (instead of sequence length + 1) and the last token will be masked.  

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
P&C LA tutorial fixes
State: closed | Pull request: yes
Labels: name: cherry-pick

Signed-off-by: Matvei Novikov <mattyson.so@gmail.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Fixing RADTTS training - removing view buffer and fixing accuracy issue
State: closed | Pull request: yes

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>

A view to the bias was cached as a buffer - that led to cpu/gpu mismatch on backward pass when training - also the caching was wrong - removed. That led to script() breaking for export - had to change script routine to do trace on PartialConv first.
Training had NaNs when using partial convolutions in mixed precision mode - had to move from epsilon to masking to fit in float16 range without damping.
Reverting submodules.py back to original and re-doing script optimizations in a cleaner manner, fixed the issues.

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[ASR] Audio processing base, multi-channel enhancement models
State: closed | Pull request: yes
Labels: name: feature

name: ASR

# What does this PR do ?

This PR adds a base class for audio processing models, an enhancement model, a number of audio-related modules and tests.

**Collection**: ASR

# Changelog 
- Audio processing base model `AudioToAudioModel`
- `EncMaskDecAudioToAudioModel` subclass for speech enhancement
- Audio modules for spectrogram transforms, mask estimation based on RNN (`MaskEstimatorRNN`), mask-based processing (`MaskReferenceChannel` and `MaskBasedBeamforming`), `SDRLoss`
- Training script for `EncMaskDecAudioToAudioModel` in `speech_enhancement.py`
- Audio processing script for `AudioToAudioModel` in `process_audio.py`

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #5196
Bug fix (removing old compute consumed samples)
State: closed | Pull request: yes

Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
P&C LA tutorial fixes
State: closed | Pull request: yes

Signed-off-by: Matvei Novikov <mattyson.so@gmail.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix issues with PL 1.8
State: closed | Pull request: yes

# What does this PR do ?

Attempts to fix all issues after many deprecations/lightning lite becoming the backbone for a lot of the underlying pytorch lightning code.

cc @titu1994 

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation
Unable to deploy nemo model to riva pipeline (after conversion to riva)
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

Unable to deploy a fine-tuned Punctuation and Capitalization model to Riva pipeline

**Steps/Code to reproduce bug**

1. Fine tune a nemo punctuation and capitalization model. 
2. Convert it to rive using this guide: https://docs.nvidia.com/deeplearning/riva/user-guide/docs/model-overview.html
3. Deploy it using nvcr.io/nvidia/riva/riva-speech:2.7.0 and it is giving an empty response. 

Deploy command
NV_GPU=1 nvidia-docker run -ti --name riva_source -p 8008:8008 -p 50051:50051 -p 5800:8000 -p 58001:8001 -v <localtion_models>:/data/models -e MODEL_REPOSITORY="--model-repository /data/models" -e INTERIM_RESULTS="--interim-results"  -e CUSTOM_ITN="--custom-itn" --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 nvcr.io/nvidia/riva/riva-speech:2.4.0-server tritonserver --log-verbose=0 --strict-model-config=true --model-repository /data/models --cuda-memory-pool-byte-size=0:1000000000 --exit-on-error=false

This was working while using riva 1.10-beta server

A  helpful guide on on how to craft a minimal bug report  http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports. 

My ourput logs have the following. Please note the output contents are empty
model_name: "riva-punctuation-en-US"
id: "1"
inputs {
  name: "PIPELINE_INPUT"
  datatype: "BYTES"
  shape: 1
  shape: 1
}
outputs {
  name: "PIPELINE_OUTPUT"
}
raw_input_contents: "\372\000\000\000in a new password i can take it from here alrighty is there anything else i can help you with them thank you very much ms rostow it was my pleasure to assist you thank you for using optum r x and if you chose our survey it will be next okay thank you"

model_name: "riva-punctuation-en-US"
model_version: "1"
id: "1"
outputs {
  name: "PIPELINE_OUTPUT"
  datatype: "BYTES"
  shape: 1
  shape: 1
}
raw_output_contents: "\000\000\000\000"

[[b'']]

**Expected behavior**

Deploy the model correctly and output content is a punctuated input string

**Environment overview (please complete the following information)**

 - Environment location: nvcr.io/nvidia/riva/riva-speech:2.4.0-server
 - Method of NeMo install: install from source
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
nvcr.io/nvidia/riva/riva-speech:2.4.0-server
- OS version
- PyTorch version
- Python version

**Additional context**

Add any other context about the problem here.
Example: GPU model
Unable to resume ASR training
State: closed | Pull request: no

I am trying to resume my training by running the training command,it pick the last checkpoint properly and while stat training it show this error

![image](https://user-images.githubusercontent.com/107035118/200524914-f55b1b7a-d353-4224-9e45-c7b2ad146afc.png)

Environment details

OS version : Ubuntu 20.04 LTS
PyTorch version: 1.13.0a0+08820cb
Python version: 3.8.0
Nemo Version : 1.12.0
Additional context

I ran my model on 2 GPU's :
RTX 3090 and 
24267MiB
fix data loader with comma seperated manifest
State: closed | Pull request: yes
Labels: name: stale

# What does this PR do ?

Fix #5349 
 
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [x] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
can't use dali with comma seperated manifest
State: closed | Pull request: no
Labels: name: bug

name: stale

When using nvidia dali to train speech to text,

```python

cfg = copy.deepcopy(char_model.cfg)

"""
read cleaned manifest
"""
train_manifest_data = read_manifest(train_manifest_cleaned)
train_charset = get_charset(train_manifest_data)
dev_manifest_data = read_manifest(dev_manifest_cleaned)
dev_charset = get_charset(dev_manifest_data)

train_dev_set = set.union(set(train_charset.keys()), set(dev_charset.keys()))



# Setup train, validation, test configs
with open_dict(cfg):
  # Train dataset  (Concatenate train manifest cleaned and dev manifest cleaned)
  cfg.train_ds.manifest_filepath = f'{train_manifest_cleaned},{dev_manifest_cleaned}'
  cfg.train_ds.labels = list(train_dev_set)
  cfg.train_ds.normalize_transcripts = False
  cg.train_ds.batch_size = 32
  cfg.train_ds.use_dali = True
  cfg.train_ds.num_workers = 8
  cfg.train_ds.pin_memory = True
  cfg.train_ds.trim_silence = True

char_model.setup_training_data(cfg.train_ds)
```

![image](https://user-images.githubusercontent.com/1567200/200490593-dfa9ad0e-fcc3-4aa7-b735-9e4c597cb4a7.png)
bugfix in volume loss for CTC models
State: closed | Pull request: yes
Labels: name: bug

# What does this PR do ?
fixes bug in computation of volume loss

Add a one line overview of what this PR aims to accomplish.
fixes bug in computation of volume loss

**Collection**: [Note which collection this PR will affect]
ASR
# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Pcla tutorial fixes
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[DOC] added ipython dependency to support IPython.sphinxext extension
State: closed | Pull request: yes

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

This is to fix the doc build error below,
```
Could not import extension IPython.sphinxext.ipython_console_highlighting (exception: No module named 'IPython')
```

[root-cause] Previously, TTS doc added a block `.. ipython::` to show the list of models (https://github.com/NVIDIA/NeMo/blob/main/docs/source/tts/checkpoints.rst), and added extension (https://github.com/NVIDIA/NeMo/blob/main/docs/source/conf.py#L113-L114).

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Megatron Export Update
State: closed | Pull request: yes

Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com>

# What does this PR do ?

Updates export process for Megatron-based models

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Changed ORT optimization due to a bug in onnxruntime (https://github.com/microsoft/onnxruntime/issues/12579)

# Usage

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
only log true validation loss not sanity check
State: closed | Pull request: yes
Labels: name: stale

Signed-off-by: arendu <adithya.r@gmail.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[STT] Add Ru ASR Conformer-CTC and Conformer-Transducer
State: closed | Pull request: yes
Labels: name: cherry-pick

Signed-off-by: Sasha Meister <ameister@nvidia.com>

# What does this PR do ?

[STT] Add Ru ASR Conformer-CTC and Conformer-Transducer

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[STT] Add Ru ASR Conformer-CTC and Conformer-Transducer
State: closed | Pull request: yes

Signed-off-by: Sasha Meister <ameister@nvidia.com>

# What does this PR do ?

[STT] Add Ru ASR Conformer-CTC and Conformer-Transducer

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Support for finetuning and finetuning inference with .ckpt files & batch size refactoring
State: closed | Pull request: yes

# What does this PR do ?

Adds support for the following:

1. To be able to do fine-tuning and fine-tuning eval with `.ckpt` files as well as `.nemo files.
2. Refactoring fine-tuning to allow for `check_val_every_n_epochs` > 1 and no more additional checks needed for `trainer.val_check_interval`. This also fixes the progress bar when grad acc > 1.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[STT] add Ru ASR conformers to models collections
State: closed | Pull request: yes

Signed-off-by: ssh-meister <sasha.meister.work@gmail.com>

Add Ru ASR Conformer-CTC Large and Conformer-Transducer Large to NeMo models collections

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[STT] add Ru ASR conformers to models collections
State: closed | Pull request: yes

# What does this PR do ?

Add Ru ASR Conformer-CTC Large and Conformer-Transducer Large to NeMo models collections

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #5314
Fixing training/script issues for radtts, also removing grad from caches
State: closed | Pull request: yes

Fixing training/script issues for radtts, also removing grad from constant/cache buffers

1. A view to the bias was cached as a buffer - that led to cpu/gpu mismatch on backward pass when training - also the caching was wrong - removed. That led to script() breaking for export - had to change script routine to do trace on PartialConv first.
2. Training had NaNs  when using partial convolutions in mixed precision mode - had to increase epsilon for mask ratio to fit in float16 range.3. 
3. While we are there, removed gradients from cache buffers in some other modules. 

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
RuntimeError "Tensors must have same number of dimensions: got 1 and 2" in Speaker Diarization inference example file.
State: closed | Pull request: no
Labels: name: bug

name: stale

File ~/anaconda3/envs/hh/lib/python3.8/site-packages/nemo/collections/asr/data/audio_to_label.py:179, in _vad_frame_seq_collate_fn(self, batch)
    177 start = torch.zeros(append_len_start)
    178 end = torch.zeros(append_len_end)
--> 179 sig = torch.cat((start, sig, end))
    180 sig_len += slice_length
    182 if has_audio:

RuntimeError: Tensors must have same number of dimensions: got 1 and 2

Is anyone else seeing this error? 

I am using the Pytorch version = 1.13.0
RuntimeError Tensors must have same number of dimensions: got 1 and 2  
State: closed | Pull request: no
Labels: name: bug

File ~/anaconda3/envs/pyannotate/lib/python3.8/site-packages/nemo/collections/asr/data/audio_to_label.py:179, in _vad_frame_seq_collate_fn(self, batch)
    177 start = torch.zeros(append_len_start)
    178 end = torch.zeros(append_len_end)
--> 179 sig = torch.cat((start, sig, end))
    180 sig_len += slice_length
    182 if has_audio:

RuntimeError: Tensors must have same number of dimensions: got 1 and 2
add precommit hood to automatic sort entries in requirements.
State: closed | Pull request: yes

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

the files changes after running below,
``` bash
$ pre-commit run --all-files
```

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
How to do quantization on the ASR model
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

I am trying to do Post training static quantization on ASR model, following example here: (https://pytorch.org/blog/introduction-to-quantization-on-pytorch/#post-training-static-quantization) but keeps on getting following error. 
Two questions: 
1) Is there a way to bypass this error and get this working?
2) How to know what is the input shape for "stt_en_conformer_ctc_xlarge" model? Is there any API through which I know the shape?


```
File "asr_ctc.py", line 20, in <module>
    print(model.transcribe([audio_filepath], batch_size=1)[0])
 ---
 ----
 ---
 NotImplementedError: Could not run 'quantized::conv2d.new' with arguments from the 'CPU' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'quantized::conv2d.new' is only available for these backends: [Negative, UNKNOWN_TENSOR_TYPE_ID, QuantizedXPU, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, SparseCPU, SparseCUDA, SparseHIP, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, SparseVE, UNKNOWN_TENSOR_TYPE_ID, NestedTensorCUDA, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID].
```

**Steps/Code to reproduce bug**
```
# Streaming with CTC based models
import torch 
import nemo.collections.asr as nemo_asr

device = 'cpu'
model = nemo_asr.models.EncDecCTCModelBPE.from_pretrained("stt_en_conformer_ctc_xlarge", map_location=device)
model = model.to(device)
model.eval()
audio_filepath = './datasets/mini-dev-clean/LibriSpeech/dev-clean-2-processed/2428-83699-0042.wav'
model.qconfig = torch.quantization.get_default_qconfig()
torch.quantization.prepare(model, inplace=True)
torch.quantization.convert(model, inplace=True)
print("The transcribed speech is: ")
print(model.transcribe([audio_filepath], batch_size=1)[0])

```

**Expected behavior**

Expected transcribed output 

**Environment overview (please complete the following information)**

 - Environment location: [Bare-metal]
 - Method of NeMo install: [pip install Nemo].

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- Ubuntu 22.04
- PyTorch = 1.12.1
- Python version = 3.8

**Additional context**

Add any other context about the problem here.
Example: CPU model
Fixes bugs with loss averaging with `validation_drop_last=False` for Megatron GPT
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)