renwei2024/nemo-github-issues huggingface dataset:
Voice_Activity_Detection.ipynb tutorial is broken
State: closed | Pull request: no
Labels: name: bug

**Voice_Activity_Detection.ipynb tutorial is broken**

If you open file: 
https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/Voice_Activity_Detection.ipynb

it will give an error:
**"Invalid Notebook"**
**"The Notebook Does Not Appear to Be Valid JSON"**
Adding PR#5663 to r1.15.0
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Support NeMo Forced Aligner for several hours of audio
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Add CPU offloading and memory tricks to let NFA work for up to 2 hours of audio via offloading

**Collection**: [Tools, ASR]

# Changelog 
- Add granular CPU offloading of tensors 
- Replace all repeat with expand to prevent unnecessarily allocating memory
- Add support for local attention

# Usage

```python

python align.py \
    model_path=<Citrinet model path> \
    model_downsample_factor=8 \
    manifest_filepath=<Manifest filepath> \
    transcribe_device='cuda' \
    viterbi_device='cpu' \
    batch_size=1 \
    transcribe_dtype="fp16" \  # bf16 if supported
    transcribe_amp=true \
    viterbi_dtype="fp16" \  # CPU offload supports fp16 everywhere
    cpu_offload_transcript_probs=True \
    cpu_offload_viterbi_probs=True \
    cpu_offload_viterbi_pointers=True \
    model_change.conformer.self_attention_model="rel_pos_local_attn" \  # Support Conformer (medium length audio) with local attention
    model_change.conformer.att_context_size=[64,64]

```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Fix memory allocation of NeMo Multi-speaker Data Simulator
State: closed | Pull request: yes
Labels: name: ASR

name: Speaker Tasks

Signed-off-by: stevehuang52 <heh@nvidia.com>

# What does this PR do ?

Fix memory leak and improve speed of the simulator.


**Collection**: [ASR]

# Changelog 
- Fix memory leak by cleaning up buffers and variables after each session is created
- Improve speed by generating candidate speakers and their audios before launching multiprocess
- Enforce spawn as multiprocessing method
- Improve some data loading logic to avoid unnecessarily traversing the same list many times.
- Adding multi-gpu (CUDA devices) support 
- Reduced parallel processing memory usage by sampling noise source samples in advance


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Megatron positional encoding alibi fix
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

This PR fixes a bug in ALiBi implementation.

ALiBi paper: https://arxiv.org/abs/2108.12409

Results shows that ALiBi is comparable in performance to RPE, and is faster by 7% from RPE and 8% slower than learnable abs embeddings during training. 
See plots below. ALiBi (this PR) in green, ALiBi (before fix) in pink, learned absolute embeddings in red, and RPE in cyan.


![Screen Shot 2023-01-25 at 17 32 00](https://user-images.githubusercontent.com/7453913/214607208-fcc652b2-4ae0-42b5-864c-d01eb62e7996.png)
![Screen Shot 2023-01-25 at 17 32 50](https://user-images.githubusercontent.com/7453913/214607215-c41b7bc8-33e6-4378-8356-f811f22e3c02.png)
![Screen Shot 2023-01-25 at 17 33 01](https://user-images.githubusercontent.com/7453913/214607219-6bd338e0-e088-4891-ac4e-edd1fa314d16.png)
![Screen Shot 2023-01-25 at 17 33 27](https://user-images.githubusercontent.com/7453913/214607227-1480cd98-c17e-453d-bf56-5ae6f1e3d053.png)
![Screen Shot 2023-01-25 at 17 36 03](https://user-images.githubusercontent.com/7453913/214607229-f395a390-ffe8-4fd8-b55e-090623fd0b34.png)
![Screen Shot 2023-01-25 at 17 36 17](https://user-images.githubusercontent.com/7453913/214607232-6a87f3c8-aee7-4f2a-81bc-f38167932ff5.png)



**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
indentation fix
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

# What does this PR do ?

JSON indentation fix

**Collection**: VAD

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
indentation fix
State: closed | Pull request: yes
Labels: name: ASR

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

# What does this PR do ?

JSON indentation fix

**Collection**: VAD

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
[TTS][DE] updated IPA dictionary and heteronyms
State: closed | Pull request: yes
Labels: name: TTS

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

updated with more entries for both dictionary and heteronyms.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
remove notebook
State: closed | Pull request: yes
Labels: name: NLP

Signed-off-by: ericharper <complex451@gmail.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Initialize MegatronBertModel
State: closed | Pull request: no

I would like to initialize a **MegatronBertModel** instead of **TextClassificationModel**, for a classification task. Basically I need to use MegatronBERT based model and tokenizer. I did not find any corresponding config file, how do I configure the YAML config file to achieve this?

Something like this:
```
config = OmegaConf.load(yaml_config)
config.model.language_model.pretrained_model_name = 'biomegatron345m_biovocab_50k_cased'
trainer = pl.Trainer(**config.trainer)
lmodel = MegatronBertModel(config.model, trainer)
```
Add segmentation export to Audacity label file
State: closed | Pull request: yes

# What does this PR do ?

This PR adds the export of the segmentation file to a label file that can directly be imported in Audacity to quickly and easily assess the quality of the segmentation.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* No usage change. The label file will be automatically generated.


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
NFA samples fix
State: closed | Pull request: yes

# What does this PR do ?

Correct NFA README typo and fix an NFA error that would give incorrect timestamps when the input audio had a different sampling rate to that of the ASR model.

**Collection**: tools/nemo_forced_aligner

# Changelog 
- Correct NFA README typo: 'samples' -> 'seconds'
- Fix timestamps bug: change variable 'audio_sr' -> 'model.cfg.sample_rate'

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ N/A ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ N/A ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ N/A ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Addressing question raised in https://github.com/NVIDIA/NeMo/discussions/5848.
Multi-task finetuning of NeMo-Megatron GPT models
State: closed | Pull request: yes
Labels: name: TTS

name: stale

name: NLP

name: common

# What does this PR do ?

Adds the ability to fine-tune NeMo-Megatron trained GPT models on multiple downstream datasets.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
update torchmetrics args confusionmatrix
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Fix issue with torchmetrics ConfusionMatrix function

**Collection**: ASR

# Changelog 
- add task arg

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
update torchmetrics args confusionmatrix
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fix issue with torchmetrics ConfusionMatrix function

**Collection**: ASR

# Changelog 
- add task arg

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
fix max len generation t5
State: closed | Pull request: yes

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update NeMo Multi-Run docs
State: closed | Pull request: yes
Labels: name: core

name: cherry-pick

Signed-off-by: smajumdar <titu1994@gmail.com>

# What does this PR do ?

Add documentation for NeMo multi-run support

**Collection**: [Core]

# Changelog 
- Add a fix for printing the error stack trace if there are multiple list of lists.
- Add documentation for NeMo Multi-Run + some tips and tricks

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [x] Documentation
Fix torchaudio installation
State: closed | Pull request: yes

Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>

# What does this PR do ?

Fixes Docker builds with `torchaudio` library:
- Fixes installation in `pytorch:22.04-py3`-based and other old containers (silently failed, which resulted in no `torchaudio` at all in the container)
- The build will fail if `torchaudio` is not installed at all (no CPU version). Warning if the GPU version is not compiled.

**NB**: we will still need to update this script after PyTorch 2.0 release.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix segmenting for pcla inference
State: closed | Pull request: yes
Labels: name: NLP

Signed-off-by: Matvei Novikov <mattyson.so@gmail.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Adding new features and speed up for multi-speaker data simulator
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

This PR adds a few features to the existing multispeaker data simulator

**Collection**: [Note which collection this PR will affect]
ASR

# Changelog 

- Added a feature to make 1 speaker sessions
- 50-100x speed up by saving loaded wav to dictionary (hashtable) ( 2000ms/session to 20-50ms/session)
- Parallel processing has been added for speed up (up to num_workers=12, it speeds up)
- A few minor bug fixes

# Usage


```python
python multispeaker_simulator.py --config-path='conf' --config-name='data_simulator.yaml' \
  data_simulator.random_seed=42 \
  data_simulator.manifest_filepath=./train-clean-100-align.json \
  data_simulator.outputs.output_dir=./test_rir \
  data_simulator.background_noise.add_bg=True \
  data_simulator.background_noise.background_manifest=./bg_noise.json
  data_simulator.rir_generation.use_rir=True
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in ASR

# Additional Information
* Related to # (issue)
Speed up and added feature to multi-speaker data simulator
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

This PR adds a few features to the existing multispeaker data simulator

**Collection**: [Note which collection this PR will affect]
ASR

# Changelog 

- Added a feature to make 1 speaker sessions
- 50~100x speed up by saving loaded wav to dictionary (hashtable)
- Parallel processing has been added for speed up (up to num_workers=12, it speeds up)
- A few minor bug fix

# Usage


```python
python multispeaker_simulator.py --config-path='conf' --config-name='data_simulator.yaml' \
  data_simulator.random_seed=42 \
  data_simulator.manifest_filepath=./train-clean-100-align.json \
  data_simulator.outputs.output_dir=./test_rir \
  data_simulator.background_noise.add_bg=True \
  data_simulator.background_noise.background_manifest=./bg_noise.json
  data_simulator.rir_generation.use_rir=True
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [] Did you write any new necessary tests?
- [] Did you add or update any necessary documentation?
- [] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in ASR

# Additional Information
* Related to # (issue)
Update NeMo Multi-Run docs
State: closed | Pull request: yes
Labels: name: core

Signed-off-by: smajumdar <titu1994@gmail.com>

# What does this PR do ?

Add documentation for NeMo multi-run support

**Collection**: [Core]

# Changelog 
- Add a fix for printing the error stack trace if there are multiple list of lists.
- Add documentation for NeMo Multi-Run + some tips and tricks

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [x] Documentation
make validation accuracy reporting optional for adapters/ptuning
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

accuracy reporting is ~10-20x slower for prompt learning and adapter learning for t5. This PR optionally disables it.

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[BugFix] Updated CTC decoders installation in tutorial
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Fixes `ctc-decoders` installation in `Offline_ASR` notebook.

**Collection**: ASR

# Changelog 
- Updated CTC decoders installation

# Usage

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [X] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)

Signed-off-by: Vitaly Lavrukhin <vlavrukhin@nvidia.com>
TPMLP for T5-based models
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
TPMLP for T5-based models
State: closed | Pull request: yes
Labels: name: NLP

Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Missing documentation for EncDecRNNTModel.transcribe
State: closed | Pull request: no

https://github.com/NVIDIA/NeMo/blob/0336b8dad4fa63880e3b15805829f0a9d8057363/nemo/collections/asr/models/rnnt_models.py#L238

There is no clear statement on return types of `EncDecRNNTModel.transcribe()` method. The text seems to be clipped (sentence is not completed)
Streaming conformer CTC export
State: closed | Pull request: yes
Labels: name: core

name: ASR

# What does this PR do ?

Work in progress:
- Export unified (encoder and decoder) onnx model for cache-aware conformer CTC models.
  - truncate encoded_output and caches where needed for easier deployment
- Use consistent chunk size etc so that first chunk is treated like other chunk whenever possible.  This simplifies deployment. 

**Collection**: ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
Export conformer-ctc with cache-aware streaming
`python scripts/export.py --max-batch 32 --check-tolerance 1.0 --runtime-check --streaming_support --cache_support <path/to/model.nemo> <outut.onnx>`

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add SSL import functionality for Audio Lexical PNC Models
State: closed | Pull request: yes
Labels: name: NLP

Signed-off-by: Daniel Egert <degert@nvidia.com>

# What does this PR do ?

This PR allows the audio lexical PNC model to also import pretrained SSL models for the audio encoder, and allows importing any audio encoder which is on disk, and not just on NGC

**Collection**: [Note which collection this PR will affect]
NLP

# Changelog 
- Changes punctuation_capitalization_lexical_audio_model.py to support `.restore_from` instead of just `.from_pretrained`
- Changes punctuation_capitalization_lexical_audio_model.py to support SSL models for the encoder by correct removing the SSL decoder_losses
- Changes punctuation_capitalization_model.py to support inference with a list of bytes instead of just list of strings for audio filepaths on disk. This allows using a bytestream which is necessary for Riva sharded tar files
- Changes punctuation_capitalization_infer_dataset.py to support the same bytestream changes
- Changes punctuation_capitalization_model.py to remove superfluous debug print statements outside nemo logging which were accidentally left in

# Usage
You can now specify `model.audio_encoder.pretrained_model` to be a .nemo file on disk rather than just the name of a model in NGC. This is set in the config yaml of the audio lexical model.

```yaml
model:
  audio_encoder:
    pretrained_model: /path/to/my_ssl_model.nemo  # this now works
    freeze:
      is_enabled: false # If set to True weights of audio encoder will not be updated during training.
      d_model: 256 # Input dimension of MultiheadAttentionMechanism and PositionwiseFeedForward
      d_ff: 1024 # Hidden dimension of PositionwiseFeedForward
      num_layers: 4 # Number of additional Conformer layers
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [X] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[BugFix] Updated CTC decoders installation in tutorial
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fixes `ctc-decoders` installation in `Offline_ASR` notebook.

**Collection**: ASR

# Changelog 
- Updated CTC decoders installation

# Usage

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [X] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)

Signed-off-by: Vitaly Lavrukhin <vlavrukhin@nvidia.com>
Unnecessary rng shuffle in x-denoiser sample generation
State: closed | Pull request: no
Labels: name: stale

I recognize there's a lot of copy-pasta from the classic t5 denoising function, but this shuffle is unnecessary, or there's a bug

https://github.com/NVIDIA/NeMo/blob/b69f61d4c05ce7ee35d1474025998ca6e5c20d72/nemo/collections/nlp/data/language_modeling/megatron/dataset_utils.py#L497

no big deal, just thought I'd flag
remove transformer version upper bound
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

remove transformer version upper bound

**Collection**: nlp, asr, tts

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
validation batch sizing and drop_last controls
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?
This PR allows use to set the batch sizes for validation. This in turn allows the use to set `validation_drop_last=False` in the config. Allowing more control over the validation loss calculation and reporting.

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Radtts dur transformer
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.
It will add IPA config files for RADTTS and ADD transformer support for duration prediction. 
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.
it adds example/tts/config/rad_tts-feature_pred_ipa and example/tts/config/rad_tts-decoder_ipa

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Nemo ASR training: Unable to get/save best top n checkpoints, and exp_dir not setting
State: closed | Pull request: no
Labels: name: bug

name: stale

Hi All,

The expdir and save_top_k under exp_manager are not working as expected. 

1. "expdir" is getting created, but nothing is being written. I tried setting logger to False, even then nothing is being written to "expdir".

2. Always only the last checkpint is saved irrespective of "save_top_k" value. And, only one checkpoint is save, and it is not the best.
In addition, setting these values in config files. I am passing them during training as below:

expdir = '/data/manju/tamil/exp/logs/expDir'
params['exp_manager']['exp_dir'] = expdir
params['exp_manager']['save_top_k'] = 6

Pls suggest @titu1994 . Thanks
[ASR] Configurable metrics for audio-to-audio + removed experimental decorators
State: closed | Pull request: yes
Labels: name: ASR

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

# What does this PR do ?

This PR adds an option to configure metrics for `AudioToAudio` models.

**Collection**: ASR

# Changelog 
- Added `AudioMetricWrapper` to support batches with examples with variable input lengths
- Added unit tests
- Added example of using metrics in `multichannel_enhancement.yaml`
- Removed experimental decorators

# Usage
Metrics for `val` and `test` can be configured by adding `model.metrics`, for example

```
model:

  ...

  metrics:
    val:
      sdr: # output SDR
        _target_: torchmetrics.audio.SignalDistortionRatio
    test:
      sdr_ch0: # SDR on output channel 0
        _target_: torchmetrics.audio.SignalDistortionRatio
        channel: 0
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
Concat dataset and aistore support for label models
State: closed | Pull request: yes
Labels: name: ASR

name: common

# What does this PR do ?

Add support for concat dataset and aistore  for label models. Ideally, we should implement both of the features in a single place, such that all of our datasets can automatically support them. But that requires some additional refactoring, so just adding partial support to a few datasets for now.

**Collection**: ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)