renwei2024/nemo-github-issues huggingface dataset:
conv_pointwise_type parameter 
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Adding the conv_pointwise_type parameter that to ConformerEncoder enabling the use of nn.Linear() for a 10% acceleration 

**Collection**: ASR

# Changelog 
Adding the conv_pointwise_type parameter to 
- nemo/collections/asr/modules/conformer_encoder.py : adding the conv_pointwise_type parameter and propagating it to ConformerLayer
- nemo/collections/asr/parts/submodules/conformer_modules.py: adding conv_pointwise_type parameter to ConformerLayer and propagating it to ConformerConvolution.
- in ConformerConvolution, if conv_pointwise_type is 'linear', use nn.Linear() instead of nn.Conv1d()

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
+model.encoder.conv_pointwise_type='linear'
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Switch to NVIDIA Megatron repo
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

name: CI

# What does this PR do ?

Move core lib to NVIDIA/Megatron-LM from custom fork

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Reduce workers on NMT CI
State: closed | Pull request: yes
Labels: name: cherry-pick

name: CI

# What does this PR do ?

Reduce data workers on NMT CI for speedup

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
FC models in menu
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: ASR

# Changelog 
- Adds FC models to the catalog

# Usage
When loading pre-trained models, we can now specify:
stt_en_fastconformer_transducer_large
stt_en_fastconformer_ctc_large
stt_en_fastconformer_transducer_large_ls
stt_en_fastconformer_ctc_large_ls


```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [X ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Reduce workers on NMT CI
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Reduce data workers on NMT CI for speedup

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
how to finetune fastpitch_finetune_adapters.py ?
State: closed | Pull request: no
Labels: name: stale

I try the same config with fastpitch_finetune in tutorial,but it doesn't work
the code I run with:
![image](https://user-images.githubusercontent.com/56793829/233610378-0f84bbb2-09bf-41c1-a9c2-7da65a236d3e.png)

![image](https://user-images.githubusercontent.com/56793829/233610207-614f8b57-4476-4adc-aaf5-225700bb9ac4.png)
Manual garbage collection with an interval
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Set garbage collection interval manually. This helps align garbage collection execution across GPUs, which mitigates CPU overhead dramatically.
Using an integer value for `gc_interval` set the garbage collection interval in global-batch steps.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Try removing spawn from scripts
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Commeting out `mp.set_start_method("spawn", force=True)` from scripts and just using persistent_workers

**Collection**: NLP
# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Added/updated new Conformer configs
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Added some new configurations and updated some of the older configuration files for Conformer models.

# Changelog 
- Added configs for cache-aware fastconformer and hybrid ASR.
- Added more info on batch size and precision suggestions in the configs.

**PR Type**:
- [X] New Feature
- [ ] Bugfix
- [ ] Documentation
Fix cache aware hybrid bugs
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fixes some bugs to support Hybrid and FastConformer variants of cache-aware streaming models.

# Changelog 
- Fixed a bug which prevented switching between decoders for Hybrid models
- Added support of Hybrid models to the cache-aware streaming inference scripts
- Added the feature to switch between decoders to the cache-aware streaming simulation script
- Added support of causal downsamling for dw_striding approach.

# Usage
Users may use the --set_decoder "rnnt" or "ctc" to change the decoder with speech_to_text_cache_aware_streaming_infer.py.
They may use asr_model.change_decoding_strategy(decoder_type='ctc' or 'rnnt') to change the decoding for a Hybrid model in general.

**PR Type**:
- [X] New Feature
- [X] Bugfix
- [ ] Documentation
Switch to NVIDIA Megatron repo
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

Move core lib to NVIDIA/Megatron-LM from custom fork

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add Frame-VAD to ASR+VAD pipeline
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

This is the third PR for Frame-VAD. Please merge the previous two before this: https://github.com/NVIDIA/NeMo/pull/6441, https://github.com/NVIDIA/NeMo/pull/6463

This PR adds Frame-VAD to ASR+VAD pipeline, and also adds the drop-frame mode to ASR+VAD, which previously only supports masking mode.

**Collection**: [ASR]

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Add Frame-VAD examples and utils
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

The second PR of Frame-VAD. Please merge the first PR first:  https://github.com/NVIDIA/NeMo/pull/6441

**Collection**: [ASR]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Support in external sample mapping for Megatron datasets
State: closed | Pull request: yes
Labels: name: NLP

This PR adds support in external sample mapping (to be able to compute on the fly).

# What does this PR do ?

The added support does not add any new functionality, but allows providing the sample mapping externally to be able to avoid loading dataset samples.

**Collection**: Megatron `dataset_utils`.

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Meagtron encoder decoder fix for empty validation outputs
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

This PR fixes a bug where `MegatronLMEncoderDecoderModel.validation_epoch_end` might get empty outputs.

**Collection**: nlp

# Changelog 
- nemo/collections/nlp/models/language_modeling/megatron_lm_encoder_decoder_model.py:validation_epoch_end

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Meagtron encoder decoder fix for empty validation outputs
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

This PR fixes a bug where `MegatronLMEncoderDecoderModel.validation_epoch_end` might get empty outputs.

**Collection**: nlp

# Changelog 
- nemo/collections/nlp/models/language_modeling/megatron_lm_encoder_decoder_model.py:validation_epoch_end

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Enable ONNX export of 5B GPT trained with TE FP8 modules
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Adds export functionality for GPT model trained with FP8 enabled

**Collection**: [NLP]

# Changelog 
- Add export script
- Add export config
- Update export_utils to enable support for this model
- Bugfix: Typecast precision check for 16 and 32

# Usage

```bash
python3 examples/nlp/language_modeling/megatron_gpt_export.py gpt_model_file=5b_fp8_tp1.nemo onnx_model_file=5b_fp8_tp1.onnx
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [X] Did you add or update any necessary documentation?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [X] Documentation
Fix an invalid link in get_data.py of ljspeech
State: closed | Pull request: yes

# What does this PR do ?

It fixes an invalid link in get_data.py of ljspeech. Usage of the link in line 63 leads to downloading a html file not a tsv file, so we need to change it to a raw link.

**Collection**: [TTS]

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Fix an invalid link in get_data.py of ljspeech
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

It fixes an invalid link in get_data.py of ljspeech. Usage of the link in line 63 leads to downloading a html file not a tsv file, so we need to change it to a raw link.

**Collection**: [TTS]

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
BERT pre-training mp fork to spawn
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Uses spawn for mp for BERT pre-training

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix to set model in the eval mode during validation with virtual pipelining
State: closed | Pull request: yes
Labels: name: cherry-pick

name: stale

name: NLP

…lining

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Cache and reuse inputs when no reset
State: closed | Pull request: yes
Labels: name: cherry-pick

name: stale

name: NLP

# What does this PR do ?

Cache inputs (attention mask, loss mask, and position ids) and reuse.
This mitigates CPU-side overhead.

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Support Swiglu in TP PP Conversion
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Add support for swiglu in weight conversion script

**Collection**: [NLP]

# Changelog 
- Update TP PP weight conversion logic with swiglu config

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Fix replace_bos_with_pad not found
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

minor fix for replace_bos_with_pad not found error in CI

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix to set model in the eval mode during validation with virtual pipelining
State: closed | Pull request: yes
Labels: name: NLP

…lining

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix scheduler params registration
State: closed | Pull request: yes
Labels: name: core

name: cherry-pick

# What does this PR do ?

Fixes an issue with scheduler param registration.
The only way to override the scheduler args is via Hydra / OmegaConf at the moment, all defaults are picked directly from the constructor of the scheduler instead of the defaults of the dataclass. 

In effect, it sidesteps the validation and defaults of the dataclass of the scheduler entirely. There is no significant effect as it can still be fully overridden from the yaml config.

**Collection**: [Core]

# Changelog 
- Fix issue with scheduler dataclass registration
- Fix issue with scheduler dataclass validation with config overrides

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Fix scheduler params registration
State: closed | Pull request: yes
Labels: name: core

# What does this PR do ?

Fixes an issue with scheduler param registration.
The only way to override the scheduler args is via Hydra / OmegaConf at the moment, all defaults are picked directly from the constructor of the scheduler instead of the defaults of the dataclass. 

In effect, it sidesteps the validation and defaults of the dataclass of the scheduler entirely. There is no significant effect as it can still be fully overridden from the yaml config.

**Collection**: [Core]

# Changelog 
- Fix issue with scheduler dataclass registration
- Fix issue with scheduler dataclass validation with config overrides

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
New noise_norm perturbation based on Riva work
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

This adds a new normalisation-based noise perturbation to Nemo ASR based on experiments done by Riva

**Collection**: ASR

# Changelog 
- nemo/collections/asr/data/audio_to_text.py
Added new logic to check for infinite datasets in RandomizedChainDataset

- nemo/collections/asr/data/audio_to_text_dataset.py
Amended the process_augmentations API to pass global_rank and world_size

- nemo/collections/asr/parts/preprocessing/perturb.py
Added new NoiseNormPerturbation class
Fixed up the AugmentationDataset to make it an Iterable instead of Iterator and bring it into line with all other IterableDataset classes in Nemo
Added passing of global_rank and world_size to main perturbation constructor for those perturbations who might need this info (the new one uses it)

- nemo/utils/model_utils.py
Added a new bypass mechanism for multi-validation dataloaders which allows you to flatten all validation datasets into one single one, tested and working for both manifest.jsons and tarred audio sets. I know this one will be controversional, but it's something we very much need on Riva.

# Usage
* You can use the new perturbation class as follows:

```yaml
++model.train_ds.augmentor.noise_norm.prob=0.3
++model.train_ds.augmentor.noise_norm.snr_samples=[5,10,15]
++model.train_ds.augmentor.noise_norm.norm_to_db=-25.0
++model.train_ds.augmentor.noise_norm.rng=123
++model.train_ds.augmentor.noise_norm.manifest_path=['/path/to/tars1/tarred_audio_manifest.json','/path/to/tars2/tarred_audio_manifest.json']
++model.train_ds.augmentor.noise_norm.audio_tar_filepaths=['/path/to/tars1/audio__OP_0..255_CL_.tar','/path/to/tars2/audio__OP_0..127_CL_.tar']
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [X] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
Tensor-parallel communication overlap with userbuffer backend
State: closed | Pull request: yes
Labels: name: core

name: TTS

name: cherry-pick

name: ASR

name: NLP

name: CI

# What does this PR do ?

Add (1) interfaces to TE and initialized (2) process group setting to support tensor-parallel communication overlap with userbuffer backend.

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
Set `ub_tp_comm_overlap` to `True`

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix replace_bos_with_pad not found
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

minor fix for replace_bos_with_pad not found error in CI

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
BERT pre-training mp fork to spawn
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Uses spawn for mp for BERT pre-training

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add Frame-VAD model and datasets
State: closed | Pull request: yes
Labels: name: ASR

name: common

# What does this PR do ?

Add model and dataset files for frame-VAD, as well as related tests. 

Future PRs:
- Add Frame-VAD examples: train/infer, configs
- Update ASR+VAD pipeline to support Frame-VAD
- Add tutorials and update docs

**Collection**: [ASR]

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Add frame vad
State: closed | Pull request: yes
Labels: name: stale

name: ASR

name: common

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
error in making tokenizer
State: closed | Pull request: no
Labels: name: bug

Hi, I wanna fine-tune the multilingual model. For this purpose, I downloaded a pre-trained english model.
Now according to the tutorials in your repository, first of all we should train the tokenizer. I want to train spe bpe.
I made manifest files like the manifest exists in tutorial and run the code like you said.
But when I run the code, I encounter the below error:

'''
trainer_interface.cc(350) LOG(INFO) SentenceIterator is not specified. Using MultiFileSentenceIterator.
trainer_interface.cc(181) LOG(INFO) Loading corpus: /home/multilingual/dataset/deepmine_voices/text_corpus/document.txt
trainer_interface.cc(406) LOG(INFO) Loaded all 0 sentences
trainer_interface.cc(422) LOG(INFO) Adding meta_piece: <unk>
trainer_interface.cc(427) LOG(INFO) Normalizing sentences...
Traceback (most recent call last):
  File "NeMo/scripts/tokenizers/process_asr_text_tokenizer.py", line 323, in <module>
    main()
  File "NeMo/scripts/tokenizers/process_asr_text_tokenizer.py", line 301, in main
    tokenizer_path = __process_data(
  File "NeMo/scripts/tokenizers/process_asr_text_tokenizer.py", line 246, in __process_data
    tokenizer_path, vocab_path = create_spt_model(
  File "/usr/local/lib/python3.8/dist-packages/nemo/collections/common/tokenizers/sentencepiece_tokenizer.py", line 372, in create_spt_model
    sentencepiece.SentencePieceTrainer.Train(cmd)
  File "/usr/local/lib/python3.8/dist-packages/sentencepiece/__init__.py", line 989, in Train
    SentencePieceTrainer._Train(arg=arg, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/sentencepiece/__init__.py", line 945, in _Train
    return SentencePieceTrainer._TrainFromString(arg)
  File "/usr/local/lib/python3.8/dist-packages/sentencepiece/__init__.py", line 923, in _TrainFromString
    return _sentencepiece.SentencePieceTrainer__TrainFromString(arg)
RuntimeError: Internal: src/trainer_interface.cc(428) [!sentences_.empty()]
'''

I want to fine-tune it on a persian dataset.
Could you please help me to solve this problem?
add config and char tokenizer for russian tts
State: closed | Pull request: yes
Labels: name: common

# What does this PR do ?

Add config and char tokenizer for russian tts. It contains russian alphabet + ipa alphabet. Each unicode character is regarded as separate token, including stress.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Support Swiglu in TP PP Conversion
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add support for swiglu in weight conversion script

**Collection**: [NLP]

# Changelog 
- Update TP PP weight conversion logic with swiglu config

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Update NeMo_TTS_Primer.ipynb
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Fix a mistake in line 782. FFT calculates the magnitude and phase of each frequency bin not frequency band. Moreover, frequency bins in FFT are not related to pitch. So, we should change _**band (ie. pitch)**_ to _**bin**_. 

**Collection**: [TTS]

# Changelog 
tutorials/tts/NeMo_TTS_Primer.ipynb
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

## Who can review?
@blisc & @okuchaiev
[TTS] Fix aligner nan loss in fp32
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Fix FastPitch training appear `nan` loss in aligner loss when precision=32.  

**Collection**: [TTS]

# Changelog 
- `nemo/collections/tts/losses/aligner_loss`

# Usage
* To reproduce the bug, we can run `FastPitch and Mixer-TTS Training` tutorial by adding `trainer.precision=32` in training command. 

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
How to reproduce the same result as in the paper, EER=0.66?
State: closed | Pull request: no
Labels: name: bug

i get the eer=0.80, this is my code

`#!/bin/sh
CUDA_VISIBLE_DEVICES=3 python extract_speaker_embeddings.py --manifest=voxceleb1_test_manifest.json --model_path='./speakerverification_en_titanet_large/model' --embedding_dir='./speakerverification_en_titanet_large/embedding'

CUDA_VISIBLE_DEVICES=3 python voxceleb_eval.py --trial_file='./speech_database/voxceleb/vox_trial_pairs/veri_test.txt' --emb='./speakerverification_en_titanet_large/embedding/embeddings/voxceleb1_test_manifest_embeddings.pkl' `

and i found the model is:
Found existing object .cache/torch/NeMo/NeMo_1.17.0/titanet-l/11ba0924fdf87c049e339adbf6899d48/titanet-l.nemo.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 37720/37720 [00:00<00:00, 110401.53it/s]
0.80 
So I want to know what went wrong? thanks very much!!