renwei2024/nemo-github-issues huggingface dataset:
enabling diverse datasets in val / test
State: closed | Pull request: yes

# What does this PR do ?

Allows to specify multiple datasets with different configs  in validation and test 

ALL

# Changelog 
When processing val / test dataset configs, if a dict is detected instead of a list, overload the entire dataset config, not just the manifest path.

# Usage
Specify multiple, different dataset configs as shown below.

```yaml
  validation_ds:
    ds_item:
    - name: en
      manifest_filepath:
      - ${d}/en/val_test/fisher/audio_manifest_dev_clean_en.json
      - ${d}/en/val_test/europarl/audio_manifest_dev_clean_en.json
      sample_rate: ${model.sample_rate}
      batch_size: 16
      shuffle: false
      num_workers: 8
      pin_memory: true
      max_duration: 20.0
      min_duration: 0.1
      use_start_end_token: false
    - name: es
      manifest_filepath:
      - ${d}/es/nemo_sp_asr_set_3pt0/dev/fisher/dev_fisher_manifest_es.json
      - ${d}/es/nemo_sp_asr_set_3pt0/dev/mcv12/dev_mcv12_manifest_es.json
      - ${d}/es/nemo_sp_asr_set_3pt0/dev/mls/dev_mls_manifest_es.json
      - ${d}/es/nemo_sp_asr_set_3pt0/dev/voxpopuli/dev_voxpopuli_manifest_es.json
      sample_rate: ${model.sample_rate}
      batch_size: 32
      shuffle: false
      num_workers: 8
      pin_memory: true
      max_duration: 10.0
      min_duration: 1.0
      use_start_end_token: false

```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix for CTC decoder setup
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

The PR adds support of recent `distutils` for CTC decoder installation.

**Collection**: ASR (optional CTC beam search decoder)

# Changelog 
- Fixed CTC decoder installation to support recent distutils

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [X] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
added RPE + fixed RMSNorm
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

- Added RPE for MegatronNMT models
- Fixed an issue with RMSNorm

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix for CTC decoder setup
State: closed | Pull request: yes

# What does this PR do ?

The PR adds support of recent `distutils` for CTC decoder installation.

**Collection**: ASR (optional CTC beam search decoder)

# Changelog 
- Fixed CTC decoder installation to support recent distutils

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [X] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Export Fixes for Megatron Models
State: closed | Pull request: yes
Labels: name: core

name: TTS

name: ASR

name: NLP

name: Speaker Tasks

name: CI

name: common

# What does this PR do ?

- Adds RPE support for NMT export
- Fixed RMSNorm export in Megatron

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Adds RPE support for NMT export
- Fixed RMSNorm export in Megatron

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[BugFix] Fix the wrong branch name in speaker diarization inference notebook
State: closed | Pull request: yes
Labels: name: Speaker Tasks

# What does this PR do ?

Fix the wrong branch name in speaker diarization inference notebook

**Collection**: [Note which collection this PR will affect]


# Changelog 
It should be BRANCH="r1.17.0" but it was BRANCH="main" in r1.17.0

# Usage
Run Speaker_Diarization_Inference.ipynb


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in ASR
MixedFusedRMSNorm Export Fix
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

Replaces MixedFusedRMSNorm with PyT equivalent during export process

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Nemo Toolkit install fail
State: closed | Pull request: no
Labels: name: bug

name: stale

Hello, 
I run command: 
`pip install "nemo_toolkit[all]"
`
And I get more errors, 

the first one is: 

`
->nemo_toolkit[all]) (0.9.1)
Building wheels for collected packages: fasttext
  Building wheel for fasttext (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [61 lines of output]
      /home/grega/miniconda3/envs/nemo/lib/python3.8/site-packages/setuptools/dist.py:720: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead
`

the second one is: 

` 
    RuntimeError: Unsupported compiler -- at least C++11 support is needed!
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for fasttext
  Running setup.py clean for fasttext
Failed to build fasttext
Installing collected packages: fasttext, black, beautifulsoup4, anyio, aiosignal, texterrors, starlette, sphinx, numba, mdit-py-plugins, matplotlib, markdown, hydra-core, httpcore, GitPython, Flask, botocore, aiohttp, wandb, transformers, torchmetrics, s3transfer, pybtex-docutils, pyannote.database, paramiko, librosa, kornia, ipython, httpx, google-auth-oauthlib, gdown, g2p-en, flask-restful, fastapi, tensorboard, sphinxcontrib-bibtex, sentence-transformers, pytorch-lightning, pyannote.metrics, ipywidgets, gradio, boto3, nemo_toolkit
  Running setup.py install for fasttext ... error
  error: subprocess-exited-with-error
  
  × Running setup.py install for fasttext did not run successfully.
  │ exit code: 1
  ╰─> [64 lines of output]
`

the last one is: 

```
      RuntimeError: Unsupported compiler -- at least C++11 support is needed!
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> fasttext

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

```

Please help me. 

Thank you.
Bugfix for BF16 grad reductions with distopt
State: closed | Pull request: yes
Labels: name: bug

name: core

name: NLP

# What does this PR do ?

https://github.com/NVIDIA/NeMo/pull/5920 adds support for BF16 grad reductions with distopt, with embedding grad reductions done in FP32. @mikolajblaz found some bugs and it turns out the FP32 reductions were not being done at all. This PR fixes those issues. When I run GPT-3 175B, I confirm the embedding grads are now optimized with the FP32 optimizer. Loss values are the same with FP32 and BF16 grad reductions, up to 50 steps and within numerical accuracy.

This turned out messier than I would have liked. It would have been better to integrate distopt support for multiple grad dtypes into Apex.

**Collection**: NLP

# Changelog 
- Debug distopt support for BF16 grad reductions

# Usage

Set the optimizer to `distributed_fused_adam` in the config file:

https://github.com/NVIDIA/NeMo/blob/aaa0ccacf5b4d204ed18f6052e55ff6f20575996/examples/nlp/language_modeling/conf/megatron_gpt_config.yaml#L207

Configure the optimizer with `grad_sync_dtype: bf16`.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Fixes bugs from https://github.com/NVIDIA/NeMo/pull/5920
* Depends on https://github.com/NVIDIA/apex/pull/1575
* Depends on https://github.com/NVIDIA/TransformerEngine/pull/75
MixedFusedRMSNorm Export Fix
State: closed | Pull request: yes

# What does this PR do ?

Replaces MixedFusedRMSNorm with PyT equivalent during export process

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
What is "text_no_preprocessing" used for in manifest.json exactly?
State: closed | Pull request: no
Labels: name: stale

Hello. I tried using extract_sup_data.py script, where my manifest.json file had only these 4 keys: "audio_filepath", "text", "text_no_preprocessing" and "duration". The script didn't work, but when I removed "text_no_preprocessing" key it worked. I guess it would have initially worked if I had additional "text_normalized" key. So my questions are:

- What is the purpose of "text_no_preprocessing" key? 
- How the model recognizes punctuation in input sentence when manifest.json only has "text" keys?

Thank you!
updated ipa_cmudict files with the latest for tts
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?
updated ipa-cmudict files for TTS
Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[ASR]:fixed augmentor arguments for transcribe functionality of Hybrid CTC-RNNT model
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Fixes issues in nemo/collections/asr/models/hybrid_rnnt_ctc_models.py regarding Augmentor parameters to enable transcribe functionality for hybrid RNNT-CTC models

Collection: [ASR]


# Changelog 
- in the function transcribe() of class EncDecHybridRNNTCTCModel, added 
    * augmentor argument in function call (line 103)
    * augmentor argument in the config for calling _setup_transcribe_dataloader() (line 179)

# Usage
* You can potentially add a usage example below

```python
python  NeMo/examples/asr/transcribe_speech.py \
    model_path="path to hybrid model" \
    pretrained_name="name of hybrid model from NGC directory" \
    audio_dir="path to folder of audio files" \
    dataset_manifest="path to manifest" \
    output_filename="output filename" \
    batch_size=32 \
    compute_timestamps=False \
    compute_langs=False \
    cuda=0 \
    amp=True \
    append_pred=False \
    pred_name_postfix="another model name for output filename, if required"
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fixed regression on feature training after onnx support
State: closed | Pull request: yes
Labels: name: TTS

name: stale

# What does this PR do ?
Resolves a bug with feature training for radtts 


**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[ASR]:fixed augmentor arguments for transcribe functionality of Hybrid CTC-RNNT model
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fixes issues in nemo/collections/asr/models/hybrid_rnnt_ctc_models.py regarding Augmentor parameters to enable transcribe functionality for hybrid RNNT-CTC models

Collection: [ASR]


# Changelog 
- in the function transcribe() of class EncDecHybridRNNTCTCModel, added 
    * augmentor argument in function call (line 103)
    * augmentor argument in the config for calling _setup_transcribe_dataloader() (line 179)

# Usage
* You can potentially add a usage example below

```python
python  NeMo/examples/asr/transcribe_speech.py \
    model_path="path to hybrid model" \
    pretrained_name="name of hybrid model from NGC directory" \
    audio_dir="path to folder of audio files" \
    dataset_manifest="path to manifest" \
    output_filename="output filename" \
    batch_size=32 \
    compute_timestamps=False \
    compute_langs=False \
    cuda=0 \
    amp=True \
    append_pred=False \
    pred_name_postfix="another model name for output filename, if required"
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add experimental MPS device support for ASR inference
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Add support for MPS device (Apple Silicon M-series GPU) for ASR inference

**Collection**: [ASR]

# Changelog 
- allow to select MPS device in `transcribe_speech.py`

# Usage

MPS device will be automatically selected in `transcribe_speech.py` or `speech_to_text_eval.py` if `allow_mps=true` flag is explicitly set, and MPS device is available.
Will work only with PyTorch >= 2.0, for now env variable `PYTORCH_ENABLE_MPS_FALLBACK=1` should be set.

Tested with models:
- https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/stt_en_conformer_ctc_large
- https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/stt_en_conformer_transducer_large

```shell
PYTORCH_ENABLE_MPS_FALLBACK=1 python examples/asr/transcribe_speech.py \
    allow_mps=true \
    <other script params>

PYTORCH_ENABLE_MPS_FALLBACK=1 python examples/asr/speech_to_text_eval.py \
    allow_mps=true \
    <other script params>
```


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
VITS HiFiTTS doc
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Update docs and model for HiFiTTS version

**Collection**: [TTS]

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
option to delay dataset .bin numpy buffer mmap creation in megatron
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Adding Transcribe functionality for hybrid_rnnt_ctc_models
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fixes issues in nemo/collections/asr/models/hybrid_rnnt_ctc_models.py regarding Augmentor parameters to enable transcribe functionality for hybrid RNNT-CTC models


**Collection**: [ASR]

# Changelog 
- in the function transcribe() of class EncDecHybridRNNTCTCModel, added 
    * augmentor argument in function call (line 103)
    * augmentor argument in the config for calling _setup_transcribe_dataloader() (line 179)

# Usage
* You can potentially add a usage example below

```python
python  NeMo/examples/asr/transcribe_speech.py \
    model_path="path to hybrid model" \
    pretrained_name="name of hybrid model from NGC directory" \
    audio_dir="path to folder of audio files" \
    dataset_manifest="path to manifest" \
    output_filename="output filename" \
    batch_size=32 \
    compute_timestamps=False \
    compute_langs=False \
    cuda=0 \
    amp=True \
    append_pred=False \
    pred_name_postfix="another model name for output filename, if required"
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
add template for taskname=taskname
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Fix TTS audio preprocessing script
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Fix some bugs and improve usability of TTS preprocessing script.

**Collection**: [TTS]

# Changelog
- Replace hydra runner based script with regular python script that uses OmegaConf to parse .yaml config. The hydra script used AudioPreprocessingConfig as input, but in some situations this might not work because classes in the "scripts" directory are typically not installed or importable.
- Update the audio trimmer class paths because they were moved to "tts.parts.preprocessing", and moved them from "scripts" to "examples".
- Support relative paths in manifest.
- Ignore min_duration and max_duration filters when they are not provided.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
add template for taskname=taskname
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
LinAlg Error When Running Clustering Diarizer
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

Diarization continues until clustering, upon which I get this error.
"linalg.eigh: The algorithm failed to converge because the input matrix is ill-conditioned or has too many repeated eigenvalues (error code: 529)"

**Steps/Code to reproduce bug**

I'm loosely following the notebook at https://github.com/NVIDIA/NeMo/blob/main/tutorials/speaker_tasks/Speaker_Diarization_Inference.ipynb except i turned oracle vad off.

Here's the diarization part of my notebook
https://colab.research.google.com/drive/1zN9LZ5-hPEI15K1NWl2obvEapJbTWd9P?usp=sharing
Add fix to resume pre-training when val outputs is empty
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

Resuming pre-training sometimes fails immediately complaining of empty validation outputs tensor. This PR provides a fix for this issue.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
sampler padding fix for per-micro-batch input loader
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix for T5 finetuning when starting with pad instead of bos
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Fixes a bug that makes T5 decode use bos_id even when asked to replace with pad.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix args share token embeddings in Float16Module
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

This PR fixes the args passing problems with Float16Module

**Collection**: [Note which collection this PR will affect]
nlp

# Changelog 
Changing how Float16Module uses share_token_embeddings, if it detects it in the module it should use that and not overwrite it

# Problem fixed
Float16Module ignores the share_token_embeddings flag set by the module because it calls [init](https://github.com/NVIDIA/NeMo/blob/14effd79731ec76a047aefcb3ac4d15c0a7330dd/nemo/collections/nlp/modules/common/megatron/module.py#L262) with no arguments which causes the flag to default to true because MegatronModule sets the default to [True](https://github.com/NVIDIA/NeMo/blob/14effd79731ec76a047aefcb3ac4d15c0a7330dd/nemo/collections/nlp/modules/common/megatron/module.py#L46) which subsequently causes checks like [this](https://github.com/NVIDIA/NeMo/blob/14effd79731ec76a047aefcb3ac4d15c0a7330dd/nemo/collections/nlp/models/language_modeling/megatron_gpt_model.py#L255) to pass despite the user specifying False in the module. Instead Float16Module should inherit this flag from the underlying module whenever possible. 
  
**PR Type**:
- [x] Bugfix
updating Conformer CTC and RNNT BPE config files docstring 
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?
updating Conformer CTC and RNNT BPE config files docstring to include xlarge model details.

**Collection**: ASR

# Changelog 
- updating docstring for conformer BPE config files with xlarge models.


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
GPT P tuning Eval changes (#5952)
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

* GPT P tuning Eval changes

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
restore path for p-tuning
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Pr allows you to pretrain a prompt-encoder.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
GPT P tuning Eval changes (#5952)
State: closed | Pull request: yes
Labels: name: NLP

* GPT P tuning Eval changes

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
How to use speaker diarization without writing input to manifest and output to .rttm file ?
State: closed | Pull request: no

Thanks for your great repository. I have a question about speaker diarization. 
I do the inference steps according to this tutorial: https://github.com/NVIDIA/NeMo/blob/main/tutorials/speaker_tasks/Speaker_Diarization_Inference.ipynb

According to this tutorial, audio will be written to manifest file, output will be saved in .rttm format in out_dir ( manifest file and out_dir are defined in _**diar_infer_telephonic.yaml**_  . Is there a direct way to pass the input as audio metadata (path, duration, ...) and give the results without writing the manifest and  the .rttm files?
Avoid unnecessarily accessing data loader with pipeline parallelism
State: closed | Pull request: yes
Labels: name: bug

name: cherry-pick

name: NLP

# What does this PR do ?

I've been experiencing a hang in the first validation step when running GPT with interleaved pipeline parallelism.  This first shows up in https://github.com/NVIDIA/NeMo/pull/6049, so it seems that accessing the data loader in middle pipeline stages is the root cause.

**Collection**: NLP

# Changelog 
- Avoid unnecessarily accessing data loader with pipeline parallelism

# Usage
No change in usage

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to https://github.com/NVIDIA/NeMo/pull/6049