renwei2024/nemo-github-issues huggingface dataset:
[TTS] fix bugs for chinese and german tutorials.
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

remove whitelist path.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Adding interCTC loss to hybrid models
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Adding interCTC loss to hybrid models. To solve issue that ctc and hybrid models use different names for things that we need (e.g., wer vs ctc_wer), I'm now directly passing all necessary "main" class variables to the interctc_setup method (except for self.cfg). I'm not sure if there is much sense in still keeping interctc as mixin, given that it does not access anything from the main class anymore, but not making any changes for now.

Additionally, there are 2 slight bugs in the original code. First, inter_ctc_loss was incorrectly calculated in the logs (main loss was used, not inter_ctc there). Note that this is purely a logging thing and does not affect training. Second, main loss was not multiplied by the correct coefficient, which does affect training to some extent. So, the calculation should have been something like `loss = final_loss * 0.7 + inter_loss * 0.3`, while it was just `loss = final_loss + inter_loss * 0.3`

**Collection**: ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Notebook fixes
State: closed | Pull request: yes
Labels: name: TTS

name: cherry-pick

name: ASR

name: NLP

name: Speaker Tasks

name: CI

# What does this PR do ?

Doc fixes and typos for tutorials

**Collection**: [ASR]

# Changelog 
- Fix incorrect line end of ASR adapters
- Fix typos


# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Notebook fixes
State: closed | Pull request: yes
Labels: name: TTS

name: ASR

name: NLP

name: Speaker Tasks

name: CI

# What does this PR do ?

Doc fixes and typos for tutorials

**Collection**: [ASR]

# Changelog 
- Fix incorrect line end of ASR adapters
- Fix typos


# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Add persistent workers to GPT
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

When using spawn, the dataloader must have persistent_workers=True, otherwise validation will be very slow (all workers will be reinitialized every validation)

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Megatron GPT model finetuning
State: closed | Pull request: yes
Labels: name: NLP

name: CI

name: common

# What does this PR do ?

Adds the ability to fine-tune Megatron GPT Models.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Cache aware streaming nfa
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?
Add cache aware streaming model in nemo force aligner

Should be easy for ctc segmentation tool to call and get the log probs as well.

Most of the codes are copied from speech_to_text_cache_aware_streaming_infer.py.

**Collection**: [Note which collection this PR will affect]
ASR

# Changelog 


# Usage
* You can potentially add a usage example below
```
nemo_model="nemo_models/cache_aware_ctc.nemo"
cd NeMo/tools/nemo_forced_aligner/
python3 align.py \
        batch_size=2 \
        model_path=$nemo_model \
        model_downsample_factor=4 \
        manifest_filepath=$manifest \
        output_dir=./streaming_ctm_with_pred \
        align_using_pred_text=true \
        simulate_cache_aware_streaming=true
```


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Try to add nvidia sans to custom.css
State: closed | Pull request: yes

# What does this PR do ?

This PR is trying to get our docs to use the NVIDIA Sans font

**Collection**: Core

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add persistent workers to GPT
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

When using spawn, the dataloader must have persistent_workers=True, otherwise validation will be very slow (all workers will be reinitialized every validation)

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] fix broken tutorial for Tacotron2
State: closed | Pull request: yes
Labels: name: TTS

name: cherry-pick

# What does this PR do ?

the tutorial was broken due to https://github.com/NVIDIA/NeMo/pull/5982.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Upper bound lightning till 2.0.0 support is fixed.
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Pin Pytorch Lightning >= 1.9 but < 2.0.0
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

A fresh install of NeMo will break when using Lightning 2.0.0 because NeMo calls `from pytorch_lightning.trainer.connectors.checkpoint_connector import CheckpointConnector` but `CheckpointConnector` has been moved to `_CheckpointConnector` in 2.0.0 (see [here](https://github.com/Lightning-AI/lightning/blob/ad1c994d96c420d8c2def96354e5f7bc551afb98/src/lightning/pytorch/trainer/connectors/checkpoint_connector.py#L45)).

**Steps/Code to reproduce bug**

A fresh install of NeMo which uses Lightning 2.0.0 by default will throw an import error. 

**Additional context**

Pinning to 1.9.3 would solve this until a refactor can be done.
IndexError: index 0 is out of bounds for dimension 1 with size 0
State: closed | Pull request: no
Labels: name: bug

name: stale

[/usr/local/lib/python3.9/dist-packages/nemo/collections/asr/parts/preprocessing/features.py](https://localhost:8080/#) in forward(self, x, seq_len, linear_spec)
    407         # do preemphasis
    408         if self.preemph is not None:
--> 409             x = torch.cat((x[:, 0].unsqueeze(1), x[:, 1:] - self.preemph * x[:, :-1]), dim=1)
    410 
    411         # disable autocast to get full range of stft values

IndexError: index 0 is out of bounds for dimension 1 with size 0

Getting this issue while executiong the chunks
Keep encountering the KeyError message while trainer.fit()
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

Hi, thanks for providing such a helpful toolkit. I'm currently following the tutorial from [Nvidia-examples](https://colab.research.google.com/github/NVIDIA/NeMo/blob/stable/tutorials/asr/ASR_with_NeMo.ipynb#scrollTo=3FT0klSV268p), and applying my own Mandarin Chinese corpus. Since our dataset is Mandarin Chinese, the only different setting is the 'labels' in the configuration files. Thus, I directly change the labels through the command below:
`params['model']['train_ds']['labels'] = [' ']+list(unique_char)+params['labels']`
`params['labels'] = [' ']+list(unique_char)+params['labels']`
where `unique_char` is a set that includes all the unique characters in our train_manifest.

p.s. I preprocess all the texts of each audio to a single character form as below:
`{audio_filepath: example.wav, duration: 15, text: '你 好 嗎'}`
Hence, aggregating all the text information from train_manifest to form `unique_char` should include all the characters.

However, I still met similar error messages while doing trainer.fit() as below:
```
  File "/opt/conda/anaconda3/envs/nemo/lib/python3.8/site-packages/nemo/collections/asr/metrics/wer.py", line 1074, in decode_ids_to_tokens
    token_list = [self.labels_map[c] for c in tokens if c != self.blank_id]
  File "/opt/conda/anaconda3/envs/nemo/lib/python3.8/site-packages/nemo/collections/asr/metrics/wer.py", line 1074, in <listcomp>
    token_list = [self.labels_map[c] for c in tokens if c != self.blank_id]
KeyError: 1669
```
BTW, it would run several iterations in Epoch 0 to pop out this error message.
Is there anyone facing similar questions that can help me fix the problem?
Appreciate your help and time, thank you!
[TTS] fix broken tutorial for Tacotron2
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

the tutorial was broken due to https://github.com/NVIDIA/NeMo/pull/5982.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue) https://github.com/NVIDIA/NeMo/issues/6050
[TTS] fixed code scanning alert 885
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

fixed code scanning alert: https://github.com/NVIDIA/NeMo/security/code-scanning/885

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue) https://github.com/NVIDIA/NeMo/issues/5936
[TTS] fixed code scanning alert 905
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

fixed code scanning alert https://github.com/NVIDIA/NeMo/security/code-scanning/905
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue) https://github.com/NVIDIA/NeMo/issues/5935
[TTS] fixed code scanning alert 836
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

fixed code scanning alert  https://github.com/NVIDIA/NeMo/security/code-scanning/836
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue) https://github.com/NVIDIA/NeMo/issues/5934
[TTS] fixed code scanning alert #774
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

fixed code scanning alert https://github.com/NVIDIA/NeMo/security/code-scanning/774
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue) https://github.com/NVIDIA/NeMo/issues/5932#issuecomment-1461156445
Bug fixing for bucketing dataset
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fixes a bug in the ASR bucketing which made the training to hang at the end of first epoch. 

**Collection**: ASR

# Changelog 
- Fixed the bucketing bug by making the bucketing dataloaders to loop over the samples indefinitely.

**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation
Allow disabling document shuffling for GPT
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add `model.data.shuffle_documents` flag which allows disabling document shuffling for GPT dataset

**Collection**: NLP

# Changelog 
-Add `model.data.shuffle_documents`  flag which allows disabling document shuffling for GPT dataset

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Virtual tokens applied at the leftmost positions.
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?
removes functionality to insert virtual tokens arbitrarily which makes code complex and has no performance benefit. This PR removes this ability and simple adds virtual tokens in the start.

Makes P-tuning for RETRO models easier.

Removes taskname_ids from dataset creation.

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update Tacotron2_Training.ipynb
State: closed | Pull request: yes
Labels: name: TTS

removed comma in the end of the list to prevent error paring this file as jupiter notebook.

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix scanning alert bugs
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Address codeQl alert: https://github.com/NVIDIA/NeMo/security/code-scanning/772



**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue): https://github.com/NVIDIA/NeMo/issues/5933
* Related to PR: https://github.com/NVIDIA/NeMo/pull/6177
[WIP] add buffered chunked streaming for nemo force aligner
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?
To support buffered chunked streaming inference for nemo neural force aligner.

**Collection**: [Note which collection this PR will affect]
ASR

# Changelog 
- add an argument (keep_logits default is False) for transcribe function in `FrameBatchASR` to keep the inference logits.
- return the logits(log probs) if keep_logits is set to True.

# Usage


```python
python align.py \
        pretrained_name="stt_en_citrinet_1024_gamma_0_25" \
        batch_size=32 \
        model_downsample_factor=8 \
        manifest_filepath=$manifest \
        use_buffered_chunked_streaming=true \
        output_dir=./buffered_streaming_test_pred \
        total_buffer_in_secs=10 \
        align_using_pred_text=true
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update ASR Data Simulator
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Add session and segment augmentation, add logging of params and metadata of each session, fix implicit bug in RIR perturbation.


**Collection**: [ASR]

# Changelog 
- Add audio augmentation on whole session and each speech segment
- Add logging generator params to output dir.
- Add logging session metadata
- Fix implicit bugs in RIR perturb, allowing individual control on foreground and background noise

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [x] Bugfix
- [ ] Documentation
Custom functions for distopt communication in interleaved pipeline parallelism
State: closed | Pull request: yes
Labels: name: feature

name: core

name: NLP

# What does this PR do ?

As discussed in https://github.com/NVIDIA/apex/pull/1611, we are experiencing excessive GPU idling when training GPT with the distributed optimizer and interleaved pipeline parallelism. That PR addresses the issue by making sure all distopt communication is aligned across the pipeline-parallel group, and this PR adds NeMo support to enable those perf optimizations.

**Collection**: NLP

# Changelog 
- Add functions for interleaved pipeline parallel schedule to trigger async distopt communication

# Usage
Set the optimizer to `distributed_fused_adam` in the config file:

https://github.com/NVIDIA/NeMo/blob/3d72e81327f9080a50a84e0a78589f8d5059e66e/examples/nlp/language_modeling/conf/megatron_gpt_config.yaml#L204

This optimization is only relevant when training GPT or BERT with virtual pipeline stages.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

Pinging @erhoo82.

# Additional Information
* Depends on https://github.com/NVIDIA/apex/pull/1611
Filter p-tuning by example length
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Previously only training and validation examples longer than the max encoder length were filtered out. This PR makes it possible to filter out by a specified length. This will save training time, as larger batch sizes will be possible after removing a small % of training and validation examples.

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Synthetic dataloading for Megatron T5 and GPT models
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

Enables testing functionality and performance of training code using a synthetic version of existing datasets.

**Collection**: NLP (language_modeling)

# Changelog 
- Added SyntheticGPTDataset which produces the same size and datatypes of output as GPTDataset
- Added SyntheticT5Dataset which produces the same size and datatypes of output as T5Dataset

# Usage

```bash
HYDRA_FULL_ERROR=1 python NeMo/examples/nlp/language_modeling/megatron_gpt_pretraining.py \
    model.data.data_impl=synthetic \
    model.data.data_prefix=[]

HYDRA_FULL_ERROR=1 python NeMo/examples/nlp/language_modeling/megatron_t5_pretraining.py \
    model.data.data_impl=synthetic \
    model.data.data_prefix=[]
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Spellchecking ASR customization model
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add spellchecking ASR customization model and scripts for data preparation and evaluation.

**Collection**: [Note which collection this PR will affect]
nlp

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
See examples/nlp/spellchecking_asr_customization/README.md

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
New feature

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fixed scanning alert bugs
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Fixes these issues
#5933 https://github.com/NVIDIA/NeMo/issues/5933 
#5937  https://github.com/NVIDIA/NeMo/issues/5937

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS][refactor] update tutorial import paths.
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

update import paths in TTS tutorials.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix missing import statement - gc
State: closed | Pull request: yes
Labels: name: ASR

Signed-off-by: limdongjin [geniuslim27@gmail.com](mailto:geniuslim27@gmail.com)

# What does this PR do ?

Simillar #6001 PR. 
I fix missing import statement in a non-CUDA environment for 'r1.16.0' branch.

**Collection**: ASR

# Changelog 
- Add import gc statement for non-CUDA environment

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[Tools] Update RIR generation scripts
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

This PR adds a few enhancements to RIR generation scripts.

**Collection**: Tools

# Changelog 
- Fix: reduce room size if evaluation of params fails
- Added: randomized mic placement
- Added: diffuse noise generation
- Added: an option to specify the format and subtype for saved audio

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Tn fix docs
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)