renwei2024/nemo-github-issues huggingface dataset:
Docs g2p update
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [X ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[ASR] Separate Audio-to-Text (BPE, Char) dataset construction
State: closed | Pull request: yes
Labels: name: ASR

Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>

# What does this PR do ?

Separate Audio-to-Text Char and BPE dataset construction:
- this allows to use datasets for further mixing (e.g. in hybrid ASR-TTS models #5659)
- removes duplicated code from CTC, RNNT (Char and BPE) models

**Collection**: [ASR]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
from nemo.collections.asr.data.audio_to_text_dataset import get_audio_to_text_bpe_dataset_from_config

dataset = get_audio_to_text_bpe_dataset_from_config(...)  # construct BPE dataset from parameters
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add Esperanto ASR example
State: closed | Pull request: yes
Labels: name: documentation

name: ASR

Signed-off-by: andrusenkoau <andrusenkoau@gmail.com>

# What does this PR do ?

Adds ASR example for training Esperanto Conformer-CTC-large model.

**Collection**: ASR

# Changelog 
- Adds Esperanto example to docs/source/asr/examples/

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation
Fixing masking in RadTTS bottleneck layer
State: closed | Pull request: yes
Labels: name: TTS

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>

Mask/length vector was ignored in bottleneck layer before.

**Collection**: [Note which collection this PR will affect]
TTS 

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Adithyare/linear combination
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Docs g2p update
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [X ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
TTS inference with Heteronym classification model, hc model inference refactoring
State: closed | Pull request: yes
Labels: name: TTS

name: CI

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ X] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix wrong label mapping in batch_inference for label_model
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fix groundtruth label mapping in `batch_inference` in `label_models.py` and add test. 

**Collection**: ASR

# Changelog 
-  The groundtruth label `batch_inference` was mapped wrongly. Fixed it
-  Add test for `batch_inference`

# Usage
* You can potentially add a usage example below

```python
from nemo.collections.asr.models import EncDecSpeakerLabelModel 
model = EncDecSpeakerLabelModel.from_pretrained('langid_ambernet')
embs, logits, gt_labels, mapped_labels = model.batch_inference(<path_to_manifest>)
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to https://github.com/NVIDIA/NeMo/pull/5278
Megatron export triton update
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- updating biased gelu 
- minor updates to mimic nemo nmt

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Missing "create_tarred_dataset.py" script for Duplex Text Normalization
State: closed | Pull request: no
Labels: name: bug

The script for creation **TarredDataset** described in the [Duplex Text Normalization Doc](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/nlp/text_normalization/nn_text_normalization.html) is missing. 
According to the documentation it should be located: 
`examples/nlp/duplex_text_normalization/create_tarred_dataset.py`
[ASR][Test] Enable test for cache audio with a single worker
State: closed | Pull request: yes

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

# What does this PR do ?

Enable a test which was disabled in #5615.

**Collection**: ASR tests

# Changelog 
- Enabled the test using a single worker

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #5607 and #5615
Update Tacotron2 NGC checkpoint load to latest version
State: closed | Pull request: yes
Labels: name: TTS

name: cherry-pick

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

# What does this PR do ?

Second version of #5748, applied to r1.15.0 this time.
Tacotron2's `list_available_models()` was still pointing to a previous version, so it would load that instead of the newest one.
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [x] Documentation

# Additional Information
* First noticed in #5714
[Fix] ConformerEncoder forward when length is None
State: closed | Pull request: yes
Labels: name: bug

name: ASR

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

# What does this PR do ?

This PR fixes a bug in `forward` of `ConformerEncoder` when `length is None`

**Collection**: ASR

# Changelog 
- Fixed initialization of `length` with `new_full`

# Usage

Currently, the following
```
import torch
from nemo.collections.asr.modules.conformer_encoder import ConformerEncoder

# create layer
conformer_encoder = ConformerEncoder(feat_in=10, n_layers=1, d_model=16)

# run
out, out_length = conformer_encoder(audio_signal=torch.rand(10, 9, 32), length=None)
```

results in a `TypeError`:

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "${PATH}/miniforge3/envs/nemo_env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1482, in _call_impl
    return forward_call(*args, **kwargs)
  File "${NEMO}/nemo/core/classes/common.py", line 1087, in __call__
    outputs = wrapped(*args, **kwargs)
  File "${NEMO}/nemo/collections/asr/modules/conformer_encoder.py", line 416, in forward
    length = audio_signal.new_full(
TypeError: new_full(): argument 'size' (position 1) must be tuple of ints, not int
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update Tacotron2 NGC checkpoint load to latest version
State: closed | Pull request: yes
Labels: name: TTS

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

# What does this PR do ?

Second version of #5748, applied to r1.15.0 this time.
Tacotron2's `list_available_models()` was still pointing to a previous version, so it would load that instead of the newest one.
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [x] Documentation

# Additional Information
* First noticed in #5714
TypeError: _run_hydra() missing 1 required positional argument: 'args'
State: closed | Pull request: no
Labels: name: bug

While training tacotron2, I end up with the following error.

```
    
Traceback (most recent call last):
  File "tacotron2.py", line 44, in <module>
    main()  # noqa pylint: disable=no-value-for-parameter
  File "/usr/local/lib/python3.8/dist-packages/nemo/core/config/hydra_runner.py", line 98, in wrapper
    _run_hydra(
TypeError: _run_hydra() missing 1 required positional argument: 'args'
[ ]
```
Just before that, I fixed the following issue using this [solution](https://github.com/NVIDIA/NeMo/issues/3915):
```
noqa pylint: disable=no-value-for-parameter File "/usr/local/lib/python3.7/dist-packages/nemo/core/config/hydra_runner.py", line 103, in wrapper strict=None, 

TypeError: _run_hydra() got an unexpected keyword argument 'strict'
```

After removing the argument `strict=None`, I get the above error: `TypeError: _run_hydra() missing 1 required positional argument: 'args'`. How to fix it? Thank you!
Fix EMA topk checkpoint deletion
State: closed | Pull request: yes
Labels: name: bug

# What does this PR do ?

Fixes case where we need to also delete the EMA checkpoints from previous runs.

**Collection**: Common

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Add EMA Docs, fix common collection documentation
State: closed | Pull request: yes
Labels: name: documentation

name: common

# What does this PR do ?

Adds simple documentation for EMA. Also fixes the docs on main due to an incorrect import in the EMA class.

**Collection**: Common

# Changelog 
- Add EMA documentation

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation
add esperanto example
State: closed | Pull request: yes
Labels: name: documentation

name: ASR

Signed-off-by: andrusenkoau <andrusenkoau@gmail.com>

# What does this PR do ?

Adds ASR example for training Esperanto Conformer-CTC-large model.

**Collection**: ASR

# Changelog 
- Adds Esperanto example to docs/source/asr/examples/

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation
GPU memory leak in nemo asr_model.transcribe()
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

GPU memory usage increases over the time
![Screenshot from 2023-01-08 23-10-52](https://user-images.githubusercontent.com/104126711/211239138-45bc947c-1910-459f-bd87-ced91070c1cd.png)



**Steps/Code to reproduce bug**

You can reproduce this bug using this notebook [Colab Notebook](https://colab.research.google.com/drive/1PqW0MqVyzCJ53fM1uwGrNvEcsmJN8Q6R?usp=sharing)

You can take any random WAV audio file and make sure you run it for 40 min to 1 hour. The Memory error should reproduce like mine.


**Expected behavior**

Memory usage aways should same after 1st batch processing complete

**Environment overview (please complete the following information)**

Google Colab
Errors when running first cell in 00_NeMo_Primer in Google Colab
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

When running the first cell in tutorials/00_NeMo_Primer.ipynb on Google Colab, I get the following errors:

```
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow 2.9.2 requires protobuf<3.20,>=3.9.2, but you have protobuf 3.20.3 which is incompatible.
tensorflow 2.9.2 requires tensorboard<2.10,>=2.9, but you have tensorboard 2.11.0 which is incompatible.
```

I then get FileNotFound exceptions when running later cells trying to import NeMo collections

**Steps/Code to reproduce bug**

1. Click link on https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/starthere/tutorials.html for NeMo Fundamentals
2. Switch Runtime to use GPU Hardware Accelerator 
3. Run first cell

**Expected behavior**

Expected dependencies to import without issue and to be able to go through the rest of the tutorial

**Environment overview (please complete the following information)**

 - Environment location: Collab
 - Method of NeMo install: pip install (specified in notebook with the commands:
 ```BRANCH = 'r1.14.0'
!python -m pip install git+https://github.com/NVIDIA/NeMo.git@$BRANCH#egg=nemo_toolkit[all]```

**Environment details**
- Python version 3.8.16

**Additional context**

Seems NeMo wants protobuf<4,>=3.20.2, but tensorflow 2.9.2 wants < 3.20.  Same for tensorboard.
[TTS][DE] refine grapheme-based tokenizer and fastpitch training recipe on thorsten's neutral datasets.
State: closed | Pull request: yes
Labels: name: TTS

name: common

# What does this PR do ?
```
1. refine `GermanCharsTokenizer` to support only graphemes as inputs by removing sentence-level phoneme representation;
2. refine `GermanCharsTokenizer` to preserve mixed cases from the original input graphemes;
3. add a new Thorsten's 22.10 dataset;
4. revise the thorsten's voice neutral datasets preparation script to support two versions of thorsten's voice datasets in a single script.
5. succeeded in running through model training. 
6. fixed some typos.
```

**TODO in separate PRs**
```
1. update NGC URL with new model ckpts. 
2. update the tutorial accordingly.
3. train grapheme-based models on HUI-ACG datasets and update NGC releases.
```
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue): https://github.com/NVIDIA/NeMo/issues/5400
Set the stream position to 0 for pydub
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

This PR sets the stream position to 0 to use pydub as fallback if SoundFile fails to read an audio file. 

When using tarred datasets, `audio_file` is a file-like object, not a path-like object. In this situation, to use pydub as fallback, we need to set the stream position of `audio_file` to 0 since SoundFile have advanced the position while reading it.

**Collection**: ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Expose conditional log prob method for enc-dec models.
State: closed | Pull request: yes
Labels: name: stale

name: NLP

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Exposes a method `log_prob()` to get the conditional log prob `p(y|x)` for enc-dec models.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Adithyare/prompt learning seed
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?
adds explicit random seed init for all prompt learning and adapter learning based methods.

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update Tacotron2's list_available_models()
State: closed | Pull request: yes
Labels: name: TTS

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

# What does this PR do ?

Tacotron2's `list_available_models()` was still pointing to a previous version, so it would load that instead of the newest one.

**Collection**: TTS

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

# Additional Information
* As noticed in #5714
Adithyare/zero shot eval prompt learning
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
link fixed
State: closed | Pull request: yes
Labels: name: TTS

name: cherry-pick

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
link fixed
State: closed | Pull request: yes
Labels: name: TTS

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
typo and link fixed
State: closed | Pull request: yes
Labels: name: TTS

name: cherry-pick

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
All files present in manifest contains silence, aborting next steps
State: closed | Pull request: no
Labels: name: feature

name: stale

**Describe the bug**

All files present in manifest contains silence, aborting next steps. 
When I run the diarization with an audio file which contains no speech, it terminates the program and throws an error

**Steps/Code to reproduce bug**

Run diarization on any audio wav file which contains no speech/audio


**Expected behavior**

Rather than program terminating, I would like a blank output and a blank rttm file.
Or how do I circumvent the "raise ValueError" in the script?
[TTS] Update Spanish TTS model to 1.15
State: closed | Pull request: yes
Labels: name: TTS

Signed-off-by: Ryan <rlangman@nvidia.com>

# What does this PR do ?

Update FastPitch Spanish config from 1.14.0 to 1.15.0, which contains IPA support and improvements from speaker level pitch normalization.

https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/tts_es_multispeaker_fastpitchhifigan/version

**Collection**: [TTS]

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
typo and link fixed
State: closed | Pull request: yes
Labels: name: TTS

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update 00_NeMo_Primer.ipynb
State: closed | Pull request: yes

Fixed a minor typo in primer tutorial.

Signed-off-by: schaltung <amoreno@gmail.com>

# What does this PR do ?

Fixed a minor typo: "White" <-- "While".

**Collection**: None

# Changelog 
- Single-word typo fix in the text markup of the jupyter notebook.

# Usage
* You can potentially add a usage example below


# Before your PR is "Ready for review"
**Pre checks**:
- [] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Mismatch in the dimensions length of log probabilities of Nemo ASR transcribe function and Vocab size
State: closed | Pull request: no
Labels: name: bug

name: stale

Hi All,
I am working on pyctcdecode integration with Nemo ASR models. However, there is a mismatch in the dimensions of the log probabilities matrix output by Nemo ASR transcribe function and the length of vocabulary size. Because of this I am unable to proceed further. Any suggestions, will be very helpful.

asr_model1 = nemo_asr.models.EncDecCTCModelBPE.from_pretrained(model_name="stt_en_conformer_ctc_medium")

print(len(asr_model1.decoder.vocabulary)) ---> this outputs "1024"

logits = asr_model1.transcribe(["/data/manju/tamil/data/segWav/tvs_test_9976337304-in-Speaker_1-11.wav"], logprobs=True)[0]

print(logits.shape) --> This outputs " (36, 1025)". Here, it should have produced "1024" in place of "1025"

Pls suggest @titu1994  @jbalam-nv . Thanks
Don't add output directory twice when creating shared sentencepiece tokenizer
State: closed | Pull request: yes
Labels: name: NLP

Signed-off-by: Patrick Simianer <patrick@lilt.com>

# What does this PR do ?

As the title says this is a small fix: The output dir was already added to `encoder_tokenizer_model` on line 789 in the same file.

**Collection**: NLP

# Changelog 
- Bugfix for creating shared sentencepiece tokenizer.

# Usage

The error can be triggered by running preprocessing with NeMo for an MT data set:
```
#!/bin/bash

python ../nemo/examples/nlp/machine_translation/enc_dec_nmt.py \
  -cn aayn_base \
  do_training=false \
  model.preproc_out_dir=./preproc_dir/ \
  model.train_ds.use_tarred_dataset=true \
  model.train_ds.lines_per_dataset_fragment=1000000 \
  model.train_ds.num_batches_per_tarfile=200 \
  model.train_ds.src_file_name=../europarl-v7.de-en.en \
  model.train_ds.tgt_file_name=../europarl-v7.de-en.de \
  model.validation_ds.src_file_name=../valid.en \
  model.validation_ds.tgt_file_name=../valid.de \
  model.encoder_tokenizer.vocab_size=32000 \
  model.decoder_tokenizer.vocab_size=32000 \
  model.encoder_tokenizer.library=sentencepiece \
  model.encoder_tokenizer.training_sample_size=9999 \
  model.decoder_tokenizer.library=sentencepiece \
  model.decoder_tokenizer.training_sample_size=9999 \
  ~model.test_ds \
  trainer.accelerator='cpu' \
  +trainer.fast_dev_run=true \
  exp_manager=null
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
Sanitize params before DLLogger log_hyperparams
State: closed | Pull request: yes

# What does this PR do ?

Allows DLLogger to work with types of non-builtin containers.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
Esperanto example
State: closed | Pull request: yes
Labels: name: core

name: TTS

name: ASR

name: NLP

name: Speaker Tasks

name: CI

name: common

# What does this PR do ?

Adds ASR example for training Esperanto Conformer-CTC-large model.

**Collection**: ASR

# Changelog 
- Adds Esperanto example to docs/source/asr/examples/

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation
Irish TN/ITN
State: closed | Pull request: yes

# What does this PR do ?

Opening to show provenance, closing immediately.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add keep_initializers_as_inputs to _export method
State: closed | Pull request: yes
Labels: name: core

Signed-off-by: Patrick Simianer <patrick@lilt.com>

# What does this PR do ?

In https://github.com/NVIDIA/NeMo/pull/5052 I forgot to add the `keep_initializers_as_inputs` arg to the `_export` method.

**Collection**: core

# Changelog 
- Simple fix.

# Usage
* See https://github.com/NVIDIA/NeMo/pull/5052

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #5052