renwei2024/nemo-github-issues huggingface dataset:
Reached max decoder steps
State: closed | Pull request: no
Labels: name: bug

Hello,

I'm training a model on Tacotron2. I tried little datasets, large datasets, but always I get the same messages.

Reached max decoder steps XXXX

By default there are 1000 steps. I tried the default configuration and also I tried greater values.

I thought it was caused because the labels property in tacotron2.yaml. But I grouped all the letters in my dataset and every letter is on the "labels" property.

Also I tried adding a dot at the end of each sentence. Absolutely all of the sentences are between 1 and 10 seconds.

The dataset is all in spanish. I tried to train minutes, hours, etc. I mean in the json file I tried from 10000 registers to 100 registers.

When training it always notice Reached max decoder steps XXXX

Do you know why is it happening? Do you know how to solve it?

Thank you!
Asr noise data scripts
State: closed | Pull request: yes

# What does this PR do ?

Adds scripts 1) download and prepare noise data from Demand database and 2) to add noise to an audio dataset at various SNRs 
**Collection**: [ASR]

# Changelog 
- Added script to download and prepare manifest for noise data from Demand noise database.
- Added script to add noise to an audio dataset at various SNRs

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Can't download NVIDIA Models
State: closed | Pull request: no
Labels: name: bug

I am trying the NEMO_Primer from this GitHub repository. I have tried downloading the model on Google Colab however if I try to run the demo on Jupiter notebook locally, the model cant be downloaded. Please see image to see error. This error pops up when passing a pre-trained model name and instantiate a model directly from the cloud:

citrinet = nemo_asr.models.EncDecCTCModelBPE.from_pretrained('stt_en_citrinet_512')

or 

quartznet = nemo_asr.models.EncDecCTCModel.from_pretrained(model_name="stt_en_quartznet15x5")

<img width="972" alt="Screenshot 2022-02-14 at 11 10 41 AM" src="https://user-images.githubusercontent.com/90296566/153793932-26dc0258-0225-4542-9322-bc7a496b4059.png">
Numpy error
State: closed | Pull request: no
Labels: name: bug

Hello,

I Installed a clean Ubuntu 20.04, installed nvidia drivers 440 for RTX 3080. Then I followed the instructions to train a model with Tacotron2.

numpy fails with the next error:
valueerror: numpy.ndarray size changed, may indicate binary incompatibility. expected 96 from c header, got 88 from pyobject 

I tried the default version of numpy and also I tried pip install --upgrade numpy==1.21.0

I really don'tknow how to solve the problem and I write this to ask for help.

Do you know how to solve it?

Thank you!

David
LM adapted T5 dataset
State: closed | Pull request: yes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Adds the ability to use the prefix-LM objective for T5 which reuses the GPT dataset class that already exists.

**Collection**: NLP

# Changelog 
- Adds at T5LMAdaptedDataset class that inherits from GPTDataset that just splits a text sequence into encoder and decoder components
- Re-uses the existing build_train_valid_test_datasets functions that exist for BERT and T5 for this class.
- Moves some dataset utils to a new file to prevent circular imports between gpt_dataset.py and dataset_utils.py.

# Usage

Add the `dataset_type` flag to either `t5` or `t5_prefix_lm` in `megatron_t5_config.yaml`.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Significantly speed up RNNT loss on CUDA
State: closed | Pull request: yes

Signed-off-by: smajumdar <titu1994@gmail.com>

# What does this PR do ?

Speed up CUDA RNNT loss 

**Collection**: [ASR]

# Changelog 

- Fix issue with data movement of costs tensor from llForward to pytorch data view in numba
- This alone costs a linear loop (scaling with batch size) that is roughly 10x the kernel costs themselves.
- Fix by writing a small kernel to copy the data and update the costs.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [x] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
[TOOLS] fix number of channels>1 issue
State: closed | Pull request: yes

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add tutorial paths
State: closed | Pull request: yes

# What does this PR do ?

Update readme to include streaming inference directory for ASR.

**Collection**: [ASR]

# Changelog 
- Update readme.md to contain link to streaming / buffered asr
- Update URL in tutorial for streaming / buffered transducer models

**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
add onnx support for speaker models
State: closed | Pull request: yes

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

# What does this PR do ?

* adds onnx support for speaker models 

**Collection**: [speaker_tasks]

# Changelog 
- Added forward_for_export function 
- Fixed previous asr onnx exportable tests 

# Usage
To generate onnx speaker model from titanet nemo 
```bash
python export.py <path/to/titanet-l.nemo> <path/to/titanet-l.onnx>
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add ssl tutorial to tutorial docs page
State: closed | Pull request: yes

Signed-off-by: sam1373 <samuelkriman@gmail.com>

# What does this PR do ?

Add ASR ssl tutorial to the tutorial docs page.

**Collection**: ASR

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
.wav file contains data in an unknown format
State: closed | Pull request: no

I am testing out the NVIDIA NeMo tutorial for ASR_with_Nemo.ipynb on google colab. The dataset in the demo from an4 works fine for me. However when I try to upload a .wav file from my local pc, the code returns an error "RuntimeError: Error opening '020010009.wav': File contains data in an unknown format." This is odd because I tried uploading a .sph file and use their code to convert the .sph file into a .wav file and it outputs the same error. I opened up the .sph file from the an4 dataset and opened up my .sph file and even though both are .sph files, the file content looks different. Seems like the demo only works on the an4 audio dataset and nothing else. Can anyone help?
Changes to support mean n-gram size masking for T5
State: closed | Pull request: yes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Allows configuring the mean n-gram size for T5 span masking when using a geometric distribution.

**Collection**: NLP

# Changelog 
- Used the fact that the expectation of the geometric distribution is 1/p to set p appropriately for a given expected value of n-gram size.

# Usage

To get a mean n-gram size of 3 while masking (like suggested in the T5 paper), thee data configuration for T5 can look like this:

```
    max_ngram_size: ${seq_length_dec}
    mean_ngram_size: 3
    geometric_dist: True
    permutation: False
    whole_work_masking: True
    favor_longer_ngrams: False
```

```python
# N/A same run script as T5, just requires changing the yaml config like above.
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update adobe analytics
State: closed | Pull request: yes

# What does this PR do ?
Update analytics code.

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update adobe analytics
State: closed | Pull request: yes

# What does this PR do ?
Update adobe analytics code.

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add a function that writes cluster label in diarization pipeline
State: closed | Pull request: yes

# What does this PR do ?

This PR adds a function that writes cluster label. This cluster label (speaker label for each segment) is needed to provide estimated segment label for overlap-aware diarization module or target-speaker ASR.

**Collection**: asr (but actually speaker_tasks)

# Changelog 
- Removed unnecessary line in speaker_utils.py
- Added a line that adds segment info and cluster label
- Added lines that write files to speaker_outputs.

# Usage
Please refer to [Diarization docs](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/asr/speaker_diarization/datasets.html)

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
Update citrinet_512.yaml
State: closed | Pull request: yes

Added num workers, cause without that argument any training goes really slow

# What does this PR do ?
reduced training time

# Changelog 
- Add specific line by line info of high level changes in this PR.



# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] Bugfix

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Found out that training on 1.6.0 starts with num workers == 0.
[TTS] Update Aligner, add new methods to AlignmentEncoder
State: closed | Pull request: yes

Signed-off-by: Oktai Tatanov <oktai.tatanov@gmail.com>

# What does this PR do ?

**Collection**: [tts]

# Changelog 
- refactoring of Aligner code and config, replacing deprecated datasets with `TTSDataset` 
- adding three methods to AlignmentEncoder which can be useful for Neural G2P research
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
update torchaudio in Dockerfile to match torch version
State: closed | Pull request: yes

Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com>

# What does this PR do ?

This PR fixes the version mismatch between torchaudio and torch (0.10 vs 1.11) in Dockerfile.
If the version of torchaudio does not match the version of torch, then it loses its CUDA backend and the following errors may occur:
```
NotImplementedError: Could not run 'torchaudio::rnnt_loss' with arguments from the 'CUDA' backend.
```

# Changelog 
Dockerfile
- `--branch release/0.10` -> `--branch release/0.11`
Tn es
State: closed | Pull request: yes

# What does this PR do ?

Adds Text Normalization for spanish language to nemo_text_normalization

**Collection**: Nemo Text Normalization

# Changelog 
- Adds Text Normalization for Spanish language. Verbalizers and classifiers are available for following classes
- - Cardinal
- - Decimal
- - Ordinal
- - Fraction
- - Money
- - Measure
- - Date
- - Time
- - Electronic
- - Whitelist

Also includes a localization option of es-amer, which changes formatting rules to accommodate tendencies in Central American orthography. (e.g. Use of periods to group cardinals instead of commas - as is customary for other Spanish speaking locales.)

Includes updated es_pytests for text normalization and edits to export_grammar.sh and normalize.py to allow deployment. All tests have passed in nemo docker environment.

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [Y ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [Y ] Did you write any new necessary tests?
- [Y ] Did you add or update any necessary documentation?
- [ ]Y Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ Y] New Feature

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
- * N. B.  Sparrowhawk deployment is altering white-space following periods, so TN is unable to manage both am/pm along with time zones. That is -> 10 a.m. est will cause an error. However, 10.00 h est is stable. This issue is Sparrowhawk related so have been unable to debug.
[TTS] FastPitch training tutorial
State: closed | Pull request: yes

# Addition of FastPitch training tutorial

Please have one of the following:

* New feature: (short feature description) FastPitch training tutorial.
This PR adds FastPitch tutorial to the same notebook as MixerTTS as both generate spectrogram. This tutorial goes through the data download, preprocess, FastPitch import and training pipeline.
Tutorial was added with careful consideration to make minimal changes to MixerTTS tutorial.

## Before your PR is "Ready for review"
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or updated any necessary documentation?


If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
[TN] ranking lm
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ X] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
FastPitch fine-tunning using TDataset
State: closed | Pull request: no
Labels: name: feature

The tutorial on FastPitch fine-tunning (`NeMo/tutorials/tts/`) is based on the `fastpitch_align.yaml` and `AudioToCharWithPriorAndPitchDataset`, which will soon be deprecated. Is there any guidance how to finetune a pretrained FastPitch model based on the `fastpitch_align_v1.05.yaml` and thus `TDataset`?
Remove compression from .nemo files
State: closed | Pull request: yes

# What does this PR do ?

This PR removes compression from .nemo files.
The compression is not desirable because:
 * It takes awfully long to compress multi-billion parameter language models
 * It does not actually save much space. For example a large CTC conformer uncompressed is 464Mb vs 431Mb compressed

This is a change to core save_to/restore_from functionality of all models.
**It is backwards compatible - older checkpoints (with compression) will still work**

# Usage
* Usage remains exactly the same.
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Generalized the P-tuning method to support various NLP tasks
State: closed | Pull request: yes

# What does this PR do ?

It generalizes the P-tuning method to support various NLP tasks including text-cls, NER, text summation, Q/A etc. It supports multiple task with one prompt encoder model. It also includes a tutorial notebook to show how to use it.

# Changelog 
- Added the decoder for the p-tuning model so it can generate any length of text.
- The prompt encoder can be conditioned on the task-tags, so it supports multiple tasks simultaneously. 
- Added the p-tune dataset. The user just need to write their own data processor to generate customized prompts for different tasks. 

# Usage
Check the notebook `https://github.com/NVIDIA/NeMo/blob/generalized_gpt_ptuning/tutorials/nlp/PTune_multiple_NLP_tasks.ipynb`



  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
WIP: Megatron t0
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Clarifying documentation
State: closed | Pull request: yes

# What does this PR do ?

Updated Optimizers & schedulers documentation.

**Collection**: all

# Changelog 
- minor fix to documentation

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #3549

Signed-off-by: Iztok Lebar Bajec ilb@fri.uni-lj.si
[TTS] Fix UnivNet LibriTTS pretrained location
State: closed | Pull request: yes

# What does this PR do ?

Mini-PR :) - Fixes the NGC location of the UnivNet LibriTTS pretrained model.

**Collection**: TTS

# Changelog 
- Fixes the NGC location of the UnivNet LibriTTS pretrained model.


# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [X] Did you write any new necessary tests?
- [X] Did you add or update any necessary documentation?
- [X] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation
./reinstall.sh crashes due to not being able to uninstall llvmlite
State: closed | Pull request: no
Labels: name: bug

Starting off of `nemo:1.5.1` container, cloning the NeMo repo to a folder inside of it and calling `./reinstall.sh` fails with
```
ERROR: Cannot uninstall 'llvmlite'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
```
`pip install -e` on the other hand succeeds installing `nemo:1.7.0rc0` and `numpy:1.22.2`, the rest of the packages remain untouched.

It seems that `./reinstall.sh` which used to work fine, a week or so ago when following the same procedure to upgrade to `nemo:1.6.0rc` redeveloped issue #841. The solution remains the same, first call
```
pip install --ignore-installed llvmlite
```
followed by `./reinstall.sh`. In this case, apart `llvml`, the following packages are updated
```
ftfy-6.0.3 nemo-toolkit-1.7.0rc0 numba-0.55.1 pytorch-lightning-1.5.9 sacrebleu-2.0.0 setuptools-59.5.0
```
Interestingly `numpy` in this case is left at `1.21.5`.
using NGC containers [ NeMo/Riva] there is no nemo2riva
State: closed | Pull request: no
Labels: name: bug

```
 docker run --runtime=nvidia -it --rm -v --shm-size=16g -p 8888:8888 -p 6006:6006 --ulimit memlock=-1 --ulimit stack=67108864 nvcr.io/nvidia/nemo:1.5.1

```
```
oot@adbd9847582b:/workspace/nemo# pip3 install nvidia-pyindex
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: nvidia-pyindex in /opt/conda/lib/python3.8/site-packages (1.0.9)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
root@adbd9847582b:/workspace/nemo# pip3 install nemo2riva
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
ERROR: Could not find a version that satisfies the requirement nemo2riva (from versions: none)
ERROR: No matching distribution found for nemo2riva
root@adbd9847582b:/workspace/nemo# pip3 install nemo2riva-1.9.0-beta-py3-none-any.whl
WARNING: Requirement 'nemo2riva-1.9.0-beta-py3-none-any.whl' looks like a filename, but the file does not exist
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
ERROR: nemo2riva-1.9.0-beta-py3-none-any.whl is not a supported wheel on this platform.

```
where do I get the tool?
Also tried with 
```
nvcr.io/nvidia/riva/riva-speech:1.8.0-beta-server
```
Update branch in notebooks to main
State: closed | Pull request: yes

Signed-off-by: ericharper <complex451@gmail.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix failing tests inside of the 22.01 container in PR 3571
State: closed | Pull request: yes

Signed-off-by: fayejf <fayejf07@gmail.com>

# What does this PR do ?

Reproduce and fix failing tests inside of the 22.01 container in [PR 3571 ](https://github.com/NVIDIA/NeMo/pull/3571)

**Collection**: ASR label dataloader and tarred label dataset

# Changelog 
- Fix failing tests due to 22.01 container upgrade

# Usage
* You can potentially add a usage example below
None
```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to [# (issue in PR 3571 )](https://github.com/NVIDIA/NeMo/pull/3571)
W&B Causing GPT3 Run to Crash
State: closed | Pull request: no

Weights and Biases is causing some GPT-3 runs to fail. This seems to occur randomly. When launching multiple identical jobs, the first two can fail because of this and the third one works.

Traceback:

> Traceback (most recent call last):
  File "/opt/conda/bin/wandb", line 8, in <module>
    sys.exit(cli())
  File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/wandb/cli/cli.py", line 86, in wrapper
    return func(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/wandb/cli/cli.py", line 235, in login
    wandb.login(relogin=relogin, key=key, anonymous=anon_mode, host=host, force=True)
  File "/opt/conda/lib/python3.8/site-packages/wandb/sdk/wandb_login.py", line 65, in login
    configured = _login(**kwargs)
  File "/opt/conda/lib/python3.8/site-packages/wandb/sdk/wandb_login.py", line 268, in _login
    wlogin.configure_api_key(key)
  File "/opt/conda/lib/python3.8/site-packages/wandb/sdk/wandb_login.py", line 154, in configure_api_key
    apikey.write_key(self._settings, key)
  File "/opt/conda/lib/python3.8/site-packages/wandb/sdk/lib/apikey.py", line 223, in write_key
    api.clear_setting("anonymous", globally=True, persist=True)
  File "/opt/conda/lib/python3.8/site-packages/wandb/apis/internal.py", line 75, in clear_setting
    return self.api.clear_setting(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/wandb/apis/internal.py", line 19, in api
    self._api = InternalApi(*self._api_args, **self._api_kwargs)
  File "/opt/conda/lib/python3.8/site-packages/wandb/sdk/internal/internal_api.py", line 78, in __init__
    self._settings = Settings(
  File "/opt/conda/lib/python3.8/site-packages/wandb/old/settings.py", line 23, in __init__
    self._global_settings.read([Settings._global_path()])
  File "/opt/conda/lib/python3.8/configparser.py", line 697, in read
    self._read(fp, filename)
  File "/opt/conda/lib/python3.8/configparser.py", line 1082, in _read
    raise MissingSectionHeaderError(fpname, lineno, line)
configparser.MissingSectionHeaderError: File contains no section headers.
file: '/root/.config/wandb/settings', line: 1
'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
wandb: Appending key for api.wandb.ai to your netrc file: /root/.netrc
Add env variable to force transformers to run offline during CI
State: closed | Pull request: yes

# What does this PR do ?
Add env variable to force transformers to run offline during CI

**Collection**: [bugfix]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add NeMo version to all new .nemo files
State: closed | Pull request: yes

Signed-off-by: smajumdar <titu1994@gmail.com>

# What does this PR do ?

Adds current nemo version to all nemo files generated. 

# Note

The version will point to the *earliest* known version of NeMo that was compatible with this model. This does not preclude later versions being compatible/incompatible with the checkpoint, so it is up to the user to determine which version would work best, but it can be assumed that no version prior to the noted version will support the checkpoint.

**Collection**: [Core]

**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Fix typo and update minimal version of scipy
State: closed | Pull request: yes

Signed-off-by: Oktai Tatanov <oktai.tatanov@gmail.com>

**Collection**: [DOC]

# Changelog 
- fix typo in README
- fix minimal version for SciPy (it is necessary to use `scipy.stats`
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Add initial support for Buffered RNNT Scripts
State: closed | Pull request: yes

# What does this PR do ?

This PR adds support for buffered RNNT evaluation, so that models can be evaluated on audio of any length by models which are normally restricted to small audio duration.

**Collection**: [ASR]

# Changelog 
- Refactor the asr_chunked_inference into ctc and rnnt sub folders
- Add speech_to_text_buffered_infer_rnnt.py script to perform RNNT buffered inference using two approaches.
  - "Middle token" algorithm
  - "Longest common subsequence merge" algorithm
- Refactor RNNT greedy batched decoding to significantly improve decoding speed after training.
  - This now is automatically applied right after performing `RNNTModel.change_decoding_strategy()`

# Usage
* You can potentially add a usage example below

```sh
Script to perform buffered inference using RNNT models.

# Middle Token merge algorithm

python speech_to_text_buffered_infer_rnnt.py \
    --asr_model="<Path to a nemo model>" \
    --test_manifest="<Path to a JSON manifest>" \
    --model_stride=4 \
    --output_path="." \
    --total_buffer_in_secs=10.0 \
    --chunk_len_in_ms=8000 \
    --device="cuda:0" \
    --batch_size=128


# Longer Common Subsequence (LCS) Merge algorithm

python speech_to_text_buffered_infer_rnnt.py \
    --asr_model="<Path to a nemo model>" \
    --test_manifest="<Path to a JSON manifest>" \
    --model_stride=4 \
    --output_path="." \
    --use_lcs_merge \
    --lcs_alignment_dir=<OPTIONAL: Some path to store the LCS alignments> \
    --total_buffer_in_secs=10.0 \
    --chunk_len_in_ms=8000 \
    --device="cuda:0" \
    --batch_size=128

# NOTE:

    You can use `DEBUG=1 python speech_to_text_buffered_infer_rnnt.py ...` to print out the
    ground truth text and predictions of the model.
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

# Additional Information
* Notebook explaining the parts of the script will be released shortly.
MTEncDecModel preprocessing
State: closed | Pull request: no
Labels: name: feature

An issue with regard to preprocessing (Moses, Jieba, etc.) in the MTEncDecModel:

If the language code is supplied to `MTEncDecModel.translate()` , preprocessing will occur. On the other hand, if the language code is not supplied ('ignore'), the directional tokens ('\<es\>', etc.) cannot be appended. This makes it impossible to have a multilingual model that does not apply preprocessing, which is relatively slow.

It would be a sufficient workaround to refactor how this method decides to do preprocessing (e.g., `preprocess=False`), but it should probably be an attribute of the model itself whether preprocessing is needed. When using a model from NGC, how would a user know whether to use `source_lang="ignore"` or `source_lang="en"`? 

It looks like there are a few other places in the model where pre- and post-processing are deeply-entrenched as well. It would be nice to see all of this separated so a user can instantiate a model with `pre_and_post_process=False` and take control of how the text is processed.
[TTS] Convert entry from UTF-16 to UTF-8
State: closed | Pull request: yes

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

# What does this PR do ?

Updates the DÉJÀ entry in CMUdict to the correct encoding (UTF-8), as mentioned in #3594.
**Collection**: TTS

# Changelog 
Bytes `D\xc9J\xc0` updated to `D\xc3\x89J\xc3\x80`.

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Different Speakers Reciting Similar Lines
State: closed | Pull request: no

Hi,

I have about 100 hours of pre-processed data for male and female speakers. There is some overlap in the lines the speakers are reciting.

I was wondering if having different speakers saying the same exact text would negatively impact the training of an automatic speech recognition model? Would I have to remove the audio and text files where the speakers say the same thing? Each speaker sounds quite different from the other.

Thank you,

Enrico
Port Variational Inference with Adversarial Learning (VITS) to NeMo TTS
State: closed | Pull request: yes

# Overview

This change adds [Variational Inference with adversarial learning (VITS)](https://github.com/jaywalnut310/vits) to the collection of TTS models.

VITS uses the `monotonic_align` module, which requires Cython to be installed prior to building (i.e. run `pip install cython` before running `./reinstall.sh`).
[TTS] Fix incorrect usage of TTSDataset in some files and fix one-line bug in NVIDIA's CMUDict
State: closed | Pull request: yes

Signed-off-by: Oktai Tatanov <oktai.tatanov@gmail.com>

# What does this PR do ?

**Collection**: [tts]

# Changelog 
- Fix bug in NVIDIA's CMUDict: word `INTERNATIONAL` used non-existent phoneme
- Fix incorrect usage of TTSDataset in some files
  
**PR Type**:
- [x] Bugfix
ITN: SH bug fixes for telephone
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
apex quick fix
State: closed | Pull request: yes

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)