renwei2024/nemo-github-issues huggingface dataset:
[TTS] FastPitch training: speed up align_prior_matrix calculation
State: closed | Pull request: yes

# What does this PR do ?

Speed up align_prior_matrix calculation and don't save it to disk. They take lots of space (tens of gigabytes per experiment) and it is perfectly fine to compute them on-the-fly. Matrix calculation speed-up is significant.

```python
%timeit beta_binomial_prior_distribution_numpy_old(80, 300)
226 ms ± 5.86 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

%timeit beta_binomial_prior_distribution_new_torch(80, 300)
586 µs ± 22 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)
```

**Collection**: TTS

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below



# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Adapter tuning for Megatron GPT models
State: closed | Pull request: yes

# What does this PR do ?

This PR provides support to train and evaluate Adapter Infused GPT Models. The adapters supported are based off Houlsby et al.  https://arxiv.org/pdf/1902.00751.pdf for NLP tasks. Some minor modifications were made to support Pipeline parallel training and inference.

**Collection**: [NLP]

# Changelog 
- Added a training script for Adapter Infused training of a GPT Model.
- Added an evaluation script that loads a base GPT Model and it's corresponding adapter weights and runs inference.
- Added an Adapter Wrapper model which handles insertion of Adapters, loading and saving of Adapters for a base GPT Model.

# Usage
* Training Usage:

```
python megatron_gpt_adapter_tuning.py \
            model.data.train_ds=["PATH TO TRAINING .jsonl FILE"] \
            model.data.validation_ds=["PATH TO VALIDATION .jsonl FILE"] \
            model.language_model_path="PATH TO BASE GPT MODEL .nemo FILE" \
            name="NAME OF TRAINING RUN" \
            exp_manager.exp_dir="DIR TO SAVE CHECKPOINTS and ADAPTER .nemo FILE" \
            trainer.max_epochs=2
```

* Inference Usage:
```
python megatron_gpt_adapter_eval.py \
            gpt_model_file="PATH TO GPT MODEL .nemo FILE" \
            adapter_model_file="PATH TO ADAPTER MODEL .nemo FILE (generated by training script: ./megatron_gpt_adapter_tuning.py)" \
            data_paths=["PATH TO A .jsonl FILE CONTAINING PROMPTS"] \
            output_file="PATH TO OUTPUT FILE TO DUMP PREDICTIONS"
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
Jenkins Tests to train and evaluation Adapter Support for tp=2 and pp=2 GPT Models.
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)

# Future Work
* Refactor class hierarchy between Adapters, P-Tuning and Prompt Tuning.
Fix bug intent slot classification tokenizer to dialogue1
State: closed | Pull request: yes

# What does this PR do ?


Fix bug intent slot classification tokenizer
**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Davidm megatron decoder memory
State: closed | Pull request: yes

# What does this PR do ?

Adding decoder memory to increase inference performance

**Collection**: [Note which collection this PR will affect]

# Changelog 
- changes in Megatron decoder and related files
- added use_memory for translate function in megatron_nmt

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [X] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update Aligner model and tutorial to add NGC checkpoint loading
State: closed | Pull request: yes

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

# What does this PR do ?

Adds NGC model loading for the RAD-TTS Aligner.

**Collection**: TTS
# Changelog 
- Add newly published NGC checkpoint to list of pretrained Aligner models
- Update tutorial to load this checkpoint instead of a local one

# Usage

```python
from nemo.collections.tts.models import AlignerModel

aligner = AlignerModel.from_pretrained("tts_en_radtts_aligner")
```
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation
Adding RETRO model Faiss sharding index and KNN sharding index
State: closed | Pull request: yes

# What does this PR do ?
To handle very large dataset, e.g. hundreds of gigabyte to terrabyte compressed raw data, we need multiple nodes to create sharding index  and combine them together.
It enhance the KNN index data structure to handling sharding index.

In this PR, it creates GPU Faiss index in 3 stages. 
  1. stage 0, train Faiss index structure. 
  2. stage 1, create Faiss sharding index
  3. stage 2, merge sharding index into one.

It creates KNN index in 2 stages.
  1. stage 1, create KNN sharding index
  2. stage 2, merge KNN sharding index into one.
  
It includes some unit tests to cover the KNN sharding index, dedup funcitons etc.

I have been used it to successfuly create Faiss and KNN index for 350G dataset in a slurm cluster enviroment.
added/fixed export for Megatron models
State: closed | Pull request: yes

Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com>

# What does this PR do ?

Fixed Megatron export for updated Megatron models

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Changed tokens_head and row parallel layers to linear during the export process

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
v0.0.2, [WIP]
State: closed | Pull request: yes
Labels: name: stale

Added comparation tool to SDE, work in progress

# What does this PR do ?

Adds new page to SDE

**Collection**: [Note which collection this PR will affect]
ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python data_explorer.py FTAll_books_train.json -c1 Conf_test.json -c2 Context_test.json
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [yes] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Stateless decoder for RNN-T
State: closed | Pull request: yes

# What does this PR do ?

This PR adds stateless decoder support for RNN-T ASR models.

**Collection**: ASR

# Changelog 
- adds a stateless decoder class for RNN-T ASR models.
- adds another set of asr model tests for partial decoding.
- small bus fixes for asr greedy decode during partial decoding

# Usage

To launch an RNN-T model with a stateless decoder, change the training script by adding the following lines with recommended settings,

```
model.decoder._target_=nemo.collections.asr.modules.StatelessTransducerDecoder \
+model.decoder.context_size=2 \
model.decoder.normalization_mode=layer \
~model.decoder.random_state_sampling \
~model.decoder.blank_as_pad \
```
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Updates to NeMo Megatron OSS docs
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [*] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Speaker Verification
State: closed | Pull request: no

Hi!

I trained the model using:
https://github.com/NVIDIA/NeMo/blob/main/tutorials/speaker_tasks/Speaker_Identification_Verification.ipynb

```
import nemo.collections.asr as nemo_asr
from omegaconf import OmegaConf
import torch
import pytorch_lightning as pl
import os

speaker_model = nemo_asr.models.EncDecSpeakerLabelModel.load_from_checkpoint('/home/denis/ttt/result/TitaNet/2022-08-05_08-47-00/checkpoints/TitaNet--val_loss=2.3548-epoch=9-last.ckpt')
#decision = speaker_model.verify_speakers('/home/denis/ttt/data/an4/wav/an4_clstk/mjes/cen7-mjes-b.wav','/home/denis/ttt/data/an4/wav/an4_clstk/mjes/cen2-mjes-b.wav')
decision = speaker_model.verify_speakers('/home/denis/ttt/data/an4/wav/an4_clstk/mjes/cen7-mjes-b.wav','data/an4/wav/an4test_clstk/fcaw/an406-fcaw-b.wav')

print(decision)
```
It works!

I can compare two files. Is there a method to compare all my speakers (many files) with another single file?
Do I need to use a loop or another method?
Train LM with a large file
State: closed | Pull request: no
Labels: name: bug

name: stale

I try to train ~5.6G data with ~700M validation 

Using this below command:

 python /workspace/data/NeMo/examples/nlp/language_modeling/bert_pretraining.py 
        --config-name=/workspace/data/NeMo/examples/nlp/language_modeling/conf/bert_pretraining_from_text_config.yaml 
        model.train_ds.data_file="/workspace/data/NeMo/lm/data/public-data/train.txt"  
        model.validation_ds.data_file="/workspace/data/NeMo/lm/data/public-data/val.txt"  
        model.train_ds.batch_size=128 
        model.optim.lr=5e-5 
        trainer.max_epochs=1 
        trainer.gpus=1 
      
Then it show this error

GPU available: True, used: True
TPU available: False, using: 0 TPU cores
Using native 16bit precision.
[NeMo I 2022-08-09 02:50:20 exp_manager:216] Experiments will be logged at /workspace/data/NeMo/public-data-lm/nemo_experiments/PretrainingBERTFromText/2022-08-09_02-50-20
[NeMo I 2022-08-09 02:50:20 exp_manager:563] TensorboardLogger has been set up
  0%|                                                              | 0/1 [00:48<?, ?it/s]
Traceback (most recent call last):
  File "/workspace/data/bew/NeMo/examples/nlp/language_modeling/bert_pretraining.py", line 31, in main
    bert_model = BERTLMModel(cfg.model, trainer=trainer)
  File "/opt/conda/lib/python3.6/site-packages/nemo/collections/nlp/models/language_modeling/bert_lm_model.py", line 63, in __init__
    super().__init__(cfg=cfg, trainer=trainer)
  File "/opt/conda/lib/python3.6/site-packages/nemo/core/classes/modelPT.py", line 127, in __init__
    self.setup_training_data(self._cfg.train_ds)
  File "/opt/conda/lib/python3.6/site-packages/nemo/collections/nlp/models/language_modeling/bert_lm_model.py", line 192, in setup_training_data
    else self._setup_dataloader(train_data_config)
  File "/opt/conda/lib/python3.6/site-packages/nemo/collections/nlp/models/language_modeling/bert_lm_model.py", line 235, in _setup_dataloader
    short_seq_prob=cfg.short_seq_prob,
  File "/opt/conda/lib/python3.6/site-packages/nemo/collections/nlp/data/language_modeling/lm_bert_dataset.py", line 90, in __init__
    sentence_indices[filename] = array.array("I", newline_indices)
OverflowError: unsigned int is greater than maximum

I can train with the small size data, training is fine but the large file I got this error

**Environment details**
- Ubuntu
- Pytorch 1.8.0a0+17f8c32
- Pytorch-lightning 1.2.10
- Python 3.6.10
- DGX-V100 32G
- NeMO 1.0.0rc1
[WEBSITE] Introduction post
State: closed | Pull request: yes

Signed-off-by: smajumdar <smajumdar@nvidia.com>

# What does this PR do ?

Add introductory post to nvidia.github.io/NeMo/ 

**Collection**: [Docs]

# Changelog 
- Add introductory post
- Add support for MathJax

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation
Megatron decoder mems
State: closed | Pull request: yes

# What does this PR do ?

Adds decoder memory for faster inference

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [X] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Port Huggingface T5v1_1 weights to NeMo-Megatron
State: closed | Pull request: yes
Labels: name: stale

# What does this PR do ?

Adds a conversion script and related compatibility args to port Huggingface T5v1_1 weights to NeMo-Megatron.

**Collection**: NLP

# Changelog 
- Adds a state dict conversation script.
- Adds necessary args to yaml configs to make sure the forward passes in HF and NeMo result in identical results.

# Usage
* You can potentially add a usage example below

```bash
python scripts/nlp_language_modeling/hf_t5_to_nemo_coverter.py \
    --hf_model_name google/ul2 \
    --nemo_state_dict /path/to/nemo_ul2_state_dict.pt
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix to fetch config file
State: closed | Pull request: yes

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

# What does this PR do ?

Fix to get the titanet-finetune.conf file before running it. 

**Collection**: Speaker Tasks

# Changelog 
Previous code line was commented, now uncommented to get the required config file

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Need some info on spanish asr ctc confirmer large model.
State: closed | Pull request: no
Labels: name: stale

Can I get more data on the dataset cleaning process? 

I am a non spanish speaker 😅.

Like voxpoplui datasets says 120hrs after cleaning. I downloaded the dataset it has 150 hrs. Need to know which 30 hrs to clean please. 

also in model description it says trained the confirmer large ctc  on 7000 hrs of pretrained model. But it points to model trained on 16k hrs. 

Thanks for the help
Adding new tokens to pretrained asr vocab 
State: closed | Pull request: no

Is it possible to add new tokens to the tokeniser of a pretrained model. say I want to fine tune on some new data, which contains few new tokens eg. punct or nums can I add these to the decoder vocabulary?

Here I don't want to change the entire vocabulary so the initial decoder weights will remain the same hence I can reuse them, but for the new tokens the decoder weights can be randomly initialised and learned during fine-tuning 

The only other way I knew to add new tokens was to change the entire tokeniser. but then I am unable to use the decoder.
Fix bug intent slot classification tokenizer to dialogue
State: closed | Pull request: yes

# What does this PR do ?

Fix bug intent slot classification tokenizer to dialogue

**Collection**: nlp

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Megatron encode function with RPE fix
State: closed | Pull request: yes

# What does this PR do ?

Fixes a bug with RPE and encode() in token_level_encoder_decoder.

**Collection**: NLP

# Changelog 
- Adds `enc_self_attention_relative_position_bias` to the `self.encoder()` call in `megatron_encoder_decoder.py`

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix typo (cartographic -> catastrophic)
State: closed | Pull request: yes

Fix #4554. There is a typo. Instead of "cartographic", it should be "catastrophic".

# What does this PR do ?

Fix #4554. There is a typo. Instead of "cartographic", it should be "catastrophic".

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
TTS tokenizers moved to collections.common.tokenizers
State: closed | Pull request: yes

# What does this PR do ?

Move TTS tokenizers to collections.common.tokenizers so they can be used in cross-collections pipelines.

For implementing ASR/ST model training which generates samples on-the-fly with TTS, we need to use TTS tokenizers. Moving them to collections.common.tokenizers allows to write a wrapper which allows to use them directly in ASR and NLP datasets. New location also seems more logical as:
1) TTS tokenizers are the only tokenizers located outside of collections.common.tokenizers
2) TTS tokenizers are currently located in collections.tts.torch but the only torch function they use is torch.distributed

Ideally, I would recommend refactoring them around TokenizerSpec in the future to have unified implementation of all tokenizers.

**Collection**: [Note which collection this PR will affect]
collections.tts and collections.common.tokenizers

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Prompt Learning Notebook Bug Fix
State: closed | Pull request: yes

Prompt Learning Notebook Bug Fix

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Adds back ability to handle dataset example dicts directly passed into the .generate method for prompt learning

**Collection**: NLP

# Changelog 
- Updated GPTPromptLearningDataset class
- Updated GPTPromptLearningModel class's generate function
- Updated prompt learning unit test

# Usage

```python
test_examples = [
    {"taskname": "intent_and_slot", "utterance": "tell me who will win the next presidential election"},
    {"taskname": "intent_and_slot", "utterance": "i would like to pickup a veggie sub with a cookie from subway"},
    {"taskname": "intent_and_slot", "utterance": "email happy new year to john"},
    {"taskname": "intent_and_slot", "utterance": "set the alarm to seven am for work"},
    {"taskname": "sentiment", "sentence": "The products have a low salt and fat content ."},
    {"taskname": "sentiment", "sentence": "The agreement is valid for four years ."},
    {"taskname": "sentiment", "sentence": "Diluted EPS rose to EUR3 .68 from EUR0 .50 ."},
    {"taskname": "sentiment", "sentence": "The company is well positioned in Brazil and Uruguay ."},
    {"taskname": "sentiment", "sentence": "Profit before taxes decreased by 9 % to EUR 187.8 mn in the first nine months of 2008 , compared to EUR 207.1 mn a year earlier ."},
]

response = model.generate(inputs=test_examples, length_params=None)
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Data Simulator
State: closed | Pull request: yes

# What does this PR do ?

Adds a multispeaker audio session simulator and corresponding documentation and tutorials.

Collections: ASR, Tools

# Changelog 
- Adds data simulator
- Adds a manifest_utils file and restructures speaker_utils
- Adds config files for simulating data as well as a tutorial and documentation

# Before your PR is "Ready for review"
**Pre checks**:
- [ x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ x] New Feature
- [ ] Bugfix
- [ x] Documentation


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
[TTS] incremented the version number to 22.08 in tutorials.
State: closed | Pull request: yes

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[ASR] Add pretrained ASR models for Croatian
State: closed | Pull request: yes

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

# What does this PR do ?

This PR adds pre-trained models for Croatian.

**Collection**: [ASR]

# Changelog 
- updated `PretrainedModelInfo` and docs
- minor (cosmetic) modifications to `checkpoint_averaging.py`

# Usage
Tested with the following

```python
from nemo.collections.asr.models import ASRModel

for type in ['ctc', 'transducer']:
    model = ASRModel.from_pretrained(f'stt_hr_conformer_{type}_large', map_location='cpu')

    print(f'\nmodel_{type}.summarize()')
    print(model.summarize())
    print()
```

Pretrained models downloaded from NGC.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
What & where is the .nemo file?
State: closed | Pull request: no

Hello!

I would like to run the train_kenlm.py file but I do not understand all the arguments it requires: 

I read the documentation (https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/asr/asr_language_modeling.html and I ran install_beamsearch_decoders.sh as well, so I have the kenlm bin root but I have no idea how to get the root of the .nemo file and what it is about?

Thanks in advance!
[TTS] use consistent spline interpolation for fastpitch and hifigan.
State: closed | Pull request: yes


Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

Apply consistent spline interpolation when generating mel-spectrograms for finetuing hifigan.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[Speech Data Explorer] An option to explicitly specify the base dir
State: closed | Pull request: yes

[Speech Data Explorer] An option to explicitly specify the base dir for manifests with relative paths


**Collection**: Tools

# Changelog 
- Use `args.audio_base_path` instead of `args.manifest`
- If `args.audio_base_path` is not provided (`None`), then default to the location (parent dir) of the manifest

# Usage

```python
python data_explorer.py /path/to/a/modified/manifest.json --audio-base-path /path/to/audio_data
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] fixed wrong pronunciations for r1.11.
State: closed | Pull request: yes

# What does this PR do ?

Mirror the bugfix from https://github.com/NVIDIA/NeMo/pull/4669

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix Fastpitch Export
State: closed | Pull request: yes

# What does this PR do ?

Fix fastpitch export

**Collection**: TTS

# Changelog 
- Fix fastpitch export
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
Pynini dependency fix
State: closed | Pull request: yes

# What does this PR do ?

Switching to 2.1.5 pynini version, reverting to pynini install via conda, pip will still be used to Linux systems. Added guard to pytests to skip TN/ITN tests when pynini is not installed.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
upgrade to PTL 1.7
State: closed | Pull request: yes

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

# What does this PR do?

Upgrade PTL version to 1.7.2 

**Collection**: All

# Changelog 

-  Default max_steps is -1 now instead of None  
-  DDPPlugin has been moved from plugins to strategies and renamed as DDPStrategy -> renamed NLPDDPPlugin to NLPDDPStrategy
- override lightning module trainer property  to make it compatible with our existing models, since it's been removed from PTL 1.7 
- Following args have been removed from Trainer 

> - prepare_data_per_node
> - checkpoint_callback - replaced with enable_checkpointing
> - process_position - now part of TQDMprogressbar callback
> - stochastic_weight_avg - now part of callback
> - flush_logs_every_n_steps
> - weights_summary - replaced with callback
> - terminate_on_nan
> - log_gpu_memory - part of DeviceGPUStats callback
> - Progressbar refresh rate - part of TQDMProgressbar callback

TODO:
- [x] remove Cuda override function, once https://github.com/Lightning-AI/lightning/pull/14128 was released as part of 1.7.2 

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [x] Bugfix
- [x] Documentation
Removed trailing spaces in CI test
State: closed | Pull request: yes

Signed-off-by: Virginia Adams <vadams@nvidia.com>

The T5 prompt learning CI test had some formatting errors keeping the CI tests from running. This should fix them

Add a one line overview of what this PR aims to accomplish.

**Collection**: NLP

# Changelog 
- Updated Jenkins file

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Remove type annotation
State: closed | Pull request: yes

# What does this PR do ?

Remove type annotations in the `calc_length` function since it causes errors when scripting Conformer models.

**Collection**: ASR

This is the error:

```
[NeMo E 2022-08-03 11:47:46 exportable:116] jit.script() failed!\
    Variable 'add_pad' is annotated with type float but is being assigned to a value of type Tensor:
      File "/Users/allabana/.virtualenvs/meval/lib/python3.8/site-packages/nemo/collections/asr/parts/submodules/subsampling.py", line 149
    def calc_length(lengths, padding, kernel_size, stride, ceil_mode, repeat_num=1):
        """ Calculates the output length of a Tensor passed through a convolution or max pooling layer"""
        add_pad: float = (padding * 2) - kernel_size
        ~~~~~~~ <--- HERE
        one: float = 1.0
        for i in range(repeat_num):
```

This is an example usage that causes the error:

```py
pre_trained_model_name = "stt_en_conformer_ctc_small"
model = EncDecCTCModelBPE.from_pretrained(pre_trained_model_name, map_location='cpu')
model.eval()
model.export(f"/tmp/{model._get_name()}.ts", check_trace=True, try_script=True)
```

# Changelog 
- Remove type annotations for `add_pad`/`one` in `nemo.collections.asr.parts.submodules.subsampling.calc_length`
- Add type annotations to `calc_length` method args.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.