renwei2024/nemo-github-issues huggingface dataset:
eval_beamsearch_ngram.py fails (pack_hypotheses invalid data type 'str')
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

I trained QuartzNet acoustic model + KenLM 2-ngram model, now I run eval_beamsearch_ngram.py to evaluate it. 

It seems like beam_search_func in BeamSearchDecoderWithLM outputs tuples (confidence, text) (the doc agrees), so cand.y_sequence is a resulting string. 

but later on the function pack_hypothesis in ctc_beam_decoding does "cand.y_sequence = torch.tensor(cand.y_sequence, dtype=torch.long)" and expects it to be int labels, so I get an error. Meanwhile greedy decoding works perfectly.

**Steps/Code to reproduce bug**

I am just following the steps here https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/asr/asr_language_modeling.html

python eval_beamsearch_ngram.py nemo_model_file=*quartznet_model_from_default_config* \
           input_manifest="" \
           kenlm_model_file=*scorer from train_kenlm* \
           beam_width=[500] \
           beam_alpha=[0.82] \
           beam_beta=[0.87] \
           preds_output_folder= \
           probs_cache_file=null \
           decoding_mode=beamsearch_ngram


**Expected behavior**

I expected the eval_beamsearch_ngram.py script work with default QuartzNet and default KenLM trained model. I am not exactly sure if I am not specifying the correct params for it. 

**Environment overview (please complete the following information)**

 - Environment location: linux
 - Method of NeMo install: pip install from tutorial

**Environment details**

Otherwise, please provide:
- OS version: ubuntu 18.04.5 LTS (Bionic Beaver)
- PyTorch version 1.13.1
- Python version 3.8.12

**Additional context**

nemo toolkit 1.16.0rc0
ctc-decoders version is 1.1
Remove 8k model mention and indicate training time
State: closed | Pull request: yes

# What does this PR do ?

Remove 8k model mention and indicate estimated training time for NeMo LLM dataset preparation tool.

Also adds a model flag to recommend_hyperparameters function, which is not used now but can be used to recommend hyperparameters based on model chosen later on.

**Collection**: None

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] update German NGC models trained on Thorsten Datasets
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

* deprecated the first german single-speaker fastpitch+hifigan model that hacked `GermanCharTokenizer` by manually mixing graphemes and phonemes. This model was trained on Thorsten's neutral 21.02 dataset.
* added two new fastpitch+hifigan models that applied the refined `GermanCharsTokenizer` that only uses grapheme-only tokens. These two models were trained on Thorsten's neutral 21.02 and 22.10 (newly released by Thorsten), respectively.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix k2 and torchaudio installation (Docker, macOS). Cherry-pick (#6094)
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Cherry-pick from #6094 

Fix `torchaudio` installation, fix `k2` installation in some cases. Make `torchaudio` and `k2` not required by default.

**Collection**: [CI]

# Changelog 

- add build arguments `REQUIRE_TORCHAUDIO` and `REQUIRE_K2` for `torchaudio` and `k2` in Dockerfile
  - `false` (by default): library will be installed if possible, errors ignored
  - `true`: if library not installed correctly, end build with error
- fix `k2` installation script on macOS
- fix `k2` installation in Docker (`k2` depends on `torchaudio`)
- fix `torchaudio` installation for latest PyTorch versions
- also fixes bug https://nvbugswb.nvidia.com/NvBugs5/SWBug.aspx?bugid=3940510&cmtNo= (`apt update && apt install -y libavdevice-dev` should be also used before executing install script)

# Usage

```shell
# build container, ignore k2/torchaudio installation error
docker build --tag "nemo:latest" .

# ignore k2 installation error, fail on torchaudio installation error
docker build --tag "nemo:latest" --build-arg REQUIRE_TORCHAUDIO=true --build-arg REQUIRE_K2=false .
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Move Spanish polyphones from heteronym to dictionary
State: closed | Pull request: yes

# What does this PR do ?

Spanish does not have heteronyms, only polyphones (eg. region-specific pronunciations of the same word). To make this clearer, I am removing the heteronyms file and placing the alternative pronunciations into the dictionary.

**Collection**: [TTS]

# Changelog 
- Delete es_ES and es_LA heteronym files
- Move words in heteronym files to pronunciation dictionary with multiple entries.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
CS bugfix
State: closed | Pull request: yes
Labels: name: bug

name: ASR

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.
Fixes bug in processing multilingual / codeswitched tarred datasets

**Collection**: [Note which collection this PR will affect]
ASR
# Changelog 
- Add specific line by line info of high level changes in this PR
- isinstance(args[0], Iterable) replaced by List
- Because a string is an Iterable!

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
tacotron2.yaml config file buggy
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**
When initiating tacotron.py for training, the following error comes up:

Error executing job with overrides: []
Error in call to target 'nemo.collections.tts.data.tts_dataset.TTSDataset':
FileNotFoundError(2, 'No such file or directory')
full_key: train_ds.dataset

**Steps/Code to reproduce bug**

python tacotron2.py

(hardcoded the training and validation dataset paths in tacotron2.yaml file but also tried giving the path from the cli)

**Expected behavior**

Training starts.

**Environment overview (please complete the following information)**

 - Environment location: on-site desktop
 - Method of NeMo install: tries installing both with pip and from source.
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version: Ubuntu 20.04 and Windows 11 (dual-boot)
- PyTorch version: 1.13.1
- Python version: 3.8.0

**Additional context**

Add any other context about the problem here.
Example: GPU model
Adjust stochastic depth dropout probability calculation
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do?

This PR adjusts calculation of stochastic depth dropout probability.
Currently, calculation fails for 1-layer ConformerEncoder with the default setup.

The culprit is calculation of the probability in `compute_stochastic_depth_drop_probs ` for `linear` mode:

```
layer_drop_probs += [l / (L - 1) * stochastic_depth_drop_prob for l in range(L)]
```

since in this case `L-1 == 0`.

## Proposed solution

Change calculation for the linear mode to the following
```
# we start with 1/L * drop_prob and and end with the desired drop probability.
layer_drop_probs += [l / L * stochastic_depth_drop_prob for l in range(1, L + 1)]
```

This can handle setup with a single layer and is consistent with the probability calculation for the `uniform` mode.

### Example
An example of calculated probabilities:

```
num_layers = 7
stochastic_depth_start_layer = 3
stochastic_depth_drop_prob = 0.5
```

| layer |  0  |  1  |  2  |  3  |  4  |  5  |  6  |
| ----- | --- | --- | --- | --- | --- | --- | --- |
| uniform         | 0.0 | 0.0 | 0.0 | 0.5   | 0.5   | 0.5   | 0.5 |
| linear original | 0.0 | 0.0 | 0.0 | 0.0   | 0.167 | 0.333 | 0.5 |
| linear proposed | 0.0 | 0.0 | 0.0 | 0.125 | 0.25  | 0.375 | 0.5 |



**Collection**: ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[QUERY] : How to extract embedding of an audio file?
State: closed | Pull request: no

Hi, first of all, thanks for the amazing toolkit. 
My query is pretty basic. I am trying to extract the embeddings of an audio file after passing it through a fine-tuned .nemo model. I saw that this support is available for [`Speaker Identification task`](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/asr/speaker_recognition/results.html#speaker-embedding-extraction). How can I emulate it for ASR? 

My trained model is a `Conformer CTC BPE` type model.
Ngram lm fusion for RNNT maes decoding
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Ngram LM fusion for RNNT modified adaptive expansion search (maes) decoding.

**Collection**: [ASR]

# Changelog 
- Add ngram_lm option to maes decoding algorithm 
- Add EncDecRNNTBPEModel and EncDecRNNTModel to kenlm_utils.py for ngram lm building
- Add new file eval_beamsearch_ngram_transducer.py (base on old Vahid's branch for ngram rnnt beam search) for testing the ngram RNNT decoding

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Fix enhancer usage in ASR-TTS examples
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Fixes enhancer usage in ASR-TTS examples

**Collection**: [ASR]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix enhancer usage in ASR-TTS examples
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fixes enhancer usage in ASR-TTS examples

**Collection**: [ASR]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Save model parallel .nemo in ExpManager
State: closed | Pull request: yes
Labels: name: core

# What does this PR do ?
save_best_model and always_save_nemo will work with tp/pp > 1 with this PR.

**Collection**: [NLP,ASR]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fixing data simulator argument and bash scripting error
State: closed | Pull request: yes
Labels: name: cherry-pick

name: Speaker Tasks

# What does this PR do ?

Repeating https://github.com/NVIDIA/NeMo/pull/6087 for r1.16.0

This PR fixes a line containing bash command "mv".
The line is unnecessary and creating an error.
Also added "rm" command that removes tar file, in case where users run the cell again.


**Collection**: [Note which collection this PR will affect]
ASR

# Changelog 
Removed "mv" line.


# Usage
This is a ipython notebook.
You can run it at `tutorials/speaker_tasks/Speaker_Diarization_Training.ipynb`.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS][refactor] Part 3 - nemo.collections.tts.g2p.models
State: closed | Pull request: yes
Labels: name: TTS

name: CI

# What does this PR do ?

Task 3: break down nemo.collections.tts.g2p.models and save to nemo.collection.tts.models 
  - base.py   # move and merge tts/g2p/models/g2p_model.py::G2PModel class here because it is the base model for g2p.
  - g2p_ctc.py  # move and rename tts/g2p/models/ctc_g2p.py here. File name uses the same format as fastpitch_*.py
  - g2p_t5.py  # move and rename tts/g2p/models/t5_g2p.py here.
  - g2p_heteronym_classification.py  # move and rename tts/g2p/models/heteronym_classifiation.py here.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fixing data simulator argument and bash scripting error
State: closed | Pull request: yes
Labels: name: Speaker Tasks

# What does this PR do ?

Repeating https://github.com/NVIDIA/NeMo/pull/6087 for r1.16.0

This PR fixes a line containing bash command "mv".
The line is unnecessary and creating an error.
Also added "rm" command that removes tar file, in case where users run the cell again.


**Collection**: [Note which collection this PR will affect]
ASR

# Changelog 
Removed "mv" line.


# Usage
This is a ipython notebook.
You can run it at `tutorials/speaker_tasks/Speaker_Diarization_Training.ipynb`.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
update readme with new badges
State: closed | Pull request: yes

# What does this PR do ?

This new badges will look like below,
1. added |pypi| |pyversion| |codeql||downloads|;
2. remove lgtm because it is deprecated. 

![Screen Shot 2023-02-24 at 11 19 11 AM](https://user-images.githubusercontent.com/1646669/221271758-3b93737a-d421-474f-8833-a2f5e81bfdf3.png)

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```



# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add missing parameters to ModelCheckpoint CallbackParams
State: closed | Pull request: yes
Labels: name: stale

# What does this PR do ?
Adds missing parameters to ModelCheckpoint CallbackParams, to achieve feature parity with PTL. See ModelCheckpoint API here: https://pytorch-lightning.readthedocs.io/en/stable/api/pytorch_lightning.callbacks.ModelCheckpoint.html

**Collection**: exp_manager

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
P-tuning refactor Part 3/N 
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS][refactor] Part 2 - nemo.colletions.tts.parts
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Task 2: re-organize nemo.collection.tts.parts
  - utils
    - tts_dataset_utils.py  # move torch/helpers.py here; merge tts/data/data_utils.py here.
    - g2p_utils.py  # move g2p/data/data_utils.py here * Evelina may comment.
    - helpers.py  # move tts/helpers/helpers.py here. Note that copyright header mixed with BSD 3-Clause. So not sure how to merge to ones with only Nvidia copyright. 
    - splines.py  # move tts/helpers/splines.py here. Same as above, it mixed Nvidia and MIT copyrights.

  - preprocessing
    - audio_trimmer.py  # move data/audio_trimmer.py here

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Cuda OutOfMemory issue when following Colab tutorial
State: closed | Pull request: no
Labels: name: bug

name: stale

Hi,
I am trying to follow the tutorial at: [Speaker_Identification_Verification.ipynb](https://github.com/NVIDIA/NeMo/blob/main/tutorials/speaker_tasks/Speaker_Identification_Verification.ipynb)

I create a new Colab environment with 15GB GPU attached and run all of the cells in that sheet. It runs until cell 21, which is the command:
`trainer.fit(speaker_model)`

That is where I get this error:
`OutOfMemoryError: CUDA out of memory. Tried to allocate 468.00 MiB (GPU 0; 14.76 GiB total capacity; 13.17 GiB already allocated; 47.88 MiB free; 13.59 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF`
[TTS][refactor] Part 6 - remove nemo.collections.tts.torch.README.md and tts_dataset.yaml
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Task 6: remove nemo/collection/tts/torch/README.md and tts_dataset.yaml.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS][refactor] Part 1 - nemo.collections.tts.data
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

This is a series of refactoring tasks forth. 
Task 1: re-organize nemo.collections.tts.data
```
tts_dataset.py  # move torch/data.py here
g2p_dataset.py  # combine g2p/data/ctc_g2p.py, g2p/data/t5_g2p.py, and g2p/data/heteronym_classification_data.py

# 2. update import paths.
```

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS][refactor] Part 7 - move module from model file.
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Task 7: 

1. remove duplicate import of `torch`.
2. `GreedyCTCDecoder` is actually a module and should move to nemo/collection/tts/modules.
3. Lack of exposure from import (see below graph). For example, `fastpitch_ssl` and `ssl_tts` should be added into `__init__.py`.
![packages_tts_models](https://user-images.githubusercontent.com/1646669/221059764-398ddb9f-6f43-46d7-993f-16f1a5a7ce87.png)

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Spectrogram Enhancer: support arbitrary input length
State: closed | Pull request: yes
Labels: name: bug

name: TTS

name: cherry-pick

# What does this PR do ?

Make the spectrogram enhancer work on arbitrary length spectrograms irrespective of lengths during training. This a convolutional model (=locally-connected network), really no reason why it should not support arbitrary lengths

Original StyleGAN 2 architecture starts from a fixed-size noise-initialized tensor and generate images. To support arbitrary length spectrograms, the architecture was modified to start from a rather long tensor and chop off part of necessary length early on. This was not a good decision.

This PR makes the move from a long starting tensor to a tensor with a fixed length of 1.

**Collection**: TTS

# Changelog 
- Change length of the starting ("initial") tensor in the enhancer to 1
- Remove `max_spectrogram_length` from the model and the config

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Fixes #6007
[TTS] Create skeleton for TTS inference API
State: closed | Pull request: yes
Labels: name: TTS

name: stale

# What does this PR do ?

Add prototype implementation of TTS pipeline that is intended to be used for inference before and after model training.

These are far from finalized. The intent is for this to be experimental and to slowly incorporate them into our inference workflows (eg. for preprocessing, evaluation, analysis, deployment) and expand and iterate on the interfaces and design as we figure out what needs to be changed.

**Collection**: [TTS]

# Changelog
- Add TTS pipeline and module definitions and documentation.
- Add module implementations using existing backends.
- Add unit and functional tests for all modules. 

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Megatron Encoder-Decoder Sampler Function
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

This PR adds support in external sampler function to Megatron encoder-decoder.

**Collection**: [Note which collection this PR will affect]

# Changelog 

1. Added support in a custom token sampling function to `MegatronLMEncoderDecoderModel.decode`.
2. Added greedy and top k / top p token sampling methods.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix k2 and torchaudio installation (Docker, macOS)
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do?

- add build arguments `REQUIRE_TORCHAUDIO` and `REQUIRE_K2` for `torchaudio` and `k2` in Dockerfile
  - `false` (by default): library will be installed if possible, errors ignored
  - `true`: if library not installed correctly, end build with error
- fix `k2` installation script on macOS
- fix `k2` installation in Docker (`k2` depends on `torchaudio`)
- fix `torchaudio` installation for latest PyTorch versions
- also fixes bug https://nvbugswb.nvidia.com/NvBugs5/SWBug.aspx?bugid=3940510&cmtNo= (`apt update && apt install -y libavdevice-dev` should be also used before executing install script)

NB: This PR doesn't fix `k2` + `Cuda 12.0` compilation errors.

**Collection**: [CI]

# Changelog 
- `k2` installation script is compatible with macOS
- `k2` installation script fails if error occurred during installation
- `Dockerfile`: add build arguments (see description above)
- `torchaudio` installation script - fixes for latest PyTorch versions (ffmpeg, match release branch, correct version)


# Usage
```shell
# build container, ignore k2/torchaudio installation error
docker build --tag "nemo:latest" .

# ignore k2 installation error, fail on torchaudio installation error
docker build --tag "nemo:latest" --build-arg REQUIRE_TORCHAUDIO=true --build-arg REQUIRE_K2=false .
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Improving text memmap generated index files error messages
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

1. Improved testing of text memmap generated index files, and improved error message when files are missing.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add batch_size to prompt_learning generate
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Adds batch size to generate function in prompt_learning class
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add batch_size to prompt_learning generate
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Adds batch size to generate function in prompt_learning class
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix enhancer usage
State: closed | Pull request: yes
Labels: name: TTS

name: cherry-pick

name: ASR

# What does this PR do ?

Fix enhancer usage in ASR-TTS models

**Collection**: [ASR, TTS]

# Changelog 
- suppress type checking for enhancer usage

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)