renwei2024/nemo-github-issues huggingface dataset:
Refactor bias act fusion
State: closed | Pull request: yes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Refactors bias activation fusion logic and adds the config option to the remaining yaml files.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Adds/replaces `bias_gelu_fusion` with `bias_activation_fusion` in remaining yaml configs.
- Refactors some of the logic within `transformer.py` and `lm_encoder_decoder.py`.

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Are there any documentation for applying inverse text normalization in another language?
State: closed | Pull request: no
Labels: name: feature

I am currently looking for a way to apply a state-of-the-art inverse text normalization method on Danish. 

I installed and tested NeMo for English and it works quite well. In the paper it is also mentioned that it is possible to apply the idea to a different language. 

But I cannot see any documentation/guide to be able to do so. Could you please help me out?
fixes invalid_ids error on Megatron decode function
State: closed | Pull request: yes
Labels: name: stale

Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com>

# What does this PR do ?

fixes invalid_ids error on Megatron decode function by changing invalid tokens to <unk>

**Collection**: NLP

# Changelog 
- small bug fix

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
fixes invalid_ids error on Megatron decode function
State: closed | Pull request: yes

# What does this PR do ?

fixes invalid_ids error on Megatron decode function

**Collection**: NLP

# Changelog 
- filter invalid tokens from the predicted tokens

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
Correct support for dataclasses in default module dim
State: closed | Pull request: yes

Signed-off-by: smajumdar <smajumdar@nvidia.com>

# What does this PR do ?

Fix issue with adapters where config is a dataclass and cannot be iterated on correctly.

**Collection**: [ASR]

# Changelog 
- Add a check and force cast to Dict Config for input to the function.


# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Docs for NeMo Adapters
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Core]

# Changelog 
- Adds documentation about NeMo adapters.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation
[TTS] bugfix for chars as inputs without is_phoneme entry.
State: closed | Pull request: yes

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

`is_phoneme may not exist in JSON files if only considering chars as inputs.`

**Collection**: [Note which collection this PR will affect]
`nemo.collections.tts.torch.data`

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Refactor for punctuation model
State: closed | Pull request: yes

# What does this PR do ?

Remove depreciated comments and implement unimplemented features

**Collection**: NLP

# Changelog 
- Removed mentions of `data_dir` from config as it doesn't exists anymore and changed it to `ds_item`
- Added the ability to add new punctuation marks to data preparation script `examples/nlp/token_classification/data/prepare_data_for_punctuation_capitalization.py `
- Removed unnecessary file extension check in `nemo/collections/nlp/data/token_classification/punctuation_capitalization_dataset.py`

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [X] New Feature
- [ ] Bugfix
- [X] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add wandb as dependency
State: closed | Pull request: yes

Signed-off-by: smajumdar <smajumdar@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Core]

# Changelog 
- Adds wandb as a dependency specifically for core

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.
Set headscale false
State: closed | Pull request: yes

# What does this PR do ?

Sets headscale=False by default in ParallelAttention

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Sets headscale=False by default in ParallelAttention

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Refactor caching for punctuation model
State: closed | Pull request: yes
Labels: name: bug

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?

Remove torch.distributed.is_initialized() guard from the dataset as it always returns False.  Try to load features only if the pkl file exists, else create the cache file.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
wav2vec ASR
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

I have been trying to get wav2vec ASR training up and running. I am mainly trying to use the model defined in

```
examples/asr/experimental/wav2vec/configs/wav2vecCTC.yaml
```

I also tried to get a model with wav2vec_pretrain.yaml running, but I get the same error as outlined below.

The call I use is:

```
python /workspace/nemo/examples/asr/asr_ctc/speech_to_text_ctc.py \
  --config-path=/workspace/nemo/examples/asr/experimental/wav2vec/configs \
  --config-name=wav2vecCTC \
  exp_manager.name=wav2vec \
  +exp_manager.exp_dir=/experiments \
  +exp_manager.version=20220613-1105 \
  exp_manager.resume_if_exists=true \
  exp_manager.resume_ignore_no_checkpoint=true \
  +exp_manager.checkpoint_callback_params.save_best_model=true \
  model.train_ds.manifest_filepath=/data/train.nemo \
  model.train_ds.batch_size=16 \
  model.train_ds.max_duration=17 \
  +model.train_ds.min_duration=1 \
  model.validation_ds.manifest_filepath=/data/validation.nemo \
  model.validation_ds.batch_size=16 \
  model.test_ds.manifest_filepath=/data/test_old.nemo \
  model.test_ds.batch_size=16 \
  model.tokenizer.dir=/data/out/tokenizer_spe_bpe_v128 \
  model.tokenizer.type=bpe
```

The error is get is:

```
KeyValidationError('ListConfig indices must be integers or slices, not str\n    full_key: [0][emb_dim]\n    object_type=list')
```

It seems like something is wrong with `emd_dim`. I leave this at its default in the `embedding_dim` parameter in the YAML file. Even if I set this explicitly I get the same error. Attached is also the full output of the call: [output_20220613-1105.txt](https://github.com/NVIDIA/NeMo/files/8890509/output_20220613-1105.txt).

Another minor thing I also noticed is that the command call crashes even earlier if I do not provide a test set. Based on the YAML file test set should not be mandatory as it is set to null and not ??.

I am aware that this is an experimental functionality, so any help will be greatly appreciated.

**Steps/Code to reproduce bug**

Full command call can be found in the previous section.

**Expected behavior**

I expect the wav2vec model to start the training process.

**Environment overview (please complete the following information)**

I am using the official NVIDIA pytorch 22.01 container, the used NeMo version is the current version of the master branch.

**Environment details**

I am using the official NVIDIA pytorch 22.01 container, the used NeMo version is the current version of the master branch.

**Additional context**

Will provided any additional info if required.
ITN doesn't work for a specific expression about time like 'ten to eleven'
State: closed | Pull request: no
Labels: name: feature

name: TN/ITN

**Is your feature request related to a problem? Please describe.**

I am trying to convert normalized asr model outputs into written form using _**inverse_normalizer.inverse_normalize**_. 
Whereas some intractable problems arose when dealing a specific expression about time like **"* to *"**.

When I convert this verbal form **"ten to eleven"**.
The ouput is **"ten to eleven"**  which is inconsistent with the expected output **"10:50"**.

If there is a sentence containing such expression about time, it will cause a more erroneous result.
For example:
The ITN output of **"set an alarm at ten to eleven pm"** is **''set an alarm at ten to 11:00 p.m."**

This is a general problem in my dataset.
I'd like to know how to solve this problem.
Should I add some specific custom rules?
Thanks a lot for any answer!

**Environment overview**
- Environment location: Docker
- Method of NeMo install: from source
Restore from W&B artifact
State: closed | Pull request: no
Labels: name: feature

**Is your feature request related to a problem? Please describe.**

I was wondering if it's possible to add first class support for W&B artifacts to the `modelPT.restore_from()` method?
Currently, I run something like the following if I want to download a NeMo model saved on W&B:

```py
from urllib.parse import urlparse

model_path = "wandb://{TEAM}/{PROJECT}/{MODEL_NAME}:{LABEL}"
url_path = urlparse(model_path)
if url_path.scheme == "wandb":
    artifact_name = f"{url_path.netloc}{url_path.path}"
    model_path = download_model_from_wandb(artifact_name, "EncDecCTCModel")
```

and `download_model_from_wandb` would look something like

```py
from pathlib import Path
import wandb
from nemo.utils import logging, model_utils

ARTIFACT_DIR = "/tmp"

def download_model_from_wandb(artifact_name: str, model_class: str) -> Path:
    api = wandb.Api()
    artifact = api.artifact(artifact_name)
    model_name = artifact.name.split(":")[0]  # Remove the tag/label
    model_name = Path(model_name).stem
    root = ARTIFACT_DIR / model_name
    nemo_path = root / f"{model_name}.nemo"
    artifact.download(root=root)
    return nemo_path
```

Maybe we could use some logic similar to the above with `restore_from()` to load the downloaded model?

**Describe the solution you'd like**

I would think that this could live in the [`restore_from()`](https://github.com/NVIDIA/NeMo/blob/main/nemo/core/classes/modelPT.py#L258) method itself, but it seems there's a new [`SaveRestoreConnector`](https://github.com/NVIDIA/NeMo/blob/main/nemo/core/connectors/save_restore_connector.py) so not sure if it's more logical to put something like this there.

**Additional context**

Can open a PR for this if it is welcome.
Make position embedding expansion specific to a batch to avoid checkpoint size mismatches
State: closed | Pull request: yes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

1. Makes position embedding expansion specific to a batch to avoid checkpoint size mismatches.
2. Logs a warning when position embeddings are being expanded.

**Collection**: NLP

# Changelog 
See "What does PR do" above.

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Raise error if trainer object is None for MegatronBaseModel
State: closed | Pull request: yes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Raises an error if a trainer is not passed to `MegatronBaseModel`

**Collection**: NLP

# Changelog 
- Raises an error if a trainer is not passed to `MegatronBaseModel`.

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix electronic bug, new time ITN rule
State: closed | Pull request: yes

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Auto-switch the input dimensions in the conformer encoder adapter to correct value
State: closed | Pull request: yes

Signed-off-by: Shantanu Acharya <shantanua@nvidia.com>

# What does this PR do ?

Fixed the issue where the auto-switching functionality of the `ConformerEncoderAdapter` input feature dimension was missing.

**Collection**: ASR

# Changelog 

File changed: `nemo/collections/asr/modules/conformer_encoder.py`

Function added: `_update_adapter_cfg_input_dim`
Updates the `in_features` value in the adapter config with the correct value i.e. `self.d_model`

Function modified: `add_adapter`
Calls the `self._update_adapter_cfg_input_dim` function to update the adapter input feature dimensions.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in NeMo ASR.
Text Normalization Corrupted Output
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

Tags are propagated to the output text during normalization.

**Steps/Code to reproduce bug**


```python
from nemo_text_processing.text_normalization import normalize

tn = normalize.Normalizer("lower_cased")

text = "google.com is related to apple.com"
print(tn.normalize(example))

"""
google dot com " right brace right brace TOKENS left brace NAME: "IS" right brace TOKENS left brace NAME: "RELATED" right brace TOKENS left brace NAME: "TO" right brace TOKENS left brace ELECTRONIC left brace DOMAIN: "APPLE dot COM
"""

example = "google.com is related to com.apple"
print(tn.normalize(example))

"""
google dot com is related to com.apple
"""


```
**Expected behavior**

The first example should return `google dot com is related to apple dot com`, the second one is ok as I believe one would need to add rules to handle such a "strange" case.

**Environment overview (please complete the following information)**

 - Environment location: [Bare-metal, Docker, Cloud(specify cloud provider - AWS, Azure, GCP, Collab)]
 - Method of NeMo install: `git clone https://github.com/NVIDIA/NeMo`
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version
- PyTorch version: 1.12.0
- Python version: 3.8.12

**Additional context**

Add any other context about the problem here.
Example: GPU model
Speedup the speech commands dataset processing script
State: closed | Pull request: yes

Signed-off-by: Shantanu Acharya <shantanua@nvidia.com>

# What does this PR do ?

Improves the speed of the google speech commands dataset processing script by adding multiprocessing support.

**Collection**: dataset processing scripts

# Changelog 
File changed: `scripts/dataset_processing/process_speech_commands_data.py`

Functions added:
- `__construct_filepaths`: Refactored the filepath construction of the audio samples into a separate function
- `__construct_silence_set`: Used for multiprocessing - Constructs the silence set
- `__rebalance_files`: Used for multiprocessing - rebalances the number of files in each class
- `__prepare_metadata`: Used for multiprocessing - create the manifest entry for a particular audio file

Functions modified: `__process_data`
- Accepts a new argument `num_processes` which determines the number of processes to use for multiprocessing
- Refactored the filepath construction, silence set construction, rebalancing and metadata construction logic to separate functions as described above

For writing the silence files, changed `librosa.output` to `soundfile.write` since in newer versions of librosa, the output module has been removed.

# Usage

```python
python process_speech_commands_data.py \
    --data_root=<absolute path to where the data should be stored> \
    --data_version=<either 1 or 2, indicating version of the dataset> \
    --class_split=<either "all" or "sub", indicates whether all 30/35 classes should be used, or the 10+2 split should be used> \
    --num_processes=<number of processes to use for data preprocessing> \
    --rebalance \
    --log
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the NeMo.
Create create_diarization_dataset_librispeech.py
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Which version of nemo to install in order to reduce dependencies?
State: closed | Pull request: no

I'm currently using nemo only for text normalization - I only import nemo_text_processing.text_normalization.normalize.Normalizer.

I install nemo toolkit using nemo-toolkit[all]==1.8.2. Is there any way to reduce the number of dependencies, for example by not installing [all]? I haven't found anything in the docs.

Thanks for any help in advance.
training self-supervised conformer
State: closed | Pull request: no
Labels: name: bug

Hi, I use `examples/asr/speech_pretraining/speech_pre_training.py` code to train a self-supervised using conformer on my custom dataset. I just changed the manifest files address in conf file. When I run the code, I encounter the below issue.
I use NeMo version 1.8.0. Also I test it on version 1.9.0 but the same problem occured.

**Environment details:**
torch 1.9.0a0+c3d40fd
pytorch-lightning 1.6.4
Python 3.8.10
NeMo 1.8.0

![Screenshot from 2022-06-07 14-22-48](https://user-images.githubusercontent.com/105767588/172352592-ed10430a-6e29-4723-a6e9-d9b7d96316d8.png)
Bits of RADTTS support
State: closed | Pull request: yes

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>

# What does this PR do ?

Those are modifications needed to support RadTTS net - which, for now, cannot be integrated into Nemo.
Adding those support bits to Nemo will make it easier to make internal RADTTS implementation a  Nemo extension, rather than a fork.   

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] enforced pin_memory = True
State: closed | Pull request: yes

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

The default `false` will slow down the data transfer to CUDA-enabled GPUs. See reference for details,

[1] https://discuss.pytorch.org/t/when-to-set-pin-memory-to-true/19723
[2] https://developer.nvidia.com/blog/how-optimize-data-transfers-cuda-cc/

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# if using x.cuda(), you should call x.cuda(non_blocking=True)
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Don't wrap model with ddp when using O2
State: closed | Pull request: yes

# What does this PR do ?

This PR checks if the NeMo Megatron model is using the O2 recipe. If so, we will not wrap the model with DDP.

When wrapping the model with DDP, gradient buckets are created which take up GPU memory. This is not needed since the O2 recipe creates its own buckets.

This should reduce memory used by NeMo Megatron.

**Collection**: NLP

# Changelog 
- Update `NLPDDPPlugin`.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Error in  multinode training
State: closed | Pull request: no
Labels: name: bug

**Following is the error log .please suggest the solution**  

[NeMo W 2022-06-04 14:58:08 exp_manager:895] The checkpoint callback was told to monitor a validation value and trainer's max_steps was set to -1. Please ensure that max_steps will run for at least 1 epochs to ensure that checkpointing will not error out.
Initializing distributed: GLOBAL_RANK: 0, MEMBER: 1/16
Initializing distributed: GLOBAL_RANK: 1, MEMBER: 2/16
Initializing distributed: GLOBAL_RANK: 1, MEMBER: 2/16
Initializing distributed: GLOBAL_RANK: 2, MEMBER: 3/16
Initializing distributed: GLOBAL_RANK: 3, MEMBER: 4/16
Initializing distributed: GLOBAL_RANK: 2, MEMBER: 3/16
Initializing distributed: GLOBAL_RANK: 4, MEMBER: 5/16
Initializing distributed: GLOBAL_RANK: 5, MEMBER: 6/16
Initializing distributed: GLOBAL_RANK: 4, MEMBER: 5/16
Initializing distributed: GLOBAL_RANK: 5, MEMBER: 6/16
Initializing distributed: GLOBAL_RANK: 3, MEMBER: 4/16
Initializing distributed: GLOBAL_RANK: 6, MEMBER: 7/16
Initializing distributed: GLOBAL_RANK: 7, MEMBER: 8/16
Initializing distributed: GLOBAL_RANK: 7, MEMBER: 8/16
Initializing distributed: GLOBAL_RANK: 6, MEMBER: 7/16
Error executing job with overrides: []
Traceback (most recent call last):
  File "/workspace/nemo/marathi/train_bpe.py", line 74, in main
    trainer.fit(asr_model)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 770, in fit
    self._call_and_handle_interrupt(
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 723, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 811, in _fit_impl
    results = self._run(model, ckpt_path=self.ckpt_path)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1171, in _run
    self.strategy.setup_environment()
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/strategies/ddp.py", line 152, in setup_environment
    self.setup_distributed()
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/strategies/ddp.py", line 205, in setup_distributed
    init_dist_connection(self.cluster_environment, self._process_group_backend)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/utilities/distributed.py", line 355, in init_dist_connection
    torch.distributed.init_process_group(torch_distributed_backend, rank=global_rank, world_size=world_size, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torch/distributed/distributed_c10d.py", line 610, in init_process_group
    _store_based_barrier(rank, store, timeout)
  File "/opt/conda/lib/python3.8/site-packages/torch/distributed/distributed_c10d.py", line 246, in _store_based_barrier
    raise RuntimeError(
RuntimeError: Timed out initializing process group in store based barrier on rank: 1, for key: store_based_barrier_key:1 (world_size=16, worker_count=8, timeout=0:30:00)
MTEncDecModel translate drops text, when input text is large
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

Is there a limit to the maximal text input length when calling `MTEncDecModel.translate()`? I mean the length of individual strings or size of List. Is it model config dependant? E.g. is it determined by `max_sequence_length`? If so, is this the number of tokens after tokenisation with `encoder_tokenizer` or the length of the string (chars with spaces)? I am asking this as I have noticed that when passing large text blocks (e.g. 3000 chars with spaces), the translation does work, but a large portion of input text is dropped.

**Steps/Code to reproduce bug**

Run `MTEncDecModel.translate` with list containing one large block of text.

**Expected behavior**

The entire input be translated, or a warning printed out that the output is/will be truncated.

**Environment details**

container:  `pytorch:22.04-py3` + `nemo:1.8.2 (./reinstal.sh)`
GPU: T4
Torchscripting Transducer model results in " Module 'FilterbankFeatures' has no attribute 'stft' "
State: closed | Pull request: no

```python
from nemo.collections.asr.models import (
    EncDecRNNTModel,
)
import torch.quantization.quantize_fx as quantize_fx

asr_model = EncDecRNNTModel.restore_from(<path>)

asr_model.eval()

quantized_model = torch.quantization.quantize_dynamic(
        asr_model, qconfig_spec={torch.nn.Linear}, dtype=torch.qint8, inplace=True
    )
scripted_model = torch.jit.script(quantized_model)
torch.jit.save(scripted_model, "hindi_medium_rnnt.pt")
 ```
 I am trying to convert a nemo rnnt transducer model to torchscript for testing mobile deployment and the torch.jit.script results in the following error:
 
```
Module 'FilterbankFeatures' has no attribute 'stft' (This function exists as an attribute on the Python module, but we failed to compile it to a TorchScript function.
The error stack is reproduced here:
local variable 'idx' referenced before assignment:
  File "/home/gowtham/anaconda3/envs/android-wsl/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 327
        # disable autocast to get full range of stft values
        with torch.cuda.amp.autocast(enabled=False):
            x = self.stft(x)
                ~~~~~~~~~ <--- HERE

        # torch returns real, imag; so convert to magnitude
```

1. I couldn't find lot of related issues online, so wanted to check if you have recommended ways to convert nemo conformed transducer models to torchscript? 

2. I also tried using [nemo_model.export](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/core/export.html) method  like this:
```python
quantized_model.export(
       "hindi_medium_rnnt.pt", try_script=True, verbose=True
    )
```
^ this also runs into multiple jit.script() failed errors but eventually produces encoder.pt and decoder-joint.pt files which are however not usable with torch.jit.load methods
Merge r1.9.0 main
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Torchaudio installation fix
State: closed | Pull request: yes

Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com>

# What does this PR do ?

Torch and torchaudio versions must match. Othervise, there will be no CUDA support for things like RNNT loss.
This PR fixes the issue.
Also, this PR automates picking the latest torchaudio release.

# Changelog 
- The commands in the Dockerfile are replaced with a separate script whick installs and tests torchaudio.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

@titu1994 @okuchaiev
Update container to 22.05
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Finetune T5 on the prefix-lm objective
State: closed | Pull request: yes

# What does this PR do ?

Adds a script and yaml config to fine-tune a pre-trained T5 model on the prefix-LM (LM adaptation) objective.

**Collection**: NLP

# Changelog 
- Add pre-training script and config

# Usage
* You can potentially add a usage example below

```python python megatron_t5_lm_adaptation_finetune.py -cn megatron_t5_lm_adaptation_finetune model.pretrained_model_path=/path/to/pretrained_t5.nemo model.data.data_prefix=.....```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fuse grad division into async grad allreduce
State: closed | Pull request: yes

# What does this PR do ?

Fuse gradient division into torch.distributed.allreduce(). 

# Changelog 
This change is made for async grad AR in class MainParamsOptimizerWrappe.

# Usage
**This fusion requires at least PyTorch 22.04.**
This fusion is enabled by default, you can turn it off by setting grad_div_ar_fusion=False

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)