renwei2024/nemo-github-issues huggingface dataset:
Fix tokenizer file caching where torch.distributed may not be initialized yet
State: closed | Pull request: yes
Labels: name: NLP

Other ranks with multi-node training may reach this code before rank 0 and torch.distributed has not been initialized and does not wait for download

# What does this PR do ?

Fix to be flexible to download by other ranks if not initialized and errors looking for file that does not exist yet

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage

```
  File "/opt/NeMo/nemo/collections/nlp/modules/common/tokenizer_utils.py", line 207, in get_nmt_tokenizer
    return get_tokenizer(tokenizer_name=model_name, vocab_file=vocab_file, merges_file=merges_file)
  File "/opt/NeMo/nemo/collections/nlp/modules/common/tokenizer_utils.py", line 134, in get_tokenizer
    return AutoTokenizer(
  File "/opt/NeMo/nemo/collections/common/tokenizers/huggingface/auto_tokenizer.py", line 81, in __init__
    raise ValueError(
ValueError: Unable to instantiate HuggingFace AUTOTOKENIZER for gpt2. Exception: Unable to load vocabulary from file. Please check that the provided vocabulary is accessible and not corrupted.
[NeMo I 2023-07-17 16:42:10 megatron_init:297] Rank 0 has embedding rank: 0
23-07-17 16:42:10 - PID:215 - rank:(0, 0, 0, 0) - microbatches.py:39 - INFO - setting number of micro-batches to constant 1
[NeMo I 2023-07-17 16:42:10 tokenizer_utils:204] Getting Megatron tokenizer for pretrained model name: megatron-gpt-345m, custom vocab file: None, and merges file: None
[NeMo I 2023-07-17 16:42:10 megatron_utils:208] Downloading from https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-vocab.json to /mount/workspace/megatron/megatron-gpt-345m_vocab
[NeMo I 2023-07-17 16:42:10 megatron_utils:208] Downloading from https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-merges.txt to /mount/workspace/megatron/megatron-gpt-345m_merges
[NeMo I 2023-07-17 16:42:10 tokenizer_utils:130] Getting HuggingFace AutoTokenizer with pretrained_model_name: gpt2, vocab_file: /mount/workspace/megatron/megatron-gpt-345m_vocab, merges_files: /mount/workspace/megatron/megatron-gpt-345m_merges, special_tokens_dict: {}, and use_fast: False
```

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #7016
PyYAML<6 breaks the installation on colab
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

PyYAML<6 in requirements breaks the installation on Google Colab. Removing the PyYAML<6 requirement from requirements/requirements_lightning.txt solves the problem.

**Steps/Code to reproduce bug**

Just run the first cell of https://github.com/NVIDIA/NeMo/blob/stable/tutorials/nlp/Punctuation_and_Capitalization.ipynb

**Expected behavior**

Successful installation.

**Environment overview (please complete the following information)**

 - Environment location: Google Colab
 - Method of NeMo install: pip
Fix typo and branch in tutorial
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Fixes typo in ASR-TTS tutorial.
Also changes branch to r1.20.0 for installation.

Add a one line overview of what this PR aims to accomplish.

**Collection**: [ASR]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
mixed task inference batching
State: closed | Pull request: yes
Labels: name: stale

name: NLP

name: CI

# What does this PR do ?

- Support to allow mixed tasks into a single inference batch.
- Supports layer selection for LoRa and Adapters
- Supports weight tying for LoRa and Adapters

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Getting "view size is not compatible with input tensor's size and stride" due to missing ".contiguous()" in megatron-core dependency
State: closed | Pull request: no
Labels: name: bug

name: stale

Trying the latest `stable`  Nemo release on wikitext by following the README instructions I run into the following error

```
  File "/disks/yaroslav/envs/nemo2/lib/python3.9/site-packages/torch/cuda/amp/autocast_mode.py", line 123, in decorate_bwd
    return bwd(*args, **kwargs)
  File "/disks/yaroslav/envs/nemo2/lib/python3.9/site-packages/megatron/core/tensor_parallel/layers.py", line 284, in backward
    total_input = total_input.view(total_input.shape[0] * total_input.shape[1],
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.
```

The work-around is to open `/disks/yaroslav/envs/nemo2/lib/python3.9/site-packages/megatron/core/tensor_parallel/layers.py file and change line 224 from`

```
        total_input = total_input.view(total_input.shape[0] * total_input.shape[1],
                                       total_input.shape[2])
```
to instead read

```
        total_input = total_input.contiguous()
        total_input = total_input.view(total_input.shape[0] * total_input.shape[1],
                                       total_input.shape[2])
```

This file is part of the following package:

```
megatron-core             0.2.0                    pypi_0    pypi
```
Remove pyyaml
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

Remove PyYAML 

**Collection**: [Core]

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Refined export_config
State: closed | Pull request: yes
Labels: name: TTS

name: ASR

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Remove pyyaml
State: closed | Pull request: yes

# What does this PR do ?

Remove PyYAML 

**Collection**: [Core]

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Fix PyYAML
State: closed | Pull request: yes

# What does this PR do ?

Fixes broken PyYAML installation by fixing the version <= 5.3.1

See the issue: https://github.com/yaml/pyyaml/issues/724

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix Cython build dependency
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix typo in ASR-TTS tutorial
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fix typo in ASR-TTS tutorial (for main branch)

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix typo and branch in tutorial
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fixes typo in ASR-TTS tutorial.
Also changes branch to r1.20.0 for installation.

Add a one line overview of what this PR aims to accomplish.

**Collection**: [ASR]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fixed tutorial's name
State: closed | Pull request: yes

# What does this PR do ?

Fixes ASR-TTS tutorial name in tutorials.rst

**Collection**: docs

# Changelog 
- Fixed tutorial's name

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [X] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [X] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
Problems Installing
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

I have been using NeMo in google collab and after I disconnected and deleted the runtime and moved over to a new account when I ran out of free GPU time, I haven't been able to install NeMo since, not even on my host computer.

**Steps/Code to reproduce bug**
>Input
```
#@title
!nvidia-smi
!python -m pip install --upgrade pip
!pip install torch
#Set the NeMo branch or version
%env BRANCH=main
# If you're using Colab and not running locally, uncomment and run this cell.
%rm -rf sample_data
!apt-get update && apt-get install -y libsndfile1 ffmpeg
!pip install Cython
!python -m pip install git+https://github.com/NVIDIA/NeMo.git@$BRANCH#egg=nemo_toolkit[all]
```
>Ouput
```
Collecting wandb (from nemo_toolkit[all])
  Obtaining dependency information for wandb from https://files.pythonhosted.org/packages/87/26/66e944b17aa06a4c9df9850a6a5c56378cb4f9b3acf3452ace7dfd895c13/wandb-0.15.5-py3-none-any.whl.metadata
  Downloading wandb-0.15.5-py3-none-any.whl.metadata (8.2 kB)
Collecting hydra-core<1.3,>=1.2.0 (from nemo_toolkit[all])
  Downloading hydra_core-1.2.0-py3-none-any.whl (151 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.1/151.1 kB 307.1 MB/s eta 0:00:00
Collecting omegaconf<2.3,>=2.2 (from nemo_toolkit[all])
  Downloading omegaconf-2.2.3-py3-none-any.whl (79 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.3/79.3 kB 228.3 MB/s eta 0:00:00
Collecting pytorch-lightning<=1.9.4,>=1.9.0 (from nemo_toolkit[all])
  Downloading pytorch_lightning-1.9.4-py3-none-any.whl (827 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 827.8/827.8 kB 361.7 MB/s eta 0:00:00
Collecting pyyaml<6 (from nemo_toolkit[all])
  Downloading PyYAML-5.4.1.tar.gz (175 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 175.1/175.1 kB 337.4 MB/s eta 0:00:00
  Installing build dependencies ... done
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
```

**Expected behavior**

NeMo to install build dependencies

**Environment overview**

 - Environment location: [Google Collab](https://colab.research.google.com/drive/10d36J0nbFh6hdwFh_M90Y-oPrf5TBB3C?usp=sharing)
 - Method of NeMo install: [Pip from source](https://github.com/NVIDIA/NeMo/tree/main#pip-from-source)

**Environment details**

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.6 LTS"
PyTorch 2.0.1+cu118
Python 3.10.12
Support for byte streams or other formats than .wav files
State: closed | Pull request: no
Labels: name: stale

**Is your feature request related to a problem? Please describe.**
Using nemo toolkit in a production environment is not really an option, since you usually don't work
with files but streams, byte arrays etc. This is problematic since all classes and functions in Nemo
expect .wav files to work with.

**Describe the solution you'd like**
Change classes and functions to accept additional file formats like IOBase.

**Describe alternatives you've considered**

Rewriting the repo, but since this is a massive change it would invalidate any future versions of Nemo.
Get the confidence value for the classified output
State: closed | Pull request: no
Labels: name: stale

Hello,

I am trying to get the confidence value with which the output labels are generated. Like for example, given a *.wav as input, I am able to extract some label from the model. But I would like to know the confidence level with which the labels are extracted for a single audio. I would need this information for some kind of thresholding in my application. 

Please suggest some ideas on how to get that information. Also please do comment if you need more information.
A little confusion about learning rate decay parameters for fastpitch and hifigan
State: closed | Pull request: no
Labels: name: TTS

name: stale

Thanks for the continual development and maintenance of this code. 
After modifying min_lr and max_epoch from default, I trained fastpitch for 500 epochs and hifigan for 52 epochs on my dataset, which consists of 25,584 audio clips totaling 44 hours. I found no decay in the learning rate.

fastpitch:
![fastpitch](https://github.com/NVIDIA/NeMo/assets/29267562/3bcd562f-2c20-4c47-89a9-ef42001b8bb2)
```
  optim:
    name: adamw
    lr: 1e-3
    betas: [0.9, 0.999]
    weight_decay: 1e-6

    sched:
      name: NoamAnnealing
      warmup_steps: 1000
      last_epoch: -1
      d_model: 1
      min_lr: 2e-4   #modified,default N/A

trainer:
  num_nodes: 1
  devices: 1
  accelerator: gpu
  strategy: null    #modified,default ddp
  precision: 16
  max_epochs: 500    #modified,default 1000
  accumulate_grad_batches: 1
  # gradient_clip_val: 1000.0
  enable_checkpointing: False
  logger: false
  log_every_n_steps: 5
  check_val_every_n_epoch: 1
  benchmark: false
  num_sanity_val_steps: 0
```

hifigan:
![hifigan](https://github.com/NVIDIA/NeMo/assets/29267562/6d22a307-d91a-4f0f-9c44-1fb8c2f217d5)
```
  optim:
    _target_: torch.optim.AdamW
    lr: 0.0002
    betas: [0.8, 0.99]

    sched:
      name: CosineAnnealing
      min_lr: 1e-5
      warmup_ratio: 0.02

  max_steps: 2500000
  l1_loss_factor: 45
  denoise_strength: 0.0025

trainer:
  num_nodes: 1
  devices: 1
  accelerator: gpu
  strategy: ddp
  precision: 32
  max_steps: ${model.max_steps}
  accumulate_grad_batches: 1
  enable_checkpointing: False  # Provided by exp_manager
  logger: false # Provided by exp_manager
  log_every_n_steps: 100
  check_val_every_n_epoch: 5
  benchmark: false
  max_epochs: 100    #modified,default N/A
```

Although the default hifigan configuration also specifies a min_lr, it still reaches the 2e-4 rate set by adamw , which my fspitch setting couldn't. Could this be due to the difference between the cosine and noam decay strategies?

Moreover, I am aware that the maximum steps for the default hifigan exceed what my dataset can reach. Nevertheless, the results obtained without learning rate decay were not disappointing. I would like to know if the default setting expects annealing to be performed?
Training Kerple Parameters with Constraints
State: closed | Pull request: no
Labels: name: bug

name: stale

Hi,

I see that NeMo implements [Kerple](https://arxiv.org/pdf/2205.09921.pdf). According to the paper, there are some constraints on the valid values for its learnable parameters. For example, [`self.kerple_p`](https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/nlp/modules/common/megatron/position_embedding/kerple_relative_position_embedding.py#L69) needs to fall into `(0, 2]`. Though there is no details on how the authors post such constraint during the training, I would expect we need to use projected gradient descent type methods to learn these parameters. But I don't see such related code in NeMo. Do I miss anything? Thanks.
Add support for Numba FP16 RNNT Loss (#6991)
State: closed | Pull request: yes
Labels: name: core

name: ASR

Cherry pick RNNT FP 16 Compat to NeMo 1.20

Signed-off-by: smajumdar <titu1994@gmail.com>

# What does this PR do ?

Adds Numba FP 16 support to NeMo 1.20

**Collection**: [ASR]

# Usage

## Install dependencies
[Numba](https://github.com/numba/numba) 0.57+ (conda install numba=0.57.1 -c conda-forge)
[CUDA Python](https://nvidia.github.io/cuda-python/install.html)

After that, train RNNT model with trainer.precision=16

```python
# Set environment variables 
# export NUMBA_CUDA_USE_NVIDIA_BINDING=1
# export STRICT_NUMBA_COMPAT_CHECK=0

from nemo.core.utils import numba_utils

print(numba_utils.numba_cuda_is_supported(numba_utils.__NUMBA_MINIMUM_VERSION_FP16_SUPPORTED__))  # Should be true

# Should also be True. 
# You can also pass True as an argument to this function to possibly get a reason as to why FP 16 is not supported.

print(numba_utils.is_numba_cuda_fp16_supported())
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
T5 metrics fix
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
TE bug fix
State: closed | Pull request: yes
Labels: name: bug

name: cherry-pick

name: stale

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
How to avoid downloading from huggingface
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**
I've made fine-tuning custom Bert model from huggingface for TokenClassification, saved the best checkpoint as .nemo file as the result.
Then I want to load that checkpoint for model inference 
```
model = TokenClassificationModel.restore_from("/tmp/model/best_ckpt.nemo")
```
but at first nemo downloads the original model from HF 
```
[NeMo W 2023-07-14 03:24:30 lm_utils:80] ai-forever/ruBert-large is not in get_pretrained_lm_models_list(include_external=False), will be using AutoModel from HuggingFace.
Downloading: 100%|██████████| 1.71G/1.71G [01:12<00:00, 23.5MB/s] 
```
and only then restores it from the checkpoint. 
```
[NeMo I 2023-07-14 03:26:04 save_restore_connector:247] Model TokenClassificationModel was successfully restored from /tmp/model/best_ckpt.nemo
```
Is it necessary do download the original model from HF? I can't find the way to avoid it
Fix caching bug in causal convolutions for cache-aware ASR models
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fixed the caching bug in the convolution layers for cache-aware models

# Changelog 
- Fixed the caching bug in the convolution layers for cache-aware models

**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation
Added bool types to neural_types export
State: closed | Pull request: yes
Labels: name: core

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.
adds 'BoolType' to exportables from `neural_type.elements`

**Collection**: [Note which collection this PR will affect]
Core


# Before your PR is "Ready for review"
**Pre checks**:
- [Y ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ N] Did you write any new necessary tests?
- [ N] Did you add or update any necessary documentation?
- [ N] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ Y] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
bug fix of initializing NLPDDPStrategy class
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

fix issue https://github.com/NVIDIA/NeMo/issues/7025


**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update SDP docs page with a new documentation link
State: closed | Pull request: yes

# What does this PR do ?

Update SDP docs page with a new documentation link.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix install_beamsearch_decoders.sh
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

fix install_beamsearch_decoders.sh

**Collection**: ASR

# Changelog 
- install_beamsearch_decoders.sh work with and without root user
- provide NEMO_PATH in the tutorial Offline_ASR.ipynb

**PR Type**:
- [V ] Bugfix

# Additional Information
* Related to # 4186721
TE bug fix
State: closed | Pull request: yes
Labels: name: bug

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix missing import for GPT SFT
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
NLPDDPStrategy class init function incorrectly passing args to parent DDPStrategy class 
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

The [init function](https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/nlp/parts/nlp_overrides.py#L94) of NLPDDPStrategy class is passing args 
```
parallel_devices, cluster_environment, checkpoint_io
```
without `key=val` pattern to parent class. But the [parent class DDPStrategy](https://github.com/Lightning-AI/lightning/blob/master/src/lightning/fabric/strategies/ddp.py#L54-L57) is expecting the args in the order of 
```
accelerator: Optional[Accelerator] = None,
parallel_devices: Optional[List[torch.device]] = None,
cluster_environment: Optional[ClusterEnvironment] = None,
checkpoint_io: Optional[CheckpointIO] = None,
```

This caused a bug of custom checkpoint_io instance incorrectly being assigned to cluster_environment var.

**Steps/Code to reproduce bug**


```
s3_checkpoint_io = S3CheckpointIO()
strategy = NLPDDPStrategy(
    checkpoint_io=s3_checkpoint_io,
)
```


A  helpful guide on on how to craft a minimal bug report  http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports. 


**Expected behavior**

A clear and concise description of what you expected to happen.

**Environment overview (please complete the following information)**

 - Environment location: [Bare-metal, Docker, Cloud(specify cloud provider - AWS, Azure, GCP, Collab)]
 - Method of NeMo install: [pip install or from source]. Please specify exact commands you used to install.
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version
- PyTorch version
- Python version

**Additional context**

Add any other context about the problem here.
Example: GPU model
[DRAFT] add changes to allow for new te features
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add ASR with TTS Tutorial. Fix enhancer usage.
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [ASR]

# Changelog 
- Tutorial for using hybrid ASR-TTS Models (`ASRWithTTSModel`)
- fix enhancer loading in `ASRWithTTSModel`

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Is there a list of models that can export ONNX?
State: closed | Pull request: no
Labels: name: stale

'vad_multilingual_frame_marblenet' can.t export. is there a restriction on exporting models for some reason?

```python
import nemo.collections.asr as nemo_asr
model = nemo_asr.models.EncDecFrameClassificationModel.from_pretrained(model_name="vad_multilingual_frame_marblenet")

model.export('test.onnx', onnx_opset_version=12)
```
error:
'EncDecFrameClassificationModel' object has no attribute 'output_names'
fix install_beamsearch_decoders.sh
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

fix install_beamsearch_decoders.sh

**Collection**: ASR

# Changelog 
- install_beamsearch_decoders.sh work with and without root user
- provide NEMO_PATH in the tutorial Offline_ASR.ipynb

**PR Type**:
- [V ] Bugfix

# Additional Information
* Related to # 4186721
add async + distopt to sft
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add torchfix linter
State: closed | Pull request: yes
Labels: name: core

name: TTS

name: ASR

name: NLP

name: CI

# What does this PR do ?

Adds the torchfix linter to all PRs 

**Collection**: [Core]

# Changelog 
- Scans the codebase python files to analyse torch fixes / improvements

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Fix race condition for downloading cache when executing with multi-node
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

When downloading the tokenizer from cache with multi-node, some ranks may see that the path does not exist yet and multiple ranks will attempt to download causing errors.

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* Training 5b/20b that uses the megatron cache works well with single node. ptuning 5b/20b with multi-node this error arises.


```
Traceback (most recent call last):
  File "/usr/lib/python3.8/shutil.py", line 791, in move
    os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: '/root/.cache/torch/megatron/megatron-gpt-345m_vocab0037lvum.tmp' -> '/root/ (1).cache/torch/megatron/megatron-gpt-345m_vocab'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "megatron_gpt_prompt_learning.py", line 91, in main
...
  File "/opt/NeMo/nemo/collections/nlp/modules/common/megatron/megatron_utils.py", line 211, in _download
    wget.download(url, path)
  File "/usr/local/lib/python3.8/dist-packages/wget.py", line 534, in download
    shutil.move(tmpfile, filename)
  File "/usr/lib/python3.8/shutil.py", line 811, in move
    copy_function(src, real_dst)
  File "/usr/lib/python3.8/shutil.py", line 435, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.8/shutil.py", line 264, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/root/ (1).cache/torch/megatron/megatron-gpt-345m_vocab'
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)