renwei2024/nemo-github-issues huggingface dataset:
ptuning inference table bug fix
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?
Inference caching was using the wrong embedding size.

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix output decoding in text_generation_utils.generate for bytes
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

Some tokenizers don't have attributes `.tokenizer` or `.tokenizer.byte_decoder`, which results in crashing for these tokenizers (e.g., TikTokenizer and probably most tokenizers implemented in NeMo that derived from TokenizerSpec).

This PR improves this logic to:
1. First checking whether the output type is bytes; otherwise we are good.
2. For bytes outputs, safety checking whether tokenizer.tokenizer.byte_decoder; otherwise simply decode the byte array to string.

# Changelog 
- Fix output decoding in text_generation_utils.generate for bytes

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

@MaximumEntropy

# Additional Information
* Related to # (issue)
[TTS] Append pretrained FastPitch & SpectrogamEnhancer pair to available models
State: closed | Pull request: yes
Labels: name: TTS

name: cherry-pick

# What does this PR do ?

Add aliases for pretrained English FastPitch and SpectrogramEnhancer models (already published on NGC)
https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/tts_en_fastpitch_spectrogram_enhancer_for_asr_finetuning

Will be used in upcoming ASR-TTS tutorial.

**Collection**: TTS

# Changelog 
- Add `tts_en_fastpitch_for_asr_finetuning` model to list of FastPitch models
- Add `tts_en_spectrogram_enhancer_for_asr_finetuning` model to list of SpectrogramEnhancer models

# Usage
```python
import nemo
from nemo.collections.tts.models import FastPitchModel, SpectrogramEnhancerModel

fastpitch = FastPitchModel.from_pretrained(model_name="tts_en_fastpitch_for_asr_finetuning")
enhancer = SpectrogramEnhancerModel.from_pretrained(model_name="tts_en_spectrogram_enhancer_for_asr_finetuning")

```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #6955
[TTS] Append pretrained FastPitch & SpectrogamEnhancer pair to available models
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Add aliases for pretrained English FastPitch and SpectrogramEnhancer models (already published on NGC)
https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/tts_en_fastpitch_spectrogram_enhancer_for_asr_finetuning

Will be used in upcoming ASR-TTS tutorial.

**Collection**: TTS

# Changelog 
- Add `tts_en_fastpitch_for_asr_finetuning` model to list of FastPitch models
- Add `tts_en_spectrogram_enhancer_for_asr_finetuning` model to list of SpectrogramEnhancer models

# Usage
```python
import nemo
from nemo.collections.tts.models import FastPitchModel, SpectrogramEnhancerModel

fastpitch = FastPitchModel.from_pretrained(model_name="tts_en_fastpitch_for_asr_finetuning")
enhancer = SpectrogramEnhancerModel.from_pretrained(model_name="tts_en_spectrogram_enhancer_for_asr_finetuning")

```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #6955
fix install_beamsearch_decoders
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

fix install_beamsearch_decoders.sh

**Collection**: ASR

# Changelog 
- fix install_beamsearch_decoders.sh

# Usage
* install_beamsearch_decoders.sh work with and without root user 
* provide NEMO_PATH in the tutorial Offline_ASR.ipynb  

**PR Type**:
- [V ] Bugfix

# Additional Information
* Related to # 4186721
Fixed bug in preprocess data when specifying folder of json files
State: closed | Pull request: yes
Labels: name: stale

# What does this PR do ?

There was a bug in the `preprocess_data_for_megatron.py` script which caused it to crash when specifying a folder of `json` files that aren't gzipped. 

**Collection**: NLP

# Changelog 
- Modified line in `preprocess_data_for_megatron.py` which was causing a crash when specifying folder with non gzipped `json` files

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [X] Did you write any new necessary tests?
- [X] Did you add or update any necessary documentation?
- [X] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
remove hard coded input and output fields
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
R1.19.1
State: closed | Pull request: yes
Labels: name: core

name: TTS

name: ASR

name: NLP

name: Speaker Tasks

name: CI

name: common

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
sft with pi
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
RoPE length extrapolation with interpolation
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: NLP

# Changelog 
- Added the ability to do length extrapolation with RoPE via interpolation.
- Added a continue training script for GPT

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fixing an issue with confidence ensembles
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

* Adds a new parameter to transcribe_speech.py to allow computing the alignment without timestamps (significant speed up for conf ensembles).
* Makes sure the confidence is always computed on top of the normalized logits (i.e. after softmax). Was not the case for Transducer based mode.
* Relaxes test constraint to reduce random failures.

**Collection**: ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Load ub_cfg from hydra config
State: closed | Pull request: yes
Labels: name: core

name: NLP

# What does this PR do ?

Remove the need to load the yaml module inside GPT model and use hydra config instead.

**Collection**: nlp

# Changelog 
- For ub_tp_comm_overlap_cfg config field you no longer provide a path to the config but the whole config instead.

# Usage
If you add to your main config:
```yaml
defaults:
  [...]
  - optional tp_overlap@model.ub_tp_comm_overlap_cfg:
```
then you can load the config from a known location like this:
` ++  tp_overlap@model.ub_tp_comm_overlap_cfg=<tp_config}`

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Import Value Error
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

Getting a Error while importing "import nemo.collections.asr as nemo_asr" the error states "ValueError: mutable default <class 'nemo.core.classes.mixins.adapter_mixin_strategies.ResidualAddAdapterStrategyConfig'> for field adapter_strategy is not allowed: use default_factory" Tried uninstalling and reinstalling the library the problem still exits.

**Steps/Code to reproduce bug**

ScreenShot added

**Expected behavior**

A clear and concise description of what you expected to happen.

**Environment overview (please complete the following information)**

 - Environment location: Local Env (created specific new env as well)
 - Method of NeMo install: pip install nemo_tooklit[all]

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version
- PyTorch version
- Python version

**Additional context**

Add any other context about the problem here.
Example: Getting a Error while importing "import nemo.collections.asr as nemo_asr" the error states "ValueError: mutable default <class 'nemo.core.classes.mixins.adapter_mixin_strategies.ResidualAddAdapterStrategyConfig'> for field adapter_strategy is not allowed: use default_factory" Tried uninstalling and reinstalling the library the problem still exits.


**Screenshot**
![image](https://github.com/NVIDIA/NeMo/assets/61668526/b1be2fa6-26bb-4243-a37b-810461a5b6d9)
fixes for spellmapper
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

1. Fix bug that can affect QA (wrong file extension). 2. Other small non-critical fixes.  
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Trained TTS Vitsmodel Restricted to 11 Seconds of Audio Generation
State: closed | Pull request: no
Labels: name: bug

name: TTS

By following [this tutorial](https://github.com/NVIDIA/NeMo/blob/main/tutorials/tts/Vits_Training.ipynb), I trained a TTS VitsModel.
While performing inference of the trained TTS model to convert text into speech, I observed a limitation where the system generates only up to 11 seconds of audio. So, when dealing with longer texts (e.g., equivalent to 90 seconds of audio), it just produces only first 11 seconds of equivalent audio.
On the other hand, when the text is short and equivalent to 0 to 11 seconds of audio, the system generates audio of the exact length.

Is there a workaround available to generate the complete audio for longer texts?

Note:

I'm using the latest codebase to implement the TTS model.
The audio data I am working with varies in length from 0.1 seconds to 63 seconds.
A clear and concise description of what the bug is.

**Steps/Code to reproduce bug**

I loaded the Vits TTS model and infer with a long text.


**Expected behavior**

It was supposed to return a audio with whole text utterance.

**Environment overview (please complete the following information)**

 - Environment location: GCP
 - Method of NeMo install: from source

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version: Debian GNU/Linux 10 (buster)
- PyTorch version: 1.12.1+cu113
- Python version: 3.10

**Additional context**

Example: GPU model: 8xA100 GPUS
Change Jenkins timeout
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

The no activity timeout didn't work so changing to total duration of 8 hours. This is because timeout is including time in queue.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Cache handling without input tensors mutation
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Hybrid conformer export
State: closed | Pull request: yes
Labels: name: core

name: cherry-pick

name: ASR

# What does this PR do ?

Allows export of Hybrid Models with either RNNT decoder (in two parts) or with CTC decoder as single encoder-decoder module. 
Usage: python scripts/export.py  file.nemo file.onnx --config decoder_type=[ctc|rnnt] 

**Collection**: [ASR]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
python scripts/export.py file.nemo file.onnx --config decoder_type=[ctc|rnnt]

# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fixes for spellmapper
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

1. Fix bug that can affect QA (wrong file extension). 2. Other small non-critical fixes.  
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
"all_probs" is not functional in text-generation-server
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

Use the following script to make a request will result in crashing in NeMo text-generation-server:

```
import json
import requests

port_num = 5555
headers = {"Content-Type": "application/json"}

def request_data(data):
    resp = requests.put('http://localhost:{}/generate'.format(port_num),
                        data=json.dumps(data),
                        headers=headers)
    resp = resp.json()
    if "sentences" not in resp:
        return f"Error: {resp}"
    return resp['sentences']

data = {
    "sentences": ["Tell me an interesting fact about space travel."],
    "tokens_to_generate": 32,
    "temperature": 1.00,
    "add_BOS": True,
    "top_k": 0,
    "top_p": 0.7,
    "greedy": False,
    "all_probs": True, # <-- problematic
    "repetition_penalty": 1.0,
    "min_tokens_to_generate": 0,
    "end_strings": [], # stop sequence
}

for sentence in request_data(data):
    print(sentence)

```

**Steps/Code to reproduce bug**

1. Launch a service (see environment below for the command).
2. Run the above script to make a request.

**Expected behavior**

The proper response as shown in the NeMo API document.

**Environment overview (please complete the following information)**

1. Download a checkpoint from HuggingFace and put it under `/data`.
2. Launch a docker

```
MODEL=/data/nemo-megatron-gpt-1.3B/nemo_gpt1.3B_fp16.nemo
TP_SIZE=1
VOLUME=/data

docker run --rm -it --gpus all --shm-size=20g --ipc=host \
    --ulimit memlock=-1 --ulimit stack=67108864 --network=host \
    -v $VOLUME:/data \
    nvcr.io/ea-bignlp/nemofw-training:23.04.1-py3:latest \
    python3 /opt/NeMo/examples/nlp/language_modeling/megatron_gpt_eval.py \
    gpt_model_file=$MODEL server=True \
    tensor_model_parallel_size=$TP_SIZE \
    trainer.devices=$TP_SIZE \
    port=5555
```

**Additional context**

The server side crashes at the line https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/nlp/modules/common/text_generation_utils.py#L430:

```
  File "/opt/NeMo/nemo/collections/nlp/modules/common/text_generation_utils.py", line 429, in synced_generate
    torch.distributed.broadcast(full_logits, src, group)
  File "/usr/local/lib/python3.8/dist-packages/torch/distributed/distributed_c10d.py", line 1422, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch/distributed/distributed_c10d.py", line 1526, in broadcast
    _check_single_tensor(tensor, "tensor")
  File "/usr/local/lib/python3.8/dist-packages/torch/distributed/distributed_c10d.py", line 568, in _check_single_tensor
    raise RuntimeError(
RuntimeError: Invalid function argument. Expected parameter `tensor` to be of type torch.Tensor.
```

This line is weird because it attempts to broadcast `full_logits`, but this variable seems not properly defined:

```
    for tokens, lengths, output_logits, full_logits in batch_token_iterator:
        context_length += 1
```

cc @yidong72 @michalivne @ericharper
Add support for Numba FP16 RNNT Loss
State: closed | Pull request: yes
Labels: name: core

name: ASR

# What does this PR do ?

Add fp16 support to RNNT Numba Loss

**Collection**: [ASR]

# Changelog 
- 

# Usage

## Install dependencies
[Numba](https://github.com/numba/numba) 0.57+ (conda install numba=0.57.1 -c conda-forge)
[CUDA Python](https://nvidia.github.io/cuda-python/install.html)

After that, train RNNT model with trainer.precision=16

```python
# Set environment variables 
# export NUMBA_CUDA_USE_NVIDIA_BINDING=1
# export STRICT_NUMBA_COMPAT_CHECK=0

from nemo.core.utils import numba_utils

print(numba_utils.numba_cuda_is_supported(numba_utils.__NUMBA_MINIMUM_VERSION_FP16_SUPPORTED__))  # Should be true

# Should also be True. 
# You can also pass True as an argument to this function to possibly get a reason as to why FP 16 is not supported.

print(numba_utils.is_numba_cuda_fp16_supported())
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [x] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Questions regarding training conformer transducer model.
State: closed | Pull request: no

Hello,

Thank you for your work and all the help provided so far with other issues too. 

I am training a conformer transducer model and I have a couple of questions that might help me achieve better results:

1. I am finetuning the **stt en conformer transducer medium** model (whole model) on 225h of romanian language ( 200h training, 20h validation). I am at epoch 100 and the val_wer keeps decreasing, as well as the training wer. However, the performances on the eval set are still far away in performance comparing to our Kaldi model (10-15% more WER). How many epochs are usually necessary for convergence of the transducer medium model and are 200 hours of training data enough for a conformer model?

2. Do you use any type of cross-validation or simply randomly split the training set in train/val? 

3. At the moment I am using specaugment and spec cutout, should I also add noise impulses/background noise from some datasets too as augmentation? Could you please point what I should add in the config file exactly or if there is an example I could follow?

4. Lastly, I am using an LR of 1 with Noam annealing and 10% warm up ratio, is it a big LR for finetuning? ( from what I understood, you used 5 as LR) I am unsure if I should lower it to 0.1 maybe or 0.01.

Training takes quite a long time on our setup and we can't afford to test many configurations, that's why these questions may be pretty basic, but any insight is greatly appreciated!
Fixing an issue with confidence ensembles
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

* Adds a new parameter to transcribe_speech.py to allow computing the alignment without timestamps (significant speed up for conf ensembles).
* Makes sure the confidence is always computed on top of the normalized logits (i.e. after softmax). Was not the case for Transducer based mode.
* Relaxes test constraint to reduce random failures.

**Collection**: ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add end_strings to SamplingParams
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?
Makes end_strings part of SamplingParams and makes it configurable.

**Collection**: Text Generation Server

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
duration limit
State: closed | Pull request: yes
Labels: name: stale

name: ASR

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
I am still getting the error.
State: closed | Pull request: no

              I am still getting the error
Unexpected keyword arguments: `compute_on_step`
  File "/home/muizz/nemo_workspace/Neural_netw.py", line 9, in <module>
    asr_model = nemo_asr.models.EncDecCTCModel.from_pretrained(model_name="QuartzNet15x5Base-En")
ValueError: Unexpected keyword arguments: `compute_on_step`

what do i do to avoid this

_Originally posted by @muizzhaque in https://github.com/NVIDIA/NeMo/issues/6977#issuecomment-1623474893_
Hybrid conformer export
State: closed | Pull request: yes
Labels: name: core

name: ASR

# What does this PR do ?

Allows export of Hybrid Models with either RNNT decoder (in two parts) or with CTC decoder as single encoder-decoder module. 
Usage: python scripts/export.py  file.nemo file.onnx --config decoder_type=[ctc|rnnt] 

**Collection**: [ASR]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
python scripts/export.py file.nemo file.onnx --config decoder_type=[ctc|rnnt]

# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Minor MPT-7B fixes and creation script update
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Updates the script used to create a Megatron MPT-7B checkpoint with fixes, and adds support to MegatronBaseModel for special tokens to be passed at runtime.

**Collection**: NLP

# Changelog 
- Updates the Megatron MPT-7B creation script with some bug fixes and a config file baseline
- Adds support for HuggingFace special tokens to be declared in Hydra/Yaml and passed to MegatronBaseModel

# Usage
You can specify the new tokens in a config YAML like so:

```yaml
...
model.tokenizer.special_tokens.pad_token="<PAD>"
model.tokenizer.special_tokens.mask_token="<MASK>"
...
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [X] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information

The MegatronBaseModel changes are necessary because the HuggingFace tokenizer which is used by MPT-7B (and indeed all other HF tokenizers) does not have special tokens built in, tokens such as PAD and MASK. This leads to a crash when prompt-learning with MPT-7B, because there is no pad token, and no way to add one at runtime. With this fix to MegatronBaseModel, you can now pass a pad token at runtime, which allows prompt-learning with MPT-7B (or any model which uses a HF tokenizer).
Remove `compute_on_step` from metrics
State: closed | Pull request: yes
Labels: name: core

name: cherry-pick

name: ASR

name: NLP

name: common

# What does this PR do ?

Remove deprecated arg compute_on_step from torchmetrics

**Collection**: [ASR, NLP]

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Cache handling without input tensors mutation
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Remove `compute_on_step` from metrics
State: closed | Pull request: yes
Labels: name: core

name: ASR

name: NLP

name: common

# What does this PR do ?

Remove deprecated arg compute_on_step from torchmetrics

**Collection**: [ASR, NLP]

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Implemented generic kv-pair setting of export_config from args
State: closed | Pull request: yes
Labels: name: core

name: ASR

# What does this PR do ?

This will be a basis for several export extensions that involve user input. 
As an example, I have changed handling  of --cache_support arg in export.py to use the new method.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Unexpected keyword arguments: `compute_on_step`   File "/home/muizz/nemo_workspace/Neural_net.py", line 9, in <module>     asr_model = nemo_asr.models.EncDecCTCModel.from_pretrained(model_name="QuartzNet15x5Base-En") ValueError: Unexpected keyword arguments: `compute_on_step`
State: closed | Pull request: no
Labels: name: bug


I've been trying to transcribe a wav file of mono channel and rated a 16khz but this error shows up all the time
below is my code:
import nemo.collections.asr as nemo_asr
asr_model = nemo_asr.models.EncDecCTCModel.from_pretrained(model_name="QuartzNet15x5Base-En")
emb = asr_model.transcribe('./right_channel.wav')
print(emb)

error:
Unexpected keyword arguments: `compute_on_step`   File "/home/muizz/nemo_workspace/Neural_net.py", line 9, in <module>     asr_model = nemo_asr.models.EncDecCTCModel.from_pretrained(model_name="QuartzNet15x5Base-En") ValueError: Unexpected keyword arguments: `compute_on_step`
rnnt and char utils
State: closed | Pull request: yes

# What does this PR do ?

fix ngram_merge using rnnt nemo model.
fix char level ngram creation.

**Collection**: ASR

# Changelog 
- scripts/asr_language_modeling/ngram_lm/ngram_merge.py count perplexity of a test set using tokenizer from RNNT model
- scripts/asr_language_modeling/ngram_lm/kenlm_utils.py create word-level ngram model using char-level nemo model

```python
# Add a code snippet demonstrating how to use this 
```

  
**PR Type**:
- [ V] Bugfix
[pre-commit.ci] pre-commit suggestions
State: closed | Pull request: yes
Labels: name: stale

<!--pre-commit.ci start-->
updates:
- [github.com/pre-commit/pre-commit-hooks: v4.3.0 → v4.4.0](https://github.com/pre-commit/pre-commit-hooks/compare/v4.3.0...v4.4.0)
- [github.com/psf/black: 19.10b0 → 23.3.0](https://github.com/psf/black/compare/19.10b0...23.3.0)
<!--pre-commit.ci end-->