renwei2024/nemo-github-issues huggingface dataset:
Fix notebook bad json
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fix bad JSON in notebook.

**Collection**: [ASR]

# Changelog 
- Correct the comma

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Patch transcribe and support offline transcribe for hybrid model
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Patch transcribe_util and support offline transcribe for hybrid model

**Collection**: ASR

# Changelog 
- Fix wrong decoding in write_transcript
- Add support for hybrid offline transcribe in asr_evaluator

# Usage
* You can potentially add a usage example below

```python
python transcribe_speech.py 
model_path={hybrid_model} 
decoder_type='ctc'/'rnnt'
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue) https://github.com/NVIDIA/NeMo/pull/6063/files#r1178635367
Add scores for FastConformer models
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Update scores table for Fast Conformer models

**Collection**: [ASR]

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Add scores for FastConformer models
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Update scores table for Fast Conformer models

**Collection**: [ASR]

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Support large inputs to Conformer and Fast Conformer
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Adds support for large inputs (large batches and large samples) to Fast Conformer models, working around this issue https://github.com/pytorch/pytorch/issues/80020
For regular Conformer, this PR adds support for larger batches.

**Collection**: ASR

# Changelog 
-  In subsampling.py, if the input is too large, try to split it by batch. if not, try to split it by channels (only models with depthwise-separable convolutions)

# Usage
* encoder.subsampling_conv_chunking_factor=1 (auto-chunking, default) or -1 (disabled) or to a specific value (power of 2)
* Also, you can load a pre-trained model and then call model.change_subsampling_conv_chunking_factor(subsampling_conv_chunking_factor=N) to perform inference on large batches / large samples

```python
# set
encoder.subsampling_conv_chunking_factor=1 
# (1 means auto-chunking, default); -1 (disabled) or to a specific value (power of 2)
# Also, you can load a pre-trained model and then call model.change_subsampling_conv_chunking_factor(subsampling_conv_chunking_factor=N) 
# to perform inference on large batches / large samples
# channel wise chunking will attempted only if the model uses depthwise-separable convolutions.
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Support large inputs to Conformer
State: closed | Pull request: yes
Labels: name: TTS

name: ASR

name: NLP

name: common

# What does this PR do 
Helps support large batches and input, working around https://github.com/pytorch/pytorch/issues/80020


**Collection**: [Note which collection this PR will affect]
ASR
# Changelog 
- If the input is detected to be too big, tries to split it by batch and if that does not work, by time

# Usage
* n/a

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [X ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Turn autocast off when precision is fp32
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Fix for autocast when precision is fp32

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add FastConformer Hybrid ASR models for EN, ES, IT, DE, PL, HR, UA, BY
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

This PR adds the FastConformer Hybrid ASR models (EN, ES, IT, DE, PL, BY, HR, UA) to the PreTrained models catalog

**Collection**: ASR

# Changelog 
- Added the FC Hybrid models to nemo/collections/asr/models/hybrid_rnnt_ctc_bpe_models.py
- Updated the ASR docs at NeMo/docs/source/asr/results.rst
- Created new results tables for BY (NeMo/docs/source/asr/data/benchmark_by.csv), UA (NeMo/docs/source/asr/data/benchmark_ua.csv)

# Usage
When loading pre-trained models, we can now specify:
- stt_en_fastconformer_hybrid_large_pc
- stt_de_fastconformer_hybrid_large_pc
- stt_it_fastconformer_hybrid_large_pc
- stt_es_fastconformer_hybrid_large_pc
- stt_hr_fastconformer_hybrid_large_pc
- stt_by_fastconformer_hybrid_large_pc
- stt_ua_fastconformer_hybrid_large_pc
- stt_pl_fastconformer_hybrid_large_pc
- stt_en_fastconformer_transducer_large
- stt_en_fastconformer_ctc_large


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Change Megatron Enc Dec model to use persistent_workers
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Change Megatron Enc Dec model to use persistent_workers.
Seems to be needed for bigNLP CI - fails for persistent_workers issue if not provided

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Pass `.scale` instead of scaler object to core
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Fixes GradScaler issue when using torch's GradScaler

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Patch transcribe and support offline transcribe for hybrid model
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Patch transcribe_util and support offline transcribe for hybrid model

**Collection**: ASR

# Changelog 
- Fix wrong decoding in write_transcript
- Add support for hybrid offline transcribe in asr_evaluator

# Usage
* You can potentially add a usage example below

```python
python transcribe_speech.py 
model_path={hybrid_model} 
decoder_type='ctc'/'rnnt'
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue) https://github.com/NVIDIA/NeMo/pull/6063/files#r1178635367
Add FastConformer Hybrid ASR models for EN, ES, IT, DE, PL, HR, UA, BY
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

This PR adds the FastConformer Hybrid ASR models (EN, ES, IT, DE, PL, BY, HR, UA) to the PreTrained models catalog

**Collection**: ASR

# Changelog 
- Added the FC Hybrid models to nemo/collections/asr/models/hybrid_rnnt_ctc_bpe_models.py
- Updated the ASR docs at NeMo/docs/source/asr/results.rst
- Created new results tables for BY (NeMo/docs/source/asr/data/benchmark_by.csv), UA (NeMo/docs/source/asr/data/benchmark_ua.csv)

# Usage
When loading pre-trained models, we can now specify:
- stt_en_fastconformer_hybrid_large_pc
- stt_de_fastconformer_hybrid_large_pc
- stt_it_fastconformer_hybrid_large_pc
- stt_es_fastconformer_hybrid_large_pc
- stt_hr_fastconformer_hybrid_large_pc
- stt_by_fastconformer_hybrid_large_pc
- stt_ua_fastconformer_hybrid_large_pc
- stt_pl_fastconformer_hybrid_large_pc
- stt_en_fastconformer_transducer_large
- stt_en_fastconformer_ctc_large


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Change Megatron Enc Dec model to use persistent_workers
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Change Megatron Enc Dec model to use persistent_workers.
Seems to be needed for bigNLP CI - fails for persistent_workers issue if not provided

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[Tools] Update RIR generation scripts
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

This PR adds a few enhancements to RIR generation scripts.

**Collection**: Tools

# Changelog 
- Fix: reduce room size if evaluation of params fails
- Added: randomized mic placement
- Added: diffuse noise generation
- Added: an option to specify the format and subtype for saved audio

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
Fix for T5 FT model
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Fix for T5 FT model tensor shape

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Pass `.scale` instead of scaler object to core
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Fixes GradScaler issue when using torch's GradScaler

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
GPT fp16 inference fix
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Fixes GPT inference when trainer.precision is 16.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
GPT fp16 inference fix
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Fixes GPT inference when trainer.precision is 16.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Add script for text preprocessing
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

A lot of people already have a local script which does text processing & normalization before training. To do it quickly one needs to use multiprocessing or joblib, as our text normalizer does not support batch processing.

It only does normalization and optional lower casing of the final output right now, but could be extended with additional processing functionality as needed.

**Collection**: [TTS]

# Changelog 
- Create text preprocessing script
- Add example config file for English text normalization

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Fix typos (#6523)
State: closed | Pull request: yes
Labels: name: ASR


* Fix typos

Signed-off-by: smajumdar <titu1994@gmail.com>

---------

Signed-off-by: smajumdar <titu1994@gmail.com>
(cherry picked from commit 5468077f5127be1a4c88065de2544f4268b9a6e4)

# What does this PR do ?


**Collection**: [Docs]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
TDT model pull request
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Token-and-Duration Transducer (TDT, https://arxiv.org/abs/2304.06795) implementation.

**Collection**: 
ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage

To train a TDT model, use config file at examples/asr/conf/conformer/tdt/conformer_tdt_bpe.yaml. You can change the $durations parameter to customize your model, e.g.

```
durations=[0,1,2,3]
num_durations=4  # this needs to match the previous durations variable.

python examples/asr/asr_transducer/speech_to_text_rnnt_bpe.py \
        [options similar to launching conventional Transducer models] \
        model.loss.tdt_kwargs.fastemit_lambda=0.001 \
        model.loss.tdt_kwargs.omega=0.1 \
        model.model_defaults.tdt_durations=$durations \
        model.model_defaults.num_tdt_durations=$num_durations \
        model.decoding.strategy='greedy'   # Greedy decoding is recommended to get more accurate validation WERs during training. It's already set in the config file as default. We list it here to emphasize its importance.

```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
How to get the intermediate features from the pre-trained checkpoint
State: closed | Pull request: no

Hi @titu1994, @VahidooX, thank you for your hard work.

I am trying out an experiment for the knowledge distillation task by utilizing the official pre-trained checkpoint ([stt_en_conformer_ctc_large_ls](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/stt_en_conformer_ctc_large_ls)) for the teacher model.

I would like to extract some intermediate features from this pre-trained teacher model (i.e., the self-attention layers in the conformer blocks), but all the intermediate layers in `ConformerEncoder` module seem to be overwritten with the variable `audio_signal`.

Is there a way to get these intermediate layers from the pre-trained checkpoint you provide?
I'd be grateful if you could provide some tutorials or tips.
Thank you.
ModuleNotFoundError: No module named 'kenlm_utils'
State: closed | Pull request: no

hi, 
while trying to import "kenlm_utils" i face this error:

ModuleNotFoundError: No module named 'kenlm_utils'

and after manually adding kenlm_utils.py to my directory i encounter the  following error:

"UnsupportedOperation                      Traceback (most recent call last)
[<ipython-input-29-cec13af5ea4e>](https://localhost:8080/#) in <cell line: 83>()
     82 
     83 if __name__ == '__main__':
---> 84     main()

3 frames
[/usr/lib/python3.10/subprocess.py](https://localhost:8080/#) in _get_handles(self, stdin, stdout, stderr)
   1636             else:
   1637                 # Assuming file-like object
-> 1638                 c2pwrite = stdout.fileno()
   1639 
   1640             if stderr is None:

UnsupportedOperation: fileno"

any help is really appreciated.
thanks!
fixes #6532
State: closed | Pull request: yes
Labels: name: stale

name: common

# What does this PR do ?

See issue https://github.com/NVIDIA/NeMo/issues/6532

**Collection**: [Note which collection this PR will affect]
tokenizers

# Changelog 
- See comments in the Issue https://github.com/NVIDIA/NeMo/issues/6532

# Usage
* You can potentially add a usage example below

```python
from nemo.collections.common.tokenizers.tabular_tokenizer import TabularTokenizer
from nemo.collections.common.tokenizers.column_code import ColumnCodes

END_OF_TEXT = '
'
NEW_LINE = '\n'
DELIMITER = ','

# see example of tab_structure and example arrays here: 
# https://github.com/NVIDIA/NeMo/pull/6533/files#diff-2eb09eb882bbc75d60d887b51950ed0fb3331d5c4248e9e0b0ec0a566609c041R133

column_codes = ColumnCodes.get_column_codes(tab_structure, example_arrays)
tabular_tokenizer = TabularTokenizer(column_codes, special_tokens=[NEW_LINE, END_OF_TEXT], delimiter=DELIMITER)
token_ids = tabular_tokenizer.encode(doc['text'])
output = tabular_tokenizer.decode(token_ids)
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # [(issue)](https://github.com/NVIDIA/NeMo/issues/6532)
[FEA] Add partial row tabular tokenizer, 'best' transform for FloatCode, and refactor
State: closed | Pull request: no
Labels: name: stale

**Is your feature request related to a problem? 
TabularTokenizer and ColumnCode api needs QOL improvements. It currently does not support partial row generation because of [this](https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/common/tokenizers/tabular_tokenizer.py#L132) and [this](https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/common/tokenizers/tabular_tokenizer.py#L164), impacting certain generative applications. because it expects all the columns and the columns to be in order.

**Describe the solution you'd like**
- add 'best' transform into FloatCode, and include new transforms, with some simple stats/heuristics for selecting a 'best' transform
- refactor column_coder.py into datatype_code and column_code to separate out the datatypes that a column has from the column_code.
- refactor Code into a code_spec, and add a NumericCode base class inheriting from Code. Push fill_all and has_nan into NumericCode as those params are only useful for numeric dataytypes
- add a partial row encoder and decoder for the tabular tokenizer
- rename some class methods to be more clear, ex. array_convert_to_int to transform_data_to_int_array to more accurately describe the function
- Fix `bug` where the [FloatCode accuracy ](https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/common/tokenizers/column_coder.py#L231) leads to cases where the decoded output is less accurate than it needed to be.
- remove `column_coder.py` and replace it with `datatype_code`. Uses `code` over `coder` since `code` is for a mapping of the dtype col itself.
- write tests

A clear and concise description of what you want to happen.
Tests will clarify usage. ex. 
```
from datatype_code import FloatCode

float_code = FloatCode('name', code_len=3, start_id=0, fill_all=True, base=100, has_nan=True, transform='identity')
float_code.compute_code(data_series)
```
This keeps the new code that users will interface with inline with the current API.

**Describe alternatives you've considered**
putting `compute_code` into the into a `fit_transform`, and encode as `transform` and decode as `inverse_transform` to make the API similar to sklearn.

A clear and concise description of any alternative solutions or features you've considered.
- instead of tokenizing the text token by token. Group the tokens by column and tokenize by column in a vectorizable fashion. This speeds up decoding a lot. Even converting to a DataFrame and doing a `df.replace` or `map` will speed up over existing method. The caveat is this will not allow for partial row generation.
- other vector based methods or parallel lookup methods to speed up encoding and decoding as this consumes quite a bit of time.
GPT FP16 Inference Fix
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Automatically sets O2=False when running inference with fp16.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix for T5 FT model
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Fix for T5 FT model tensor shape

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Restore GPT support for interleaved pipeline parallelism
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

This restores support for interleaved pipeline parallelism after https://github.com/NVIDIA/NeMo/pull/6393.

**Collection**: NLP

# Changelog 
- Restore custom synchronization logic to GPT training and validation steps
- Fix bug where all model chunks in first pipeline stage were created with embeddings
- Support dynamic attention masks in pipeline parallelism

# Usage

Set `virtual_pipeline_model_parallel_size > 1`:
https://github.com/NVIDIA/NeMo/blob/5468077f5127be1a4c88065de2544f4268b9a6e4/examples/nlp/language_modeling/conf/megatron_gpt_config.yaml#L51

This requires `pipeline_model_parallel_size > 2`  and `virtual_pipeline_model_parallel_size*pipeline_model_parallel_size` divides `num_layers`.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

Pinging @aklife97.

# Additional Information
* Restores functionality broken in https://github.com/NVIDIA/NeMo/pull/6393
* Depends on https://github.com/NVIDIA/Megatron-LM/pull/331
Updating num_workers in lm_encoder_decoder
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Updating the num_workers from 0 to the one mentioned in the config

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Purely GPU-based optimized RNN-T greedy batched decoding
State: closed | Pull request: yes
Labels: name: stale

name: ASR

# What does this PR do ?

A highly optimized implementation for RNN-T greedy batched decoding with GPUs.

**Collection**: [Note which collection this PR will affect]
ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

The method could be invoked by setting  rnnt_decoding.cuda_optimized as True, e.g.

```
python examples/asr/transcribe_speech.py \
        model_path=$model \
        dataset_manifest=$manifest  \
        rnnt_decoding.strategy=greedy_batch \
        +rnnt_decoding.cuda_optimized=True \
        batch_size=$batch \
        output_filename=$out_file
```

This gives significant inference speed-up for large batch sizes. E.g. Librispeech Transducer-large model with stateless decoders running inference on test-other, with batch=128,

| inference type  | WER | decoding time |
| --------------- | ----- | --------------- |
| baseline greedy batch | 4.70 | 119s |
| GPU optimized | 4.64 | 86s |


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update quartznet_15x5.yaml
State: closed | Pull request: yes
Labels: name: stale

name: ASR

# What does this PR do ?
tb logging leads to error

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Not doing CastToFloat by default
State: closed | Pull request: yes

# What does this PR do ?

Removing casts around norms from the set of default pre-export replacements and moving them to a separate function.
Latest TRT should not need that anymore - also the casts may prevent some fusion.
That function may still be used from Nemo2Riva explicitly should it be needed on case-by-case basis. 

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix typos
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fix typos

**Collection**: [ASR]

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation