renwei2024/nemo-github-issues huggingface dataset:
Cache aware streaming infer error with size mismatched
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

I trained a Cache aware streaming Conformer CTC model, and decode with
```
CUDA_VISIBLE_DEVICES="0" python examples/asr/asr_cache_aware_streaming/speech_to_text_cache_aware_streaming_infer.py \
    --asr_model=/home/xx/NeMo/exp/libri_mini_conformer_ctc_small_streaming/Conformer-CTC-BPE/2023-06-25_01-05-04/checkpoints/Conformer-CTC-BPE.nemo \
    --manifest_file=/home/xx/NeMo/data/libri_960h/dev_clean_2.json \
    --batch_size=1 \
    --use_amp \
    --debug_mode
```

but got a error:
```
  File "/home/xx/NeMo/nemo/collections/asr/parts/submodules/multi_head_attention.py", line 211, in forward
    key, value, query = self.update_cache(key=key, value=value, query=query, cache=cache, cache_next=cache_next)
  File "/home/xx/NeMo/nemo/collections/asr/parts/submodules/multi_head_attention.py", line 154, in update_cache
    cache_next[self._cache_id, :, -q_keep_size:, :] = query[:, :q_keep_size, :]
RuntimeError: The expanded size of the tensor (9988) must match the existing size (200) at non-singleton dimension 1.  Target sizes: [1, 9988, 176].  Tensor sizes: [200, 176]
```

**Steps/Code to reproduce bug**
Use lated NeMo code and use code like below to train a streaming model:
```
python examples/asr/asr_ctc/speech_to_text_ctc_bpe.py \
    --config-path /home/xx/NeMo/examples/asr/conf/conformer \
    --config-name conformer_ctc_bpe_small.yaml \
    model.train_ds.manifest_filepath=/home/xx/NeMo/data/libri_960h/train_clean_100
    model.validation_ds.manifest_filepath=/home/xx/NeMo/data/libri_960h/dev_clean.
    model.tokenizer.dir=/home/xx/NeMo/data/eng_bpe_128 \  # extracted from pretrained model stt_en_conformer_ctc_small_ls
    model.tokenizer.type=bpe \
    model.optim.lr=0.1 \
    model.optim.sched.warmup_steps=20000 \
    trainer.devices=2 \
    trainer.accelerator="gpu" \
    trainer.strategy="ddp" \
    trainer.max_epochs=100 \
    exp_manager.exp_dir=/home/xx/NeMo/exp/libri_mini_conformer_ctc_small_streaming
```

and then use code like below to decode:
```
CUDA_VISIBLE_DEVICES="0" python examples/asr/asr_cache_aware_streaming/speech_to_text_cache_aware_streaming_infer.py \
    --asr_model=/home/xx/NeMo/exp/libri_mini_conformer_ctc_small_streaming/Conformer-CTC-BPE/2023-06-25_01-05-04/checkpoints/Conformer-CTC-BPE.nemo \
    --manifest_file=/home/xx/NeMo/data/libri_960h/dev_clean_2.json \
    --batch_size=1 \
    --use_amp \
    --debug_mode
```

**Expected behavior**

Normal decoding.

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- Red Hat 8.4.1-1
- PyTorch 1.11.0
- Python 3.8.17

**Additional context**
full log: 
[xx.log](https://github.com/NVIDIA/NeMo/files/11859985/xx.log)
ptuning oom fix
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

The prompt learning dataset was not padding sequences to a multiple of 8 which forced a non-optimized version of softmax to be used resulting in more memory use and OOM errors. This PR pads to a multiple of 8 to reduce memory. 

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix gpt trainer test
State: closed | Pull request: yes
Labels: name: NLP

name: CI

name: common

# What does this PR do ?

- Fix gpt trainer inference_step to compute generation results during validation and test
- Add dataset with metadata, to output in prediction file
- Add token_f1 metrics for Scrolls

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update container info in  README.rst
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix FP8 scales conversion issue from non-nemo checkpoint to nemo checkpoint
State: closed | Pull request: yes
Labels: name: ASR

name: NLP

# What does this PR do ?

Converting a non-nemo checkpoint to NeMo with Transformer Engine modules results in truncating of scales in `_extra_state`. This WAR temporarily fixes the issue by manually toggling the `fp8` attribute in TE modules. 

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix fast-glu activation in change partitions
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Change `swiglu` to all fast glu activation for partition conversion. 

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Make sure asr_model.change_attention_model is run if either cfg.model_path or cfg.pretrained_name is specified
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?
Makes sure `asr_model.change_attention_model` is run regardless of whether `cfg.model_path` or `cfg.pretrained_name` is specified.

**Collection**: ASR

# Changelog 
- Moved `asr_model.change_attention_model` call outside of if-branch.

# Usage
* You can potentially add a usage example below

```bash
python NeMo/examples/asr/transcribe_speech.py \
        pretrained_name="stt_en_fastconformer_ctc_large" \
        dataset_manifest=$src_manifest \
        ++model_change.conformer.self_attention_model="rel_pos_local_attn" \
        ++model_change.conformer.att_context_size="[128, 128]"
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [N/A] Did you write any new necessary tests?
- [N/A] Did you add or update any necessary documentation?
- [N/A] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Reduce number of nodes produced in ConformerEncoder export
State: closed | Pull request: yes
Labels: name: ASR

This commit breaks up the output cache tensor into a list of tensors. Each layer which writes to the output cache now writes to its own tensor. The output cache tensors are then stacked into one tensor before being returned, so ConformerEncoder still has the same inputs and outputs as before.

The reason for this change is from an ONNX perspective, updating a tensor creates a new copy with the changes. So minimizing the sizes of the tensors that get updated makes the exported ONNX model more efficient.

The number of nodes produced in the ONNX export is also reduced. In the example I tested, the exported ONNX model had 11.484 nodes before this change, and only 4,892 nodes after this change.

Signed off by: Ilya Sherstyuk <isherstyuk@nvidia.com>
Support about Streaming inference of Conformer Transducer?
State: closed | Pull request: no
Labels: name: stale

Does Nemo support about streaming inference of Conformer Transducer?

From my view, as long as Conformer Transducer trained using Cache-aware Streaming Conformer, then it can be used on streaming situation.

But from this: https://github.com/NVIDIA/NeMo/issues/3144#issuecomment-962420001, it seems we can train a Cache-aware Streaming based Conformer Transducer model to use it in streaming situation.

So my question is:
1. Can I train a Cache-aware Streaming based Conformer Transducer model using https://github.com/NVIDIA/NeMo/blob/main/examples/asr/conf/conformer/cache_aware_streaming/conformer_transducer_bpe_streaming.yaml?
2. Can I inference using script from https://github.com/NVIDIA/NeMo/blob/main/examples/asr/asr_cache_aware_streaming/speech_to_text_cache_aware_streaming_infer.py with a Confomer Transducer model, which means does speech_to_text_cache_aware_streaming_infer.py support Cache-aware Streaming based Conformer Transducer model?
3. If I want use a Conformer Transducer model in streaming situation, what can I do for now? Because from Librispeech 960 experiments, it seems Conformer Transducer is best with respect to WER?

Thanks for any tips.
Support BLOOM for finetuning?
State: closed | Pull request: no
Labels: name: stale

I noticed that you referenced the ALiBi in the flash-attn module, but I couldn't find any mention or description of its usage in your project. I'm curious to know if you have incorporated support for models like BLOOM for fine-tuning and converting their checkpoints from Huggingface format to .nemo or MegatronLM.
Fix flash-attention
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

1. Set default `apply_query_key_layer_scaling=False`
2. Fix flash_attention_triton attention_bias broadcast issue (reduce memory usage)
3. Add testing for flash cross-attention suggested by @yidong72 

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
removed some tests
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Enable `rpe` methods in bert-like models
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?
- Add extrapolatable position embedding to bert-like model, based on the recent changes introduced in #6666 
- I am already using this change in the BioNeMo framework and running experiments to test how `rpe` techniques help bert-like model to extrapolate to longer sequences. 

**Collection**: [NLP]

# Changelog 
- BERT: 
   - nemo/collections/nlp/models/language_modeling/megatron_bert_model.py
   - nemo/collections/nlp/models/language_modeling/megatron/bert_model.py
# Usage

```python
model.position_embedding_type='' # Options ['learned_absolute', 'rope', 'alibi', 'xpos', 'sandwich', 'kerple']
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
hot fix SDE
State: closed | Pull request: yes

# What does this PR do ?

Got report about unexpected behaviour. fixed

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [+] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Best way to get started for real-time ASR through microphone 
State: closed | Pull request: no
Labels: name: stale

Hello,

I want to implement real time inferencing on microphone audio and saw that the Cache-aware Streaming Conformer models are being used for this in NeMo, I wanted to know if there are any available models for real-time transcription though mic and any example scripts to implement this in python. 

Thanks!
Finetuning ASR on another language (tokenizer)
State: closed | Pull request: no

Hello, 

I would like to finetune the stt_en_conformer_transducer_medium on 200 hours of another language (romanian). I am at the tokenization step and I am unsure of what would be the best practice:

1. Prepare the tokens using the text from the training audio transcripts (around 2 million words, not necessarily different). I would use sentencepiece, with unigram and vocab of 1024.
2. Prepare the tokens using a big text corpus of around 50 million words, one that is usually used for training LMs. Same configuration.

I am unsure if it is ok to train the tokenizer on more text than the training audios, I was not able to find if this is an OK practice and can lead to better/worse results. I was thinking that it might generalize better if it has more possible tokens. The tokenizer I obtained on just 200 hours looked rather limited, with lots of big words that are not actually that common outside of the training set. However, I am scared that it might not learn to pronounce some tokens based on the audio dataset, but I consider this rather unlikely.

One last question would be: What are the "##" , "_", "_de", etc. tokens representing? 

Thank you in advance!
Enables manifest sharding for bucketed ASR datasets
State: closed | Pull request: yes
Labels: name: stale

name: ASR

# What does this PR do ?
Enables manifest sharding for bucketed ASR datasets

Add a one line overview of what this PR aims to accomplish.

**Collection**:  ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS][ZH] add Chinese TTS recipes based on IPA symbol sets.
State: closed | Pull request: yes
Labels: name: TTS

name: common

# What does this PR do ?

This PR provides 4 options for Chinese phoneme symbol sets, 24-final pinyin and its IPA conversion, 36-final pinyin and its IPA conversion. From my experiments, the IPA symbol set converted from 36-final pinyin produces the best sounds in terms of naturalness and prosody among these 4 setups. It's up to user's judge to pick either recipe.  
* add new pinyin and ipa dictionaries with 36 finals.
* add yaml configs for 24-final pinyin and ipa.
* add copyright header
* add a directory level 24finals to discriminate from 36 finals.

```
$ pytest --nightly tests/collections/tts/models/
========================================= 27 passed, 3 warnings in 330.76s (0:05:30) ==========================================
```

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
add end-of-file-fixer hook
State: closed | Pull request: yes
Labels: name: TTS

name: stale

name: ASR

name: NLP

name: Speaker Tasks

name: CI

# What does this PR do ?

enforce this hook to ensure that a file is either empty, or ends with one newline. This feature can help avoiding review code changes relevant to newlines.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
add a pre-commit hook to trim trailing whitespace.
State: closed | Pull request: yes
Labels: name: core

name: TTS

name: stale

name: ASR

name: NLP

name: Speaker Tasks

name: CI

name: common

# What does this PR do ?

enforce this hook to avoid reviewing useless code changes relevant to trailing whitespaces.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Add output audio format to preprocessing
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Add option in TTS audio preprocessing to convert audio to desired output format. Default to flac since it is lossless and uses much less space than wav.

Adding this to save on disk space for large datasets, and because I was hitting cryptic dataloader errors when training with mp3 files.

**Collection**: [TTS]

# Changelog 
- Add `output_format` CLI parameter.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Update setup_os2s_decoders.py
State: closed | Pull request: yes
Labels: name: stale

# What does this PR do ?

Install ctc_decoders without distutils duplication to avoid `AttributeError: module 'distutils' has no attribute 'ccompiler'`.

**Collection**: ASR

# Changelog 
- Fix install_beamsearch_decoders.sh to work outside container

# Usage

```
# make NeMo venv
conda create --name nemo python==3.8.10
conda activate nemo

# install NeMo
sudo apt-get update && apt-get install -y libsndfile1 ffmpeg
git clone https://github.com/NVIDIA/NeMo
cd NeMo
./reinstall.sh

# install ctc-decoders
./scripts/asr_language_modeling/ngram_lm/install_beamsearch_decoders.sh $(pwd)
```
  
**PR Type**:
- [ x ] Bugfix

# Who can review?
@titu1994
Fix k2 build topo helper
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fix `build_topo` function for `k2`-based topologies. 
This solution fixes failing tests `tests/collections/asr/k2/test_ctc.py::TestCTCLossK2::test_case_small_blank_last` + `tests/collections/asr/k2/test_rnnt.py::TestRNNTLossK2::test_case_small_blank_last`

**Collection**: [ASR]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update distopt API for coalesced NCCL calls
State: closed | Pull request: yes
Labels: name: bug

name: core

name: CI

# What does this PR do ?

The Pytorch API for coalescing NCCL calls has changed in 2.1.0 (see https://github.com/pytorch/pytorch/pull/98793), which required some changes in the Apex wrapper functions (see https://github.com/NVIDIA/apex/pull/1663).

**Collection**: NLP

# Changelog 
- Use updated API for coalescing NCCL calls in distributed optimizer

# Usage
Set the optimizer to `distributed_fused_adam` in the config file.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* PyTorch API change in https://github.com/pytorch/pytorch/pull/98793
* Apex API change in https://github.com/NVIDIA/apex/pull/1663
Start using ModelParallelConfig from Megatron Core
State: closed | Pull request: yes
Labels: name: core

name: NLP

name: CI

# What does this PR do ?

This PR is adding the ModelParallelConfig arguments to be used with the next release of Megatron Core.


**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
removed unnecessary print
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?
removes unnecessary log

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update container for import action
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Updates the base container for import action. Testing if this helps the NLP import test which is currently timing out due to solver issues with pip.

Edit: The issue was with installing and uninstalling nemo for each of the domains. This was wreaking havoc on the solver so we instead test each domain in parallel from a clean environment.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
typo fix from #6666
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

fixes small error introduced in #6666 

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
inference.compute_logprob=True fails
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

setting compute_logprob=Ture in the megatron_gpt_eval.py script is failing.

**Steps/Code to reproduce bug**
1. checkout main
2. Run ```python examples/nlp/language_modeling/megatron_gpt_eval.py gpt_model_file=path_to_nemo_gpt_model/megatron_gpt.nemo 'prompts=[How to fix GPU memory? A:]' tensor_model_parallel_size=1 inference.tokens_to_generate=32 trainer.precision=16 inference.compute_logprob=True```

This is the error:
```
Traceback (most recent call last):
  File "examples/nlp/language_modeling/megatron_gpt_eval.py", line 273, in main
    response = model.generate(
  File "/home/adithyare/NeMo/nemo/collections/nlp/models/language_modeling/megatron_gpt_model.py", line 1100, in generate
    return megatron_gpt_generate(self.cuda(), inputs, self.tokenizer, length_params, sampling_params)
  File "/home/adithyare/NeMo/nemo/collections/nlp/modules/common/text_generation_utils.py", line 108, in megatron_gpt_generate
    compute_attention_mask=sampling_params.get("compute_attention_mask", True) ** strategy_args,
TypeError: unsupported operand type(s) for ** or pow(): 'bool' and 'dict'
```

Please list *minimal* steps or code snippet for us to be able to reproduce the bug.

A  helpful guide on on how to craft a minimal bug report  http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports. 


**Expected behavior**

A clear and concise description of what you expected to happen.

**Environment overview (please complete the following information)**

 - Environment location: [Bare-metal, Docker, Cloud(specify cloud provider - AWS, Azure, GCP, Collab)]
 - Method of NeMo install: [pip install or from source]. Please specify exact commands you used to install.
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version
- PyTorch version
- Python version

**Additional context**

Add any other context about the problem here.
Example: GPU model
Do Contextnet and Squeezeformer support streaming training
State: closed | Pull request: no

So far, I know all ASR model but Contextnet and Squeezeformer in Nemo support streaming  training.

So I want to ask : Do Contextnet and Squeezeformer support streaming training?

BTW, by streaming  training, I mean chunk based like Cache-aware Streaming Conformer or model support streaming input itself during both training and inference stage, not the strategy just in inference stage like Buffered streaming.

Thanks in advance for anyone who give tips on this.
Import Enum for chatbot component
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update mcore requirement to 0.2.0
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update mcore requirement to 0.2.0
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
transcribing with stt_en_fastconformer_ctc_large fails with oom on RAM (not gpu memory) on large manifest
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

Transcribing with `stt_en_fastconformer_ctc_large` fails with oom on RAM (not gpu memory) on large manifest.
I run 
```
python ${NEMO_PATH}/examples/asr/transcribe_speech.py \
      pretrained_name="stt_en_fastconformer_ctc_large" \
      dataset_manifest=${part}.json \
      output_filename=pred_fast_ctc.$part.json \
      batch_size=128
```
where input manifest contains 168'000+ records (small phrases).
This is run on Slurm where I have to specify the required amount of RAM for the process (this is common RAM, not gpu-memory).
If I specify 15G, the process fails on 46% of transcription, for 20G - same on 64%. So this is not connected with specific record in manifest - seems to be a cumulative memory consumption.

I tested other models on the same manifest (`stt_en_conformer_transducer_large`, `stt_en_conformer_ctc_large`, `stt_en_quartznet15x5`), all work fine.

Error message:
```
Detected 1 oom-kill event(s) in StepId=753012.0 cgroup. Some of your processes may have been killed by the cgroup out-of-memory handler.
srun: error: nodeXXX: task 0: Out Of Memory
```

**Steps/Code to reproduce bug**

Please list *minimal* steps or code snippet for us to be able to reproduce the bug.

A  helpful guide on on how to craft a minimal bug report  http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports. 


**Expected behavior**

A clear and concise description of what you expected to happen.

**Environment overview (please complete the following information)**

 - Environment location: [Bare-metal, Docker, Cloud(specify cloud provider - AWS, Azure, GCP, Collab)]
 - Method of NeMo install: [pip install or from source]. Please specify exact commands you used to install.
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Environment details**

OS version: CentOS Stream 8
PyTorch version: 1.13.0+cu117
Python version: 3.9
GPU model: Titan_X