renwei2024/nemo-github-issues huggingface dataset:
Added num decoder blocks in megatron export
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
stt_ru_conformer_ctc_large Text hypothesis does not match word timestamps.
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

It is with this model "stt_ru_conformer_ctc_large" that I have problems. Can't get timestamps, I'm executing the code:

from nemo.collections.asr.parts.utils.decoder_timestamps_utils import ASRDecoderTimeStamps
asr_decoder_ts = ASRDecoderTimeStamps(cfg.diarizer)
asr_model = asr_decoder_ts.set_asr_model()
word_hyp, word_ts_hyp = asr_decoder_ts.run_ASR(asr_model)

and I get an error:

Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "/usr/local/lib/python3.8/dist-packages/nemo/collections/asr/parts/utils/decoder_timestamps_utils.py", line 674, in run_ASR_BPE_CTC
     text, char_ts, word_ts = werbpe_ts.ctc_decoder_predictions_tensor_with_ts(
   File "/usr/local/lib/python3.8/dist-packages/nemo/collections/asr/parts/utils/decoder_timestamps_utils.py", line 98, in ctc_decoder_predictions_tensor_with_ts
     word_ts, word_seq = self.get_ts_from_decoded_prediction(decoded_prediction, hypothesis, char_ts)
   File "/usr/local/lib/python3.8/dist-packages/nemo/collections/asr/parts/utils/decoder_timestamps_utils.py", line 148, in get_ts_from_decoded_prediction
     assert len(word_ts) == len(hypothesis.split()), "Text hypothesis does not match word timestamps."
AssertionError: Text hypothesis does not match word timestamps.


**Expected behavior**

Get timestamps.

**Environment overview (please complete the following information)**

  - Environment location: Docker
  - Commands used: docker run --gpus all -ti --shm-size=8g nvcr.io/nvidia/pytorch:23.02-py3

**Environment details**
i use docker
[Core] return_config=True now extracts just config, not full tarfile
State: closed | Pull request: yes
Labels: name: core

name: NLP

# What does this PR do ?

Make return_config=True now only extract out the yaml files from the tarfile, preventing opening and file extractions of all files (should significantly speedup return_config=True for Billion parameter models).

**Collection**: [Core, ASR, NLP, TTS]

# Changelog 
- Add flag to extract just config from core internals

# Usage

```python
model_config = ModelClass.restore_from(..., return_config=True)
OR
model_config = ModelClass.from_pretrained(..., return_config=True)
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Fix radtts sort r17
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

This is a duplicate PR of https://github.com/NVIDIA/NeMo/pull/6320 merged in main branch, but the release branch needs this fix as well.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Cherry-pick recent distopt commits
State: closed | Pull request: yes
Labels: name: core

name: cherry-pick

name: NLP

name: CI

# What does this PR do ?

This PR cherry-picks several recent distopt commits from the `main` branch into `r1.17.0`. 

**Collection**: NLP

# Changelog 
- GPT support for BF16 grad reductions (#5920)
- Add custom functions to launch distopt communication in interleaved pipeline parallelism (#6183)
- Bugfix for BF16 grad reductions with distopt (#6340)

# Usage

Set the optimizer to `distributed_fused_adam` in the config file:

https://github.com/NVIDIA/NeMo/blob/aaa0ccacf5b4d204ed18f6052e55ff6f20575996/examples/nlp/language_modeling/conf/megatron_gpt_config.yaml#L207

Configure the optimizer with `grad_sync_dtype: bf16`.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
remove pinned numba version in Dockerfile
State: closed | Pull request: yes
Labels: name: cherry-pick

name: CI

# What does this PR do ?

remove pinned numba version in Dockerfile

**Collection**: ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
remove pinned numba version in Dockerfile
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

remove pinned numba version in Dockerfile

**Collection**: ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Bugfix for BF16 grad reductions with distopt
State: closed | Pull request: yes
Labels: name: bug

name: core

name: NLP

# What does this PR do ?

https://github.com/NVIDIA/NeMo/pull/5920 adds support for BF16 grad reductions with distopt, with embedding grad reductions done in FP32. @mikolajblaz found some bugs and it turns out the FP32 reductions were not being done at all. This PR fixes those issues. When I run GPT-3 175B, I confirm the embedding grads are now optimized with the FP32 optimizer. Loss values are the same with FP32 and BF16 grad reductions, up to 50 steps and within numerical accuracy.

This turned out messier than I would have liked. It would have been better to integrate distopt support for multiple grad dtypes into Apex.

**Collection**: NLP

# Changelog 
- Debug distopt support for BF16 grad reductions

# Usage

Set the optimizer to `distributed_fused_adam` in the config file:

https://github.com/NVIDIA/NeMo/blob/aaa0ccacf5b4d204ed18f6052e55ff6f20575996/examples/nlp/language_modeling/conf/megatron_gpt_config.yaml#L207

Configure the optimizer with `grad_sync_dtype: bf16`.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* This PR is identical to https://github.com/NVIDIA/NeMo/pull/6297, but merges into `main` instead of `r1.17`. See that PR for discussions.
* Fixes bugs from https://github.com/NVIDIA/NeMo/pull/5920
* Depends on https://github.com/NVIDIA/apex/pull/1575
* Depends on https://github.com/NVIDIA/TransformerEngine/pull/75
TTS for Korean tokenizer/phonemizer
State: closed | Pull request: no
Labels: name: stale

I am looking to train mixer-tts model on korean dataset. I see in my config.yaml i need to complete the normalization, tokenization and phonemization. Is there a tutorial for how to train model and preprocess data for language not listed in your tts_tokenizers and then input those changes/new functions in the config.yaml?
simplified notebook for p-tuning
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Simplified the notebook for p-tuning. Sticks to a simple single task training and inference.
 
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix RNNT Joint narrow()
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Fixes a but in the signature of narrow().
Fixes #6328 

**Collection**: [ASR]

# Changelog 
- Casts the length input of all narrow() to int() explicitly

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Fix RNNT Joint narrow()
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fixes a but in the signature of narrow().
Fixes #6328 

**Collection**: [ASR]

# Changelog 
- Casts the length input of all narrow() to int() explicitly

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Minor updates to MegatronGPTPromptLearningModel
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

Fixes typos in docstring. Removed unused imports

**Collection**: NLP

# Changelog 
- fix docstring typos; remove unused imports

# Usage
n/a

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
n/a
Pin the version to hopefully fix rtd build
State: closed | Pull request: yes

# What does this PR do ?
Fix RTD using this: https://github.com/geopandas/dask-geopandas/pull/247

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
user selected max_seq_len should be less than model's max_seq_len
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

user selected max_seq_len should be less than model's max_seq_len
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Megatron hidden transformations
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

This PR adds support in applying transformation to the hiddens in Megatron Encoder-Decoder model, and augmenting a corresponding loss. This enables support in latent variable models, like VAE and MIM.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can add the following to your config :
hidden_transforms: [MegatronGaussianHiddenTransform()] and it will apply a gaussian transform to the hiddens.

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Added num decoder blocks in megatron export
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add ability to enable/disable act ckpt and seq parallelism in GPT
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?
Adds functions to enable and disable activation checkpointing and sequence parallelism. This way, we can enable them during training and disable them during inference/generation.

**Collection**: nemo/collections/nlp/language_modeling

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
simplified notebook for p-tuning
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Simplified the notebook for p-tuning. Sticks to a simple single task training and inference.
 
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Restore Thutmose Tagger tutorial, move it to nlp
State: closed | Pull request: yes
Labels: name: NLP

Restore Thutmose Tagger tutorial, move it to nlp (earlier it was in text_processing)

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Move checkpoint consolidation to cpu
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

This aims to move the megatron checkpoint to nemo conversion to cpu rather than using pytorch distributed following additions introduced [here](https://github.com/NVIDIA/NeMo/pull/6218).  This is useful for large models where the user may have checkpoints to consolidate but don't want to use the massive amount of gpus just for conversion.

**Collection**: [NLP]

# Changelog 
- Change megatron_ckpt_to_nemo.py to use CPU only

# Usage
```python
python3 megatron_ckpt_to_nemo.py \
     --checkpoint_folder <path_to_PTL_checkpoints_folder> \
     --checkpoint_name <checkpoint_name> \
     --nemo_file_path <path_to_output_nemo_file> \
     --tensor_model_parallel_size <tensor_model_parallel_size>\
     --pipeline_model_parallel_size <pipeline_model_parallel_size> \
     --model_type <model type>
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Correctly determine ZERO_LEVEL_SPEC_DB_VAL which affects normalization for first signal buffer
State: closed | Pull request: yes
Labels: name: ASR

name: common

# What does this PR do ?
If asr_model has preprocessor of type `AudioToMelSpectrogramPreprocessor`, `AudioToSpectrogram`, `AudioToMFCCPreprocessor`, etc., the `StreamingFeatureBufferer` incorrectly determines whether it uses log scale.

Instead of checking for 'log' attribute directly on preprocessor instance, it should check on the `preprocessor.featurizer` instance, which exposes the `log` property.

Therefore, the first few frames (until the generated silence is replaced by an actual audio signal) get incorrectly normalized and the model output is poor or absent.

First frame with incorrect `ZERO_LEVEL_SPEC_DB_VAL`:
![image](https://user-images.githubusercontent.com/8578790/228567021-62578f28-4b56-49b7-9f6b-952d0370d55b.png)


First frame with correct `ZERO_LEVEL_SPEC_DB_VAL`:
![image](https://user-images.githubusercontent.com/8578790/228566722-0cda3fe4-3a88-4c33-94fc-893ba4322680.png)

**Collection**: asr (`nemo.collections.asr.parts.utils.streaming_utils`)

# Changelog 
- `nemo.collections.asr.parts.utils.streaming_utils:365`
Checking for `asr_model.preprocessor.log` always resulted in False, as the `AudioToMelSpectrogramPreprocessor` doesn't define `log` property. Therefore I added an additional check for `asr_model.preprocessor.featurizer.log`.

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
Fixing bug in unsort_tensor
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
added RPE + fixed RMSNorm
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

- Added RPE for MegatronNMT models
- Fixed an issue with RMSNorm

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Make AIS CLI installation optional
State: closed | Pull request: yes
Labels: name: cherry-pick

name: CI

# What does this PR do ?

This PR makes installation of AIS CLI optional.

**Collection**: Dockerfile

# Changelog 
- Update `Dockerfile` to install AIS CLI only if `REQUIRE_AIS_CLI == true`
- Added an installer script for AIS CLI

# Usage
AIS CLI can be installed by setting `REQUIRE_AIS_CLI=true`, for example:

```bash
docker ... --build-arg REQUIRE_AIS_CLI=true
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Create functions for TTS preprocessing without dataloader
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Add modules, scripts, and tests for computing TTS pitch & energy prior to model training.

**Collection**: [TTS]

# Changelog 
- Add compute_features.py script as an alternative to extract_sup_data.py, that is standalone without depending on TTSDataset.
- Add "parts.preprocessing.features.py" with functions for computing pitch, energy, and mel spectrogram. Functions use numpy arrays for non-batched preprocessing, but could also have Tensor based inputs/outputs if it helps in the future.
- Add parts.preprocessing.features.TTSFeature as a clean and minimalistic alternative to tts_data_types to be used in the next iteration of TTS data loader.
- Add example feature configurations for 22khz and 44khz audio.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Fix some typos in Multitask_Prompt_and_PTuning notebook
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Fix some typos in Multitask_Prompt_and_PTuning.ipynb

**Collection**: NLP

# Changelog 
- Fixed typos in notebook text

# Usage
n/a


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
n/a
Add a knob to control loss broadcast
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

Add a knob to control loss broadcast. It's enabled by default.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Make AIS CLI installation optional
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

This PR makes installation of AIS CLI optional.

**Collection**: Dockerfile

# Changelog 
- Update `Dockerfile` to install AIS CLI only if `REQUIRE_AIS_CLI == true`
- Added an installer script for AIS CLI

# Usage
AIS CLI can be installed by setting `REQUIRE_AIS_CLI=true`, for example:

```bash
docker ... --build-arg REQUIRE_AIS_CLI=true
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Questions About Pre-trained Models for ACEVC
State: closed | Pull request: no
Labels: name: stale


Hi. Thank you very much for your implementation on ACEVC. I'm interested in this Zero-shot Voice Conversion project. I tried to train the model following the document on the website https://paarthneekhara.github.io/ace/code.html. However, it takes much time because of my poor hardware conditions. Are there any pretrained models for ACEVC,such as the pre-trained Conformer-SSL in step 1 or the HiFiGAN model in step 3. I'm sincerely thankful with your help.
chore: minor cleanup
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Minor cleaning, followup from #5837 

@VahidooX I think I've address you comments. Let me know if I missed something.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

@VahidooX 

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[BugFix] Fix multi-processing bug in data simulator
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fix multi-processing bug in data simulator

**Collection**: 
ASR 

# Changelog 
- Add specific line by line info of high level changes in this PR.
- Changed silence sampling logic to follow overlap sampling logic
- Changed perturbator config structure to include `add_xxx_aug` in side `xxxxxx_augmentor`
- Some comment changes
- Fixed annotation file writing mechanism to fix multi-processing issue.
- Added more unit-tests for util functions 

# Usage

```python
This script creates a synthetic diarization session using the provided audio dataset with ctm files.
Usage:
  python <NEMO_ROOT>/tools/speech_data_simulator/multispeaker_simulator.py  \
    num_workers=10 \
    data_simulator.random_seed=42 \
    data_simulator.manifest_filepath=manifest_with_alignment_file.json \
    data_simulator.outputs.output_dir=./simulated_data \
    data_simulator.outputs.output_filename=sim_spk2_sess20 \
    data_simulator.session_config.num_sessions=1000 \
    data_simulator.session_config.num_speakers=2 \
    data_simulator.session_config.session_length=20 \
    data_simulator.background_noise.add_bg=False \
    data_simulator.background_noise.background_manifest=background_noise.json \
    data_simulator.background_noise.snr=40 \

Check out parameters in ./conf/data_simulator.yaml.
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
update and deprecate warning for Mic notebook 
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

add warning about "The incompatibility of components could lead to failure of running this notebook locally with container, we might deprecate this notebook and provide a better tutorial in soon releases."


**Collection**: [ASR]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[BugFix] Fix the wrong branch name in speaker diarization inference notebook
State: closed | Pull request: yes
Labels: name: cherry-pick

name: Speaker Tasks

# What does this PR do ?

Fix the wrong branch name in speaker diarization inference notebook

**Collection**: [Note which collection this PR will affect]


# Changelog 
It should be BRANCH="r1.17.0" but it was BRANCH="main" in r1.17.0

# Usage
Run Speaker_Diarization_Inference.ipynb


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in ASR
update and deprecate warning for Mic notebook 
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

add warning about "The incompatibility of components could lead to failure of running this notebook locally with container, we might deprecate this notebook and provide a better tutorial in soon releases."


**Collection**: [ASR]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)