renwei2024/nemo-github-issues huggingface dataset:
Add new text segmentation library for better TTS quality
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?
When running below snippet for using FastPitch model trained from AISHELL-3 (the `aishell3.nemo` is from [here](https://github.com/RobinDong/nemo_models/blob/main/models/aishell3.nemo))
```python
from nemo.collections.tts.models import FastPitchModel
from nemo.collections.tts.models import HifiGanModel

spec_generator = FastPitchModel.restore_from("~/aishell3.nemo")
model = HifiGanModel.from_pretrained(model_name="tts_zh_hifigan_sfspeech")

# Generate audio
import soundfile as sf
import torch
with torch.no_grad():
    parsed = spec_generator.parse("这些新一代的中央处理器不只效能惊人。")
    spectrogram = spec_generator.generate_spectrogram(tokens=parsed, speaker=1)
    audio = model.convert_spectrogram_to_audio(spec=spectrogram)
    if isinstance(audio, torch.Tensor):
        audio = audio.to('cpu').numpy()

# Save the audio to disk in a file called speech.wav
sf.write("speech.wav", audio.T, 22050, format='WAV')
```
The `jieba` will segment the input text to
```python
['这些', '新一代', '的', '中央处理器', '不', '只', '效能', '惊人', '。']
```
and the phoneme for '只' is "zhi1", which is incorrect in this context.

But using `pkuseg`, the input text will be segmented to
```python
['这些', '新一代', '的', '中央', '处理器', '不只', '效能', '惊人', '。']
```
and the phonemes for '只' is "zhi3", which is correct.

# Changelog 
- Add a new text-segmentation-library to make better TTS quality for the Chinese language

# Usage
Setting `word_segmenter` from `jieba` to `pkuseg` in the configuration file `fastpitch_config.yaml`: 
```python
...
spec_generator = FastPitchModel.restore_from("~/aishell3.nemo", override_config_path="fastpitch_config.yaml")
...
```
The TTS sample that use `jieba`: [before.zip](https://github.com/NVIDIA/NeMo/files/12817139/before.zip)
The TTS sample that use `pkuseg`: [after.zip](https://github.com/NVIDIA/NeMo/files/12817141/after.zip)

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [x] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation


## Who can review?

@XuesongYang @blisc
Add ddp_find_unused_parameters=True and change accelerator to auto
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Group-residual vector quantizer
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

This PR adds group-residual vector quantizer as in http://arxiv.org/abs/2305.02765

**Collection**: TTS

# Changelog 
- Added group-RVQ
- Added unit tests for RVQ

# Usage
To use, add the following in the config file

```yaml
  vector_quantizer:
    _target_: nemo.collections.tts.modules.encodec_modules.GroupResidualVectorQuantizer
    num_codebooks: 4 # total number of codebooks
    num_groups: 2 # number of groups
    codebook_dim: 128 # dimension of the audio encoder output
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
bugfix: trainer.gpus, trainer.strategy, trainer.accelerator
State: closed | Pull request: yes
Labels: name: TTS

name: cherry-pick

name: ASR

name: NLP

# What does this PR do ?

fixed trainer.devices and trainer.accelerator.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
add fc large ls models
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Add fc large LS only trained models to pretrained list and docs

**Collection**: ASR

# Changelog 
- Add to pretrained list

# Usage

```python
from nemo.collections.asr.models import ASRModel
model = ASRModel.from_pretrained('stt_en_fastconformer_transducer_large_ls')
# or
model = ASRModel.from_pretrained('stt_en_fastconformer_ctc_large_ls')
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation
GPT PEFT Evaluation Fixes
State: closed | Pull request: yes
Labels: name: stale

name: NLP

name: CI

# What does this PR do ?

- Fixes a small bug with GPT PEFT evaluation
- Adds BCM support for running evaluation

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Propagate mp config for continue training
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
PEFT eval fix
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Fix issue where peft weights are not loaded for distributed checkpoints

**Collection**: NLP

# Changelog 
- Load peft weights in restore_from instead of _load_state_dict_from_disk if distributed checkpoint is used

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Propagate mp config for continue training
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[ASR] Fix type error in jasper
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Currently when running VAD marblenet you sometimes randomly get the error:

```
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1514, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/nemo/collections/asr/parts/submodules/jasper.py", line 351, in forward
    self.update_masked_length(x.size(2), device=lens.device)
  File "/usr/local/lib/python3.10/dist-packages/nemo/collections/asr/parts/submodules/jasper.py", line 374, in update_masked_length
    self.lens, self.max_len = _masked_conv_init_lens(self.lens, max_len, self.max_len)
RuntimeError: _masked_conv_init_lens() Expected a value of type 'Tensor' for argument 'original_maxlen' but instead found type 'int'.
; ((Tensor, Tensor))
Cast error details: Unable to cast 64 to Tensor
terminate called without an active exception
```

This happens because in this line of code, it assigns self.max_len to an integer value:

https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/asr/parts/submodules/jasper.py#L378

Whereas in all other code paths it assigns it to a tensor:

https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/asr/parts/submodules/jasper.py#L321

https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/asr/parts/submodules/jasper.py#L374
https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/asr/parts/submodules/jasper.py#L231
https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/asr/parts/submodules/jasper.py#L234

**Collection**: [ASR]

# Changelog 
- Convert max_len to tensor.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
[ASR] RNN-T greedy decoding max_frames fix for alignment and confidence
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fixes RNN-T greedy decoding: writes blank alignment and confidence immediately.
Fixes https://github.com/NVIDIA/NeMo/issues/7484

**Collection**: [Note which collection this PR will affect]
ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to https://github.com/NVIDIA/NeMo/issues/7484
Modular SpeechLLM implementation for Sept. 2023 submission (SALM)
State: closed | Pull request: yes
Labels: name: ASR

name: NLP

name: common

# What does this PR do ?

Include Modular SpeechLLM implementation for Sept. 2023 submission (SALM). The implementation integrates a pretrained FC and a frozen LLM, and can do ASR, AST, and in-context learning for keyword boosting.

The current implementation is based off of r1.20.0 release.

**Collection**: multimodal

# Usage

refer to `workspace/run_sft_audio_lm.py`

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
Modular SpeechLLM implementation for Sept. 2023 submission (SALM) 
State: closed | Pull request: yes
Labels: name: core

name: TTS

name: ASR

name: NLP

name: Speaker Tasks

name: CI

name: common

# What does this PR do ?

Include Modular SpeechLLM implementation for Sept. 2023 submission (SALM). The implementation integrates a pretrained FC and a frozen LLM, and can do ASR, AST, and in-context learning for keyword boosting.

The current implementation is based off of r1.20.0 release.

**Collection**: multimodal

# Usage

refer to `workspace/run_sft_audio_lm.py`

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
WIP Speechllm 
State: closed | Pull request: yes
Labels: name: ASR

name: NLP

name: common

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update the TopKSequenceGenerator to support Top-P sampling
State: closed | Pull request: yes
Labels: name: stale

name: ASR

name: NLP

# What does this PR do ?

Implements Top-P sampling, also known as nucleus sampling.

I put it in with the Top-K sampler, as I felt like this kept the code cleaner – at the cost of an `if` statement in the decoding loop. I could move it to its own class as well, but since a lot of people use Top-K and Top-P together, we'd probably want that new class to support both anyway.

Let me know if there are any changes needed :)

**Collection**: NLP, ASR

# Changelog 
- Add the parameter `top_p: Optional[float]` to the `nemo.collections.nlp.modules.common.transformer.transformer_generators.TopKSequenceGenerator` and the `nemo.collections.asr.modules.transformer.transformer_generators.TopKSequenceGenerator` classes
- This allows using Top-P sampling in combination with Top-K sampling
- Adds parameters for `top_p` to the two places in code where these classes are called

# Usage
```python3
from nemo.collections.nlp.models.machine_translation import MTEncDecModel
from nemo.collections.nlp.modules.common.transformer.transformer_generators import TopKSequenceGenerator

model = MTEncDecModel.from_pretrained('nmt_en_es_transformer12x2')
top_k_and_p_search = TopKSequenceGenerator(
    model.beam_search.embedding,
    model.beam_search.decoder,
    model.beam_search.log_softmax,
    pad=model.beam_search.pad,
    bos=model.beam_search.bos,
    eos=model.beam_search.eos,
    beam_size=5,
    top_p=0.75,
)

s = 'Among the various methods of peeling a cucumber, I find only a few to be worth my precious time.'
print(model.translate([s]))
# ['Entre los diversos métodos de pelar un pepino, encuentro solo unos pocos que valen mi precioso tiempo.']

model.beam_search = top_k_and_p_search
print(model.translate([s]))
# Varies, e.g.,
# ['Entre los diferentes métodos de pelar un pepino, solo encuentro que unos pocos valen mi valioso tiempo.']
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc): No
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix import error no module name model_utils
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[NLP] Rotary embeddings issue with TransformersEngine
State: closed | Pull request: no
Labels: name: bug

name: NLP

**Describe the bug**

We observed a discrepancy between numerical outputs of the NeMo and TE implementations of GPT. It seems to come from rotary embeddings not being passed to the TE module.

**Steps/Code to reproduce bug**

In https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/nlp/modules/common/megatron/transformer.py#L1531 , the `rotary_pos_emb` argument is missing from the forward call to TE module.

**Expected behavior**

Same results with and without TE for GPT arch with rotary embeddings.

**Additional context**

Same issue seems present with checkpointing.
bugfix: pin nemo-text-process to fix Chinese normalizer error.
State: closed | Pull request: yes

# What does this PR do ?

Related to [bug report](https://nvbugswb.nvidia.com/NvBugs5/SWBug.aspx?bugid=4312419&cmtNo=), 

Chinese normalizer outputted wrong normalized text for the input, '双辽境内除东辽河、西辽河等**5**条河流', where we expect '双辽境内除东辽河、西辽河等**五**条河流'.

**Evidence**

- On the latest version, i.e. nemo-text-processing=0.2.0rc0, it always normalized it as '双辽境内除东辽河、西辽河等5条河流';

- On the previous version, i.e. nemo-text-processing=0.1.8rc0, it normalized it as expected text, '双辽境内除东辽河、西辽河等五条河流'.

**Detailed Log**

- 0.2.0rc0
```
In [1]: from nemo_text_processing.text_normalization.normalize import Normalizer

In [2]: normalizer = Normalizer(lang='zh', input_case='cased', overwrite_cache=True, cache_dir="./cache_dir")

In [3]: text = '双辽境内除东辽河、西辽河等5条河流'

In [4]: normalizer.normalize(text)
Out[4]: '双辽境内除东辽河、西辽河等5条河流'
```
- 0.1.8rc0
```
Downloading nemo_text_processing-0.1.8rc0-py3-none-any.whl (2.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.3/2.3 MB 12.5 MB/s eta 0:00:00
Installing collected packages: nemo-text-processing
  Attempting uninstall: nemo-text-processing
    Found existing installation: nemo-text-processing 0.2.0rc0
    Uninstalling nemo-text-processing-0.2.0rc0:
      Successfully uninstalled nemo-text-processing-0.2.0rc0
Successfully installed nemo-text-processing-0.1.8rc0


In [1]: from nemo_text_processing.text_normalization.normalize import Normalizer

In [2]: normalizer = Normalizer(lang='zh', input_case='cased', overwrite_cache=True, cache_dir="./cache_dir")

In [3]: text = '双辽境内除东辽河、西辽河等5条河流'

In [4]: normalizer.normalize(text)
Out[4]: '双辽境内除东辽河、西辽河等五条河流'
```
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
PEFT eval fix
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Fix issue where peft weights are not loaded for distributed checkpoints

**Collection**: NLP

# Changelog 
- Load peft weights in restore_from instead of _load_state_dict_from_disk if distributed checkpoint is used

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix ptuning crash for llama 2 ckpt
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add ddp_find_unused_parameters=True and change accelerator to auto
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix llama2 70b lora tuning bug
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Fixes a bug in LLaMa2 70b LoRA tuning. There was a modification to specifically deal with LLaMa2 70b (which uses GQA) in the old API that was not propagated during the refactor.

**Collection**: NLP

# Changelog 
- Modified LoraPEFTConfig to exactly follow the logic in the old MegatronGPTLoRAModel

# Usage

```python
python megatron_gpt_peft_tuning.py <...> model.restore_from_path=<a llama2 70b checkpoint>
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #7299 and #7308
bugfix: trainer.gpus, trainer.strategy, trainer.accelerator
State: closed | Pull request: yes
Labels: name: TTS

name: ASR

name: NLP

# What does this PR do ?

fixed trainer.devices and trainer.accelerator.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Bound transformers version in requirements
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Safeguard nemo_text_processing installation on ARM (#7485)
State: closed | Pull request: yes
Labels: name: TTS

name: ASR

name: NLP

* safeguard nemo_text_processing installing



* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update check



---------

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Get scores from inferences from the Text_Classification_Sentiment_Analysis script
State: closed | Pull request: no
Labels: name: bug

Hello everyone, I trained a model with Text_Classification_Sentiment_Analysis.ipynb and I would like to know if I can extract the scores for each sentiment e.g.:

"I answered a call" is a neutral phrase, what score does the engine understand that this phrase is neutral rather than positive or negative?
Is it possible to obtain this information?
I'm using extraction this way

```
model = nemo_nlp.models.TextClassificationModel.restore_from("path/model")

l_results = model.classifytext(queries=keys_list, batch_size=3, max_seq_length=512)
```
Added NKB
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix py3.11 dataclasses issue 
State: closed | Pull request: yes
Labels: name: core

name: TTS

name: cherry-pick

name: ASR

name: NLP

name: common

# What does this PR do ?

Fixes an issue with mutable defaults inside dataclasses across NeMo configs

Fixes #7166 

**Collection**: [Core, ASR, NLP, TTS, Tools]

# Changelog 
- Guard mecab and ipadic
- Remove dependence on mecab and ipadic from sacrebleu
- Fix dataclasses across domains and tools

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Minor blog website updates
State: closed | Pull request: yes

# What does this PR do ?

Updating metadata and docs. Changing buttons on home page.

**Collection**: NeMo github io website

# Changelog 
- explain `<!-- more -->` tag in template and README
- update date of the Numba post
- update the OpenGraph metadata of the NFA, alignment and Numba blog posts
- add documentation of OpenGraph metadata in the README
- reorder the metadata in the blog posts so it's a bit tidier
- change 3 ASR/Megatron/TTS buttons on the homepage to one 'Documentation' button.

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [N/A] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [N/A] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix ssl models ptl monitor val through logging
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Add logging of loss by adding to validation_outputs (self.log) for ssl_models

**Collection**: ASR

# Changelog 
- Add to self.log by setting to validation_step_outputs

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
adding special_tokens from tokenizer config for transformer-lm model
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

`special_tokens` are currently ignored when constructing a tokenizer for `transformer_lm_model` using the following example: https://github.com/NVIDIA/NeMo/blob/5cb76a53f21d7470773a80d62fd513defa24ddbf/examples/nlp/language_modeling/conf/bert_pretraining_from_text_config.yaml#L51

**Collection**: NLP/language-modeling

# Changelog 
- transformer_lm_model: use `special_tokens` from config when creating the tokenizer, making it backward-compatible with previous behavior (when `special_tokens` are not provided we proceed with `None`).

# Usage
* https://github.com/NVIDIA/NeMo/blob/5cb76a53f21d7470773a80d62fd513defa24ddbf/examples/nlp/language_modeling/conf/bert_pretraining_from_text_config.yaml#L51

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
 - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix metrics for SE tutorial
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

This PR fixes a bug in SE tutorial caused by initialization of metrics for audio-to-audio models.

**Collection**: ASR, Tutorials

# Changelog 
- Updated metric initialization
- Small fixes in the tutorial notebook

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[pre-commit.ci] pre-commit suggestions
State: closed | Pull request: yes
Labels: name: stale

<!--pre-commit.ci start-->
updates:
- [github.com/pre-commit/pre-commit-hooks: v4.3.0 → v4.4.0](https://github.com/pre-commit/pre-commit-hooks/compare/v4.3.0...v4.4.0)
- [github.com/psf/black: 19.10b0 → 23.9.1](https://github.com/psf/black/compare/19.10b0...23.9.1)
<!--pre-commit.ci end-->