renwei2024/nemo-github-issues huggingface dataset:
Adding server option to peft eval
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Adding server options to GPT PEFT eval

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage

```bash
python /data/NeMo/examples/nlp/language_modeling/tuning/megatron_gpt_peft_eval.py \
        model.restore_from_path=/data/megatron_converted_2b_tp1_pp1.nemo \
        model.peft.restore_from_path=/data/megatron_gpt_peft_tuning.nemo \
        model.data.test_ds.file_names=['/models/dataset/squad/squad_subsamples_formatted_tn/100/1_squad_val.jsonl'] \
        model.data.test_ds.global_batch_size=1 \
        model.data.test_ds.micro_batch_size=1 \
        model.data.test_ds.names=['squad'] \
        model.data.test_ds.tokens_to_generate=30  \
        server=True
```

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix alibi with tp>1 eval
State: closed | Pull request: yes
Labels: name: stale

name: NLP

name: common

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Adapter ipa
State: closed | Pull request: yes
Labels: name: TTS

name: NLP

# What does this PR do ?

Adds a flag in the adapter tutorial to choose between ipa and arpabet.

**Collection**: [Note which collection this PR will affect]
TTS
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.
sync O2 fix with main
State: closed | Pull request: yes
Labels: name: core

name: NLP

name: CI

* fix ptuning and lora model_parallel_config



* support deprecated models



* update megatron connit sha



---------

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix ptuning and lora model_parallel_config
State: closed | Pull request: yes
Labels: name: core

name: NLP

name: CI

# What does this PR do ?

replace dummy model_parrallel_config with the real one after initializing base model.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Draft: Add SHARP interface to Megatron Core
State: closed | Pull request: yes
Labels: name: TTS

name: stale

name: ASR

name: NLP

name: Speaker Tasks

name: CI

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

Adding SHARP interface to Megatron Core

# Changelog 
Add SHARP enable argument to parallel state function

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix adapters and ptuning for amp O2
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Fix the issues when using megatron_amp_O2 on PEFT models

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Transform adapters to fp16/bf16 when the base model uses megatron_amp_O2. When using amp_O2 we should make model parameters in fp16/bf16. Previous PR #7232 handles the precision issue in ColumnParallelLinear and RowParallelLinear in PEFT models, and this PR handles precision casting of the rest of the modules such as layernorm.
- Explicitly set the ptuning inference table to untrainable for FP16/BF16 modules. Previous whitelist only includes the name of the inference table as in a FP32 model. We add its name in FP16/BF16 models to whitelist.
- Explicitly set ptuning inference table to be untrainable in AdapterPTuningModel like in PTuningModel

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #7232
Shehzeen speechlm
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Conv1d subsampling modules: striding_conv1d and dw_striding_conv1d
State: closed | Pull request: yes
Labels: name: core

name: ASR

name: common

# What does this PR do ?

Add conv1d subsampling for ASR / lip reading

**Collection**: asr

# Changelog 
- Add 'striding_conv1d' and 'dw_striding_conv1d' subsampling options. Subsampling is performed using strided conv1d convolutions with kernel size 5 instead of doing reshape + frequency-temporal conv2d + reshape + linear projection.

# Usage
* Simply use 'subsampling: striding_conv1d in conformer yaml config for example.

```
  encoder:
      _target_: nemo.collections.asr.modules.ConformerEncoder
      feat_in: ${model.video_front_end.dim_output}
      feat_out: -1 # you may set it if you need different output size other than the default d_model
      n_layers: 18
      d_model: 512
  
      # Sub-sampling params
      subsampling: striding_conv1d # vggnet, striding, stacking or stacking_norm, dw_striding, striding_conv1d_k5, dw_striding_conv1d
      subsampling_factor: 8 # must be power of 2 for striding and vggne
...
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Integrate dist ckpt with new DistOpt state dict v2
State: closed | Pull request: yes
Labels: name: core

name: NLP

# What does this PR do ?

This MR integrates dist ckpt with recent improvements of [Apex DistributedOptimizer state dict](https://github.com/NVIDIA/apex/pull/1704).
It makes the [old integration PR](https://github.com/NVIDIA/NeMo/pull/7140) redundant.

**Collection**: NLP

# Changelog 
- Add distributed checkpointing integration with DistOpt.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
File lists for pretrain model
State: closed | Pull request: no
Labels: name: stale

Hello, I read [this](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/stt_ru_conformer_ctc_large) page and it says that you used `SOVA - RuAudiobooksDevices [260 hours] and RuDevices [75 hours] subsets`.
Could you share the file lists of these subsets? Or code to reproduce these subsets?

Thank you in advance.

@titu1994
Speechlm paarth Separate Speaker Embeddings, some fixes for 3b Model
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
loss mask aware final layer applicaiton
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

save memory during training by loss mask selection for final output layer + softmax.


**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Remove prefetch for megatron_gpt_sft_model.py
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Replaces `super().validation_step` in `inference_step` of `megatron_gpt_sft_model.py` with the `validation_step` from `megatron_gpt_model` but without prefetch as just one batch is passed to the `validation_step` in `megatron_gpt_sft_model` and prefetching causes an error.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix restore sequence parallel
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] minor fix typos and input_types
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Fixed duplicate definition of `output_types` for class `MultiPeriodDiscriminator`.

**Collection**: TTS

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update numba support in docker
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Update Numba version in Docker

**Collection**: [ASR]

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Fix skip generation
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Use generation if user set `write_predictions_to_file` to `True` or `dataset.metric.name!=loss` 

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix adapters and ptuning for amp o2
State: closed | Pull request: yes
Labels: name: core

name: NLP

# What does this PR do ?

Fix the issues when using megatron_amp_O2 on PEFT models

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Transform adapters to fp16/bf16 when the base model uses megatron_amp_O2. When using amp_O2 we should make model parameters in fp16/bf16. Previous PR #7232 handles the precision issue in ColumnParallelLinear and RowParallelLinear in PEFT models, and this PR handles precision casting of the rest of the modules such as layernorm.
- Explicitly set the ptuning inference table to untrainable for FP16/BF16 modules. Previous whitelist only includes the name of the inference table as in a FP32 model. We add its name in FP16/BF16 models to whitelist.
- Explicitly set ptuning inference table to be untrainable in AdapterPTuningModel like in PTuningModel

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #7232
Bugfix and optimization in `end_of_generation_condition()`
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

It fixes a bug in `end_of_generation_condition()` (#7187) and makes it significantly faster in some cases.

**Collection**: nlp

# Changelog 
- Fixed edge case that could make text generation stop earlier than intended
- Speed up text generation when using custom end strings

# Detailed explanation

1. Bugfix

The previous implementation did not verify that `end_string` was encoded
into a single token, which could trigger the end of generation earlier
than intended (see discussion in #7187)

2. Optimization

The previous implementation was scaling linearly with the batch size and
quadratically with the length of the generated sequence, which could
lead to a significant overhead in some situations.

The new implementation is much more efficient in "normal" situations
(where the end of generation is identified by a set of unique tokens),
and raises a warning when it needs to fallback to the inefficient string
matching case.

Note that it does not behave exactly the same as before, because we skip
the string comparison when the end strings all have unique tokens
associated to them. For instance, in the previous implementation, if the
model had generated the string
        "Some string.
"
(where "
" would really be generated as a string, and not as
a single token), then the previous implementation would have considered
it to be the end of generation (assuming `end_strings` has length > 1),
while the new one would not. The previous behavior was likely a bug
though, since we expect models to generate the special tokens associated
to end strings when they exist (for instance, the standard case
`end_strings=["
"]` has always been handled by just
comparing the last token to `eod_id`).

3. Minor change

See commit message of ab56968fd for the explanation regarding the warning that was added in `__init__()`

# Tests

Hopefully there are existing tests on CI -- I have tested this myself on my own jobs.

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information

Fixes ##7187
[TTS] Audio codec fixes
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

A few minor fixes for the audio codec training recipe.

Most important, moving the zero_gradient() call back before the loss calculations. Having it after `optim_disc.step()` results in it saving discriminator gradients during the generator `self.manual_backward(loss_gen_all)` calculation which stops the discriminator from learning (adversarial and feature losses quickly become 0).

**Collection**: [TTS]

# Changelog 
- Zero gradient before discriminator loss calculation
- Change default mel loss scale from 1.0 to 5.0 as it significantly improves ViSQOL and makes training more consistent.
- Fix name of validation time domain loss

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Add the `strategy` argument to `MegatronGPTModel.generate()`
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

The motivation is to be able to override the text generation strategy when calling `MegatronGPTModel.generate()`.

It is passed as an explicit argument rather than through `**strategy_args` so as to ensure someone cannot accidentally pass other arguments that would end up being ignored.

It is a keyword-only argument to ensure that if in the future we want to update the signature to `**strategy_args`, we can do it without breaking code.

**Collection**: nlp

# Changelog 
- Add the `strategy` argument to `MegatronGPTModel.generate()`


# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [X] New Feature
- [ ] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
Add finetuning scripts
State: closed | Pull request: yes
Labels: name: ASR

name: CI

# What does this PR do ?

Add a finetuning script along with config file for finetuning existing nemo models. 

**Collection**: ASR

# Changelog 
- Added speech_to_text_finetune.py script
- Added speech_to_text_finetune.yaml config 
- Jenkins run
- Doc strings

TODO:
* Update tutorial

# Usage
Eg: without changing tokenizer
```bash
 python speech_to_text_finetune.py \
 init_from_pretrained_model=stt_en_fastconformer_hybrid_large_pc \
 model.tokenizer.update_tokenizer=false
```
or
finetune with a new tokenizer (architecture of base model doesn't change)
```bash
 python speech_to_text_finetune.py  \
 init_from_pretrained_model=stt_en_fastconformer_hybrid_large_pc \
 model.tokenizer.update_tokenizer=true \
 model.tokenizer.dir=<path_to_tokenizer_dir> \
 model.tokenizer.type=<bpe/char>
```


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Resume checkpoint priority
State: closed | Pull request: yes

# What does this PR do ?

Fix resume from checkpoint behavior to have lower priority than experiment dir.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- resume from checkpoint behavior
- reorganize `check_resume` for prints

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add comprehensive error messages
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

Adds comprehensive error messages for cases when input has wrong type

**Collection**: nlp

# Changelog 
- Add a couple of error messages


# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Adapter ipa Tutorial and config update
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?
Updates the tutorial to support IPA and adds an adapter IPA config.

**Collection**: [Note which collection this PR will affect]
TTS
  
**PR Type**:
- [ *] New Feature
- [ ] Bugfix
- [ *] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[NLP] Bug Fix: Fix `full_logits` Initialization for `all_probs=True` & `compute_logprob=False` in `text_generation_utils.py`
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

This PR ensures that the `sample_sequence_batch` function correctly handles the scenario where `all_probs` is set to `True` but `compute_logprob` is `False`. 

Earlier the `full_logits` variable was not correctly initialized when the `all_probs=True` but `compute_logprob=False` causing a `RuntimeError` due to an attempt to broadcast a `None` value using `torch.distributed.broadcast`.


**Collection**: NLP

# Changelog 
- Updated conditions on lines 704, 754, and 792 in `text_generation_utils.py` to handle cases where `all_probs=True` but `compute_logprob=False`.
This ensures that `full_logits` is correctly initialized, preventing runtime errors during distributed broadcasting.

# Usage
With this fix, users can now set `compute_logprob` to `False` and `all_probs` to `True` in the inference section of the config. This ensures that the `sample_sequence_batch` function correctly handles this specific scenario without encountering the previously observed `RuntimeError`.

For example:

```yaml
inference:
  ...
  all_probs: True  # whether to return the log prob for all the tokens in the vocab
  ...
  compute_logprob: False  # a flag used to compute logprob of all the input text, a very special case of running inference, default False
```
# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Logging model checkpoints as artifacts in MlFlow 
State: closed | Pull request: yes

# What does this PR do ?

MlFlow is required for training and resuming runs from checkpoint. Currently, when the MlFlow logger is enabled in experiment manager it doesn't log model checkpoint as artifacts to MLFlow.
This PR simply adds one line to the `nemo.utils.loggers.MLFlowParams` to allow for logging checkpoints.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add `log_model ` to `nemo.utils.loggers.MLFlowParams`

# Usage
To enable logging checkpoints in MlFlow
```python
exp_manager:
  ...
  create_checkpoint_callback: True
  create_mlflow_logger: True
  mlflow_logger_kwargs:
  ...
      log_model: True
```


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #7257
Experiment manager MlFlow logger does not log model checkpoints as artifacts
State: closed | Pull request: no

**Is your feature request related to a problem? Please describe.**

MlFlow is required for training and resuming runs from checkpoint. Currently, when the MlFlow logger is enabled in experiment manager it doesn't log model checkpoint as artifacts to MLFlow

**Describe the solution you'd like**

One of the parameters (`log_model`) must to be passed to `pytorch_lightning.loggers.MLFlowLogger` is not defined in `nemo.utils.loggers.MLFlowParams` dataclass and cannot be set without modifying the code.

**Describe alternatives you've considered**

Doesn't seem to have an alternative

**Additional context**

Add any other context or screenshots about the feature request here.
Dummy class for ModelParallelConfig
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Sets a dummy class for type hinting when import is not found.

**Collection**: NLP
# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[DRAFT][TTS] Support tarred dataset in TTS
State: closed | Pull request: yes
Labels: name: TTS

name: stale

# What does this PR do ?

Add support for tarred datasets in TTS

**Collection**: tts

# Changelog 
- Add specific line by line info of high level changes in this PR.
Created and Iterative TarredTTSDataset class which inherits nemo.collections.asr.data.audio_to_text._TarredAudioToTextDataset.
This class adds takes in tarred dataset and manifests as input and creates batches of data for TTS models.


# Usage
* You can potentially add a usage example below

```python
# Use this config: [examples/tts/conf/fastpitch_align_v1.05_tarred.yaml] 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Checkpoint averaging for model parallel
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Adding Multilingual, Code-Switched, and Hybrid ASR models
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

This PR:

1. Adds the multilingual models (stt_multilingual_fastconformer_hybrid_large_pc, stt_multilingual_fastconformer_hybrid_large_pc_blend_eu) to the pre-trained models catalog and ASR docs
2. Adds the FastConformer Hybrid FR, RU ASR models (stt_ru_fastconformer_hybrid_large_pc, stt_fr_fastconformer_hybrid_large_pc)  to the pre-trained models catalog and ASR docs
3. Adds the code-switching models (stt_enes_conformer_transducer_large_codesw, stt_enes_conformer_ctc_large_codesw) to the ASR docs
4. Updates the versions of the FastConformer Hybrid ASR models (EN, ES, IT, DE, PL, BY, HR, UA) 

**Collection**: ASR

# Changelog 
- Added the FC Hybrid models to nemo/collections/asr/models/hybrid_rnnt_ctc_bpe_models.py
- Updated the ASR docs at NeMo/docs/source/asr/results.rst
- Created new results tables for Multilingual (NeMo/docs/source/asr/data/benchmark_multilingual.csv), Code Switching(NeMo/docs/source/asr/data/benchmark_code_switching.csv)

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation
[ASR] Fix GPU memory leak in transcribe_speech.py
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fix memory leak in utility used by `transcribe_speech.py` when transcribing partial audio segments, by moving logit tensor stored in results to CPU.

**Collection**: [ASR]

# Changelog 
- Modify `transcribe_partial_audio` to move logits to CPU when when `logprobs=False`.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation