renwei2024/nemo-github-issues huggingface dataset:
NeMo + Lhotse integration
State: closed | Pull request: yes
Labels: name: ASR

name: CI

name: common

# What does this PR do ?

This PR adds an option to leverage Lhotse for dataloading in NeMo.

**Collection**: Currently ASR only, with clear path to extend into other collections.

# Changelog 
- Support creating a Lhotse dataloader from NeMo configuration
- Support for most Lhotse-specific dataloading features, especially: dynamic bucketing, multiplexing datasets.
- Support initializing Lhotse dataloader either from Lhotse manifests, Lhotse Shar (tarred) manifests, NeMo manifests, and NeMo tarred manifests
- ASR dataloading for hybrid CTC transducer as the first example

# Usage
* See the unit tests
* NeMo's `model.{train,validation,test}_ds` sections can be extended with additional fields `use_lhotse: True/False` and `lhotse: {...}` to enable this. 

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [x] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

@titu1994 all suggestions implemented

Need to pass jenkins though

jenkins

jenkins

Thanks guys! Such a joy to click merge after all this work 😀
add values to en tts dict
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Add terms to tts ipa pronunciation dictionary for en-US to meet internal needs OOTB

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

@ekmb any updates on this?

> @ekmb any updates on this?

@mgrafu I'm good with merge once the CI is green

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

jenkins
Mengruw/fixed tgt lang v3
State: closed | Pull request: yes
Labels: name: TTS

name: cherry-pick

name: stale

name: ASR

name: NLP

name: Speaker Tasks

name: CI

name: common

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
swig_decoders dependency
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

Cannot run the n-gram beam search script.

**Steps/Code to reproduce bug**

Run the `./install_beamsearch_decoders.sh`

 ``` import swig_decoders
ModuleNotFoundError: No module named 'swig_decoders'
```

Ubuntu 22.04.3


It should have been installed as per [line](https://github.com/NVIDIA/NeMo/blob/77d1386174930239fb619a7d4660a88f2061a5b3/scripts/installers/setup_os2s_decoders.py#L138C18-L138C30).


Was it unsuccessful during installation? Can you check your logs and post the error here? 

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.

@nithinraok When I run `./install_beamsearch_decoders.sh` I'm getting `file swig_decoders.py (for module swig_decoders) not found`  for `NeMo 1.23.0`

```
writing manifest file 'ctc_decoders.egg-info/SOURCES.txt'
file swig_decoders.py (for module swig_decoders) not found
reading manifest file 'ctc_decoders.egg-info/SOURCES.txt'
writing manifest file 'ctc_decoders.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
file swig_decoders.py (for module swig_decoders) not found
creating build
creating build/lib.linux-x86_64-cpython-310
copying ctc_decoders.py -> build/lib.linux-x86_64-cpython-310
file swig_decoders.py (for module swig_decoders) not found
```

It should be installed as part of OpenSeqtoSeq decoder installers setup:https://github.com/NVIDIA/OpenSeq2Seq/blob/master/decoders/setup.py 

Relevant issue: https://github.com/PaddlePaddle/models/issues/315 

@karpov-nick do you know if users need to install it separately? 

@nithinraok @karpov-nick 

I did not face `file swig_decoders.py (for module swig_decoders) not found` when I use image `docker pull nvcr.io/nvidia/nemo:24.01.speech` But I faced `Segmentation fault issue` please check details in [10557](https://github.com/NVIDIA/NeMo/discussions/10557) 

Hi @nithinraok Any update regarding this?
NeMo MCore Llama2 support fp8
State: closed | Pull request: no
Labels: name: stale


Hi, does Nemo MCore llama2 currently support training with fp8? I got this error when training the llama model with fp8
![Screenshot from 2023-11-09 18-03-17](https://github.com/NVIDIA/NeMo/assets/25007718/57c79bc4-8610-4f45-a042-dcdc8b69bf0d)

Thanks !

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.
Add MCore T5
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
GPU usage by NEMO ASR Model
State: closed | Pull request: no

I want to limit the usage of Nemo[Asr] mode, lets say i want to limit it to 5 GB of VRAM. how can i do it? is there any way to limit this?

You can try using this before important anything from NeMo - https://github.com/pytorch/pytorch/blob/main/torch/cuda/memory.py#L121
Issue faced in Nemo Model Inference using Farward collection
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

The farward nemo collection of Nemo ASR model requires following parameters.

-  input_signal – Tensor that represents a batch of raw audio signals, of shape [B, T]. T here represents timesteps, with 1 second of audio represented as self.sample_rate number of floating point values.

- input_signal_length – Vector of length B, that contains the individual lengths of the audio sequences.

**Steps/Code to reproduce bug**
Load the model
hi_nemo_asr=nemo_asr.models.EncDecCTCModelBPE.restore_from(en_model_path).cpu()
waveform, sample_rate = torchaudio.load(speech_file[0])
input_signal_length=input_signal.shape[1]
log_prob,_,_=hi_nemo_asr.forward(input_signal=waveform,input_signal_length=waveform.shape[1])

This gives the following error
IndexError: tuple index out of range

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
-Os =Linux
- PyTorch version =2.1.0
- Python version=3.8.10



Your reproducible steps are unclear as you are using some variables which are out of scope from what described. 

However for forward, signal should be BxT and signal_length should be B size **tensors**. 

 Waveform.shape[idx] would return int but not torch tensor. 

Can you change it to `torch.tensor([waveform.shape[1]])` and retry?

@Pri4tam Assuming tha tthe above uggestions solved your bug.. Please reopen if it is unresolved.
Continue training from 'ckpt' file
State: closed | Pull request: no

Is it possible somehow to continue training from checkpoint that made by pytorch_lightning? I have been training ASR model and load pretrained model. I want to continue training from exact position when server shut down. Do I need load from model.nemo or is there more gentle way to do this?

if you are using `https://github.com/NVIDIA/NeMo/blob/main/examples/asr/speech_to_text_finetune.py` to finetune then we currently don;t support loading from ckpt file. 
However while training you can always save nemo file by toggling https://github.com/NVIDIA/NeMo/blob/d49b73cb495dfd17554b01cf1b0f6f8f1bbbbc7c/nemo/utils/exp_manager.py#L105 through config. 

Make sure you toggle and use these values accordingly: https://github.com/NVIDIA/NeMo/blob/d49b73cb495dfd17554b01cf1b0f6f8f1bbbbc7c/nemo/utils/exp_manager.py#L135..L143 

I am using https://github.com/Open-Speech-EkStep/vakyansh-nemo-experimentation/blob/main/src/finetuning/speech_to_text_ctc_bpe.py#L104 to finetune. I have trained the pre-trained model up to 30 epochs. So, can I continue training from the 'ckpt' generated at the 30th epoch?

@mukherjeesougata I did something like that:
```
    trainer = ptl.Trainer(
        #callbacks=[early_stop_callback],
        devices=-1 if device == torch.device('cuda') else 1,
        accelerator = "gpu",
        max_epochs=settings.num_epochs,
        accumulate_grad_batches=settings.accumulate_grad_batches,
        enable_checkpointing=True,
        logger=True,
        log_every_n_steps=1,
        check_val_every_n_epoch=1,
        resume_from_checkpoint=ckpt_path
    )

    trainer.fit(model)
```


@mukherjeesougata 
> I am using https://github.com/Open-Speech-EkStep/vakyansh-nemo-experimentation/blob/main/src/finetuning/speech_to_text_ctc_bpe.py#L104 to finetune. I have trained the pre-trained model up to 30 epochs. So, can I continue training from the 'ckpt' generated at the 30th epoch?

Yes.
added converter for lora checkpoints to tp1
State: closed | Pull request: yes
Labels: name: stale

# What does this PR do ?

added converter for lora checkpoints to tp1

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Tar codec
State: closed | Pull request: yes
Labels: name: TTS

Add webdataset support for VocoderDataset

**Collection**: TTS

# Changelog
- Add TarredVocderDataset class for dataset fetching
- sampler is not supported with IterableDataset
- added a conditional code to calculate correct number of max steps when webdataset is used

Known issue:
- Haven;t tested for passing multiple tarred datasets. Will be pushing a PR later to support it. 

# Usage
* add `is_sharded=True`  in cfg for train_ds to load webdataset

# Before your PR is Ready for review
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?

**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins
Mengruw/fixed tgt lang v3
State: closed | Pull request: yes
Labels: name: TTS

name: ASR

name: NLP

name: Speaker Tasks

name: CI

name: common

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins
Tar codec
State: closed | Pull request: yes
Labels: name: TTS

name: ASR

# What does this PR do ?

Add webdataset support for VocoderDataset

**Collection**: TTS

# Changelog 
- Add TarredVocderDataset class for dataset fetching
- sampler is not supported with IterableDataset
- added a conditional code to calculate correct number of max steps when webdataset is used

# Usage
* add `is_sharded=True` in cfg for train_ds to load webdataset

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Removing duplicate Megatron-LM installation
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

small fix to remove duplicate megatron-lm installation
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Added comma so I can test build process.
State: closed | Pull request: yes

# What does this PR do ?

Makes a trivial change.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Upgrade NeMo to latest mcore and TE
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

Upgrades NeMo to latest mcore and TE versions.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins
16000 sample rate VITS config help inquiry
State: closed | Pull request: no
Labels: name: bug

name: TTS

name: stale

Good day,

Could you help me to re-configure my VITS model trainer for 16000 hz speech dataset?

sample_rate: 22050
n_mel_channels: 80
n_window_size: 1024
n_window_stride: 256
n_fft: 1024
lowfreq: 0
highfreq: null
window: hann

I tried different combination but always receive something like these:
```
RuntimeError: The size of tensor a (64) must match the size of tensor b (128) at non-singleton dimension 2
RuntimeError: The size of tensor a (40) must match the size of tensor b (51) at non-singleton dimension 2
```

Thank you!


This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.
Question: can we load same size bert model to train same size gpt if yes then how?
State: closed | Pull request: no
Labels: name: stale

I have trained a bert model, I want to use its weights to train gpt model. Is it possible, if yes how

@meghmak13 
@titu1994 
@nithinraok 
@okuchaiev 

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.
Adding support for TTS generated SFT for speechlm models
State: closed | Pull request: yes
Labels: name: stale

name: common

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
generalized chat sft prompt (#7655)
State: closed | Pull request: yes
Labels: name: stale

name: NLP

name: CI

* fix dataset issues



* working version



* all passed



* refactor tests



* all pass



* working version



* use end name signal for labels



* all fixed



* update doc



* style fix



* remove unused imports



* make sure nccl not timing out



* style fix



* generate example template



* generic end of name token



* style fix



* add the chat prompt format into the config



* make sure sft working



* address reviewer comment



* fix non



* try openAI prompt



* remove unused imports



* remove human labels from the data



* use hf dataset to clean



* reviewer comments



---------

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

jenkins

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Bugfix: Args didn't have spe byte fallback
State: closed | Pull request: yes
Labels: name: stale

# What does this PR do ?

process_asr_text_tokenizer didn't have parser for args: spe_byte_fallback.

**Collection**: [ASR]

# Changelog 
- Bugfix: Added spe_byte_fallback to the args list.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

## Who can review?

@titu1994, @redoctopus, @jbalam-nv, or @okuchaiev


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Runtime error
State: closed | Pull request: no

Traceback (most recent call last):
  File "/home/user/app/app.py", line 3, in <module>
    gr.Interface.load("models/nvidia/stt_en_fastconformer_transducer_xlarge").launch()
AttributeError: 'Dependency' object has no attribute 'launch'

Its an error from gradio not nemo. Not sure what you are trying to do. 

Thanks for looking at this. 

I tried to deploy the model (https://huggingface.co/nvidia/stt_en_fastconformer_transducer_xlarge) as an gardio app in click deploy option on the right) but it is failing. It errors out during the build phase. 

I dont see it working for other models as well. Can you provide an working example for other models. 
I tried `model/segmind/SSD-1B`

You are right! I tried a few other models and they too failed with the same error. I guess it has something do with Spaces in HF. I will ask from help there.

Thanks for your help!
Update transformers cache on Jenkins
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Updating the cache by allowing transformers to be online

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

jenkins

jenkins

jenkins

jenkins
enable reading phonetic transcripts from manifest
State: closed | Pull request: yes
Labels: name: TTS

name: stale

name: common

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


2 cleaner approaches to consider:

- Define a G2P class which sets its symbols to `get_ipa_character_set(locale)` and with a forward method that returns `[c for c in text]`
- Make the g2p class an optional input to the IPATokenizer. Replace the checks with `if read_phones_from_file` to instead check `if g2p is None`

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Fix flash decoding precision
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Query and Key may have fp32 after `apply_rotary_pos_emb`, so we cast the precision to fp16 or bf16.

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins
Mengruw/fixed tgt lang ids inference/v2
State: closed | Pull request: yes
Labels: name: core

name: TTS

name: stale

name: ASR

name: NLP

name: Speaker Tasks

name: CI

name: common

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


Hi, @Davood-M 

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Does the project support QLora
State: closed | Pull request: no
Labels: name: stale

**Is your feature request related to a problem? Please describe.**

I have limit gpu resources， so I want to use qlora with model paralelism for LLM model.

**Describe the solution you'd like**

Using qlora and model paralelism or pipeline parallelism together。

**Describe alternatives you've considered**

None

**Additional context**

None


This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.

@minitu Is there any plan for QloRA support ? thanks.
Fix bug order of buckets not shuffled every epoch when using synced_r…
State: closed | Pull request: yes
Labels: name: ASR

…andomized

# What does this PR do ?

fix bug order of buckets not shuffled every epoch when using synced_randomized

**Collection**: [ASR]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix mcore conversion bug
State: closed | Pull request: yes

# What does this PR do ?

Fix mcore conversion script bug which affected GPT3 models. Also improve the UX of the script.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
python scripts/nlp_language_modeling/convert_nemo_gpt_to_mcore.py --in-folder=<input-nemo-folder> --out-file=<output-mcore-ckpt>

```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #7730
add guard if its a distributed checkpoint
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

adds guard in case we pick up a distributed checkpoint

# Changelog 
- Add specific line by line info of high level changes in this PR.

Fix issue when resuming topk from distributed checkpoint, the logic will [look](https://github.com/NVIDIA/NeMo/blob/8c061debd05837148e86fac19abf024e7210c35d/nemo/utils/callbacks/nemo_model_checkpoint.py#L92) for topk checkpoints and then delete them.

However, the [_saved_checkpoint_paths](https://github.com/NVIDIA/NeMo/blob/8c061debd05837148e86fac19abf024e7210c35d/nemo/utils/callbacks/nemo_model_checkpoint.py#L92C50-L92C73) returns paths without the .ckpt prefix as they are distributed formats, yet the code still calls [ckpt_to_dir](https://github.com/NVIDIA/NeMo/blob/8c061debd05837148e86fac19abf024e7210c35d/nemo/utils/callbacks/nemo_model_checkpoint.py#L226) which crashes since there is no .ckpt prefix on distributed checkpoints. This PR adds a guard to make sure if we see a distributed checkpoint we just return.

  
**PR Type**:
- [x] Bugfix


When resuming from topk with distributed checkpoint
Update README.rst for container update
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Graphviz fix
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

- Fixes graphviz cropped plots for notebooks
- Fixes duplicated Megatron-LM installation
- Fixes k2 library for compatibility with latest PyTorch

**Collection**: [ASR]

# Changelog 
- Now graphviz is built and installed from sources.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
setting peft eval configs correctly for logs
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

minor change to log the correct values during peft inference...

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)