renwei2024/nemo-github-issues huggingface dataset:
Update checkpoint_averaging_model_parallel.py
State: closed | Pull request: yes
Labels: name: stale

checkpoint_averaging_model_parallel.py
Description update

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Canary speechllm
State: closed | Pull request: yes
Labels: name: core

name: TTS

name: stale

name: ASR

name: NLP

name: Speaker Tasks

name: CI

name: common


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Set up microphone for Colab
State: closed | Pull request: no

Hello everyone
I want to run the 
https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/Online_Offline_Microphone_VAD_Demo.ipynb
in colab, but colab dose not access to the microphone.
How can I resolve the issue?

Hi, the real-time streaming notebooks (such as that one) won't work in Colab unfortunately. You need to run the notebooks locally. They will not be able to record your microphone audio if you run them in Colab or in a Docker container.
speech_to_text_buffered_infer: How to use timestamps 
State: closed | Pull request: no

Hello!
How can timestamps be used when using **speech_to_text_buffered_infer_ctc.py** or **speech_to_text_buffered_infer_rnnt.py** ?
Apparently compute_timestamps is not used in the code
Thank you

Sadly those two do not support time stamp calculation. You can use a CTC model with NeMo Forced Aligners to get accurate timestamps

It's still not supported, as far as a I can tell.

What's the current recommendation to use all (or most) features of https://github.com/NVIDIA/NeMo/blob/main/examples/asr/transcribe_speech.py when limited in VRAM?
Fix FSDP SHARP integration
State: closed | Pull request: yes
Labels: name: stale

name: NLP

The attribute was never implemented.

# What does this PR do ?

For NeMo NLP, the FSDP sharding strategy cannot be used currently because the `sharp` attribute for the `NLPFSDPStrategy` was not actually implemented in https://github.com/NVIDIA/NeMo/pull/7793, even though it is queried.

**Collection**:
- NLP

# Changelog 
- Implement missing `NLPFSDPStrategy.sharp` attribute.

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
Related to #7793 (pull request)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

Bla.

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

Bla.

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

Bla.

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

Bla.

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

Bla.

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

Bla.

Independently fixed in dd69c7a7d.
remove more deprecated files
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

removes deprecated scripts for peft t5 tuning.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins
Error in exporting the speaker verification model to ONNX
State: closed | Pull request: no
Labels: name: bug

name: stale

The following code
```python3
import torch
import nemo.collections.asr as nemo_asr


speaker_model = nemo_asr.models.EncDecSpeakerLabelModel.from_pretrained(model_name="ecapa_tdnn")
with torch.no_grad():
  speaker_model.eval()
  speaker_model.export('model.onnx')
```
fails to export the model using the latest master and it throws the following error logs:
```
[NeMo I 2024-01-13 11:35:35 features:289] PADDING: 16
[NeMo I 2024-01-13 11:35:36 save_restore_connector:249] Model EncDecSpeakerLabelModel was successfully restored from /root/.cache/torch/NeMo/NeMo_1.21.0rc0/ecapa_tdnn/3e0c5c4731b176aeb70c29a74d800c81/ecapa_tdnn.nemo.
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
[<ipython-input-2-3f0cb43be965>](https://localhost:8080/#) in <cell line: 6>()
      6 with torch.no_grad():
      7   speaker_model.eval()
----> 8   speaker_model.export('model.onnx')

2 frames
[/usr/local/lib/python3.10/dist-packages/nemo/utils/export_utils.py](https://localhost:8080/#) in parse_input_example(input_example)
    116 
    117 def parse_input_example(input_example):
--> 118     input_list = list(input_example)
    119     input_dict = {}
    120     # process possible kwargs

TypeError: 'NoneType' object is not iterable
```

---

Please use the following colab notebook to reproduce it

https://colab.research.google.com/drive/1hQumtSeKdDMCupcUHgrEIcX67c3AW1bj?usp=sharing

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.

it is not stale.

reopen
Revert "adding OnlineSampleMapping"
State: closed | Pull request: yes
Labels: name: NLP

Reverts NVIDIA/NeMo#8137

Nemo pipeline is broken (finetune tests failing) due to this. Hence revert.

More context/details:
- two gpt3 fine-tunning tests failed on EOS (finetune.gpt3.126m_tp1_pp1_1node_100steps_squad and finetune.gpt3.400m_improved_tp1_pp1_1node_100steps_squad). Check out the error log, it’s not the fault of context parallelism. The error comes from dataset, which is introduced by this PR https://github.com/NVIDIA/NeMo/pull/8137

[10:43](https://nvidia.slack.com/archives/D062J0U5PJ8/p1705084988269489)

Above problematic PR was merged after my context parallelism PR. So for sanity check, I also ran a pipeline with the NeMo commit ID of context parallelism merging (i.e., without above problematic PR) 
you can see those two fine-tuning test passed, so I think context parallelism code works fine

jenkins

Is there a CI test which uses context parallel? @pablo-garay @ericharper 
cc @michalivne 

jenkins

@xrennvidia yes but w/o CI its hard to debug/iterate on the PR.


jenkins
Upgrade to DLFW PyTorch 23.12
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Updates the base pytorch container

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

jenkins
Add random_seed argument to generate
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add random_seed argument to generate. By default it does not set anything, so backward compatibility is preserved.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

jenkins

jenkins
Quartznet onnx model can't run on onnxruntime(CUDA backend) benchmark
State: closed | Pull request: no
Labels: name: stale

The develop ENV is as follows:
cuda                                       11.8
cudnn                                     8.2.4
onnxruntime(cuda backend)  1.16.3

When I run my exported Quartznet onnx model  on onnxruntime(CUDA backend) benchmark, the following error occurs:
[E:onnxruntime:benchmark, cuda_call.cc:116 CudaCall] CUDNN failure 3: CUDNN_STATUS_BAD_PARAM ; GPU=0 ; file=/onnxruntime_src/onnxruntime/contrib_ops/cuda/fused_conv.cc ; line=85 ; expr=cudnnAddTensor(cudnnHandle, &alpha, Base::s_.z_tensor, Base::s_.z_data, &alpha, Base::s_.y_tensor, Base::s_.y_data)
 [E:onnxruntime:, sequential_executor.cc:514 ExecuteKernel] Non-zero status code returned while running FusedConv node. Name:'Conv_41' Status Message: CUDNN failure 3: CUDNN_STATUS_BAD_PARAM ; GPU=0 ;file=/onnxruntime_src/onnxruntime/contrib_ops/cuda/fused_conv.cc ; line=85 ; expr=cudnnAddTensor(cudnnHandle, &alpha, Base::s_.z_tensor, Base::s_.z_data, &alpha, Base::s_.y_tensor, Base::s_.y_data); 
terminate called after throwing an instance of 'Ort::Exception'
  what():  Non-zero status code returned while running FusedConv node. Name:'Conv_41' Status Message: CUDNN failure 3: CUDNN_STATUS_BAD_PARAM ; GPU=0 ;file=/onnxruntime_src/onnxruntime/contrib_ops/cuda/fused_conv.cc ; line=85 ; expr=cudnnAddTensor(cudnnHandle, &alpha, Base::s_.z_tensor, Base::s_.z_data, &alpha, Base::s_.y_tensor, Base::s_.y_data);

Thank you very much for help!



This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.
Knowledge distillation support for Nemo ASR models
State: closed | Pull request: no
Labels: name: stale

**Is your feature request related to a problem? Please describe.**
I am currently working with Nemo FastConformer Large model, but would like to deploy a smaller model on mobile devices. Instead of training from scratch, I would like to perform KD

**Describe the solution you'd like**
I would greatly appreciate it if the Nemo ASR team could implement Knowledge Distillation (KD) support or maybe guide me how to do it with a FastConformer Model :
1. Maybe an Easier API support for KD within the Nemo framework.
2. Documentation and examples demonstrating how to use KD for model compression.
3. Tools or guidelines for effectively transferring knowledge from large to small models.

**Describe alternatives you've considered**
As an alternative, I have considered manually implementing KD techniques using pytorch lightening as described in #1996. However I did not get the full picture. 

Thank you for your consideration


This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

I'm also interested in this topic. any update?

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.
Update CONTRIBUTING.md
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
MPT to NeMo conversion
State: closed | Pull request: yes
Labels: name: stale

This script converts mpt-7b to mcore (transformer engine). 

two things need to be aware:
1. layernorm instead of Low precision layernorm is used
2. no bias layernorm is not supported by transformer engine as of now

to use
```
python scripts/checkpoint_converters/convert_mpt_7b_hf_to_nemo.py
```

text generation:
```
python examples/nlp/language_modeling/megatron_gpt_eval.py             gpt_model_file=mpt_7b_mcore.nemo             inference.greedy=True                 trainer.devices=1             trainer.num_nodes=1             tensor_model_parallel_size=1             pipeline_model_parallel_size=1             prompts="[query: how much protein should a female eat]"
```
example outputs:
```
[{'sentences': ['query: how much protein should a female eat?\nI am a female, 5\'4" and weigh about 130 lbs. I am a vegetarian and I am trying to lose weight. I'], 'tokens': [['
', 'query', ':', 'Ġhow', 'Ġmuch', 'Ġprotein', 'Ġshould', 'Ġa', 'Ġfemale', 'Ġeat', '?', 'Ċ', 'I', 'Ġam', 'Ġa', 'Ġfemale', ',', 'Ġ5', "'", '4', '"', 'Ġand', 'Ġweigh', 'Ġabout', 'Ġ130', 'Ġlbs', '.', 'ĠI', 'Ġam', 'Ġa', 'Ġvegetarian', 'Ġand', 'ĠI', 'Ġam', 'Ġtrying', 'Ġto', 'Ġlose', 'Ġweight', '.', 'ĠI']], 'logprob': None, 'full_logprob': None, 'token_ids': [[0, 7267, 27, 849, 1199, 2601, 943, 247, 5343, 6008, 32, 187, 42, 717, 247, 5343, 13, 608, 8, 21, 3, 285, 14357, 670, 11084, 38818, 15, 309, 717, 247, 39203, 285, 309, 717, 2820, 281, 7168, 2801, 15, 309]], 'offsets': [[0, 0, 5, 6, 10, 15, 23, 30, 32, 39, 43, 44, 45, 46, 49, 51, 58, 59, 61, 62, 63, 64, 68, 74, 80, 84, 88, 89, 91, 94, 96, 107, 111, 113, 116, 123, 126, 131, 138, 139]]}]
```

This PR depends on https://github.com/NVIDIA/Megatron-LM/pull/668

@yaoyu-33 

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Update dependencies
State: closed | Pull request: yes
Labels: name: ASR

name: NLP

# What does this PR do ?

Updates dependencies for NeMo
Fixes #8118 

**Collection**: [Core, Common]

# Changelog 
- Remove Numpy upperbound
- Remove pydantic pin < 2

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation


jenkins
Mixtral to NeMo conversion script.
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


Could you provide the convert NeMo to huggingface script? 

@cdj0311 will try to have the NeMo to HF by next week

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

Hi @ericharper. I browsed CodeQL suggestions in this pull request. This is a good example when **Flake8** is better than CodeQL: CodeQL fails to detect undefined variables (a simple issue), see `head_num` [here](https://github.com/NVIDIA/NeMo/blob/6865c39d85c4f3b120a02fe74024ad10d5fab286/scripts/nlp_language_modeling/convert_nemo_mixtral_to_hf.py#L124), while Flake8 catches them. I strongly suggest to start using Flake8 complementary to CodeQL in CI. 

Another example are the unused imports `from megatron.core.utils import init_method_normal, scaled_init_method_normal` [here](https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/nlp/models/language_modeling/megatron_gpt_model.py#L94). This file is frequently edited btw. Possibly CodeQL misses this due to import guards, but I'm not sure. cc @ericharper
Update README.rst
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Avoid NFS consistency issues during ckpt dir creation
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

On some NFS setups, the root checkpoint directory created on rank 0 was not visible to other ranks even 30 seconds after creation, probably due to NFS caching. This PR solves it by calling `makedirs(..., exist_ok=True)` on all ranks.
Related nvbug: #4440262

**Collection**: NLP

# Changelog 
- Create root checkpoint directory on all ranks instead of rank 0 only

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

@ericharper can you review?

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

jenkins

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Improve documentation gpt_training.rst 
State: closed | Pull request: yes
Labels: name: stale

name: NLP

**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation


@yaoyu-33 

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.

/cicd
add first_val_step to mcore scheduler
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

@ericharper @athitten 
Can you help review this PR?

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins
switch to mcore dataset [with FIM support]
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

1. Switches to megatron core dataset.
2. Adds FIM support.
3. Adds starcoder config.


**Collection**: NLP

# Changelog 
- Added additional functions in oreder to use mcore ds.
- Changed NeMo ds classes to mcore ds classes (megatron_gpt_model.py).
- Created gpt_fim_dataset.py with GPTFIMDatasetConfig and GPTFIMDataset classes in order to use FIM techinque.
- Added new config for starcoder models (megatron_starcoder_config.yaml).

# Usage
Example of FIM usage. This part should be added to the data section in the model config:
```yaml
add_fim: True # Set to True to use FIM
    fim:
      # fill in the middle
      rate: 0.5 # Probability to convert a training sample into a "Fill-in-the-Middle" format. Must be between 0 and 1
      spm_rate: 0.5 # Probability that the a FIM sample uses the SPM format over the PSM format
      split_sample: null # String around which to split the sample for FIM. If None (default), FIM is applied on the sample-level
      fragment_rate: 0.5 # Rate of FIM on each fragment when fim_split_sample is not None
      no_prefix: null # Do not apply FIM to fragments that start with this prefix
      extra_tokens:
        prefix: "<fim_prefix>"
        middle: "<fim_middle>"
        suffix: "<fim_suffix>"
        pad: "<fim_pad>"
        eod: "
"
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins
Can be any improvement from using learned preprocessors?
State: closed | Pull request: no
Labels: name: stale

Duplicated: https://github.com/NVIDIA/NeMo/discussions/8133

I think about preprocessors like in this repo: https://github.com/KinWaiCheuk/nnAudio
Maybe some improvements (speed of training or inference, quality) can be reached from using learned preprocessors? (and also they can be learned also then ASR model trained)

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.
Remove typo
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix: numba.*_num_threads resets torch num_threads #8141
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Temporary fix until https://github.com/numba/numba/issues/9387 gets resolved.

**Collection**: [ASR]

# Changelog 
- wrap rnnt numba num_threads check and change to protect torch num_threads from being reset.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


Could you sign the commits in PR by following the DCOBot? 

jenkins 

jenkins

Hmm core dumped in numba unit-tests. I think ill try running them one more time in case it was a fluke, and then well need to debug more
Fix TP>1 issue for conversion script
State: closed | Pull request: yes

# What does this PR do ?

Fix issue where nemo -> hf conversion script crashes for models trained with TP > 1 

**Collection**: NLP

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #7770 #7922 


jenkins
updating default transcribe behavior to en lowercase for Canary Models
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Updated default transcribe behavior to be ASR-English-lowercase for canary models. Model will default to this if provided with a list of audio paths (paths2audio_files) with out specifying additional task details. 

**Collection**: ASR



# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix layer_norm_1p Apex call to match Apex API
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Apex _fast_layer_norm expects required memory_efficient parameter to be passed when it is called: https://github.com/NVIDIA/apex/blob/master/apex/contrib/layer_norm/layer_norm.py#L37 

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This is a duplicate: https://github.com/NVIDIA/NeMo/pull/8129
Add knobs for swiglu and rope fusion
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add knob for rope fusion
Add knob for swiglu fusion
Change knob for bias_gelu fusion

The PR is related to the PR of MLM: https://gitlab-master.nvidia.com/ADLR/megatron-lm/-/merge_requests/984

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* set `model.apply_rope_fusion: true` to enable rope fusion
* set `model.bias_activation_fusion: true` to enable bias+activation fusion, which works for both gelu and swiglu type.

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
Add @ericharper @erhoo82 for review.
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

Actually, should we default `apply_rope_fusion` to True or False?
Before this PR it would be been "False". Are there risks to changing the default behavior to "True" now?

> Actually, should we default `apply_rope_fusion` to True or False? Before this PR it would be been "False". Are there risks to changing the default behavior to "True" now?

I think we should keep it as true at most cases since rope is a separated operator and well fused in apex. MCore will check if the fused implementation is included in apex, or the config will be set to false.

> > Actually, should we default `apply_rope_fusion` to True or False? Before this PR it would be been "False". Are there risks to changing the default behavior to "True" now?
> 
> I think we should keep it as true at most cases since rope is a separated operator and well fused in apex. MCore will check if the fused implementation is included in apex, or the config will be set to false.

Cool, thanks!

jenkins

The changes have been enabled in another cleaner PR: https://github.com/NVIDIA/NeMo/pull/8184.
Close it now.
G2P CTC - allow setting vocab_file from config
State: closed | Pull request: yes
Labels: name: TTS

name: stale


# What does this PR do ?

ctc G2P model always generates the vocab_file which is only suitable for english. This allows you to set custom vocab_file thru config.

**Collection**: TTS/G2p

# Changelog 
- allow passing custom vocab file to grapheme_tokenizer



jenkins

@titu1994 what should i do to move this forward ? thank you.

G2P would be @rlangman 
Plus, we cannot write to file inside of training step or val step. File writing must be done offline somewhere else cause ddp will introduce too many cases to it. 

file writing already happened here, i didnt change anything about it, except i skip file writing if dict is provided, so i guess for DDP this should work if file is provided and not generated, so i think this should be an improvement for tht case as well

jenkins

jenkins

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.

i hope you can reopen this, i was still waiting for the review, as far as i am aware this is ready to merge

jenkins

jenkins

@ppisljar [this test](https://github.com/NVIDIA/NeMo/blob/main/Jenkinsfile#L1216) is still failing 

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
remove deprecated scripts
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?
Removes deprecated scripts.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

jenkins
adding OnlineSampleMapping
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

jenkins


jenkins