renwei2024/nemo-github-issues huggingface dataset:
[TTS] Modify codec training tutorial to allow HF download
State: closed | Pull request: yes
Labels: name: TTS

name: Run CICD

# What does this PR do ?

Modify the tutorial about training audio codec models so that the user can download and fine tune from a hugging face checkpoint.

**Collection**: [TTS]

# Changelog 
- Change previous logic using `list_available_models` to instead use `from_pretrained()`
- Replace default 16khz audio codec config with `nvidia/audio-codec-22khz
`
# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

[🤖]: Hi @rlangman 👋,

We wanted to let you know that a [CICD pipeline](https://github.com/NVIDIA/NeMo/actions/runs/12250787239) for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g
Support attention backend configuration changes
State: closed | Pull request: yes
Labels: name: stale

name: NLP

name: CI

name: Run CICD

name: skip-docs

name: no-fail-fast

# What does this PR do ?
Fix up after https://github.com/NVIDIA/Megatron-LM/commit/fa0dcc48720ed9f440f36c7973fe5e7a54c7d208

Users of megatron's TransformerConfig should no longer disable TransformerEngine attention backends through environment variables like `NVTE_FUSED_ATTN`, `NVTE_FLASH_ATTN`, and `NVTE_UNFUSED_ATTN`.
Instead, after https://github.com/NVIDIA/Megatron-LM/commit/fa0dcc48720ed9f440f36c7973fe5e7a54c7d208, users should explicitly specify the attention backend in the TransformerConfig.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.
Fix issue with Wandb update to settings parsing
State: closed | Pull request: yes
Labels: name: stale

In this PR: https://github.com/wandb/wandb/pull/8649 Wandb added a hard constraint that directories are passed as strings rather than `pathlib.Path`s. We've requested a change on their end to allow for both types. In the interim, either this change or pinning `wandb==0.16.0` resolves the issue. We should avoid using pathlib.Path with Wandb constructs for now.

<!-- pylint-output-strict-mode-false -->

beep boop 🤖: 🙏 The following files have warnings. In case you are familiar with these, please try helping us to improve the code base.

---

Your code was analyzed with PyLint. The following annotations have been identified:

```
************* Module nemo.lightning.nemo_logger
nemo/lightning/nemo_logger.py:75:0: C0301: Line too long (125/119) (line-too-long)
nemo/lightning/nemo_logger.py:98:0: C0301: Line too long (120/119) (line-too-long)
nemo/lightning/nemo_logger.py:99:0: C0301: Line too long (123/119) (line-too-long)
nemo/lightning/nemo_logger.py:104:0: C0301: Line too long (127/119) (line-too-long)
nemo/lightning/nemo_logger.py:174:0: C0301: Line too long (125/119) (line-too-long)
nemo/lightning/nemo_logger.py:207:0: C0301: Line too long (128/119) (line-too-long)
nemo/lightning/nemo_logger.py:208:0: C0301: Line too long (124/119) (line-too-long)
nemo/lightning/nemo_logger.py:213:0: C0301: Line too long (124/119) (line-too-long)

-----------------------------------
Your code has been rated at 9.46/10
```

---

Thank you for improving NeMo's documentation!

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
[BUG] wandb init path
State: closed | Pull request: yes

Wandb recently changed the init directory to be validated as a string, pathlib.Path fails. We have requested them to change this, in the interim we should either pin Wandb `wandb==0.16.0` or avoid using pathlib.Path with Wandb constructs.



<!-- pylint-output-strict-mode-false -->

beep boop 🤖: 🙏 The following files have warnings. In case you are familiar with these, please try helping us to improve the code base.

---

Your code was analyzed with PyLint. The following annotations have been identified:

```
************* Module nemo.lightning.nemo_logger
nemo/lightning/nemo_logger.py:75:0: C0301: Line too long (125/119) (line-too-long)
nemo/lightning/nemo_logger.py:98:0: C0301: Line too long (120/119) (line-too-long)
nemo/lightning/nemo_logger.py:99:0: C0301: Line too long (123/119) (line-too-long)
nemo/lightning/nemo_logger.py:104:0: C0301: Line too long (127/119) (line-too-long)
nemo/lightning/nemo_logger.py:174:0: C0301: Line too long (125/119) (line-too-long)
nemo/lightning/nemo_logger.py:207:0: C0301: Line too long (128/119) (line-too-long)
nemo/lightning/nemo_logger.py:208:0: C0301: Line too long (124/119) (line-too-long)
nemo/lightning/nemo_logger.py:213:0: C0301: Line too long (124/119) (line-too-long)

-----------------------------------
Your code has been rated at 9.46/10
```

---

Thank you for improving NeMo's documentation!
An error occurs when exporting T5G2PModel to ONNX.
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

An error occurs when exporting T5G2PModel to ONNX.

**Steps/Code to reproduce bug**

> import torch
> from transformers import PreTrainedTokenizerBase, AutoTokenizer
> 
> from nemo.collections.tts.g2p.models.t5 import T5G2PModel
> model_name = "T5G2P.nemo"
> model = T5G2PModel.restore_from(model_name, map_location=torch.device("cpu"))
> model.eval()
> 
> model.export("test.onnx")

[Error]
RuntimeError: 0 INTERNAL ASSERT FAILED at "../torch/csrc/jit/ir/alias_analysis.cpp":615, please report a bug to PyTorch. We don't have an op for aten::full but it isn't a special case.  Argument types: int[], bool, int, NoneType, Device, bool, 

Candidates:
	aten::full.names(int[] size, Scalar fill_value, *, str[]? names, ScalarType? dtype=None, Layout? layout=None, Device? device=None, bool? pin_memory=None) -> Tensor
	aten::full(SymInt[] size, Scalar fill_value, *, ScalarType? dtype=None, Layout? layout=None, Device? device=None, bool? pin_memory=None) -> Tensor
	aten::full.names_out(int[] size, Scalar fill_value, *, str[]? names, Tensor(a!) out) -> Tensor(a!)
	aten::full.out(SymInt[] size, Scalar fill_value, *, Tensor(a!) out) -> Tensor(a!)

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- Ubuntu 20.04
- PyTorch version - 2.3.0+cu118
- Python version - 3.10.12

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.
chore(beep boop 🤖): Bump `MCORE_TAG=6dfeb25...` (2024-12-09)
State: closed | Pull request: yes
Labels: name: stale

name: CI

name: Run CICD

🚀 PR to bump `NVIDIA/Megatron-LM` in `Dockerfile.ci` to `MCORE_TAG=6dfeb2597c625f5dc1ea2f564e45aee654ffca44`.  

📝 Please remember the following to-do's before merge: 
- [ ] Verify the presubmit CI  

🙏 Please merge this PR only if the CI workflow completed successfully.

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
chore(beep boop 🤖): Bump `MCORE_TAG=dc7fea9...` (2024-12-08)
State: closed | Pull request: yes
Labels: name: stale

name: Run CICD

🚀 PR to bump `NVIDIA/Megatron-LM` in `Dockerfile.ci` to `MCORE_TAG=dc7fea91a15d12cbe5084699f09935a088f172f1`.  

📝 Please remember the following to-do's before merge: 
- [ ] Verify the presubmit CI  

🙏 Please merge this PR only if the CI workflow completed successfully.

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
[ASR]: Compatibility error with NumPy v2.0
State: closed | Pull request: no
Labels: name: bug

Hi

**Describe the bug**

Compatibility error with NumPy v2.0: AttributeError: `np.sctypes was removed in the NumPy 2.0 release. Access dtypes explicitly instead.. Did you mean: 'dtypes'?`

**Steps/Code to reproduce bug**

Run this code

```python
import nemo.collections.asr as nemo_asr

# 1. Download and instantiate the Persian model
asr_model_name="nvidia/stt_fa_fastconformer_hybrid_large"
asr_model = nemo_asr.models.EncDecHybridRNNTCTCBPEModel.from_pretrained(model_name=asr_model_name)

audio_file_path = "voice_01.wav"
asr_model.transcribe([audio_file_path])
```

**Expected behavior**

Work fine!

**Environment overview (please complete the following information)**

 - Environment location: [Github Codespace)]
 - pip install nemo_toolkit==2.0.0

**Environment details**

- OS version: Linux codespaces-aa0b2f 6.5.0-1025-azure
- PyTorch version: `2.5.1`
- Python version: 3.12.7


fixed with static version set for numpy

@njfamirm what version did you fix numpy to? I'm having the same problem. I tried to downgrade to numpy 2.0.2 but had other compatibility issues. Is [ASR] not compatible with numpy 2 at all or just the latest release?

EDIT: in the end it worked with 1.23.5

> [@njfamirm](https://github.com/njfamirm) what version did you fix numpy to? I'm having the same problem. I tried to downgrade to numpy 2.0.2 but had other compatibility issues. Is [ASR] not compatible with numpy 2 at all or just the latest release?
> 
> EDIT: in the end it worked with 1.23.5

Hi, `numpy==1.26.4`
[NLP] Punctuation and capitalisation
State: closed | Pull request: no
Labels: name: bug

name: stale

I have noticed that in the section https://github.com/NVIDIA/NeMo/tree/main/examples/nlp/token_classification, a warning has been added (2 months ago), which states the section is deprecated and scheduled for removal in release 23.11. 

Is this a mistake since on ngc the container is already at version 24.09? If not, could you share information on the future of punctuation and capitalisation, especially since the task is also an integral part in Riva pipelines?

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.
chore(beep boop 🤖): Bump `MCORE_TAG=bd677bf...` (2024-12-07)
State: closed | Pull request: yes
Labels: name: stale

name: Run CICD

🚀 PR to bump `NVIDIA/Megatron-LM` in `Dockerfile.ci` to `MCORE_TAG=bd677bfb13ac2f19deaa927adc6da6f9201d66aa`.  

📝 Please remember the following to-do's before merge: 
- [ ] Verify the presubmit CI  

🙏 Please merge this PR only if the CI workflow completed successfully.

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
minor fix for nemo2 sftpeft readme
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
changed-megatron-lm-ref
State: closed | Pull request: yes
Labels: name: stale

# What does this PR do ?

Replaced instances of Megatron-LM and Megatron-LM framework reference in docs to Megatron Core

**Collection**: [Note which collection this PR will affect]

# Changelog 
- intro.rst

# Usage
no code changes

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [X] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [X] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
readme updates for Q4 2024
State: closed | Pull request: yes

# What does this PR do ?

Updated the readme Latest News section to Q4 2024, revised links, changed playbook references to tutorial references, and made a few copyedits to the text. These updates were requested by Shashank.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- modified the readme.md file

# Usage
no code documentation change

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [X] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [X] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
DAPT with NeMo 2.0 
State: closed | Pull request: yes
Labels: name: stale

# What does this PR do ?
 
See previously closed PR: https://github.com/NVIDIA/NeMo/pull/10689 


Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
fix: export weight name mapping if model is nemo model
State: open | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
BERT NeMo2 Recipe
State: closed | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


<!-- pylint-output-strict-mode-false -->

beep boop 🤖: 🙏 The following files have warnings. In case you are familiar with these, please try helping us to improve the code base.

---

Your code was analyzed with PyLint. The following annotations have been identified:

```

------------------------------------
Your code has been rated at 10.00/10
```

---

Thank you for improving NeMo's documentation!

<!-- pylint-output-strict-mode-true -->

beep boop 🤖: 🚨 The following files must be fixed before merge!

---

Your code was analyzed with PyLint. The following annotations have been identified:

```

------------------------------------
Your code has been rated at 10.00/10
```

---

Thank you for improving NeMo's documentation!

[🤖]: Hi @suiyoubi 👋,

We wanted to let you know that a [CICD pipeline](https://github.com/NVIDIA/NeMo/actions/runs/12248433090) for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g
chore(beep boop 🤖): Bump `MCORE_TAG=bd677bf...` (2024-12-06)
State: closed | Pull request: yes
Labels: name: Run CICD

🚀 PR to bump `NVIDIA/Megatron-LM` in `Dockerfile.ci` to `MCORE_TAG=bd677bfb13ac2f19deaa927adc6da6f9201d66aa`.  

📝 Please remember the following to-do's before merge: 
- [ ] Verify the presubmit CI  

🙏 Please merge this PR only if the CI workflow completed successfully.

[🤖]: Hi @ko3n1g 👋,

We wanted to let you know that a [CICD pipeline](https://github.com/NVIDIA/NeMo/actions/runs/12190479871) for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g
Update Nemo Distributed Checkpoint User Guide
State: closed | Pull request: yes

# What does this PR do ?

This PR aims to update Nemo distributed checkpoint user guide documentation, with best practices for parameter tuning, mechanism, FAQs, etc. 

**Collection**: [Note which collection this PR will affect]
Distributed checkpoint.

# Changelog 
- Update dist_ckpt.rst documentation in Nemo GitHub
- Update Introduction
- Add Mechanism with images
- Add Parameter Tuning Best Practices
- Add FAQs
- Add Glossary

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.


Updated per above suggestions, and changed the image source, please check again
ci: Re-enable `L2_Megatron_LM_To_NeMo_Conversion`
State: closed | Pull request: yes
Labels: name: CI

name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


Here's the relevant run: https://github.com/NVIDIA/NeMo/actions/runs/12185355486
Autodetect number of devices for torchrun executor
State: closed | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?

Small change to autodetect number of devices for `torchrun` executor rather than using default 8. Useful for testing in various environments.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


[🤖]: Hi @janekl 👋,

We wanted to let you know that a [CICD pipeline](https://github.com/NVIDIA/NeMo/actions/runs/12240425643) for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g

[🤖]: Hi @janekl 👋,

We wanted to let you know that a [CICD pipeline](https://github.com/NVIDIA/NeMo/actions/runs/12468343506) for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g

Hey @ananthsub. I added docstring to the edited function based on @marcromeyn's request.

Could you please approve the MR again?

[🤖]: Hi @janekl 👋,

We wanted to let you know that a [CICD pipeline](https://github.com/NVIDIA/NeMo/actions/runs/12710202776) for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g
RuntimeError: world_size (1) is not divisible by 2
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

Unable to run the hello world script `nemo llm pretrain --factory llama3_8b` like in the documentation https://github.com/NVIDIA/NeMo/blob/main/examples/llm/pretrain/README.md

I get: 
```
[rank0]:   File "/opt/megatron-lm/megatron/core/parallel_state.py", line 532, in initialize_model_parallel
[rank0]:     raise RuntimeError(f"world_size ({world_size}) is not divisible by {total_model_size}")
[rank0]: RuntimeError: world_size (1) is not divisible by 2
```

**Steps/Code to reproduce bug**

`srun -p batch -N 1 --container-image nvcr.io/nvidia/nemo:24.09 --pty bash`
The machine has 8 gpu according to nvidia-smi 
`run nemo llm pretrain --factory llama3_8b`

**Expected behavior**

I would expect it to run seamlessly 

**Environment overview (please complete the following information)**

`srun -p batch -N 1 --container-image nvcr.io/nvidia/nemo:24.09 --pty bash`
The machine has 8 gpu according to nvidia-smi 


This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.
ci: Run secrets detector on merge-commit
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix: export converts properly if no model_prefix
State: closed | Pull request: yes
Labels: name: Run CICD

Previously if model_prefix wasn't part of model_state_dict, then the nemo_model_conversion_dict would contain keys that looked like `.decoder....` with an extra "." prefix. This just makes sure model_prefix isn't empty before adding the model_prefix in.

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


<!-- pylint-output-strict-mode-false -->

beep boop 🤖: 🙏 The following files have warnings. In case you are familiar with these, please try helping us to improve the code base.

---

Your code was analyzed with PyLint. The following annotations have been identified:

```

------------------------------------
Your code has been rated at 10.00/10
```

---

Thank you for improving NeMo's documentation!

[🤖]: Hi @terrykong 👋,

We wanted to let you know that a [CICD pipeline](https://github.com/NVIDIA/NeMo/actions/runs/12185999548) for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g
chore(beep boop 🤖): Bump `MCORE_TAG=2f67f35...` (2024-12-05)
State: closed | Pull request: yes
Labels: name: Run CICD

🚀 PR to bump `NVIDIA/Megatron-LM` in `Dockerfile.ci` to `MCORE_TAG=2f67f35b4a9e5596034e93eddeed07009bcf630c`.  

📝 Please remember the following to-do's before merge: 
- [ ] Verify the presubmit CI  

🙏 Please merge this PR only if the CI workflow completed successfully.
fix protobuf version
State: closed | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


[🤖]: Hi @Victor49152 👋,

We wanted to let you know that a [CICD pipeline](https://github.com/NVIDIA/NeMo/actions/runs/12243593261) for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g
Update restore from config for gpt type continual training in NeMo1
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Some model's config is not properly updated for merging continual training + pre-training. Mainly moving restore_from config from root to under model config.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
ci: Disable HexHighEntropyString plugin
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
chore(beep boop 🤖): Bump `MCORE_TAG=844119f...` (2024-12-04)
State: closed | Pull request: yes
Labels: name: Run CICD

🚀 PR to bump `NVIDIA/Megatron-LM` in `Dockerfile.ci` to `MCORE_TAG=844119f5c856a3037ec7c7f6d6ef7b3518ceee6b`.  

📝 Please remember the following to-do's before merge: 
- [ ] Verify the presubmit CI  

🙏 Please merge this PR only if the CI workflow completed successfully.
Whisper model support in Lite
State: closed | Pull request: yes
Labels: name: Run CICD

name: skip-docs

# What does this PR do ?

Adds AutoModelSeq2Seq HF support


I made some changes to enable the training to converge (commit https://github.com/NVIDIA/NeMo/pull/11464/commits/c605d12aa89f8674e6a11141a0e7fdd4c9e80c5c):
* Preserve HF tokenizer's Whisper special tokens (prompt)
* Change the precision to bf16 (fp16 results in nan after the first weight update)
* Fix decoder_input_ids and labels for training step

[🤖]: Hi @oyilmaz-nvidia 👋,

We wanted to let you know that a [CICD pipeline](https://github.com/NVIDIA/NeMo/actions/runs/12476579837) for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g
PTQ CLI and param updates
State: closed | Pull request: yes
Labels: name: Run CICD

name: skip-docs

# What does this PR do ?

* Addressing CLI type hint issue `Unexpected error: Unserializable value /models/LLAMA31-8B-base-fp8 of type <class 'pathlib.PosixPath'>. Error occurred at path '<root>.export_config.path'.")` in [https://github.com/NVIDIA/NeMo/blob/a635607eea98b95a41f0f7a336ae083f38633630/nemo/collections/llm/quantization/quantizer.py#L83](https://github.com/NVIDIA/NeMo/blob/a635607eea98b95a41f0f7a336ae083f38633630/nemo/collections/llm/quantization/quantizer.py#L83)
* Some renaming

**Collection**: NLP / LLM

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to https://github.com/NVIDIA/NeMo-Run/pull/118 (prerequisite)


<!-- pylint-output-strict-mode-false -->

beep boop 🤖: 🙏 The following files have warnings. In case you are familiar with these, please try helping us to improve the code base.

---

Your code was analyzed with PyLint. The following annotations have been identified:

```
************* Module nemo.collections.llm.api
nemo/collections/llm/api.py:601:0: C0116: Missing function or method docstring (missing-function-docstring)

-----------------------------------
Your code has been rated at 9.98/10
```

---

Thank you for improving NeMo's documentation!

[🤖]: Hi @janekl 👋,

We wanted to let you know that a [CICD pipeline](https://github.com/NVIDIA/NeMo/actions/runs/12143270192) for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g