renwei2024/nemo-github-issues huggingface dataset:
handle mistralai/Mistral-7B-Instruct-v0.3 tokenizer correctly
State: closed | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?

Uses sentence-piece legacy to handle special tokens as part of text, this change only affects mistralai/Mistral-7B-Instruct-v0.3 


**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


<!-- pylint-output-strict-mode-false -->

beep boop 🤖: 🙏 The following files have warnings. In case you are familiar with these, please try helping us to improve the code base.

---

Your code was analyzed with PyLint. The following annotations have been identified:

```
************* Module convert_mistral_7b_hf_to_nemo
scripts/checkpoint_converters/convert_mistral_7b_hf_to_nemo.py:436:0: C0116: Missing function or method docstring (missing-function-docstring)

-----------------------------------
Your code has been rated at 9.97/10
```

---

Mitigation guide:

* Add sensible and useful docstrings to functions and methods
* For trivial methods like getter/setters, consider adding `# pylint: disable=C0116` inside the function itself
* To disable multiple functions/methods at once, put a `# pylint: disable=C0116` before the first and a `# pylint: enable=C0116` after the last.

By applying these rules, we reduce the occurance of this message in future.

Thank you for improving NeMo's documentation!

[🤖]: Hi @akoumpa 👋,

We wanted to let you know that a [CICD pipeline](https://github.com/NVIDIA/NeMo/actions/runs/12754892366) for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g
ModuleNotFoundError: No module named 'megatron.core.dist_checkpointing'
State: closed | Pull request: no
Labels: name: bug

**ModuleNotFoundError: No module named 'megatron.core.dist_checkpointing'**

I am getting `ModuleNotFoundError: No module named 'megatron.core.dist_checkpointing'`


**Steps/Code to reproduce bug**
```bash
python /home/ubuntu/NeMo/scripts/checkpoint_converters/convert_llama_hf_to_nemo.py     --input_name_or_path meta-llama/Llama-3.2-1B-Instruct     --output_path Llama-3.2-1B-Instruct.nemo     --precision=bf16

Traceback (most recent call last):
  File "/home/ubuntu/NeMo/scripts/checkpoint_converters/convert_llama_hf_to_nemo.py", line 34, in <module>
    from nemo.collections.nlp.models.language_modeling.megatron_gpt_model import MegatronGPTModel
  File "/home/ubuntu/NeMo/nemo/collections/nlp/__init__.py", line 15, in <module>
    from nemo.collections.nlp import data, losses, models, modules
  File "/home/ubuntu/NeMo/nemo/collections/nlp/models/__init__.py", line 16, in <module>
    from nemo.collections.nlp.models.duplex_text_normalization import (
  File "/home/ubuntu/NeMo/nemo/collections/nlp/models/duplex_text_normalization/__init__.py", line 15, in <module>
    from nemo.collections.nlp.models.duplex_text_normalization.duplex_decoder import DuplexDecoderModel
  File "/home/ubuntu/NeMo/nemo/collections/nlp/models/duplex_text_normalization/duplex_decoder.py", line 34, in <module>
    from nemo.collections.nlp.models.nlp_model import NLPModel
  File "/home/ubuntu/NeMo/nemo/collections/nlp/models/nlp_model.py", line 40, in <module>
    from nemo.collections.nlp.parts.nlp_overrides import NLPSaveRestoreConnector
  File "/home/ubuntu/NeMo/nemo/collections/nlp/parts/nlp_overrides.py", line 68, in <module>
    from nemo.collections.nlp.modules.common.megatron.transformer import AutocastTransformerLayer, ParallelTransformerLayer
  File "/home/ubuntu/NeMo/nemo/collections/nlp/modules/common/megatron/transformer.py", line 28, in <module>
    from nemo.collections.nlp.modules.common.megatron.adapters.parallel_adapters import (
  File "/home/ubuntu/NeMo/nemo/collections/nlp/modules/common/megatron/adapters/parallel_adapters.py", line 27, in <module>
    from megatron.core.dist_checkpointing.mapping import ShardedStateDict
ModuleNotFoundError: No module named 'megatron.core.dist_checkpointing'
```

**Expected behavior**

A clear and concise description of what you expected to happen.

**Environment overview (please complete the following information)**

 - Environment location: Cloud(LambdaLabs)
 - Method of NeMo install: from source

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version: debian
- PyTorch version: 2.4.1+cu124
- Python version: Python 3.10.16

**Additional context**
GPU model: A100 SXM4


I found the issue. Instead of installing from `main` branch, it wokred when I reinstall from `r2.*`
add dataset_root to SpecterDataModule
State: closed | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


[🤖]: Hi @suiyoubi 👋,

We wanted to let you know that a [CICD pipeline](https://github.com/NVIDIA/NeMo/actions/runs/12769533435) for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g

[🤖]: Hi @suiyoubi 👋,

We wanted to let you know that a [CICD pipeline](https://github.com/NVIDIA/NeMo/actions/runs/12830798905) for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g
Annoyed NeMo User Rant
State: closed | Pull request: no

I have been using NeMo for the last 3 years mainly for speech. While the speech part is okay to use, the NLP/LLM side is an absolute hell. There has never been a single time that I was able to install without an issue.

You try to install NeMo then mamba-ssm fails, you spend another 3 hrs trying to resolve it, you give up and delete your server for a fresh installation to end up with the same issue. Even if you solve this with the grace of God, next you need to figure out the issue with Megatron, then transformer-engine then apex. Endless non stop horrible experience. :(
Use _get_mlp_module_spec from Megatron Core rather than redefine locally
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Reuse `_get_mlp_module_spec ` from Megatron Core to improve stability. This also fixes Mixtral loading bug as the current implementation fails with:
```
  File "/opt/megatron-lm/megatron/core/transformer/moe/moe_layer.py", line 109, in __init__
    self.experts = build_module(self.submodules.experts, self.num_local_experts, self.config)
  File "/opt/megatron-lm/megatron/core/transformer/spec_utils.py", line 82, in build_module
    module = import_module(spec_or_module.module)
AttributeError: 'MLPSubmodules' object has no attribute 'module' when instantiating MoELayer when instantiating TransformerLayer
```
due to recent refactoring in Megatron https://github.com/NVIDIA/Megatron-LM/commit/aa2a45dd44516925ba5c0579eb262caf48a81a1b.

**Collection**: NLP / LLM

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Bugfix for output_generation_logits in tensorrtllm (#11820)
State: closed | Pull request: yes

# What does this PR do ?

See https://github.com/NVIDIA/NeMo/pull/11820

**Collection**: NLP / LLM

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
update nemo2sftpeft tutorial container verison
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
chore(beep boop 🤖): Bump `MCORE_TAG=c76410a...` (2025-01-13)
State: closed | Pull request: yes
Labels: name: Run CICD

🚀 PR to bump `NVIDIA/Megatron-LM` in `Dockerfile.ci` to `MCORE_TAG=c76410adec0eb1f81544344fcc2c404166e9913d`.  

📝 Please remember the following to-do's before merge: 
- [ ] Verify the presubmit CI  

🙏 Please merge this PR only if the CI workflow completed successfully.
Fix running peft with nemo2 local job
State: closed | Pull request: yes

# What does this PR do ?

Fix the error when running llama3 and llama31 peft with nemo2 local job. 

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


closing this
chore(beep boop 🤖): Bump `MCORE_TAG=726da58...` (2025-01-12)
State: closed | Pull request: yes
Labels: name: Run CICD

🚀 PR to bump `NVIDIA/Megatron-LM` in `Dockerfile.ci` to `MCORE_TAG=726da588ead74ac7f5e62324dea478c1dabb4150`.  

📝 Please remember the following to-do's before merge: 
- [ ] Verify the presubmit CI  

🙏 Please merge this PR only if the CI workflow completed successfully.
Canary ouputs English for Arabic Speech
State: open | Pull request: no
Labels: name: bug

**Describe the bug**

Hello, I am trying to infer [Canary 1b](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/riva/models/canary-riva-1b) for Arabic ASR with Riva quick start 2.18.0. According to the description it has already supported Arabic, but it outputs English instead of Arabic tokens. 

Here's a partial extract of my config.sh
```
service_enabled_asr=true
service_enabled_nlp=false
service_enabled_tts=false
service_enabled_nmt=false

asr_language_code=("ar-AR")
asr_acoustic_model=("canary")
asr_acoustic_model_variant=("1b")
use_existing_rmirs=false
```

When I used the same config to infer Parakeet 1.1b_unified_ml_cs_concat and Parakeet 1.1b_unified_ml_cs_universal, they do output Arabic tokens, so I guess the issue is within the Canary model. 

Any idea please?

@BenoitWang Can you try `ar-SA`? We may have just assigned Arabic language under a Saudi dialect tag for that round.

Hi @tbartley94, just tried but still got English outputs.

hmm, okay will look into it

@BenoitWang It looks like you're using Riva client to run Canary-1b model. If so, you need to pass language code in the client side. For example, `riva_asr_client --language_code=ar-AR --audio_file=ar-AR_sample.wav`. Could you try with `--language_code` param? 

Hi @myungjongk, thank you that works much better. However when I looked into its transcriptions, compared with parakeet-ctc-1.1b-concat (the 1st image), it still generates very often the English tokens (the 2nd image), which degrades both the WER and CER quite a lot. The 20 samples are from CommonVoice 18.0. Am I still missing something please?

<img width="645" alt="Image" src="https://github.com/user-attachments/assets/c753bae8-6338-4164-b3eb-dfd0de6fa9b2" />

<img width="645" alt="Image" src="https://github.com/user-attachments/assets/4950a925-07ba-48bd-9839-282ca13cf4d1" />

In fact we're running this [Arabic ASR leaderboard](https://huggingface.co/spaces/elmresearchcenter/open_universal_arabic_asr_leaderboard), and we find that it performs badly compared to the other models, but we do wish to include Canary if this get fixed, thanks for your help @tbartley94 @myungjongk .

Another observation is that both Canary 1b and 0.6b-turbo hallucinate quite often on Arabic speech, generating repeated tokens, which significantly increases CER.

<img width="885" alt="Image" src="https://github.com/user-attachments/assets/017e9162-8729-46a8-a647-4ea31595eb53" />

Ooh, this is a good catch. Thanks for catching this. 

@myungjongk This may be a deployment issue, I'll evaluate with the NeMo model on my end to see if there's something that didn't pop up in our evaluations.
How to use nemo docker container as base image
State: open | Pull request: no

Hi,
I'm trying to create a docker image that uses nemo as the base image.
I have a python project with some scripts (including a requirements.txt file with some additional dependencies) that I'm containerizing on that same image.

The Nemo base image is around 29GB in size but when I run pip install on the requirements file the output is around 100GB because it reinstalls everything and it takes ages to complete. 

I'm trying to learn how to use this and make some cool examples with ASR and TTS for research/personal training.

I'm pretty sure I'm doing something wrong on this dependency hell. 😀

It would be nice to have a simple dockerfile that uses nvidia/nemo as base image and a simple python project in it.

Can someone help, please?

Thanks in advance.

Hi, you can use this as the start of your dockerfile

```
ARG NEMOFW_VERSION=24.09
FROM nvcr.io/nvidia/nemo:$NEMOFW_VERSION
```

@wheynelau thanks for your reply.
That is what I'm doing. Here is my dockerfile:

```dorckerfile
FROM nvcr.io/nvidia/nemo:24.12 AS base
EXPOSE 8080

# Install PowerShell Core
RUN apt-get update && apt-get install -y wget \
    && wget -q https://packages.microsoft.com/config/ubuntu/24.04/packages-microsoft-prod.deb \
    && dpkg -i packages-microsoft-prod.deb \
    && apt-get update \
    && apt-get install -y powershell \
    && rm packages-microsoft-prod.deb \
    && rm -rf /var/lib/apt/lists/*

# Install PowerShell Core
RUN apt-get update && apt-get install -y wget \
    && wget -q https://packages.microsoft.com/config/ubuntu/24.04/packages-microsoft-prod.deb \
    && dpkg -i packages-microsoft-prod.deb \
    && apt-get update \
    && apt-get install -y powershell \
    && rm packages-microsoft-prod.deb \
    && rm -rf /var/lib/apt/lists/*

# Install dependencies aspnetcore
RUN apt-get update && apt-get install -y \
    apt-transport-https \
    software-properties-common

# Download and run the Microsoft installation script
RUN wget https://dot.net/v1/dotnet-install.sh -O dotnet-install.sh \
    && chmod +x dotnet-install.sh \
    && ./dotnet-install.sh --runtime aspnetcore --version 9.0.0

# Set the environment variables 
ENV DOTNET_ROOT=/root/.dotnet 
ENV PATH=$PATH:/root/.dotnet

WORKDIR /app
COPY ../Diarization/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY ../Diarization/audio ./audio

FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
ARG BUILD_CONFIGURATION=Release
WORKDIR /src
COPY ["Worker/Worker.csproj", "Worker/"]
RUN dotnet restore "./Worker/Worker.csproj"
COPY . .
WORKDIR "/src/Worker"
RUN dotnet build "./Worker.csproj" -c $BUILD_CONFIGURATION -o /app/build

FROM build AS publish
ARG BUILD_CONFIGURATION=Release
RUN dotnet publish "./Worker.csproj" -c $BUILD_CONFIGURATION -o /app/publish /p:UseAppHost=false

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "Worker.dll"]

```

And here is my requirements.txt file:

```
faster-whisper
pydub
demucs
soundfile
Cython
packaging
```

This image has 97.35GB and this is the issue. :)
chore(beep boop 🤖): Bump `MCORE_TAG=726da58...` (2025-01-11)
State: closed | Pull request: yes
Labels: name: Run CICD

🚀 PR to bump `NVIDIA/Megatron-LM` in `Dockerfile.ci` to `MCORE_TAG=726da588ead74ac7f5e62324dea478c1dabb4150`.  

📝 Please remember the following to-do's before merge: 
- [ ] Verify the presubmit CI  

🙏 Please merge this PR only if the CI workflow completed successfully.
[WIP] Use FSDP2 for nemo-lite
State: open | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


<!-- pylint-output-strict-mode-false -->

beep boop 🤖: 🙏 The following files have warnings. In case you are familiar with these, please try helping us to improve the code base.

---

Your code was analyzed with PyLint. The following annotations have been identified:

```
************* Module hf
examples/llm/peft/hf.py:60:0: C0301: Line too long (184/119) (line-too-long)
examples/llm/peft/hf.py:56:0: C0116: Missing function or method docstring (missing-function-docstring)
examples/llm/peft/hf.py:95:0: C0116: Missing function or method docstring (missing-function-docstring)
examples/llm/sft/hf.py:71:0: C0301: Line too long (184/119) (line-too-long)
examples/llm/sft/hf.py:137:0: C0301: Line too long (162/119) (line-too-long)
examples/llm/sft/hf.py:67:0: C0116: Missing function or method docstring (missing-function-docstring)

-----------------------------------
Your code has been rated at 9.63/10
```

---

Mitigation guide:

* Add sensible and useful docstrings to functions and methods
* For trivial methods like getter/setters, consider adding `# pylint: disable=C0116` inside the function itself
* To disable multiple functions/methods at once, put a `# pylint: disable=C0116` before the first and a `# pylint: enable=C0116` after the last.

By applying these rules, we reduce the occurance of this message in future.

Thank you for improving NeMo's documentation!
Bugfix for output_generation_logits in tensorrtllm
State: closed | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?

Gets the value of `generation_logits_available` from `inputs["output_generation_logits"][0][0]` so that it is a bool as opposed to `inputs["output_generation_logits"]` where it's a np array and can give error when the array is len > 1.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


[🤖]: Hi @athitten 👋,

We wanted to let you know that a [CICD pipeline](https://github.com/NVIDIA/NeMo/actions/runs/12718762026) for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g
[doc] [typo] The performance improvement of sequence packing is claimed to be as higher as 10x. Is it a typo?
State: open | Pull request: no
Labels: name: bug

Sequence packing is a very useful feature.
There are two places in the document (as shown in the following) that mention the performance improvement of sequence packing, but the data differs greatly. 
10x performance improvement is unthinkable.  **_Is it a typo?_**

https://docs.nvidia.com/nemo-framework/user-guide/latest/nemotoolkit/features/optimizations/sequence_packing.html

![Image](https://github.com/user-attachments/assets/aefc95d6-df0f-4d9e-8bbf-80876666233b)

https://docs.nvidia.com/nemo-framework/user-guide/latest/nemotoolkit/multimodal/mllm/sequence_packing.html

![Image](https://github.com/user-attachments/assets/1acbcdfa-62c8-457b-8d46-c04f91cdad28)
build: Fix mamba install
State: closed | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


[🤖]: Hi @ko3n1g 👋,

We wanted to let you know that a [CICD pipeline](https://github.com/NVIDIA/NeMo/actions/runs/12705954398) for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g
build: Fix build-arg
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
build: Bump PyT and TE
State: open | Pull request: yes
Labels: name: CI

name: Run CICD

name: no-fail-fast

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
ASR: Is there any checked and stable way for pretrain?
State: open | Pull request: no

Hello,
I am trying to develop ASR tools for 2 custom languages.
I used 3-4 times  speech_to_text_ctc_bpe.py for pretrain, but all cases I did not receive reslult. My best results is WER:0.98.
```
  
python speech_to_text_ctc_bpe.py \
	+config-name=../conf/fastconformer/fast-conformer_ctc_bpe \
	model.train_ds.manifest_filepath=/nemoasr/train.5 \
	model.validation_ds.manifest_filepath=/nemoasr/test.5 \
	model.tokenizer.dir=/nemoasr/tokenizer/tokenizer_spe_unigram_v512 \
	model.tokenizer.type=bpe \
	trainer.devices=-1 \
	trainer.accelerator="gpu" \
	trainer.strategy="ddp" \
	trainer.max_epochs=100 \
	model.optim.name="adamw" \
	model.optim.lr=0.001 \
	model.optim.betas=[0.9,0.999] \
	model.optim.weight_decay=0.0001 \
	model.optim.sched.warmup_steps=2000 \

```
Meanwhile I can get sufficient resluts (WER:3.5-4.5) using script. which I generated from examples in open sources of Internet (Japanese, Vietnamese).

Actually I would not to know is there any checked way for pretrain?
Thank you advance!


No one knows pretrain mechanism???
It is so-so strange
How to set lhotse mixed noise parameters in yaml
State: open | Pull request: no

if use_lhoste set False, 
yaml set noise:
```
model:
  train_ds:
    augmentor:
          time_stretch:
                .....
          gain:
               ......
          noise:
            manifest_path: background_noise.json
            .....
          white_noise:
            ......
```
How should I set it when use_lhotse set True?
chore(beep boop 🤖): Bump `MCORE_TAG=3046e33...` (2025-01-10)
State: closed | Pull request: yes
Labels: name: Run CICD

🚀 PR to bump `NVIDIA/Megatron-LM` in `Dockerfile.ci` to `MCORE_TAG=3046e33f242d076181436efd4714eae4e40e534e`.  

📝 Please remember the following to-do's before merge: 
- [ ] Verify the presubmit CI  

🙏 Please merge this PR only if the CI workflow completed successfully.
Add padding in mllama vision encoder to align with HF
State: open | Pull request: yes

# What does this PR do ?

Add padding in mllama vision encoder to align with HF

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Latest News updated for Cosmos
State: closed | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?

Add news for Cosmos and previous news/cosmos blurb written by jennifer in a previously closed PR

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


Reviewed latest updates. LGTM.
News: Cosmos WFMs support
State: closed | Pull request: yes

# What does this PR do ?

Update the readme for news about Cosmos WFMs.

**Collection**:  readme

# Changelog 
- Added new details element for Cosmos WFMs
- Added subsection for cosmos in "What's new in NeMo 2.0" (content was previously already written by Jennifer)

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix the tokenizer after llama-3
State: open | Pull request: yes

# What does this PR do ?

update 'LlamaTokenizer' → 'AutoTokenizer'

Starting from llama-3, the tokenizer has been switched to use `PreTrainedTokenizerFast` instead of `LlamaTokenizer`.
This PR fixes the tokenizer loading method, enabling it to function correctly with models released after llama-3

# Changelog 
- L21: remove unused import `open_dict`
- L22: `import LlamaTokenizer` → `import AutoTokenizer`
- L258: load tokenizer by `LlamaTokenizer` → load by `AutoTokenizer`
- L261: remove `LlamaTokenizer` parameter
- L263~265: the tokenizer load by `AutoTokenizer` is already a "Fast" tokenizer, we don't need to convert it
- L276: remove the unused variable

# Usage
```python
>>> from transformers import LlamaTokenizer
>>> try:
...     tokenizer = LlamaTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B")
... except Exception as e:
...     print(e)
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'PreTrainedTokenizerFast'. 
The class this function is called from is 'LlamaTokenizer'.
```

```python
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B")
```

# Before your PR is "Ready for review"

**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation


<!-- pylint-output-strict-mode-false -->

beep boop 🤖: 🙏 The following files have warnings. In case you are familiar with these, please try helping us to improve the code base.

---

Your code was analyzed with PyLint. The following annotations have been identified:

```
************* Module convert_llama_nemo_to_hf
scripts/checkpoint_converters/convert_llama_nemo_to_hf.py:61:0: C0116: Missing function or method docstring (missing-function-docstring)
scripts/checkpoint_converters/convert_llama_nemo_to_hf.py:241:0: C0116: Missing function or method docstring (missing-function-docstring)

-----------------------------------
Your code has been rated at 9.84/10
```

---

Mitigation guide:

* Add sensible and useful docstrings to functions and methods
* For trivial methods like getter/setters, consider adding `# pylint: disable=C0116` inside the function itself
* To disable multiple functions/methods at once, put a `# pylint: disable=C0116` before the first and a `# pylint: enable=C0116` after the last.

By applying these rules, we reduce the occurance of this message in future.

Thank you for improving NeMo's documentation!
chore(beep boop 🤖): Bump `MCORE_TAG=bafab5a...` (2025-01-09)
State: closed | Pull request: yes
Labels: name: Run CICD

🚀 PR to bump `NVIDIA/Megatron-LM` in `Dockerfile.ci` to `MCORE_TAG=bafab5a54b55f2f7309e6625cd0887756548a44b`.  

📝 Please remember the following to-do's before merge: 
- [ ] Verify the presubmit CI  

🙏 Please merge this PR only if the CI workflow completed successfully.

[🤖]: Hi @ko3n1g 👋,

We wanted to let you know that a [CICD pipeline](https://github.com/NVIDIA/NeMo/actions/runs/12681430662) for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g
Fix Gemma2 Attention Init Args
State: closed | Pull request: yes
Labels: name: NLP

name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


[🤖]: Hi @suiyoubi 👋,

We wanted to let you know that a [CICD pipeline](https://github.com/NVIDIA/NeMo/actions/runs/12679136145) for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g