renwei2024/nemo-github-issues huggingface dataset:
Latest release version 1.23.0 missing the AudioCodecModel checkpoint list.
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

Latest release version `1.23.0` is missing the `AudioCodecModel` checkpoint list.

**Steps/Code to reproduce bug**

As instructed [here](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/audio_codec_16khz_small), run the following code
```
from  nemo.collections.tts.models  import  AudioCodecModel
nemo_codec_model  =  AudioCodecModel.from_pretrained('audio_codec_16khz_small')
```

**Expected behavior**

I expect to load the 16kHz checkpoint of `AudioCodecModel`. However, on running the above code, I get the following error
```
FileNotFoundError: Model audio_codec_16khz_small was not found. Check cls.list_available_models() for the list of all available models.
```
On checking the function in the latest release version `==1.23.0`, https://github.com/NVIDIA/NeMo/blob/e772dbf53145a7b45f13440cf6e0ef51035f80dc/nemo/collections/tts/models/audio_codec.py#L646
we see that the function returns an empty list.
\
This problem is resolved on the latest commit on the `main` branch
https://github.com/NVIDIA/NeMo/blob/3c29fefe9ac442e594f1c35c0f8ecc09b5ef5015/nemo/collections/tts/models/audio_codec.py#L652

but installing `nemo_toolkit` using `pip` as instructed installs the latest release version.


**Environment overview**

 - Environment location: Linux Cloud Server
 - Method of NeMo install: `pip install nemo_toolkit['all']`

**Environment details**
- OS version: Ubuntu 22.04.3 LTS
- PyTorch version: 2.3.0
- Python version: 3.10.12



Audio codec is in NeMo 2.0.0rc0. You can get it by using pip install nemo_toolkit[asr,tts] --pre
Akoumparouli/nemo ux mixtral
State: closed | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Multimodal projection layer adapter fix for PP>1
State: closed | Pull request: yes
Labels: name: core

name: NLP

name: Multi Modal

name: Run CICD

# What does this PR do ?

When loading a non-strict dist checkpoint in NeMo it hangs at https://github.com/NVIDIA/NeMo/blob/d7bb40364c17bf322004539f851cc83df4c4c2b7/nemo/utils/callbacks/dist_ckpt_io.py#L272 at PP_rank=0. 

When loading multimodal projection layer, there will be ranks loading nothing which leads to the hang because we only load weights for rank0. Setting **validate_access_integrity** resolves this hang.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix failing RIR unit test with lhotse 1.24+
State: closed | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Remove configs not relevant to Fast Conformer-CTC
State: closed | Pull request: yes
Labels: name: stale

name: ASR

# What does this PR do ?

Removed "pred_hidden", "joint_hidden", and "pred_rnn_layers" because they are Fast Conformer-Transducer configs.

**Collection**: [ASR]

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [X] Documentation

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Tiledsiglip
State: closed | Pull request: yes
Labels: name: stale

name: Multi Modal

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Fix py3.11 dataclasses issue
State: closed | Pull request: yes
Labels: name: ASR

name: Run CICD

# What does this PR do ?

Fixes #9419.

**Collection**: [ASR]

# Changelog 
- Fix dataclasses in examples/asr/transcribe_speech.py.
- Fix dataclasses in examples/asr/speech_to_text_eval.py.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation


# Additional Information
* Related to #7166.


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.

Could you fix the formatting checks?

> Could you fix the formatting checks?

Formatting checks are passing now (after I synced my fork).

We're seeing some CI instability so the tests are taking longer than ususal. I merged main and re-triggered the failing tests (they don't seem related to your changes). Will keep monitoring this PR until it's all green.
Can we run NeMo MSDD Neural Diarizer model in realtime for realtime diarization?
State: closed | Pull request: no


**Disclaimer** 
I just want some inputs in developing a below mentioned pipeline, if it's feasible.

**Requirements**
I'm currently working on a project where I need the diarization to work in a real-time manner.  I tried using the PyAnnote Library but None of those could match the accuracy of MSDD but We weren't able to run MSDD in a real-time manner. Is there any example for how to do this.  We need to run MSDD along with Whisper model for transcription. 

check these two PRs: https://github.com/NVIDIA/NeMo/pull/5609, https://github.com/NVIDIA/NeMo/pull/7896
Although I couldn't get them to work

Thanks for your interest, no we currently don;t support online speaker diarization using MSDD. 

@nithinraok thank you nithin for the confirmation
[NeMo-UX] Adding support for mcore distributed optimizer
State: closed | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


@marcromeyn can you fix the DCO? thanks.
FastConformer-Longformer
State: closed | Pull request: no

Hi NeMo team,
Do you plan to release En Large FastConformer-Long-CTC-BPE model trained with Local Attention and Global token?

we released 1B model but not the large model here: https://huggingface.co/spaces/nvidia/parakeet-tdt_ctc-1.1b 

Thanks, I have seen this. I was interested to know about Large-FastConformer(~120M) model trained with this config:
https://github.com/NVIDIA/NeMo/blob/main/examples/asr/conf/fastconformer/long_fastconformer/fast-conformer-long_ctc_bpe.yaml
Draft: Add LoRA test with sequence parallelism
State: closed | Pull request: yes
Labels: name: CI

name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.
MCore T5 support for NeMo - Training
State: closed | Pull request: yes
Labels: name: NLP

name: CI

name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.
RAM memory leaks for EncDecCTCModelBPE at inference
State: closed | Pull request: no
Labels: name: bug

name: stale

Numbers suggests that `EncDecCTCModelBPE` has RAM memory leaks.
I'm running inference with `batch_size=1` on a directory with 10k small audio files with total duration of around 9.5 hours.

Does any body know where this excessive RAM usage comes from?

Here is a sample script:

```
import time
from pathlib import Path

import psutil
import torch
from nemo.collections.asr.models import EncDecCTCModelBPE

torch.set_num_threads(1)
proc_usage = psutil.Process()
cur_usage_rss = proc_usage.memory_info().rss / 1024 / 1024
print(f"Initial rss usage:{cur_usage_rss}")


dir_name = "/mnt/data/subset_10k"
audio_paths = Path(dir_name)


def find_leaks():
    model = EncDecCTCModelBPE.from_pretrained(model_name="stt_en_conformer_ctc_medium")
    res_file_path = f"memory_usage3_start_{time.time()}.txt"
    fd = open(res_file_path, "w")

    files = audio_paths.glob("*.wav")
    for idx, a_path in enumerate(files):
        res = model.transcribe(str(a_path), verbose=False)

        if idx % 100 == 0:
            cur_usage_rss = proc_usage.memory_info().rss / 1024 / 1024
            msg = f"{idx: <10} rss usage:{cur_usage_rss}"
            print(msg)
            print(msg, file=fd)

    fd.close()


if __name__ == "__main__":
    find_leaks()
```

And a sample output:
```
0          rss usage:1433.66796875
100        rss usage:1561.16796875
200        rss usage:1584.16796875
300        rss usage:1596.66796875
400        rss usage:1607.16796875
500        rss usage:1629.9375
600        rss usage:1636.4375
700        rss usage:1660.2890625
800        rss usage:1660.2890625
900        rss usage:1661.2890625
1000       rss usage:1672.7890625
1100       rss usage:1680.7890625
1200       rss usage:1688.7890625
1300       rss usage:1695.7890625
1400       rss usage:1697.2890625
1500       rss usage:1706.2890625
1600       rss usage:1718.09765625
1700       rss usage:1718.09765625
1800       rss usage:1721.09765625
1900       rss usage:1725.09765625
2000       rss usage:1738.59765625
```

**Expected behavior**

Constant memory consumption over time.

**Environment overview (please complete the following information)**

 - Environment location: Bare-metal
 - Method of NeMo install: `python -m pip install git+https://github.com/NVIDIA/NeMo.git@{2.0.0rc0}#egg=nemo_toolkit[all]`

**Environment details**
- OS version: Ubuntu 22.04, kernel 6.5.0-35-generic
- PyTorch version: 2.2.2
- Python version: 3.10.12


This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.

Try to run the model undel 
```
with torch.no_grad()
```
query tp_comm_disable_qkv via getattr
State: closed | Pull request: yes
Labels: name: NLP

name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


will add this to mcore.
Use TensorRT-LLM native parameter names in nemo.export module
State: closed | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?

A small renaming PR to use native TensorRT-LLM parameter names:
* `max_input_len` instead of `max_input_token`
* `max_output_len` instead of `max_output_token`

For  `export` and `forward` methods in `TensorRTLLM` class it is still possible to use old names but a deprecation warning is issued.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] Refactor
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Enable specifying alpha for PTQ INT8 SmoothQuant method
State: closed | Pull request: yes
Labels: name: NLP

name: Run CICD

# What does this PR do ?

Allow users to specifying **alpha** for INT8 SmoothQuant method. For example, setting it to 0.8 improves Llama3 (8b, 70b) MMLU accuracy results compared to default alpha=1.0.

```
| MODEL      |   ALPHA |   MMLU |
|------------|---------|--------|
| LLAMA3-70B |    0    |  0.004 |
| LLAMA3-70B |    0.25 |  0.116 |
| LLAMA3-70B |    0.5  |  0.516 |
| LLAMA3-70B |    0.7  |  0.765 |
| LLAMA3-70B |    0.75 |  0.77  |
| LLAMA3-70B |    0.8  |  0.772 |
| LLAMA3-70B |    0.85 |  0.766 |
| LLAMA3-70B |    0.9  |  0.75  |
| LLAMA3-70B |    1    |  0.728 |
| LLAMA3-8B  |    0    |  0.062 |
| LLAMA3-8B  |    0.25 |  0.227 |
| LLAMA3-8B  |    0.5  |  0.379 |
| LLAMA3-8B  |    0.7  |  0.617 |
| LLAMA3-8B  |    0.75 |  0.624 |
| LLAMA3-8B  |    0.8  |  0.625 |
| LLAMA3-8B  |    0.85 |  0.62  |
| LLAMA3-8B  |    0.9  |  0.619 |
| LLAMA3-8B  |    0.95 |  0.612 |
| LLAMA3-8B  |    1    |  0.594 |
```
**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
python megatron_quantization.py \
    ...   
    quantization.algorithm=int8_sq \
    quantization.alpha=0.5 \
    ....
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Wrap all fp8 extra states in LocalNonpersistentObject
State: closed | Pull request: yes
Labels: name: stale

name: NLP

name: Run CICD

# What does this PR do ?

This PR generalizes FP8 extra state wrapping for all tensors.

**Collection**: nlp

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
mcore distOpt restore fix
State: closed | Pull request: yes
Labels: name: NLP

name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Akoumparouli/profiling docs
State: closed | Pull request: yes
Labels: name: documentation

name: core

name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


CI passed: https://github.com/NVIDIA/NeMo/actions/runs/9440118885
Python 3.11 dataclasses ValueError
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

Some scripts still throw errors similar to #7166:
```
transcribe_speech.py 104 <module>
@dataclass

dataclasses.py 1232 dataclass
return wrap(cls)

dataclasses.py 1222 wrap
return _process_class(cls, init, repr, eq, order, unsafe_hash,

dataclasses.py 958 _process_class
cls_fields.append(_get_field(cls, name, type, kw_only))

dataclasses.py 815 _get_field
raise ValueError(f'mutable default {type(f.default)} for field '

ValueError:
mutable default <class 'nemo.collections.asr.modules.conformer_encoder.ConformerChangeConfig'> for field conformer is not allowed: use default_factory
```

**Steps/Code to reproduce bug**

`python NeMo-1.23.0/examples/asr/transcribe_speech.py`


**Expected behavior**

Run without error.

**Environment overview (please complete the following information)**

 - Environment location: Bare-metal.
 - Method of NeMo install: `pip install nemo_toolkit['asr']`.

**Environment details**

- OS version: Linux DESKTOP-4S0973M 5.15.146.1-microsoft-standard-WSL2
- PyTorch version: 2.3.0
- Python version: 3.11.9

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.
Fix: Ensure start_index is converted to a CPU-based NumPy array
State: closed | Pull request: yes
Labels: name: stale

name: ASR

# What does this PR do ?

Ensures the `start_index` is correctly converted to a CPU-based NumPy array to maintain compatibility with operations requiring NumPy arrays.

**Collection**: `RNNT Decoding`

# Changelog 
- Updated the calculation of `start_index` to include `.cpu().numpy()` conversion in the `AbstractRNNTDecoding` class.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Getting `TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.`
State: closed | Pull request: no
Labels: name: bug

I am getting `TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.` when transcribing using a transducer model by setting `return_hypotheses=True` and `compute_timestamps=True` in decoding config. If I remove `compute_timestamps=True`, I don't get this error but the char and word level timestamp information won't be included.

**Steps/Code to reproduce bug**

```python
config = nemo_asr.models.ASRModel.restore_from(model_path, return_config=True)
decoding_config = config.decoding

decoding_config.strategy = "greedy_batch"

with open_dict(decoding_config):
    decoding_config.preserve_alignments = True
    decoding_config.fused_batch_size = -1  # temporarily stop fused batch during inference.
    decoding_config.compute_timestamps = True

model.change_decoding_strategy(decoding_config)

hypothesis = model.transcribe(audio=sample_audio, batch_size=1, return_hypotheses=True)[0][0]
```

**Error**
```bash
File [~/NeMo/nemo/collections/asr/parts/submodules/rnnt_decoding.py:851], in AbstractRNNTDecoding._compute_offsets(hypothesis, token_repetitions, rnnt_token)
    849 # Construct the start and end indices brackets
    850 end_indices = np.asarray(token_repetitions).cumsum()
--> 851 start_indices = np.concatenate(([start_index], end_indices[:-1]))
.
.
.
File [~/miniconda3/envs/base/lib/python3.11/site-packages/torch/_tensor.py:1062], in Tensor.__array__(self, dtype)
   1060     return handle_torch_function(Tensor.__array__, (self,), self, dtype=dtype)
   1061 if dtype is None:
-> 1062     return self.numpy()
   1063 else:
   1064     return self.numpy().astype(dtype, copy=False)

TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.
```

**Environment overview (please complete the following information)**

 - Environment location: AWS
 - Method of NeMo install: from source

**Environment details**
- OS version: Ubuntu 22.04
- PyTorch version: 2.2.1
- Python version: 3.11
Fix README
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add option to merge distributed optimizer buckets
State: closed | Pull request: yes
Labels: name: feature

name: NLP

name: Run CICD

# What does this PR do ?

Adds an option to merge distributed optimizer buckets with GPT. By default each bucket corresponds to a Transformer layer (without interleaved pipeline parallelism) or to virtual pipeline parallel stage (with interleaved pipeline parallelism)

**Collection**: NLP

# Changelog 
- Add option to merge distributed optimizer buckets

# Usage

Run GPT, e.g. with the config at https://github.com/NVIDIA/NeMo/blob/main/examples/nlp/language_modeling/conf/megatron_gpt_config.yaml.

Enable the distributed optimizer with `model.optim.name=distributed_fused_adam` and merge buckets with `model.distributed_adam_bucket_merge_size=<N>`.

For example, if training GPT without pipeline parallelism, `model.distributed_adam_bucket_merge_size=2` will cause each distopt bucket to contain the weights of two Transformer layers.

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

Pinging @erhoo82.

# Additional Information



@ericharper 
Can you please review and merge?
ci: Enrich notifications
State: closed | Pull request: yes
Labels: name: CI

name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
cherry pick of #9266
State: closed | Pull request: yes
Labels: name: NLP

name: CI

name: Run CICD

# What does this PR do ?

cherry pick of #9266 PR.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
FSDP reduce_scatter can not overlap with compute
State: closed | Pull request: no
Labels: name: bug

name: stale

I use nemo 24.03 and i find FSDP reduce_scatter can not overlap with compute
<img width="838" alt="截屏2024-06-07 15 24 28" src="https://github.com/NVIDIA/NeMo/assets/25609661/0aa82472-88a5-4445-aa0c-0c8d8d41bb32">

```[tasklist]
### Tasks
```


no one concerned about this issue？


This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.