renwei2024/nemo-github-issues huggingface dataset:
Remove precision args in trainer due to PTL update
State: closed | Pull request: yes
Labels: name: NLP

name: Multi Modal

# What does this PR do ?

Remove precision args in trainer due to PTL update. The precision plugin and precision arg cannot be used together anymore.

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

jenkins

jenkins

jenkins
Zero-shot evaluation pipeline for mcore RETRO
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

Adding zero-shot evaluation code for new RETRO model. The evaluation code for legacy model is modified to *_legacy.py
This PR is on top of huvu/mcore_retro, which implements the new RETRO model.

The files this PR works on are:

 + examples/nlp/language_modeling/conf/megatron_retro_inference.yaml
 + examples/nlp/language_modeling/megatron_retro_eval.py
 + examples/nlp/language_modeling/megatron_retro_qatask_eval.py
 + examples/nlp/language_modeling/conf/megatron_retro_qatask.yaml
 + nemo/collections/nlp/modules/common/text_generation_strategy.py
 + nemo/collections/nlp/modules/common/text_generation_utils.py

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins
Add option to get per sample loss in CP
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Skip top_p computations when set to 1.0
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

This avoid doing useless computations (that may even filter out some tokens due to numerical approximation) when `top_p=1.0` (which is not supposed to have any effect).

**Collection**: nlp

# Changelog 
- Skip top_p computations when set to 1.0

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation
- [X] Optimization

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

jenkins

jenkins

jenkins
Correcting bullets and notes within NeMo Forced Aligner
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


Thanks !
Add optimized positional multi head stateless net (rnn-t decoder).
State: closed | Pull request: yes
Labels: name: stale

name: ASR

# What does this PR do ?

This is an optimized (without loops) version of stateless net for transducer decoder model.

**Collection**: ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ x] New Feature
- [ ] Bugfix
- [ ] Documentation

## Who can review?

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
File not found in the github repo
State: closed | Pull request: no
Labels: name: bug

name: stale

While following the official documentation - https://developer.nvidia.com/blog/adapting-p-tuning-to-solve-non-english-downstream-tasks/ , I would like to see the Multitask_Prompt_and_PTuning.ipynb notebook file for reference. But it seems the file was moved/deleted. 
![Screenshot from 2024-04-12 17-03-00](https://github.com/NVIDIA/NeMo/assets/41164884/d9e47367-28f9-4dd6-834c-a630ed59991d)
Where to it moved? where can I see this notebook
@okuchaiev 

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.
NeMo dev doc restructure
State: closed | Pull request: yes
Labels: name: ASR

name: NLP

name: Multi Modal

# What does this PR do ?

Refactor NeMo Developer Docs 1st PR. In this PR, we restructured the dev doc as screen shot below. More contents yet to be added.

<img width="327" alt="image" src="https://github.com/NVIDIA/NeMo/assets/54727607/f3d6d612-6910-4e98-9712-50d475117742">



# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Akoumparouli/low mem mixtral ckpt converter
State: closed | Pull request: yes

# What does this PR do ?
This add the option --low-ram to enable checkpoint conversion in low-ram computers for large checkpoints (e.g. 250GB).

Memory reduction is achieved by not having the full converted checkpoint and the huggingface checkpoint on memory at the same time. Instead, it saves the converted checkpoint layer by layer, remove the HF checkpoint from memory and finally restores the converted layers and saves to .nemo format.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

jenkins
Akoumparouli/add ep in yaml
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?
Add expert_model_parallel_size to yamls

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

jenkins

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Fix README header
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Consider refactoring CTC greedy decoding
State: closed | Pull request: no

**Is your feature request related to a problem? Please describe.**

As part of my work on #8673 , I have been profiling to make sure we have low latency and high throughput. I have observed that this line can sometimes take https://github.com/NVIDIA/NeMo/blob/57444ae9b63afc5419a16948f7d032a1a9f7dc7f/nemo/collections/asr/parts/submodules/ctc_greedy_decoding.py#L218 several milliseconds. To be concrete, running at batch size 10, the model is taking 167 milliseconds, while decoding is taking 29 milliseconds, with almost all of the time spent on this max() operation. This is documented below.

<img width="1244" alt="image" src="https://github.com/NVIDIA/NeMo/assets/4767568/623c66d5-57b9-4d9d-8e35-62cfad51a702">

It is not acceptable to have model inference be bottlenecked on non-GPU work like this.

I uploaded a gzipped version of the nsight systems file if you would like to view it.

[report25.nsys-rep.gz](https://github.com/NVIDIA/NeMo/files/14950389/report25.nsys-rep.gz)

The most obvious way to handle this situation is to do the "max()" operation on the GPU. I previously did a refactor that sped up CTC decoding by improving copy from GPU to CPU https://github.com/NVIDIA/NeMo/pull/8521, but it seems that is better to copy only after the max() operation is done. I did not do this in my previous PR, because (1) I didn't think it would be a concern and (2) that change is much more intrusive. _greedy_decode_logprobs is built with running on only a single element at a time rather than a batch. Maybe vmap can save us...

BTW, why is this only sometimes slow? I'm not certain, but it probably has to do with the CPU cache. It's working directly on pinned memory. The cache line may be getting evicted only sometimes?

@artbataev this may be related to your observation of TDT being faster than CTC in some cases.

If I recall correctly, max does work on GPU so I don't know why I forced it onto CPU. Maybe the slice? Either way, if it fixes the slowdown let's move the .CPU() to after max calculation.

It might have been that the logprobs can be super long for 12 hours so I wanted to preserve memory, in which case the cost of speed is not under consideration but I suppose we can make it a premature optimization and select whether to move the tensor to CPU before or after depending on length of the tensor. 

It is pretty common to run long audio CTC - NFA does it constantly. But on average, it is more common to run short audio batches 

> It might have been that the logprobs can be super long for 12 hours so I wanted to preserve memory

The tensor pointed to by "prediction" already existed on GPU. It is just being copied to CPU in this code. We should not expect any memory increase from my suggestion. In fact, we will see a CPU memory decrease.

That's true. Can you see if moving it onto CPU at the end before return is sufficient to remove this bottleneck then let's do it in your other pr
Skip validation model gradient zeroing
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Enable the AllGather of the updated parameters in the data-parallel domain with the validation step.

# Changelog 
- Skip gradient zeroing at the beginning of the validation step routine, which executes param sync
- Pass param sync func at validation step

# Usage
- Set `validation_param_sync_overlap=True`, which is `False` by default

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins
disable bprop reduce for cpl when SP is enabled
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

disable bprop reduce for cpl when SP is enabled

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


@cuichenx

jenkins

jenkins
Huvu/mcore retro eval
State: closed | Pull request: yes
Labels: name: TTS

name: NLP

name: CI

# What does this PR do ?

Adding zero-shot evaluation code for new RETRO model. The evaluation code for legacy model is modified to `*_legacy.py`
 This PR is on top of `huvu/mcore_retro`, which implements the new RETRO model.

The files this PR works on are:
 + examples/nlp/language_modeling/conf/megatron_retro_inference.yaml
 + examples/nlp/language_modeling/megatron_retro_eval.py
 + examples/nlp/language_modeling/megatron_retro_qatask_eval.py
 + examples/nlp/language_modeling/conf/megatron_retro_qatask.yaml
 + nemo/collections/nlp/modules/common/text_generation_strategy.py
 + nemo/collections/nlp/modules/common/text_generation_utils.py 

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

jenkins
support PTL >= 2.1 for chatglm
State: closed | Pull request: yes

# What does this PR do ?
support PTL >= 2.1 for chatglm

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?
@yaoyu-33 

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
NeMo License Discrepancy?
State: closed | Pull request: no
Labels: name: bug

Howdy,

So I noticed that in the readme of this GitHub Repo it states that NeMo is released under the Apache 2.0 license.
![image](https://github.com/NVIDIA/NeMo/assets/166728544/b705b16e-afbe-43e1-9038-c36e81b559aa)
This is also the license [linked](https://github.com/NVIDIA/NeMo/blob/main/LICENSE) in the repo.

However, on the NVIDIA website, the page for the [NeMo Framework Docker Container](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nemo) states that NeMo is licensed under the NVIDIA AI Product Agreement.
![image](https://github.com/NVIDIA/NeMo/assets/166728544/bf298587-4d52-4fd6-b7ea-2c60e8fa5824)

Is there a reason for this discrepancy, or could anyone provide any guidance on which license is correct?
Canary greedy and temperature decoding
State: closed | Pull request: yes
Labels: name: stale

name: ASR

name: common

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
TypeError: EncDecCTCModel.transcribe() got an unexpected keyword argument 'logprobs'
State: closed | Pull request: no
Labels: name: bug

Hi,

I am trying to use language model described in Offline_ASR.ipynb, but I am receiving error:

`TypeError: EncDecCTCModel.transcribe() got an unexpected keyword argument 'logprobs'`

My code is

`import nemo
import nemo.collections.asr as nemo_asr
import sys
import os
import ctc_decoders
import numpy as np

wave_file = ['test2.wav']
quartznet = nemo_asr.models.EncDecCTCModel.restore_from(restore_path="checkpoints/ASR-Char-Model-Language-hy.nemo",strict=False)
#print(quartznet.transcribe(wave_file,batch_size=1))
#softmax implementation in NumPy
def softmax(logits):
    e = np.exp(logits - np.max(logits))
    return e / e.sum(axis=-1).reshape([logits.shape[0], 1])

#let's do inference once again but without decoder
logits = quartznet.transcribe(wave_file, logprobs=True)[0]
probs = softmax(logits)

lm_path='/corpuses/hye/corpus/19.0/lm_tgsmall.arpa.gz'
beam_search_lm = nemo_asr.modules.BeamSearchDecoderWithLM(
    vocab=list(quartznet.decoder.vocabulary),
    beam_width=16,
    alpha=2, beta=1.5,
    lm_path=lm_path,
    num_cpus=max(os.cpu_count(), 1),
    input_tensor=False)

beam_search_lm.forward(log_probs = np.expand_dims(probs, axis=0), log_probs_length=None)`

The single difference from Offline_ASR.ipynb is nemo_asr.models.EncDecCTCModel.restore_from instead nemo_asr.models.EncDecCTCModel.from_pretrained.

Could you advise me error reason?
Thank you advance!


NeMo main has an updated version of transcribe() that supports more functionality and deprecated old arguments with new ways of doing things. 

You can use and install Nemo 1.23.0 branch to get it to work. Or you can use the following to fix transcribe on main. 

Change 
logits = quartznet.transcribe(wave_file, logprobs=True)[0]

To 

```python

Logits_hyps = asr_model.transcribe(
                self.audio_file_list, batch_size=self.asr_batch_size, return_hypotheses=True
            )  # type: List[nemo_asr.parts.Hypothesis]
            
Logits = [hyp.alignments for hyp in Logits_hyps]
```


Thank you very much!
Akoumparouli/fix sd train 2
State: closed | Pull request: yes
Labels: name: CI

name: Multi Modal

# What does this PR do ?
pass fp16 precision to UNetModel constructor if user specifies fp16 precision via CLI.

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

jenkins

jenkins

Set unet_precision when user not set it.


jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins
lora a2a after linear out when sp is enabled and parallel input
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

lora a2a after linear out when sp is enabled and parallel input

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


LGTM, We did convergence runs and convergence looks as expected on GBS=8, 32. We see measurable speedup for this change.
One minor comment:
- We should try to JIT the python code, but maybe that can happen as a separate PR.

@ericharper @cuichenx can you pls review/comment?

jenkins
Support alternative mapping TP->PP->DP
State: closed | Pull request: yes
Labels: name: core

name: ASR

name: NLP

name: CI

name: Multi Modal

# What does this PR do ?

This PR will support new parallel initialize order with megatron's new feature: TP -> PP -> DP

**Collection**: [Note which collection this PR will affect]
* NLP collection

# Changelog 
- refactor rank calculate by mcore's new interface.
- a new knob 'use_tp_pp_dp_mapping' to enable this feature

# Usage
* You can potentially add a usage example below

just make sure use enable `use-tp-pp-dp-mapping` knob.

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

Will reopen with another branch soon.
Fix Distributed Fused Adam Issues
State: closed | Pull request: yes
Labels: name: core

# What does this PR do ?

This PR fixes distributed optimizer issues:

1. Support NHWC layout, as required by Diffusion models;
2. Fix complex `zero_grad()` being not captured by CUDA graph;
3. Add option to distribute states within node;

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


> Overall looks good, although we are still hashing out the design in [NVIDIA/apex#1794](https://github.com/NVIDIA/apex/pull/1794). As discussed in [NVIDIA/apex#1794 (comment)](https://github.com/NVIDIA/apex/pull/1794#issuecomment-2050894033), I think we should set `MegatronDistributedFusedAdam._step_support_amp_scaling=False` to signal that the NeMo grad scaler can accommodate the distributed optimizer (unlike the plain PyTorch grad scaler). As a bonus, this approach fixes the grad scaling issue even without needing to update Apex.

You are probably right, to make sure it won't hurt perf I need to confirm with our usecases. Anyway, I think that relates to the changes in APEX, the changes here have no relation to gradient clipping.

jenkins

jenkins
Add TE guards for DGRAD RS overlap
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


Tested that the guard works correctly now, and will also work in 1.6 release where version would be 1.6.0.

jenkins
Fix module.training for Neva in FusedAttn backward which causes nan
State: closed | Pull request: yes
Labels: name: Multi Modal

# What does this PR do ?

Fused Attn requires module training set to True even the module self is frozen.

**Collection**: [Multimodal]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

jenkins
Akoumparouli/fix sd train
State: closed | Pull request: yes
Labels: name: CI

name: Multi Modal

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

jenkins
remove fp8 checkpoints for Attention
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Fix Checkpointing for FP8 Attention

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


@mikolajblaz can you review?

jenkins

jenkins

jenkins
Updates for TRT-LLM 0.9
State: closed | Pull request: yes

# What does this PR do ?

Updates export to TRT-LLM code for version 0.9
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation
- [x] Update



@JimmyZhang12 thanks for your review. Addressed your comments. Could you please approve?

jenkins

jenkins
How to search for pretrained model
State: closed | Pull request: no
Labels: name: stale

**Is your feature request related to a problem? Please describe.**
How can I search for a pretrained ONNX model?
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Hi, since my PC has no powerful GPU, so training any model is almost mission impossible, so I have to use some pretrained model, for example: I know one ONNX model: stt_en_quartznet15x5.onnx
But I can't find such pretrained model from ONNX zoo.

**Describe the solution you'd like**
I want to have some kind of pre-trained model search, so I can either input the model name or something to describe the pre-trained models, like: "Transcribe French", then the searching engine can return a list of pre-trained models meet the searching conditions, and provide download link, so I can download the models I want.
Please note that not everyone has the computing power to train any complicated model, not many people can host computing power like a data center.

A clear and concise description of what you want to happen.
Provide a code snippet on how new APIs/changes would be used by others.

**Describe alternatives you've considered**
No alternatives I can think about.

A clear and concise description of any alternative solutions or features you've considered.

**Additional context**

Add any other context or screenshots about the feature request here.


We don't have pretrained onnx models released in NeMo. Nvidia Riva might have .Riva files that contain a compiled model but that isn't supported in NeMo 

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.
Rachitg/fp8dpa
State: closed | Pull request: yes
Labels: name: core

name: NLP

name: CI

name: Multi Modal

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)