renwei2024/nemo-github-issues huggingface dataset:
Sync with main
State: closed | Pull request: yes
Labels: name: core

name: TTS

name: ASR

name: NLP

name: Speaker Tasks

name: CI

name: common

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Toxicity classifier
State: closed | Pull request: yes
Labels: name: core

name: stale

name: NLP

# What does this PR do ?

Merge Toxicity classifier branch into main

**Collection**: nlp

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Fix nemo_llama_to_hf conversion
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Fix convert_nemo_llama_to_hf.py:

- Correctly account for `megatron_amp_O2` flag
- Save HF model with correct precision
- Transfer over the NeMo tokenizer, instead of using the possibly-incompatible default tokenizer
- Make sure to save fast version of tokenizer
- Resize the model's embedding tensor to match the new tokenizer's vocab
- Correct a typo in how-to-use example

  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation


## Who can review?

@ericharper 

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

jenkins
.ctm in data simulator annotator compliant with RT-09 specification
State: closed | Pull request: yes
Labels: name: ASR

An attempt to fix https://github.com/NVIDIA/NeMo/issues/7445 so that the data simulator .ctm are compliant with RT-09 specification (see https://web.archive.org/web/20170119114252/http://www.itl.nist.gov/iad/mig/tests/rt/2009/docs/rt09-meeting-eval-plan-v2.pdf): 
```
<SOURCE><SP><CHANNEL><SP> <BEG-TIME><SP><DURATION><SP><TOKEN><SP>
<CONF><SP><TYPE><SP><SPEAKER><NEWLINE>
```

I have put <NA> for fields unknown e.g. <TOKEN>. 
This makes it also easy to use the generated sessions with https://github.com/lhotse-speech/lhotse

Sign-off issue. Will re-open another PR. Closing.
Fix crash when converting to mcore a model using rotary embeddings
State: closed | Pull request: yes

# What does this PR do ?

Fix crash when converting to mcore a model using rotary embeddings.

The problem is that mcore models don't register `rotary_pos_emb.inv_freq` as a buffer anymore.

I made it a command-line argument so that if the same issue occurs with other parameters / buffers, users can still run the conversion script by providing them on the command-line (instead of having to wait for a new version of NeMo).

I tested this PR on a model where the script failed previously with container `nemofw-training:23.11`.

**Collection**: nlp

# Changelog 
- Fix crash when converting to mcore a model using rotary embeddings

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

jenkins

jenkins
Fixing conversion script to work for code llama
State: closed | Pull request: yes

# What does this PR do ?

Add rotary base paramter to conversion script for llama hf to nemo

**Collection**: NLP


```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

jenkins
The process_asr_text_tokenizer file excludes an argument.
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

There is no `spe_byte_fallback` argument in process_asr_text_tokenizer.py

**Steps/Code to reproduce bug**

Just run the process_asr_text_tokenizer with any args, e.g.:
`python ./scripts/tokenizers/process_asr_text_tokenizer.py --manifest=/path/to/manifest --vocab_size=1024 --tokenizer=spe --data_root=/path/to/dataroot --spe_type=unigram


**Expected behavior**

The expected behavior is that this works, but the error thrown is 
`AttributeError: 'Namespace' object has no attribute 'spe_byte_fallback'`.
This makes sense because this commit - https://github.com/NVIDIA/NeMo/commit/1d4d3975e339df819b97d29ceafdd98f37b34906 - added three of the four required args for the argparse, excluding the `spe_byte_fallback` one.

@titu1994 
```[tasklist]
### Tasks
```


It has fixed, please re open if you face additional issues. 
https://github.com/NVIDIA/NeMo/blob/c67243521b048e7ce1446a0d45a0b6e262528e73/scripts/tokenizers/process_asr_text_tokenizer.py#L162
Fixes Notebooks for ASR
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

name: CI

# What does this PR do ?

Notebook fixes 

**Collection**: [ASR]

# Changelog 
- Update Librosa signature to use only kwargs

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Fixes Notebooks for ASR
State: closed | Pull request: yes
Labels: name: ASR

name: CI

# What does this PR do ?

Notebook fixes 

**Collection**: [ASR]

# Changelog 
- Update Librosa signature to use only kwargs

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
fix librosa display issue
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Fixes issue with librosa display issue with 0.10.1 https://github.com/librosa/librosa/issues/1763 

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
update branch name
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Update branch name

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix librosa display issue
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fixes issue with librosa display issue with 0.10.1 https://github.com/librosa/librosa/issues/1763 

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

jenkins
update branch name
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Update branch name

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins
Support for Mistral7b training?
State: closed | Pull request: no
Labels: name: stale

Mistral has been shown to be a competitive open sourced model. Many recently competitive models on the open-llm-leaderboard are derived from mistral. However, I can't seem to find the supported training pipeline of mistral as that for llama.

Is there a plan for incorporating mistral into the training pipeline? Or has it been incorporated and I am just dumb not being able to find it?

Kind thanks



This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.
Add LoRA support to all linear layers
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

Add LoRA support to all linear transformations in Attention and MLP modules in NeMo and MCore GPT models.

**Collection**: [Note which collection this PR will affect]
nlp

# Usage
To add LoRa to specific layer, just add the **target_modules** parameter to lora_tuning. The available configuration are below:

```yaml    
lora_tuning:
      target_modules: ['attention_qkv'] # default, adds LoRA to QKV projection layer
      target_modules: ['attention_qkv','attention_dense'] # adds LoRA to QKV projection and Dense layers attention
      target_modules: ['mlp_fc1','mlp_fc2']# adds LoRA to linear layers in the MLP module
      target_modules: ['attention_qkv','attention_dense','mlp_fc1','mlp_fc2'] # adds LoRA to every linear layer in attention and MLP modules
```
There are also shortcut options defined as below:
```yaml    
lora_tuning:
      target_modules: ['attention'] # same as ['attention_qkv','attention_dense']
      target_modules: ['mlp'] # same as  ['mlp_fc1','mlp_fc2']
      target_modules: ['attention','mlp'] # same as ['attention_qkv','attention_dense','mlp_fc1','mlp_fc2']
      target_modules: ['all'] # same as ['attention_qkv','attention_dense','mlp_fc1','mlp_fc2']
```
If the target_modules parameter is not provided by the user, the default value is  
```yaml 
 target_modules: ['attention_qkv']
```
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?
@arendu

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

jenkins

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.

jenkins


jenkins


jenkins


jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

i think CI issue is blocked by #8342

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins

jenkins
Mengruw/ngram deduplicate
State: closed | Pull request: yes
Labels: name: TTS

name: stale

name: ASR

name: NLP

name: Speaker Tasks

name: CI

name: common

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


Hi, @Davood-M Could you help to review? I wrote code for avoiding repetition during inference.

jenkins

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
[Fix] Fixed name of a test
State: closed | Pull request: yes

# What does this PR do ?

Rename a test that was shadowing another test.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Renamed test

# Usage
n/a

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

jenkins

jenkins
Remove host-to-device data copies in LLAMA SFT
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

Pre-create cumulative sequence argmins and the max sequence length per micro-batch on the CPU tensor. This avoids copying the GPU tensor to calculate this information on the CPU.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Add news section to README
State: closed | Pull request: yes

# What does this PR do ?

Add news section to README.


Edit:
News section will temporarily be at the top of the page. We'll move the news lower in the README as the news ages.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[NLP] Use register_artifact from superclass directly
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

Removes explicit code to call superclass method (cleanup).

**Collection**: [NLP]

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] Improvement

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to https://github.com/NVIDIA/NeMo/pull/7935


I don't know why we need this. It breaks a bunch of things at this point. Is it worth it to change it?

> I don't know why we need this. It breaks a bunch of things at this point. Is it worth it to change it?

That's quite surprising.. Let me retry the Jenkins pipeline.

I'm not insisting on that though, it's a small cleanup which should be harmless.

jenkins

Eric pointed out that default for `verify_src_exists` is different across the two classes, which I missed. Hence pipeline fails.

I'll figure that out as I think it's worth it as a simplification

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
[NLP] Remove replace_sampler_ddp (deprecated in Trainer)
State: closed | Pull request: yes

# What does this PR do ?

Removes deprecated trainer option in MPT conversion script.

**Collection**: [NLP]

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] Bugfix

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to https://github.com/NVIDIA/NeMo/pull/7935
[NLP] ArtifactItem with init=True to make it debuggable
State: closed | Pull request: yes

# What does this PR do ?

When debugging artifact items I struggled with `ArtifactItem` as it is defined with `init=False` which results in issues when printing an instance of this class.

The following minimal example illustrates the problem - this code
```
from dataclasses import dataclass


@dataclass(init=False)
class Point:
    x: int
    y: int

p = Point()
print(p)
```
results in **AttributeError: 'Point' object has no attribute 'x'**. So it doesn't make it easier to check what's going on.

**Collection**: [NLP]

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] Improvement


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to https://github.com/NVIDIA/NeMo/pull/7935


jenkins

> Thanks LGTM

Thanks! I've just rebased it, I'll merge once pipeline completes.
[NLP] Use helpers from utils_funcs.py in Llama conversion
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

An improvement to use `torch_dtype_from_precision` helper to resolve precision.

**Collection**: [NLP]

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] Improvement


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to https://github.com/NVIDIA/NeMo/pull/7935
[NLP] Minor improvements in Llama conversion script
State: closed | Pull request: yes

# What does this PR do ?

Minor extra fixes to Llama conversion script.

**Collection**: [NLP]

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] Documentation
- [x] Improvement

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to https://github.com/NVIDIA/NeMo/pull/7935
[NLP] Improve and unify loading state_dict for community models
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

- Provide a common way for loading state_dict for HF models among different conversion scripts
- Deduplicate tokenizers in the Nemo file produced: the model now comes with a single copy instead of an extra redundant tokenizer file (which is an undesired side effect of [`load_model` function](https://github.com/NVIDIA/NeMo/blob/52d50e9e09a3e636d60535fd9882f3b3f32f92ad/scripts/nlp_language_modeling/convert_hf_llama_to_nemo.py#L58))

 
**Collection**: [NLP]

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] Improvement

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to https://github.com/NVIDIA/NeMo/pull/7935


jenkins

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.
Draft: Cuda graphs greedy rnnt inference
State: closed | Pull request: yes
Labels: name: core

name: stale

name: ASR

# What does this PR do ?

Greedy RNNT inference currently sees the GPU idle 90% of the time because of CPU overheads. This experiments with using cuda graphs to do greedy inference, so that we can decoder at much higher throughputs. Very WIP. I began to work on this because I discovered that using TensorRT was not helpful at all, because it speeds up such a small portion of the code right now.

**Collection**: ASR

# Changelog 
- Will have to do after cleaning up

Not ready for serious review.

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This is now at parity with the original blank_as_pad greedy RNN-T decoding implementation. Currently, I can decode librispeech test clean in 1 min, 39 seconds, while the original implementation runs in 2 min, 21 seconds, at batch size 16. This may seem disappointing, but it is clear that there is still CPU overhead being counted because loading data off disk is not fast enough. In addition, this result is using float32, not float16.

That is to say that the WER output here is 0:

https://github.com/NVIDIA/NeMo/pull/7976/files#diff-e87be798c4c213fbdf438595c4c06898e1dabea0757aaa60334331cbf9f44cb1R83

However, this still needs to be cleaned up. It has a lot of cruft, including an attempt to use TensorRT from before my realization that the problem was CPU overhead, not slowness in the GPU implementation.

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Token count and sequence length logging for MegatronGPTSFTModel
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Adds progress bar logging for sequence length (including padding tokens) and real token count for each train step with MegatronGPTSFTModel to enable accurate calculation of tokens/sec and related metrics.  Depends upon #7945.

**Collection**: nlp/language_modeling

# Changelog 
- Alters GPTSFTDataset and GPTSFTPackedDataset to pass out a list of token counts as part of the batch
- Alters MegatronGPTSFTModel to expect, average, and log that list of token counts at the beginning of the step

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Depends upon #7945.


@vysarge 
Are you going to open this PR?

We need this update, right? 
Let's merge this PR ASAP as the change looks to be straightforward. 

@vysarge 
Please change the merge target to the main branch.
Support O2 training of PEFT and SFT
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Support O2 training (`model.megatron_amp_O2=True`) of PEFT and SFT using the refactored PEFT APIs.
This was listed as a known issue since the PEFT Refactor (#7308). This PR fixes that issue.

Code was tested on GPT and T5, PEFT and SFT. 

**Collection**: NLP

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #7308 


jenkins
Add All Multimodal Source Code Part 2: Text to image, x to nerf
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Add multimodal support, second PR, includes text to image, x to nerf source code

**Collection**: Multimodal

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

jenkins

jenkins

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

jenkins

jenkins

jenkins

jenkins
Add starcoder NeMo -> HF conversion script
State: closed | Pull request: yes
Labels: name: stale

# What does this PR do ?

Add starcoder NeMo -> HF conversion script 

**Collection**: NLP

# Usage
See script header


# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

Waiting for a solution to reconcile architectural differences between HF and Megatron before merging this..

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

closing because of new HF architecture
feat(dependabot): Add dependabot config for dockerfiles and github-actions
State: closed | Pull request: yes
Labels: name: stale

name: CI

# What does this PR do ?

This change enabled dependabot to automatically generate PRs for updated images in Dockerfiles or updated github-action versions.  This will help the maintainers keep dependencies up to date.

**Collection**: N/A

# Changelog 
- feat(dependabot): Add dependabot config for dockerfiles and github-actions

# Usage
* N/A

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x ] New Feature
- [ ] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Virtual pipeline parallel support for MegatronGPTSFTModel
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Enables running MegatronGPTSFTModel with virtual pipeline parallel when peft_scheme=null.

**Collection**: NLP (language_modeling)

# Changelog 
- Alters word embedding initialization and passing of several parameters in MegatronGPTSFTModel to accommodate virtual pipeline parallelism when self.use_peft is false
- Adds a guard in `MegatronGPTSFTModel.__init__` to raise a ValueError if self.use_peft is true
- Corrects the TP group initialization check in MegatronGPTSFTModel to account for both TE and MCore flags

# Usage
Run a GPT model with SFT and set `virtual_pipeline_model_parallel_size`, such as:

```
python examples/nlp/language_modeling/tuning/megatron_gpt_peft_tuning.py \
++model.virtual_pipeline_model_parallel_size=10 \
...
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

jenkins

jenkins
SFT patch: (1) enable sequence parallelism and (2) enable profile
State: closed | Pull request: yes
Labels: name: core

name: NLP

# What does this PR do ?

SFT patch: (1) enable sequence parallelism and (2) enable profile

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

@hsiehjackson 
I don't know the background of why this is set as False for inference.
When we remove this condition, we use sequence parallelism for both training and inference but I didn't see any issue with this.

@erhoo82 for inference do you mean generation mode instead of calculating validation loss? This assert needs config `sequence_parallel == False`. I think if you remove `_reset_sequence_parallelism_args` for  models with sequence parallel, we may get assertion error during generation.

jenkins