renwei2024/nemo-github-issues huggingface dataset:
FSDP support of SALM and BESTOW
State: closed | Pull request: yes
Labels: name: Multi Modal

# What does this PR do ?

Support FSDP in SALM and BESTOW. Thank @pyf98 for early trying

# Usage

Elaborated and tested here https://docs.google.com/document/d/1fr2UlqThlzqvqL_lpc_HIyrh0yCzL27A-QX1eCI_ojg/edit#heading=h.9o9g0iu9eu69


A few notes:
- You might need to adapt this class to support additional modules: https://github.com/zhehuaichen/NeMo/blob/speechllm-develop/nemo/collections/nlp/parts/nlp_overrides.py#L626
  - E.g., adding new modules here: https://github.com/zhehuaichen/NeMo/blob/speechllm-develop/nemo/collections/nlp/parts/nlp_overrides.py#L668
- The above class has several methods related to "state dict". Not sure if those are correct for speech LM.
- I excluded the embedding layer of LLM to avoid errors when using it elsewhere. We'd better fix it since the embedding layer can be very large. https://github.com/zhehuaichen/NeMo/commit/44056f2d1f69261c47829181e7d8e4c0c1c077de#diff-e3fcc68c97812462e588465942b917f381808fc65501b32a431b8b311fab6fd3R1691

> A few notes:
> 
> * You might need to adapt this class to support additional modules: https://github.com/zhehuaichen/NeMo/blob/speechllm-develop/nemo/collections/nlp/parts/nlp_overrides.py#L626
>   
>   * E.g., adding new modules here: https://github.com/zhehuaichen/NeMo/blob/speechllm-develop/nemo/collections/nlp/parts/nlp_overrides.py#L668
> * The above class has several methods related to "state dict". Not sure if those are correct for speech LM.
> * I excluded the embedding layer of LLM to avoid errors when using it elsewhere. We'd better fix it since the embedding layer can be very large. [zhehuaichen@44056f2#diff-e3fcc68c97812462e588465942b917f381808fc65501b32a431b8b311fab6fd3R1691](https://github.com/zhehuaichen/NeMo/commit/44056f2d1f69261c47829181e7d8e4c0c1c077de#diff-e3fcc68c97812462e588465942b917f381808fc65501b32a431b8b311fab6fd3R1691)

Thanks for the note.
SpeechLLM does not add extra nlp modules but only reusing them if that is what you mean. Locally I found the ckpt restore seems working. Will confirm with a cluster run
Copy data from GPU to CPU only once in batched_hyps_to_hypotheses for each tensor of interest.
State: closed | Pull request: yes
Labels: name: stale

name: ASR

name: Run CICD

Previously, several small GPU->CPU copies were used, which caused excess latency linearly proportional to the batch size. For small copies, it is much more efficient to do a single memory copy.

There is still a remaining issue. pack_hypotheses() appears to be copying data from GPU to CPU for the dec_state tensor, one at a time. I would like to remove that, but that is an intrusive change that affects interfaces, so I will not do it for now.

RTFx before this change on librispeech test other: 1766.645297158031
RTFx after  this change on librispeech test other: 1795.3508087085368

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Simplified key-names mapping from NeMo to TRT LLM
State: closed | Pull request: yes

# What does this PR do ?

Simplified key-names mapping in NeMo to TRT LLM export. 

**Collection**: nlp

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
rebase main
State: closed | Pull request: yes
Labels: name: stale

name: NLP

name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Alit/roberta convert
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fixed loading packed distributed nemo checkpoints
State: closed | Pull request: yes

# What does this PR do ?

Fix imports and loading of the packed, distributed NeMo checkpoints during conversion to TRT LLM.

**Collection**: nlp

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


This should be covered by https://github.com/NVIDIA/NeMo/pull/9826, so please include it for your purposes
AttributeError: 'dict' object has no attribute '_get_node_flag'
State: closed | Pull request: no
Labels: name: stale

  File "/share//code//NeMo/examples/nlp/language_modeling/megatron_ckpt_to_nemo.py", line 245, in <module>
    convert(local_rank, rank, world_size, args)
  File "/share//code//NeMo/examples/nlp/language_modeling/megatron_ckpt_to_nemo.py", line 198, in convert
    model = MegatronGPTModel.load_from_checkpoint(checkpoint_path, hparams_file=args.hparams_file, trainer=trainer)
  File "/usr/local/lib/python3.10/dist-packages/nemo/collections/nlp/models/nlp_model.py", line 380, in load_from_checkpoint
    model = ptl_load_state(cls, checkpoint, strict=strict, cfg=cfg, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/core/saving.py", line 144, in _load_state
    obj = cls(**_cls_kwargs)
  File "/usr/local/lib/python3.10/dist-packages/nemo/collections/nlp/models/language_modeling/megatron_gpt_model.py", line 243, in __init__
    super().__init__(cfg, trainer=trainer, no_lm_init=True)
  File "/usr/local/lib/python3.10/dist-packages/nemo/collections/nlp/models/language_modeling/megatron_base_model.py", line 118, in __init__
    with open_dict(cfg):
  File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.10/dist-packages/omegaconf/omegaconf.py", line 983, in open_dict
    prev_state = config._get_node_flag("struct")
AttributeError: 'dict' object has no attribute '_get_node_flag'


Thanks.

Could you add more details on how to reproduce this? We have a bug template which can be followed when creating the issue.

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.
Fix Canary's transcribe(): move to device and prompt feeding
State: closed | Pull request: yes
Labels: name: ASR

name: Run CICD

# What does this PR do ?

* Fixes an issue after introducing dataclass as a mini-batch type.
* Fixes Canary's transcribe() which for certain cases incorrectly used the contents of the "text" field in addition to the prompt (and if it was not available, it still used a default text value).

**Collection**: ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
bugfix for packed distributed checkpoints
State: closed | Pull request: yes

# What does this PR do ?

Fix imports and loading of the packed distributed checkpoints.

**Collection**: nlp

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add moe_router_pre_softmax=True to checkpoint converters & nemo-ux
State: closed | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update dev doc for features
State: closed | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?

Update dev doc for features section

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Support LLAMA 3.1 in NeMo
State: closed | Pull request: no

**Is your feature request related to a problem? Please describe.**

LLAMA3.1 introduces a new ROPE implementation, while NeMo hasn't supported it.
https://github.com/huggingface/transformers/blob/51ab25e2932da15511ced35bcbdfa92d25c4794c/src/transformers/modeling_rope_utils.py#L298

**Describe the solution you'd like**

Support the new ROPE implementation and LLAMA 3.1.

This has been added here: https://github.com/NVIDIA/NeMo/pull/9815

Please re-open or create a new issue if needed.
fix FLOPsMeasurementCallback on_train_end when experiment is None
State: closed | Pull request: yes
Labels: name: common

name: Run CICD

# What does this PR do ?

Fixes basic error in recently introduced #9932 FLOPsMeasurementCallback on_train_end method when `pl_module.logger.experiment` is None.

**Collection**: common

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation


@ar-nowaczynski  Can you check if https://github.com/NVIDIA/NeMo/pull/10125 fixes this issue.
Enable specifying kwargs to DDPStrategy via config for most models
State: closed | Pull request: yes
Labels: name: ASR

name: Run CICD

# What does this PR do ?

This change allows, amongst other things, to specify options for DDP such as `gradient_as_bucket_view=True` which saves a considerable amount of memory for 1B+ models.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


> LGTM, thanks! I like this approach and the possibility of using an explicitly instantiated strategy. Also, learned about `Mapping`, thanks :)

Thanks! I added a test and one fix, should be OK to merge now.
[Nemo CICD] Update to PT24.05
State: closed | Pull request: yes
Labels: name: stale

name: Run CICD

[Nemo CICD] Update to PT24.05

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
[NeMo-UX] Fix a serialization bug affecting fsdp
State: closed | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?

This PR recursively finds the leaves of the trainer config and calls `track_io` on the leaves. This handles the case of adding io functionality to plugins which are dictionaries. 

**Collection**: llm

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
remove assertation for models with unknown chat template
State: closed | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?

Use hard-coded chat-template for known models only, need a better strategy.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add support for overlapped gradient and parameter synchronization for GPT SFT model
State: closed | Pull request: yes
Labels: name: NLP

name: Run CICD

Add support for overlapped gradient and parameter synchronization for GPT SFT model

# What does this PR do ?

Add support for overlapped gradient and parameter synchronization

**Collection**: nlp

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Tips for training Ecapa-TDNN speaker recognition model as fast as possible?
State: closed | Pull request: no
Labels: name: stale

Hello everybody,

I would like to train an Ecapa-TDNN model on Voxceleb2 and Voxceleb2+Cnceleb2 for speaker verification/identification. I don't want it to be SOA (ex: complicated pretraining and then large margin finetuning), but I want it to be moderately performant as fast as possible, as the model performance is not the primary focus.

I can train on an A100 gpu. I was thinking of randomly cropping audios in train-set to 3s and then applying RIR+musan+(0.95-1.05) speed perturbation+ spec. augment on the fly (with 0.3 probability for each one). I would also use the AAM Softmax with s=30, m=0.2 loss. How many epochs would I need to train this model and what is the expected time for training such a model? (on Voxceleb2 for example) Is it possible to obtain good performances with such a simple configuration? (under 1% EER on VoxCeleb1-O for example)

I saw that some people apply offline augmentation to create more speakers, or do 2-stage training with pretraining and then fine-tuning, etc. But I would like to train both models in just a couple of days (as an A100 is a pretty big GPU), and I don't want to achieve peak performances, such as the more complicated pipelines, but decent performances.

Any other tips would be greatly appreciated! I would mostly like to know if I could achieve these performances in a reasonable time and any other tips for fast convergence in as few epochs as possible. :D

Thank you in advance!

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.
[FEA] Improve score_document (NumberOfLinesOfCodeFilter) and _line_statistics (PerExtensionFilter)
State: closed | Pull request: no

Hi!

I have noticed a couple of small changes in the code that can improve its performance:

By refactoring `score_document` as suggested, it will run between 2x and 3x faster, and will reduce by half the memory footprint.

By refactoring '_line_statistics' as suggested, it can run about 21% faster.

I will submit a PR in a few minutes with the changes.

Regards,
Miguel

Update:

Apologies. I opened it in the wrong repository. My bad.
Disable gradients in `swap_tensors()` within `ema.py`
State: closed | Pull request: yes
Labels: name: stale

name: common

# What does this PR do ?

Disables gradients in `swap_tensors()` within `ema.py` to prevent unintended leaf variable gradient modifications (should they arise). This was found to be necessary in certain projects that use PyTorch Lightning 2+ such as [AlphaFold3-Lightning-Hydra](https://github.com/amorehead/alphafold3-pytorch-lightning-hydra).

**Collection**: [Note which collection this PR will affect]

N/A.

# Changelog 
- Disables gradients in `swap_tensors()` within `ema.py` to prevent unintended leaf variable gradient modifications (should they arise).

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.


This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.
Feature Extraction with NeMo ASR Pipeline without Conformer Training
State: closed | Pull request: no
Labels: name: bug

name: stale

**Issue:**

I want to run the NeMo ASR pipeline to analyze the features using different parameters, datasets, and strategies. However, I do not want to run the conformer training. Instead, I only want to extract the features from the input JSON files and store them.

**Current Understanding:**

From my understanding, the current pipeline flow is as follows:

```
For each batch:
    extract features -> apply augmentation (if required) -> train -> validation
```

**Requirement:**

I need to comment out or disable the training & further parts so that the pipeline loops back to the next batch to continue feature extraction. Despite reviewing various parts of the internal code pipeline, I am unable to pinpoint where and how to make these changes.

**Version:**

NeMo version: 1.12.0rc0 (I know it's pretty old, but I have already performed a few experiments using it, so I don't want to change anything.)
Pytorch Lightning: 1.7.4

**Request for Help:**

Could you please guide me on how to disable the training & further portion while ensuring that the feature extraction continues to function as required?

Thank you.

Hi, since you want to run only inference, you could modify the prediction_step in [model code](https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/asr/models/ctc_models.py#L572) and run .predict on any model (that uses the feature extractor you are looking for) 
Reference: https://lightning.ai/docs/pytorch/stable/deploy/production_basic.html 

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.
[lhotse] Support for NeMo tarred manifests with offset field
State: closed | Pull request: yes
Labels: name: common

name: Run CICD

# What does this PR do ?

Support for NeMo tarred manifests with offset field in lhotse dataloading.

**Collection**: ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


@pzelasko I checked, caching `TarInfo`s with a single linear scan lets then extract in O(1):
```python
def taropen_build(tar_path):
    import tarfile
    tar = tarfile.open(tar_path)
    tarinfo_index = {tarinfo.name : tarinfo for tarinfo in tar.getmembers()}
    return lambda member_path, mode: tar.extractfile(tarinfo_index[member_path]) if mode == 'rb' else None

    # instead for uncompressed tars (which makes sense for audio files) can do without caching the TarInfo structure and just cache two ints per string file name, this makes it more amenable to sharing between DataLoader threads and easier to sync in tensors
    #import io
    #tarinfo_index = {tarinfo.name : (tarinfo.offset_data, tarinfo.size) for tarinfo in tar.getmembers()}
    #return lambda member_path, mode: [io.BytesIO(f.seek(e[0]) and f.read(e[1])) for f in [open(tar_path, mode)] for e in [tarinfo_index[member_path]]][0] if mode == 'rb' else None

if __name__ == '__main__':
    taropen = taropen_build('myfile.tar')
    print(taropen('mymember.txt', 'rb').read())
```

Supporting `.tar`s with wav/opus files inside would be very nice, as these small audio files can be in 10s or 100s of thousands which is always a pressure on the file system and adds time to unpacking

We already support tars with wav/opus. Linear scan is not feasible for URL data sources (cloud storage, AIStore).
AttributeError: 'MegatronGPTModel' object has no attribute 'decoder'
State: closed | Pull request: no
Labels: name: stale

#### Description

I am retraining a LLaMA3 model. Due to the limited size of my dataset, I attempted to use `freeze_updates` as referenced in the [NVIDIA NeMo documentation](https://docs.nvidia.com/nemo-framework/user-guide/latest/nemotoolkit/core/neural_modules.html#dynamic-layer-freezing). My configuration is as follows:

```yaml
freeze_updates:
  enabled: true  # set to false if you want to disable freezing
  modules:   # list all of the modules you want to have freezing logic for
    decoder: 100
```
However, I encountered the following error:
```
AttributeError: 'MegatronGPTModel' object has no attribute 'decoder'
```

I also tried changing decoder to `encoder` or `joint`, but I still faced errors. I would like to ask how to properly configure this setting?

Additionally, within the NeMo framework, is it possible to freeze specific layers, such as only the attention layer? If so, how can I achieve this? Thanks!

I'm curious if by retraining you might mean continued training? https://docs.nvidia.com/nemo-framework/user-guide/latest/llms/allmodels/continuetraining.html?highlight=continued%2520training#configure-continual-learning

Regarding your error, the decoder module will be `MegatronGPTModel.model.decoder` I believe. But you can check by inspecting the `MegatronGPTModel` 

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.
[NeMo-UX] Change default interval to step and add lr in prog_bar in nemo.lightning
State: closed | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?
Default interval for lr_schedulers was `epoch` which means the lr was never getting updated for nemo.collections.llm examples.

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix metric logging
State: closed | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Remove `rank_zero_only=True` flag, unless used as [documented](https://lightning.ai/docs/pytorch/stable/visualize/logging_advanced.html#rank-zero-only)
- Refactor train and val loss logging

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


Please update base branch to `main`.

stopping this until NeMo CI is out of maintenance mode
Nemotron Conversion script
State: closed | Pull request: yes
Labels: name: Run CICD

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
remove prepare_youmakeup.py
State: closed | Pull request: yes
Labels: name: Multi Modal


# What does this PR do ?

Hi, sorry to bother.
There might be some dataset license issues. So need to remove the dataset downloading and preprocessing script.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


The `Isort and Black Formatting / reformat_with_isort_and_black` test is not passed.
Not sure about the reason.
remove prepare_youmakeup.py
State: closed | Pull request: yes
Labels: name: Multi Modal

# What does this PR do ?

Due to license issue, we need to remove the data downloading and preprocessing script.

**Collection**: [Note which collection this PR will affect]
None of the collection will be affected; 

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)