renwei2024/nemo-github-issues huggingface dataset:
Mixture of Expert with Expert Parallel -- initialization in nemo doesn't have expert_model_parallel flag
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

The model parallel initialisation in NeMo doesn't account for `expert_model_parallel_size` in `megatron_base_model.py` which is currently supported in the latest version of Megatron-LM.

```
        initialize_model_parallel_for_nemo(
            world_size=init_world_size,
            global_rank=init_global_rank,
            local_rank=init_local_rank,
            tensor_model_parallel_size=cfg.get('tensor_model_parallel_size', 1),
            pipeline_model_parallel_size=cfg.get('pipeline_model_parallel_size', 1),
            virtual_pipeline_model_parallel_size=vp_size,
            pipeline_model_parallel_split_rank=cfg.get('pipeline_model_parallel_split_rank', 0),
            micro_batch_size=cfg.get('micro_batch_size'),
            global_batch_size=cfg.get('global_batch_size'),
            rampup_batch_size=cfg.get('rampup_batch_size', None),
            use_fp8=cfg.get('fp8', False),
            init_mpi_proc_group=cfg.get('ub_tp_comm_overlap', False),
            seed=self.cfg.get('seed', 1234),
            apex_transformer_log_level=self.cfg.get('apex_transformer_log_level', 30),
        )
```

This should have the option to set expert_parallel_size here but it's currently not supported. Any ETA when changes to NeMo will add this?



Training will fail with the following error: 

[1,1] File "~/Megatron-LM/megatron/core/parallel_state.py", line 550, in get_tensor_and_expert_parallel_group
[1,1]<stdout>:    assert (
[1,1]<stdout>:AssertionError: tensor and expert parallel group is not initialized

which makes sense since the option for expert parallel is not supported in the initialize_model_parallel_for_nemo() function.

Setup for testing uses latest updates in both Megatron-LM and NeMo-Megatron
Update Apex install command in Dockerfile
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Updates Dockerfile to install Apex correctly.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
(1) Add SHARP interface to M-CORE, (2) use send/recv to send train loss to the first rank instead of b-cast
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

(1) Add SHARP interface to M-CORE in the communicator initialization.
(2) use send/recv to send train loss to the first rank instead of b-cast. This mitigates the communication overhead.


# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add option to tune distopt bucket sizes
State: closed | Pull request: yes
Labels: name: core

name: stale

name: NLP

# What does this PR do ?

The distributed Adam optimizer's performance and memory usage is sensitive to the bucket size:

1. Larger bucket sizes improve performance (less kernel overhead and NCCL latency overhead)
2. There is wasted memory and communication if the bucket sizes don't match the param sizes
3. The bucket size should be smaller than existing buffers in the caching memory allocator, e.g. from temporary activation tensors. Otherwise the allocator will need to allocate new buffers for temporary distopt buffers (e.g. for all-gathers).

This PR attempts to mitigate (2) by adding an option that looks for the bucket size in `[bucket_cap_mb/2, bucket_cap_mp]` that minimizes waste. Note that this `bucket_cap_mb` should still be chosen based on (3) and that the optimal choice will depend on the model.

Based on https://github.com/NVIDIA/NeMo/pull/7399. Pinging @shjwudp.

**Collection**: NLP

# Changelog 
- Add option to tune distopt bucket sizes
 
# Usage

Configure the model with `model.optim.name=distributed_fused_adam`, `model.optim.tune_bucket_cap_mb=True`, `model.optim.bucket_cap_mb=<max buffer size in allocator>`:

https://github.com/NVIDIA/NeMo/blob/fb6fb2fc494afc6cbb13ab77e3829b69b6db6bb2/examples/nlp/language_modeling/conf/megatron_gpt_config.yaml#L228

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Closes https://github.com/NVIDIA/NeMo/pull/7399
Add All Multimodal Source Code
State: closed | Pull request: yes
Labels: name: NLP

This reverts commit d6900f9bc1922d086e2e388dcec6e3bd2b0f59dc.

# What does this PR do ?

Add All Multimodal Source Code

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add some docs and update scripts for ASR
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Adds docs for subsampling args in ASR Fast Conformer and updates tokenization script

**Collection**: [ASR]

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [x] Documentation
WER of Citrinet 256 is not improving.
State: closed | Pull request: no
Labels: name: bug

name: stale

Hi. I have 34 hours of ASR dataset (split in an 80-20 ratio for training and validation) in the Armenian language and want to train the ASR model using Citrinet 256 (transfer-learning from stt_en_citrinet_256). After some epochs, WER is going up. How to improve this or must I use Citrinet 512? Below are hyperparameters of training:
`tokenizer type is unigram,`
`vocab size is 1024,`
`freeze_encoder: False,`
`batch_size: 4,`
`use_start_end_token: True,`
`trim_silence: False,`
`learning rate: 0.001,`
`betas: [0.95, 0.5],`
`weight_decay: 0.001,`
`warmup_steps: None,`
`warmup_ratio: 0.1,`
`minimum learning rate: 1e-5,`
`spec_augment.freq_masks: 2,`
`spec_augment.freq_width: 27,`
`spec_augment.time_masks: 10,`
`spec_augment.time_width: 0.05,`
`epochs: 200,`
Add selection criteria for reference audios in the `GlobalStyleToken` submodule
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

- Allow specifying the criterion with which the _reference audio_ is selected for a given training/validation example

**Collection**: TTS

# Changelog 
- The current NeMo implementation selects the reference audio for a given training example as a random example from the same speaker. As discussed in #7777, this allows the GST module to extract speaker information, and _not_ speaking style information, from the reference spectrogram, thereby making it easier to fine-tune on new speakers with very little data
- On the other hand, the [GSTs paper](https://arxiv.org/abs/1803.09017) states that:
    > During training, the reference signal is ground-truth audio.
- As discussed in #7777, it is desirable to support both use cases as opposed to just one of the two

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation


## Who can review?
@subhankar-ghosh, @hsiehjackson


# Additional Information
* Related to #7777
Update `ema.py` for full Lightning 2.0 and 2.1 support
State: closed | Pull request: yes
Labels: name: stale

name: common

# What does this PR do ?

Updates `ema.py` for full Lightning 2.0 and 2.1 support, and also adds full docstrings for the EMA `Callback` and its components.

# Changelog 
- Update imports for the `EMA` `Callback`
- Add docstrings to the `EMA` `Callback`

# Usage
* Being able to use the `EMA` `Callback` in Lightning 2.0 as well as Lightning 2.1 models.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
ConditionalInput: cat along the feature dim, not the batch dim
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Fix bug in `nemo.collections.tts.modules.submodules.ConditionalInput.forward()`: concatenate `inputs` and `conditioning` along the _feature_ dimension, not the _batch_ dimension.

**Collection**: TTS

# Changelog 
- When using `ConditionalInput` in `"concat"` mode, concatenate `inputs` and `conditioning` along the _feature_ dimension, not the _batch_ dimension as before
  - Despite the misleading docstring in `ConditionalInput.forward()` (corrected in this PR), `inputs` and `conditioning` have shapes `B x T x H` and `B x T x C`, respectively (to be even more precise, `conditioning` has shape `B x 1 x C` _before_ the call to `conditioning.repeat()` and `B x T x C` _after_). Consequently, concatenating the two along the batch dimension has 2 problems:
    - If `H != C`, the concatenation will fail
    - If `H == C`, the concatenation results in a `2B x T x C` tensor
       - This causes an error downstream when calling `self.concat_proj()`, which is defined as:
`self.concat_proj = torch.nn.Linear(hidden_dim + condition_dim, hidden_dim)`
       - `self.concat_proj` is meant to map `B x T x (H+C)` tensors to `B x T x H` tensors, so passing a `2B x T x (H=C)` tensor to it results in the following error:
`RuntimeError: mat1 and mat2 shapes cannot be multiplied ((2B*T)x(H=C) and (H+C)xH)` (the actual error message contains constant values, but here I have extracted variables to make my point clearer)
  - **Solution**: concatenate along the feature dimension to obtain a `B x T x (H+C)` tensor

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation


## Who can review?
@subhankar-ghosh, @hsiehjackson

# Additional Information
* Related to #7777
set context for text memmap to fork
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

the context for memmap dataset is very slow with "spawn" so we set the context to "fork" just for memmap dataset loading.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Configure MCore logger
State: closed | Pull request: yes

# What does this PR do ?

Configure MCore logger to print messages from MCore.

Note that currently only megatron.core.dist_checkpointing uses any logging.
In case of unidentified errors with dist checkpointing, might be helpful to run with NEMO_TESTING=1.

**Collection**: NLP
# Changelog 
- Configure MCore logger to print messages from MCore.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
-  [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Attention Prior, Tarred MLS Dataset, Top K Sampling, Multitask audio synthesis dataset
State: closed | Pull request: yes
Labels: name: ASR

name: NLP

name: common

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
vera
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Create PrecisionPlugin for megatron_ckpt_to_nemo.py trainer
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
NeMo doc update: NeMo basics + improve ASR docs
State: closed | Pull request: yes
Labels: name: stale

name: ASR

# What does this PR do ?

Add some foundational NeMo docs + improve ASR docs

**Collection**: docs

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Nemo to HF converter for LLaMA model
State: closed | Pull request: yes

# What does this PR do ?

Nemo to HF converter for LLaMA model

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
python3 convert_nemo_llama_to_hf.py
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add reconfigure microbatch calculator before inference and update GBS, MBS for inference
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Create PrecisionPlugin for megatron_ckpt_to_nemo.py trainer
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Llama 2 70b convert & finetuning
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

I am following the tutorial: https://docs.nvidia.com/nemo-framework/user-guide/latest/playbooks/llama2peft.html
to finetuning llama 2 on custom dataset.

when convert the llama 2 70b-hf to .nemo with the script
```
python3 ./scripts/nlp_language_modeling/convert_hf_llama_to_nemo.py --in-file=$1 --out-file=$2
```

the output of conversion is 
![image](https://github.com/NVIDIA/NeMo/assets/22444830/f22fc475-36c4-400a-b372-c06f0da408e5)

but there exist no file named "llama2-70b.nemo" , only outputs a ``model_weights'':
![image](https://github.com/NVIDIA/NeMo/assets/22444830/ccf96378-900a-4065-9dab-dccf705e192b)

the  ``model_weights'' cannot feed to the training script:
![image](https://github.com/NVIDIA/NeMo/assets/22444830/93923895-9b1b-4fcf-87e5-ca00dfc99043)

 
How can we successfully run the tutorial ?

 
**Environment details**

- OS version: Ubuntu 18.04
- PyTorch version: '2.2.0.dev20231013+cu118'
- Python version: 3.8
- transformers: 4.33.0
- transformer_engine:  stable branch
- CUDA : 11.8
- CUDNN: 8.6

**Additional context**
GPU:  8xA100
Update transformers requirement to the latest stable version 
State: closed | Pull request: no
Labels: name: stale

Hi there!

The third-party dependency for the transformers package is outdated when looking at the requirements folder [./requirements/requirements_lightning.txt](https://github.com/NVIDIA/NeMo/blob/main/requirements/requirements_lightning.txt#L5)

Is it possible to update this dependency to the latest stable version transformers 4.34.1?
Fix bug wrt change decoding strategy for bpe models
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Fixes an issue that forgets to update cur_decoder of rnnt bpe model

**Collection**: [ASR]

# Changelog 
- Fix change decoding strategy bug
- Add test to check for repeated change to strategy

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Add reconfigure microbatch calculator before inference and update GBS, MBS for inference
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix bug wrt change decoding strategy for bpe models
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fixes an issue that forgets to update cur_decoder of rnnt bpe model

**Collection**: [ASR]

# Changelog 
- Fix change decoding strategy bug
- Add test to check for repeated change to strategy

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Change accelerator to 'auto' in nlp_checkpoint_port.py
State: closed | Pull request: yes
Labels: name: core

name: cherry-pick

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Updated installation of ctc-decoders
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

Fixed issue with installation of ctc-decoders and Offline-ASR notebook

**Collection**: `tutorials/asr`

# Changelog 
- Fixed issue with installation of ctc-decoders and Offline-ASR notebook

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [X] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Pin mcore to 0.3
State: closed | Pull request: yes
Labels: name: cherry-pick

name: CI

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add activations_checkpoint related args for model cfg in lora.ipynb
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Pin mcore to 0.3
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add Adapter and IA3 support for MCore models
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add Adapter and IA3 for MCore PEFT tuning. 
Previously, only LoRA and PTuning was supported for MCore models.
Verified training curve and eval results on Squad are identical as NeMo GPT model.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage

```bash
model.peft.peft_scheme=adapter
# OR
model.peft.peft_scheme=ia3
# are now supported for mcore models (model.mcore_gpt=True)
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add dist ckpt support for regular optimizers
State: closed | Pull request: yes
Labels: name: core

name: NLP

# What does this PR do ?

Add dist checkpoint support for regular (non O2 and non-distributed) optimizers.
This change is required to support dist ckpt for FSDP training

**Collection**: NLP

# Changelog 
- Add dist checkpoint support for regular (non O2 and non-distributed) optimizers.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Pin tensorstore version
State: closed | Pull request: yes

# What does this PR do ?
Tensorstore version 0.1.46 (published 10/18/23) breaks BF16 support for Zarr

**Collection**: NLP

# Changelog 
- pin tensorstore version to 0.1.45

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Change accelerator to 'auto' in nlp_checkpoint_port.py
State: closed | Pull request: yes
Labels: name: core

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Updated installation of ctc-decoders
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Fixed issue with installation of ctc-decoders and Offline-ASR notebook

**Collection**: `tutorials/asr`

# Changelog 
- Fixed issue with installation of ctc-decoders and Offline-ASR notebook

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [X] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
tune specific params in the base model
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

tune only a selected subset of the params of the base model.

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)