renwei2024/nemo-github-issues huggingface dataset:
Add steps for document of getting dataset 'SF Bilingual Speech'
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

When following the [document](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/tts/datasets.html#sfspeech-chinese-english-bilingual-speech) to get dataset `SFSpeech Chinese/English Bilingual Speech`, the first command
```bash
python scripts/dataset_processing/tts/sfbilingual/get_data.py \
    --data-root <your_local_dataset_root> \
    --val-size 0.1 \
    --test-size 0.2 \
    --seed-for-ds-split 100
```
 directly raise error:
```
Traceback (most recent call last):
  File "/home/xxx/NeMo/scripts/dataset_processing/tts/sfbilingual/get_data.py", line 122, in <module>
    main()
  File "/home/xxx/NeMo/scripts/dataset_processing/tts/sfbilingual/get_data.py", line 116, in main
    __process_data(
  File "/home/xxx/NeMo/scripts/dataset_processing/tts/sfbilingual/get_data.py", line 91, in __process_data
    entries = __process_transcript(dataset_path)
  File "/home/xxx/NeMo/scripts/dataset_processing/tts/sfbilingual/get_data.py", line 65, in __process_transcript
    with open(file_path / "text_SF.txt", encoding="utf-8") as fin:
FileNotFoundError: [Errno 2] No such file or directory: 'tryit/text_SF.txt'
```

The reason: `scripts/dataset_processing/tts/sfbilingual/get_data.py` actually doesn't download the dataset. The `SFSpeech Chinese/English Bilingual Speech` could only be downloaded through [Nvidia NGC](https://catalog.ngc.nvidia.com/orgs/nvidia/resources/sf_bilingual_speech_zh_en).

# Changelog
- Add steps in the document of dataset SFSpeech to download by ngc-cli tool at first

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and follow [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?
@blisc  @okuchaiev @titu1994 @XuesongYang
Fix wrong calling of librosa.get_duration() in notebook
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

When running notebook `tutorials/asr/Online_Offline_Speech_Commands_Demo.ipynb` of `main` branch in the Google Colab, it will report the error:
```
TypeError                                 Traceback (most recent call last)
[<ipython-input-18-1dea7463f5dd>](https://localhost:8080/#) in <cell line: 5>()
      3 CHANNELS = 1
      4 audio, sample_rate = librosa.load(wave_file, sr=SAMPLE_RATE)
----> 5 dur = librosa.get_duration(audio)
      6 print(dur)

TypeError: get_duration() takes 0 positional arguments but 1 was given
```

The calling of `librosa.get_duration(audio)` is wrong. This PR is fixing it.

# Changelog 
- Fix the error of calling librosa.get_duration() in notebook.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

@titu1994, @redoctopus, @jbalam-nv

# Additional Information
* Related to #7179
Fix logging to remove 's/it' from progress bar in Megatron models and add train_step_timing
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Fix audio codec type checks
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Most classes in the audio codec code had input_type() and output_type() parameters but were missing the typecheck() annotation on the forward() method to enforce them.

This commit adds the typecheck() to all relevant forward() methods and fixes their corresponding type annotations.

Also has a small change to add the vector quantizer parameters to the optimizer in the model (though the existing RVQ has no trainable parameters).

**Collection**: [TTS]

# Changelog 
- Add typecheck to forward methods
- Fix various typos in input/output types

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
fix for peft tied weights
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

add kwargs and model_parallel_config for ParallelAdaptersWeightTying class.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update to core 23.08 branch ToT
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Update to core 23.08 branch ToT

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Upper bounding ptl
State: closed | Pull request: yes

# What does this PR do ?

2.0.8 introduces an issue with the progress bar when using max_steps

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fixed trainer.strategy=auto from None.
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue): https://github.com/NVIDIA/NeMo/issues/7349
Revert comment
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Revert the commented line.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix pipeline parallel inference
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

intermediate output tensor is [s,b,h] while the final output is [b,s,h]

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
pin numba=0.57.1 to fix reinstall.sh error
State: closed | Pull request: yes

# What does this PR do ?

I observed our Dockerfile had constrained `numba>=0.57.1` (see [here](https://github.com/NVIDIA/NeMo/blob/main/Dockerfile#L86)) while `reinstall.sh` still pins `NUMBA_VERSION=0.55` (see [here](https://github.com/NVIDIA/NeMo/blob/main/reinstall.sh#L23)). When I run `reinstall.sh` from the latest main, it will raise installation error for `llvmlite` (see error below). Once I pinned `NUMBA_VERSION=0.57.1`, and run `reinstall.sh` again, the problem is resolved.

```bash
  Attempting uninstall: llvmlite
    Found existing installation: llvmlite 0.38.1
ERROR: Cannot uninstall 'llvmlite'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
```

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add steps for document of getting dataset 'SF Bilingual Speech'
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

When following the [document](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/tts/datasets.html#sfspeech-chinese-english-bilingual-speech) to get dataset `SFSpeech Chinese/English Bilingual Speech`, the first command
```bash
python scripts/dataset_processing/tts/sfbilingual/get_data.py \
    --data-root <your_local_dataset_root> \
    --val-size 0.1 \
    --test-size 0.2 \
    --seed-for-ds-split 100
```
 directly raise error:
```
Traceback (most recent call last):
  File "/home/xxx/NeMo/scripts/dataset_processing/tts/sfbilingual/get_data.py", line 122, in <module>
    main()
  File "/home/xxx/NeMo/scripts/dataset_processing/tts/sfbilingual/get_data.py", line 116, in main
    __process_data(
  File "/home/xxx/NeMo/scripts/dataset_processing/tts/sfbilingual/get_data.py", line 91, in __process_data
    entries = __process_transcript(dataset_path)
  File "/home/xxx/NeMo/scripts/dataset_processing/tts/sfbilingual/get_data.py", line 65, in __process_transcript
    with open(file_path / "text_SF.txt", encoding="utf-8") as fin:
FileNotFoundError: [Errno 2] No such file or directory: 'tryit/text_SF.txt'
```

The reason: `scripts/dataset_processing/tts/sfbilingual/get_data.py` actually doesn't download the dataset. The `SFSpeech Chinese/English Bilingual Speech` could only be downloaded through [Nvidia NGC](https://catalog.ngc.nvidia.com/orgs/nvidia/resources/sf_bilingual_speech_zh_en).

# Changelog
- Add steps in the document of dataset SFSpeech to download by ngc-cli tool at first

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and follow [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?
@blisc  @okuchaiev
Add steps for document of dataset SFSpeech Bilingual
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

When following the [document](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/tts/datasets.html#sfspeech-chinese-english-bilingual-speech) to get dataset `SFSpeech Chinese/English Bilingual Speech`, the first command
```bash
python scripts/dataset_processing/tts/sfbilingual/get_data.py \
    --data-root <your_local_dataset_root> \
    --val-size 0.1 \
    --test-size 0.2 \
    --seed-for-ds-split 100
```
 directly raise error:
```
Traceback (most recent call last):
  File "/home/xxx/NeMo/scripts/dataset_processing/tts/sfbilingual/get_data.py", line 122, in <module>
    main()
  File "/home/xxx/NeMo/scripts/dataset_processing/tts/sfbilingual/get_data.py", line 116, in main
    __process_data(
  File "/home/xxx/NeMo/scripts/dataset_processing/tts/sfbilingual/get_data.py", line 91, in __process_data
    entries = __process_transcript(dataset_path)
  File "/home/xxx/NeMo/scripts/dataset_processing/tts/sfbilingual/get_data.py", line 65, in __process_transcript
    with open(file_path / "text_SF.txt", encoding="utf-8") as fin:
FileNotFoundError: [Errno 2] No such file or directory: 'tryit/text_SF.txt'
```

The reason: `scripts/dataset_processing/tts/sfbilingual/get_data.py` actually doesn't download the dataset. The `SFSpeech Chinese/English Bilingual Speech` could only be downloaded through [Nvidia NGC](https://catalog.ngc.nvidia.com/orgs/nvidia/resources/sf_bilingual_speech_zh_en).

# Changelog
- Add steps in the document of dataset SFSpeech to download by ngc-cli tool at first

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and follow [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?
@blisc  @okuchaiev
Add migration guide for lightning 2.0 upgrade
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Use distributed optimizer support for multiple dtypes
State: closed | Pull request: yes
Labels: name: feature

name: core

name: NLP

name: CI

# What does this PR do ?

https://github.com/NVIDIA/apex/pull/1721 adds distributed optimizer support for multiple dtypes for the grad reduce-scatters and param all-gathers. This has two main applications in NeMo:

- For GPT, most grads can be reduced in BF16 but embedding grads need to be reduced in FP32 to avoid learning issues.
- For FP8 support, weight matrices can be stored in FP8 while most other parameters (e.g. biases, layernorm params, embeddings) are in BF16. In upcoming work, we would like to handle FP8 and BF16 param all-gathers in the same optimizer.

This approach is much nicer than the old approach for mixed BF16/FP32 grad reductions (https://github.com/NVIDIA/NeMo/pull/5920), and is also compatible with the distributed checkpoint format (https://github.com/NVIDIA/NeMo/pull/7116, https://github.com/NVIDIA/NeMo/pull/7281). I confirm I've been able to save and load models with different parallel configs and grad reduction dtypes.

**Collection**: NLP

# Changelog 
- Remove old distopt infrastructure for mixed BF16/FP32 grad reductions
- Add type hints to distributed optimizer wrapper

# Usage

Set the optimizer to `distributed_fused_adam` in the config file:

https://github.com/NVIDIA/NeMo/blob/f8be40b75ee1f8437b56fcc9602dc2aaddfb0643/examples/nlp/language_modeling/conf/megatron_gpt_config.yaml#L228

Configure the optimizer with `grad_sync_dtype: bf16`.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Depends on https://github.com/NVIDIA/apex/pull/1721
* Will break checkpoints for models that are using mixed BF16/FP32 grad reductions
add O2 option in gpt eval
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add an option to run amp O2 in gpt_eval/inference

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Llama load buffers in checkpoint
State: closed | Pull request: yes

# What does this PR do ?

Optionally, if state_dict contains tensors in the model's buffers, load them as well.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Change audio codec token type to TokenIndex
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Change neural type of `tokens` from `Index` to `TokenIndex`

**Collection**: [TTS]

# Changelog 
- Change token type to TokenIndex
- Cleanup unused import and file.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
enable fp32 optimizer for output_layer in mcore
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Mcore maintains both weights of embedding and output layers. When share_embeddings_and_output_weights=true, weight of output layer is not registered as _with_fp32_optimizer, then the [allreduce](https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/nlp/models/language_modeling/megatron_gpt_model.py#L752) op hangs .

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
State: closed | Pull request: no
Labels: name: bug

When I pull the Nemo image use command "docker pull nvcr.io/nvidia/nemo:23.06",I receive this message：
Error response from daemon: Get https://nvcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

How can i sovle this problem ?
Upgrade pytorch container to 23.08
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

Upgrading to the latest PyTorch container.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fixing indentation bug in indexed_dataset memory deallocation
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

This PR fixes memory deallocation for indexed_dataset when dataset is first created (currently it happens only when loaded).
Thanks to @hillst for finding the bug!

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
N-gram Language model with Hybrid transducer CTC large. 
State: closed | Pull request: no

Hi, I am trying to use N-gram language model with Hybrid Transducer CTC large. but i am getting an error.
I loaded model using the below code.
```import nemo.collections.asr as nemo_asr
asr_model = nemo_asr.models.EncDecHybridRNNTCTCBPEModel.from_pretrained(model_name="stt_en_fastconformer_hybrid_large_pc")
```

I am following this  [https://github.com/NVIDIA/NeMo/blob/acbd88257f20e776c09f5015b8a793e1bcfa584d/tutorials/asr/Offline_ASR.ipynb](tutorial) to add language model .


```lm_path="/content/5gram_correct.arpa"

beam_search_lm = nemo_asr.modules.BeamSearchDecoderWithLM(
    vocab=list(asr_model.decoder.vocabulary),
    beam_width=16,
    alpha=1, beta=1.5,
    lm_path=lm_path,
    num_cpus=max(os.cpu_count(), 1),
    input_tensor=False)
```
but i am getting this error.

`AttributeError: 'RNNTDecoder' object has no attribute 'vocabulary'`
is there any way to use N-gram language model with Hybrid Transducer CTC large?
Voice Conversion Tutorial
State: closed | Pull request: no
Labels: name: stale

I am trying to use NeMo for voice conversion and I am looking for a tutorial notebook that explains how I can do this.

Basically I want to provide 2 wav files as input and have the output be a single wav file that contains the speech from one file with the voice from the other file.

I went through the documentation for examples and tutorials and could not find how I could achieve this.
Replace 'fasttext' with 'fasttext-wheel' in requirements_nlp
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

It is not possible to install the requirements of the NLP component because the fasttext wheel cannot be build any more. However, there is a fork called fasttext-wheel that can be installed with pip. I have tested it extensively and it works with NeMo

**Steps/Code to reproduce bug**

```
pip install -r requirement/requirements_nlp.txt 
```

**Expected behavior**

To be able to install NLP component of the NeMo toolkit without having to hack the requirements_nlp file each time.

**Environment overview (please complete the following information)**

 - Environment location: Bare-metal
 - Method of NeMo install:  pip and source

```
pip install "git+https://github.com/NVIDIA/NeMo.git@main#egg=nemo_toolkit[all]"
```

or  (from source)

```
pip install -e ".[all]"
```

**Environment details**

- OS version : Manjaro Linux (current)
- PyTorch version: Nightly builds with Cuda 12.1 (pip)
- Python version: 3.8.18 (NeMo does not work with more recent Python versions) 

**Additional context**
Update save_to and restore_from for dist checkpointing
State: closed | Pull request: yes
Labels: name: core

name: NLP

# What does this PR do ?

Save_to a .nemo file with distributed checkpoint and restore_from a .nemo file with dist ckpt.

This makes it so that, for example, eval can be done with different model parallel configs on the fly.

**Collection**: NLP
# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Added Exportable bits to T5 + unit test
State: closed | Pull request: yes
Labels: name: core

name: TTS

name: stale

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Maanug/resume checkpoint priority rebased
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Remove apex and trainer dependency with single gpu
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

Remove apex, trainer, torch.distributed dependencies when run inference on single gpu.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
A demo colab:
https://colab.research.google.com/drive/1kMoBxNzTB8W0dmDzzWaMhl3WgYYXfh8N?usp=sharing


# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [X] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Move `resume_from_checkpoint` parameter: trainer -> exp_manager (for PTL 2.0)
State: closed | Pull request: yes
Labels: name: TTS

name: ASR

name: NLP

# What does this PR do ?

Fixes broken configs (PTL 2.0): move parameter `resume_from_checkpoint` to `exp_manager`, as implemented in #7159 

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
add bf16 inference support and fix seq_len stft issue
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

add bf16 inference support and fix seq_len stft issue

**Collection**: ASR

# Changelog 
- torch.floor_divide will make sure to not go into round off issues at higher precisions
- add support for amp_dtype during transcribing

# Usage

```bash
python speech_to_text_eval.py dataset_manifest=<name> amp=True amp_dtype=bfloat16
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [x] Bugfix
- [ ] Documentation
Move model precision copy
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Move model precision copy to MegatronBaseModel to reduce boilerplate.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Remove `cfg.model.precision` = `cfg.trainer.precision` copy from several examples
- Set precision in `MegatronBaseModel` which all Megatron models inherit from
- Fix some mapping of PTL 2.0 precision to torch datatypes that were exposed by above

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
- [X] Refactor

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Resume checkpoint priority
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Fix resume from checkpoint behavior to have lower priority than experiment dir.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- resume from checkpoint behavior
- reorganize `check_resume` for prints
- disable `TRANSFORMERS_OFFLINE` for some tests that were failing

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)