renwei2024/nemo-github-issues huggingface dataset:
Fix activation checkpoint
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Fix issue related to this PR https://github.com/NVIDIA/NeMo/pull/7200/

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Token_Classification-BioMegatron.ipynb Does not work on Google Colab (Model initialization failed)
State: closed | Pull request: no
Labels: name: bug

name: stale

I tried to run the colab notebook from the stable realse(`BRANCH='r1.20.0'`):https://colab.research.google.com/github/NVIDIA/NeMo/blob/stable/tutorials/nlp/Token_Classification_Named_Entity_Recognition.ipynb
On the step for initializing the model:
```
model_ner = nemo_nlp.models.TokenClassificationModel(cfg=config.model, trainer=trainer)
```
I got this error:
```
[NeMo E 2023-08-28 16:39:54 common:506] Model instantiation failed!
    Target class:	nemo.collections.nlp.models.language_modeling.megatron_bert_model.MegatronBertModel
    Error(s):	name 'set_logging_level' is not defined
    Traceback (most recent call last):
      File "/usr/local/lib/python3.10/dist-packages/nemo/core/classes/common.py", line 485, in from_config_dict
        instance = imported_cls(cfg=config, trainer=trainer)
      File "/usr/local/lib/python3.10/dist-packages/nemo/collections/nlp/models/language_modeling/megatron_bert_model.py", line 89, in __init__
        super().__init__(cfg, trainer=trainer, no_lm_init=False)
      File "/usr/local/lib/python3.10/dist-packages/nemo/collections/nlp/models/language_modeling/megatron_base_model.py", line 123, in __init__
        initialize_model_parallel_for_nemo(
      File "/usr/local/lib/python3.10/dist-packages/nemo/collections/nlp/modules/common/megatron/megatron_init.py", line 141, in initialize_model_parallel_for_nemo
        set_logging_level(apex_transformer_log_level)
    NameError: name 'set_logging_level' is not defined
    
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
[<ipython-input-25-4645443f78bc>](https://localhost:8080/#) in <cell line: 1>()
----> 1 model_ner = nemo_nlp.models.TokenClassificationModel(cfg=config.model, trainer=trainer)
````

BTW, I always got dependencies error for installing NeMo on Colab. Currently, I just ignore it.
```
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
ipython 7.34.0 requires jedi>=0.16, which is not installed.
cvxpy 1.3.2 requires setuptools>65.5.1, but you have setuptools 65.5.1 which is incompatible.
```
adding bias_dropout_add_fusion option for BERT
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Adding an option to disable `bias_dropout_add_fusion`, same as for GPT model [here](https://github.com/NVIDIA/NeMo/blob/455d2dfb3479cd30a7857db3c2e2864a921488b8/nemo/collections/nlp/models/language_modeling/megatron_gpt_model.py#L362)

**Collection**: Language Modeling -> BERT

# Changelog 
- `bert_model.py:188` - adding the flag to the `bert_model` constructor
- `bert_model.py:236` - passing the flag to the `language_model` creation
- `megatron_bert_model.py:184` - passing the flag to the `bert_model` based on config

# Usage
* `bias_dropout_add_fusion: <BOOLEAN>` can now be set in BERT model config

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
ImportError: scikit_learn.libs/libgomp-d22c30c5.so.1.0.0: cannot allocate memory in static TLS block
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

tried to run: NeMo/tutorials/00_NeMo_Primer.ipynb

got error message after this command:
import nemo.collections.asr as nemo_asr


**Environment details**
environment: jetson agx orin, jetpack 5.1.2, in a docker: [dustynv/nemo:r35.3.1](https://hub.docker.com/r/dustynv/nemo/tags)

[error_message.txt](https://github.com/NVIDIA/NeMo/files/12450309/error_message.txt)
Automatic Lip Reading Recognition (ALR) - ASR/CV (Visual ASR)
State: closed | Pull request: yes
Labels: name: core

name: ASR

name: common

# What does this PR do ?

Add a new "cv" branch for lip reading task and everything related to computer vision

**Collection**: cv

# Changelog 
- new cv branch
- 6 new cv models in cv/models: VisualEncDecCTCModelBPE, VisualEncDecCTCModel, VisualEncDecHybridRNNTCTCBPEModel, VisualEncDecHybridRNNTCTCModel, VisualEncDecRNNTBPEModel, VisualEncDecRNNTModel
- cv datasets in cv/data: VideoToBPEDataset, VideoToCharDataset, TarredVideoToBPEDataset
- cv modules in cv/modules: ResNetVideoFrontEnd, LinearProjectionVideoFrontEnd, VideoAugmentation, VideoPreprocessor
- video featurizer in cv/parts/preprocessing

# Usage
* You can potentially add a usage example below

```python
import nemo.collections.cv as nemo_cv
from omegaconf import OmegaConf

config_path = "model_config.yaml"
params = OmegaConf.load(config_path)

# hybrid CTC/RNN-T BPE lip reading model
model = nemo_cv.models.VisualEncDecHybridRNNTCTCBPEModel(cfg=params.model)

```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
High GPU usage in Google Colab
State: closed | Pull request: no
Labels: name: bug

name: stale

**Description**

While using NeMo in google golab in CPU mode the GPU spike to 100% never comes down. the same issue is in the TPU mode Also the RAM spiked to 100%

**Steps to reproduce bug**
Try to run this collab file, the same issue will be seen
````
https://colab.research.google.com/drive/1uO2cgu5wjZqEUY1wnlHCe0npe3XRDTka?usp=sharing
````

**Expected behavior**

A clear and concise description of what you expected to happen.

 - Environment Location:  Collab
 - Method of NeMo install: While running this code 

````
parsed = spec_generator.parse(input_text)
spectrogram = spec_generator.generate_spectrogram(tokens=parsed)
audio = model.convert_spectrogram_to_audio(spec=spectrogram)
````
**Environment details**

Otherwise, please provide:
- OS version
- PyTorch Version
````
pytorch-lightning==1.9.4
torch @ https://download.pytorch.org/whl/cu118/torch-2.0.1%2Bcu118-cp310-cp310-linux_x86_64.whl#sha256=a7a49d459bf4862f64f7bc1a68beccf8881c2fa9f3e0569608e16ba6f85ebf7b
````
- Python 3.10.12
move precision test
State: closed | Pull request: yes
Labels: name: stale

name: NLP

name: CI

# What does this PR do ?

Run tests after what is failing on main

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
resume checkpoint prio test
State: closed | Pull request: yes
Labels: name: stale

name: CI

# What does this PR do ?

check other tests after what is currently failing on main

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
enable selective unfreeze
State: closed | Pull request: yes
Labels: name: core

name: NLP

name: common

# What does this PR do ?

allow for selective unfreeze of PEFT params. Useful for continued training of PEFT.

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
lora merge fix for O2 names
State: closed | Pull request: yes

# What does this PR do ?

O2=True changes the names of the expected keys after merging.

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Speechlm delayparallel
State: closed | Pull request: yes
Labels: name: TTS

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
resume from checkpoint if exp dir is empty
State: closed | Pull request: yes
Labels: name: stale

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Add additional config to preprocess_text and compute_feature_stats
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Add some configuration to TTS preprocessing to make them more usable.

Most important, adding `batch_size` to text normalization multi-processing which I have found to make it about 10x faster than using the default 'auto' batch size in joblib. I do not fully understand why, but the documentation indicates that large batch sizes work for small, fast tasks:

> batch_size: int or 'auto', default: 'auto'
> The number of atomic tasks to dispatch at once to each
> worker. When individual evaluations are very fast, dispatching
> calls to workers can be slower than sequential computation because
> of the overhead. Batching fast computations together can mitigate
> this.

**Collection**: [TTS]

# Changelog 
- Add `batch_size` parameter to `preprocess_text.py`
- Add input and output field parameters to `preprocess_text.py`
- Fix boolean flag parsing for `lower_case` for `preprocess_text.py`
- Modify `compute_feature_stats.py` so that it can take a list of manifests, making it easy to compute pitch/energy stats across multiple datasets.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
ModuleNotFoundError: No module named 'ctc_decoders'
State: closed | Pull request: no
Labels: name: bug

 File "/home/mid01/anaconda3/envs/py3.8/lib/python3.8/site-packages/nemo/collections/asr/modules/beam_search_decoder.py", line 64, in __init__
    from ctc_decoders import Scorer, ctc_beam_search_decoder_batch
ModuleNotFoundError: No module named 'ctc_decoders'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "decoder_testing.py", line 38, in <module>
    beam_search = nemo_asr.modules.BeamSearchDecoderWithLM(
  File "/home/mid01/anaconda3/envs/py3.8/lib/python3.8/site-packages/nemo/collections/asr/modules/beam_search_decoder.py", line 66, in __init__
    raise ModuleNotFoundError(
ModuleNotFoundError: BeamSearchDecoderWithLM requires the installation of ctc_decoders from scripts/asr_language_modeling/ngram_lm/install_beamsearch_decoders.sh


My python version is 3.8 as https://github.com/NVIDIA/NeMo/issues/6690 said.

And I also tried 

conda create --name nemo python==3.8.10
conda activate nemo

```
# install NeMo
sudo apt-get update && apt-get install -y libsndfile1 ffmpeg
pip install Cython
git clone https://github.com/NVIDIA/NeMo
cd NeMo
./reinstall.sh

# install ctc-decoders
./scripts/asr_language_modeling/ngram_lm/install_beamsearch_decoders.sh
```

but 
./scripts/asr_language_modeling/ngram_lm/install_beamsearch_decoders.sh: line 26: cd: /workspace/nemo: No such file or directory
error occured then I get GPG error, No module named 'swig_decoders' .. 

still doesn't work.

I just want to get BeamSearchDecoder's result.
Please help me.
Replace prefetch with val iterator check in megatron models
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

1) Catch the end of `dataloader_iter` to exit gracefully from `validation_step` without having to prefetch  all `microbatches` in a step which was being done previously. This is a temporary piece of code that is needed until we have lightning's fix that takes care of catching the end of `dataloader_iter`
2) Set `limit_val_batches` and `num_sanity_val_steps` for `pretraining` to be a multiple of num of `microbatches`. `limit_val_batches` is reconfigured so that we run as many global batches or validation_steps as the user entered `limit_val_batches`. 

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[ASR] Multichannel mask estimator with flex number of channels
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

This PR adds a multichannel mask estimator which can process flexible number of channels.
It also fixes some bugs related to PTL 2.0 and librosa version.

**Collection**: ASR

# Changelog 

- added example config `beamforming_flex_channels.yaml`
- add `max_utts` `process_audio.py`
- updated `librosa.get_duration` to use `path`, since `filename` is deprecated
- bumped `librosa` requirement to `0.10.0` in `requirements_asr.py`
- added `MaskEstimatorFlexChannels` in `audio_modules.py`
- added various multichannel modules in `multichannel_modules.py`, including `ParametricMultichannelWienerFiler` and `ReferenceChannelEstimatorSNR`
- added unit tests for multichannel modules

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
migrated class CSVFieldsMemmapDataset from BioNeMo
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Adding class `CSVFieldsMemmapDataset `

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ x] Did you write any new necessary tests?
- [ x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
TTS Model and Inference Tutorial not working
State: closed | Pull request: no
Labels: name: bug

name: TTS

name: stale

**Describe the bug**

The tutorial links in https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/starthere/tutorials.html are not working for the TTS stuff like Fastpitch. There are also bugs in the code of some of the jupyternotebooks. The code in the ngc models for the tutorials also does not work if you take it from the overview tab in https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/tts_de_fastpitch_multispeaker_5.
Add seq_len_interpolation_factor support for long-context llama ckpts
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Update AudioCodec API
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

This PR extends the API for `AudioCodec`, adds comments and small fixes.

**Collection**: TTS

# Changelog 
- Added `encode` to convert a time-domain signal into a discrete representation
- Added `decode` to convert a discrete (quantized) representation to audio
- Use `quantize` to convert a continuous encoded representation into a quantized discrete representation
- Use `dequantize` to convert a discrete representation to a continuous encoded representation
- Use batch-first format for quantized discrete representation
- Updated input/output type definitions
- Added docstrings in audio_codec modules
- Added logging in audio_codec modules and callbacks

# Usage
Example of usage:

```python
# instantiate a model
model = AudioCodecModel(…)

# convert `audio_in` to disrete representation
# `tokenized` has shape (batch_size, num_codebooks, num_frames)
tokenized, tokenized_len = model.encode(audio=audio_in, audio_len=audio_in_len)

# convert a discrete representation to time-domain audio
audio_out, audio_out_len = model.decode(tokens=tokenized, tokens_len=tokenized_len)
```


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
"get_commonvoice_data.py" is still work?
State: closed | Pull request: no
Labels: name: bug

Hi,
When I follow the tutorial "ASR_CTC_Language-Finetuning. ipynb" on Colab(https://colab.research.google.com/github/NVIDIA/NeMo/blob/stable/tutorials/asr/ASR_CTC_Language_Finetuning.ipynb),
I found this tutorial a bit outdated, so I tried to modify this line of code

```
VERSION="cv corpus 6.1-2020-12-11"
LANGUAGE="ja"
```

by
```
VERSION="cv corpus 14.0-2023-06-28"
LANGUAGE="ja"
```
Discovered that it didn't work anymore.
Here is the error message:
```
Resolving voice-prod-bundler-ee1969a6ce8178826482b88e843c335139bd3fb4.s3.amazonaws.com (voice-prod-bundler-ee1969a6ce8178826482b88e843c335139bd3fb4.s3.amazonaws.com)... 52.92.161.65, 52.218.224.66, 52.92.195.193, ...
Connecting to voice-prod-bundler-ee1969a6ce8178826482b88e843c335139bd3fb4.s3.amazonaws.com (voice-prod-bundler-ee1969a6ce8178826482b88e843c335139bd3fb4.s3.amazonaws.com)|52.92.161.65|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2023-08-24 02:24:22 ERROR 403: Forbidden.

Traceback (most recent call last):
  File "/content/scripts/get_commonvoice_data.py", line 214, in <module>
    main()
  File "/content/scripts/get_commonvoice_data.py", line 181, in main
    tar = tarfile.open(target_file)
  File "/usr/lib/python3.10/tarfile.py", line 1804, in open
    raise ReadError(f"file could not be opened successfully:\n{error_msgs_summary}")
tarfile.ReadError: file could not be opened successfully:
- method gz: ReadError('empty file')
- method bz2: ReadError('not a bzip2 file')
- method xz: ReadError('not an lzma file')
- method tar: ReadError('empty file')
```

So I opened the xx file for review and found the code inside:
```
Parser.add_ Argument（
'-- version',
Default='cv corpus 5.1-2020-06-22 ',
Type=str,
Help='Version of the dataset (observable via https://commonvoice.mozilla.org/en/datasets ',
）
Parser.add_ Argument（
'-- language',
Default='en ',
Type=str,
Help='Which language to download.' (default English, '
Check https://commonvoice.mozilla.org/en/datasets For more language codes',
）
```
I don't know what to fill in for VERSION and LANGUAGE? I am on this website（ https://commonvoice.mozilla.org/en/datasets ）Didn't I find the answer I wanted in?

In addition, I would like to know if there is a relatively new tutorial that can help me train a new model. For example, if I want to train a Cantonese model, what should I do? 
I downloaded the dataset from CommonVoice and it no longer has the **dev/train/test.tsv** file. Instead, it has **other/validated/validated/reported.tsv,** times.txt, and a clip folder with audio. Does this mean I need to categorize the files in the clip into three parts of dev/train/test myself?

I hope to receive your reply as soon as possible!!!
PEFT GPT & T5 Refactor
State: closed | Pull request: yes
Labels: name: core

name: NLP

name: CI

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix precision for PTL 2.0 in ModelParallelConfig
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
specify explicitly to set pretrained model paths
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Added arpa/ipa flag. Choo
**Collection**: tts

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [x] Documentation
Adithyare/peft metric calculation
State: closed | Pull request: yes

# What does this PR do ?

replaces `squad_metric_calc.py` with `peft_metric_calc.py` which computes f1 and accuracy and rouge at once. 

**Collection**: []

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Hiddens modules documentation
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

1. Changed hiddens transformations module from `transformations` to `hiddens`.
2. Added documentations for hiddens module.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
我在测试tutorials/tts/FastPitch_ChineseTTS_Training.ipynb中，发现对于AI两个字母不起效果
State: closed | Pull request: no
Labels: name: bug

name: TTS

**Describe the bug**

A clear and concise description of what the bug is.

**Steps/Code to reproduce bug**

Please list *minimal* steps or code snippet for us to be able to reproduce the bug.

A  helpful guide on on how to craft a minimal bug report  http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports. 


**Expected behavior**

A clear and concise description of what you expected to happen.

**Environment overview (please complete the following information)**

 - Environment location: [Bare-metal, Docker, Cloud(specify cloud provider - AWS, Azure, GCP, Collab)]
 - Method of NeMo install: [pip install or from source]. Please specify exact commands you used to install.
 - If method of install is [Docker], provide `docker pull` 

https://github.com/NVIDIA/NeMo/assets/107624041/ef36be57-6419-4574-b618-d0b4c65d7e3d

& `docker run` commands used

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version
- PyTorch version
- Python version

**Additional context**

Add any other context about the problem here.
Example: GPU model
NeMo MCore llama2 support + MCore PEFT adapters
State: closed | Pull request: yes
Labels: name: core

name: NLP

name: CI

# What does this PR do ?

Support Llama2 models in mcore.
Support PEFT adapters in mcore.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* `mcore_gpt` needs to be enabled for LLAMA2.

```python
model.mcore_gpt = True
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Sync llama branch
State: closed | Pull request: yes
Labels: name: core

name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Merge UL2 bugfixes into UL2 GPT branch
State: closed | Pull request: yes
Labels: name: NLP

name: common

# What does this PR do ?


**Collection**: [NLP]

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
remove old prompt table for storing cached ptunig representations
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

caches ptuning representations in a simple embedding layer.

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Conv1d subsampling
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Add conv1d subsampling for ASR

**Collection**: asr

# Changelog 
- Add 'striding_conv1d' and 'dw_striding_conv1d' subsampling options. Subsampling is performed using strided conv1d convolutions with kernel size 5 instead of doing reshape + frequency-temporal conv2d + reshape + linear projection.

# Usage
* Simply use 'subsampling: striding_conv1d in conformer yaml config for example.

```
  encoder:
      _target_: nemo.collections.asr.modules.ConformerEncoder
      feat_in: ${model.video_front_end.dim_output}
      feat_out: -1 # you may set it if you need different output size other than the default d_model
      n_layers: 18
      d_model: 512
  
      # Sub-sampling params
      subsampling: striding_conv1d # vggnet, striding, stacking or stacking_norm, dw_striding, striding_conv1d_k5, dw_striding_conv1d
      subsampling_factor: 8 # must be power of 2 for striding and vggne
...
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
remove additional line in peft state dict
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)