renwei2024/nemo-github-issues huggingface dataset:
[TN/TTS] Add graph to tag IPA words/sentences in square brackets and leave them unchanged
State: closed | Pull request: yes

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?

IPA words/sentences in square brackets won't be normalized

**Collection**: [Note which collection this PR will affect]

# Changelog 
TN: IPA words/sentences in square brackets won't be normalized

# Usage
* You can potentially add a usage example below

```
# untouched
python normalize.py --text="[oʊ ɔɪ p ɹ s ʃ t θ]" # -> "[oʊ ɔɪ p ɹ s ʃ t θ]"

# normalized if no sq brackets
python normalize.py --text="oʊ ɔɪ p ɹ s ʃ t θ" -> "oʊ ɔɪ p ɹ s ʃ t theta"
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
NeMo Model to HF Hub Upload Tutorial
State: closed | Pull request: yes

# What does this PR do ?

Add a tutorial showing how to upload NeMo models to the Hugging Face Hub

**Collection**: [Core, Documentation]

# Changelog 
- Add tutorial notebook that shows how to upload a nemo model to the hub
- Prepare the model card with a template
- Upload results of model evaluations

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.
Add Squeezeformer to Nemo
State: closed | Pull request: no
Labels: name: feature

Squeezeformer looks promising and uses some of the reusable components from conformer backbone. Creating a placeholder for discussion and implementation.

[Official Implementation](https://github.com/kssteven418/Squeezeformer)

[Paper]([arxiv.org/pdf/2206.00888.pdf](https://arxiv.org/pdf/2206.00888.pdf))
[TTS] capture exception of non-supported windows.
State: closed | Pull request: yes

* remove redundant lines;
* declare global variables;
* capture exception of non-supported windows.

Signed-off-by: Xuesong Yang <xuesongyxs@gmail.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```bash
(nemo) xueyang@aiapps-031622:~/workspace/NeMo$ pytest --cpu tests/collections/tts/test_torch_tts.py::TestTTSDataset::test_raise_exception_on_not_supported_window
============================================================== test session starts ==============================================================
platform linux -- Python 3.8.13, pytest-7.1.1, pluggy-1.0.0 -- /home/xueyang/miniconda3/envs/nemo/bin/python
cachedir: .pytest_cache
rootdir: /home/xueyang/workspace/NeMo, configfile: setup.cfg
plugins: hydra-core-1.1.1
collected 1 item

tests/collections/tts/test_torch_tts.py::TestTTSDataset::test_raise_exception_on_not_supported_window PASSED                              [100%]
========================================================= 1 passed, 4 warnings in 3.87s =========================================================
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] add staticmethod decoration for BetaBinomialInterpolator
State: closed | Pull request: yes

Signed-off-by: Xuesong Yang <xuesongyxs@gmail.com>

# What does this PR do ?
`member function is static so it needs a decorator of static method.`

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Megatron perceiver with tensor parallelism only
State: closed | Pull request: yes

# What does this PR do ?

Implements a Megatron-based perceiver encoder with tensor parallel only.

**Collection**: NLP

# Changelog 
- Add `megatron_perceiver_encoders.py` file and add corresponding configuration options in yaml files.

# Usage

Set the following `aayn_base_megatron.yaml` or `megatron_t5_config.yaml` 

`encoder_arch: perceiver`
`hidden_steps: 32`
`num_self_attention_per_cross_attention: 2`

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
How should i use Kaldi MFCCs?
State: closed | Pull request: no
Labels: name: feature

Hi 
I have kaldi feats.ark and text, but i do not have wavs. And i need to train ASR. I guess you had scripts for that before, but you removed them. Do you have any suggestion for me?Or any way to use feats?

Thanks
Best Regards

https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/starthere/best-practices.html?highlight=kaldi#using-kaldi-formatted-data
Is it possible to write custom dataloader and embed it in the architecture of the QuartzNet model for training?
State: closed | Pull request: no
Labels: name: feature

Good evening! I have 2 questions...
1. Is it possible to write your own dataloader and embed it in the architecture of the QuartzNet model for training?
It seems to me that I saw somewhere how the default dataloader is replaced with a custom dataloader. But I can't find an example of how to do it.
2. Is it possible to somehow see which dataloader is used from the loaded model in the code?

Many thanks in advance for any insights!
Removes debug logging statements in Megatron NMT
State: closed | Pull request: yes

# What does this PR do ?

Removes logging statements that were added to debug.

**Collection**: NLP

# Changelog 
- Removes logging statements that were added to debug megatron NMT

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add NeMo Adapters tutorial to Core
State: closed | Pull request: yes

Signed-off-by: smajumdar <smajumdar@nvidia.com>

# What does this PR do ?

Adds NeMo adapters tutorial to Core

**Collection**: [Core, ASR]

# Changelog 
- Adds tutorial that explains how to add adapter support to any NeMo / PyTorch model.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation
IPA support for TTS
State: closed | Pull request: yes

# What does this PR do ?

Add IPA support to TTS collection with new tokenizer, G2P class, and CMUdict with IPA entries.

**Collection**: TTS

# Changelog 
- Adds static tokenizer and G2P that handle IPA
- Adds CMUdict version with IPA entries that can be used with the above classes to train TTS models
- New, completely revamped heteronyms file (old one will be removed in a future PR; it doesn't match up with the existing CMUdict entries we have)
- New example IPA FastPitch config

# Usage
The new IPA FastPitch config can be used practically identically to the previous configs, e.g.:
```bash
python examples/tts/fastpitch.py --config-name=fastpitch_align_ipa.yaml \
train_dataset=/LJSpeech/nvidia_ljspeech_train.json \
validation_datasets=/LJSpeech/nvidia_ljspeech_val.json \
sup_data_path=/LJSpeech/supplementary/
```
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Remove verbose flag from Dali Index Creator
State: closed | Pull request: yes

Signed-off-by: smajumdar <smajumdar@nvidia.com>

# What does this PR do ?

Fix an issue in Dali Index Creator causing a crash.

**Collection**: [ASR]

# Changelog 
- Remove verbose=False arg from DALI Index Creator.

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Adjust lr for AdamW from LAMB default
State: closed | Pull request: yes

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

# What does this PR do ?

Adjusts default learning rate down to 1e-3 due to change in default optimizer to AdamW from LAMB (see PR #4233)

**Collection**: TTS

# Changelog 
- Changes default learning rate from 1e-1 to 1e-3 for FastPitch and Mixer-TTS models
Add library name and version when downloading from the Hugging Face Hub
State: closed | Pull request: yes

# What does this PR do ?

Provides additional information for more granular metrics in the future of what version of the library is being used when downloading files.

**Collection**: [Core]

# Changelog 
- Trivial change with no functional change


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

@titu1994
Adding docs for ASR SSL
State: closed | Pull request: yes

# What does this PR do ?

Adding documents for SSL in ASR 

**Collection**: ASR

# Changelog 
- Added document in `NeMo/docs/source/asr/ssl`


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Added support for subnet export
State: closed | Pull request: yes

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>

# What does this PR do ?

Added support for subnet export: AIA-2515

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Use unique names for temporary directories in punctuation and capitalization tests
State: closed | Pull request: yes
Labels: name: bug


Use `mktemp -d` for creating temporary directories for punctuation and capitalization model tests. Previously race conditions happened if several similar tests were run on same machine.

**Collection**: nlp
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.
[collections] bugfix for capturing NotImplementedError of non-supported sup data types.
State: closed | Pull request: yes

# What does this PR do ?

Fixed the bug that does not correctly capture NotImplementedError of non-supported supplementary data types. Original implementation as shown in https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/tts/torch/data.py#L262-L263 will not raise any exception ever because the `data_type` has already been validated by looking it up in `DATA_STR2DATA_CLASS` in line 257.

**Collection**: [Note which collection this PR will affect]
`nemo/collections/tts/torch/data.py`

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```bash
# added a unit test for the function.
$ pytest --cpu tests/collections/tts/test_torch_tts.py::TestTTSDataset::test_raise_exception_on_not_supported_sup_data_types

========================================= test session starts =========================================
platform linux -- Python 3.8.13, pytest-7.1.1, pluggy-1.0.0 -- /home/xueyang/miniconda3/envs/nemo/bin/python
cachedir: .pytest_cache
rootdir: /home/xueyang/workspace/NeMo, configfile: setup.cfg
plugins: hydra-core-1.1.1
collected 1 item

tests/collections/tts/test_torch_tts.py::TestTTSDataset::test_raise_exception_on_not_supported_sup_data_types PASSED [100%]
==================================== 1 passed, 3 warnings in 2.95s ====================================
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fine-Tuning TokenClassification NER Bert Model
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

A clear and concise description of what the bug is.

**Steps/Code to reproduce bug**

While doing the Fine-Tuning of pretrained ner_en_bert model, Model Config file is not getting update with the new one. It is using the same config file on which model is pretrained on. I can see the same in logs after calling trainer.fit()
I've already tried with below settings:

model.cfg = cfg
model._cfg = cfg
model._hparams_initial = cfg



    model_from_scratch = nemo_nlp.models.TokenClassificationModel.from_pretrained("ner_en_bert")
    cfg = copy.deepcopy(model_from_scratch.cfg)
    cfg.dataset.data_dir = data_dir
    cfg.label_ids = {'O': 0, 'B-place': 1, 'B-emotion': 2, 'B-noun': 3, 'B-name': 4}
    print('BEFORE:',cfg)
    cfg.optim.lr = 0.1
    print('CFG',cfg)
    model_from_scratch.cfg.label_ids = {'O': 0,'B-place':1,'B-emotion':2,'B-noun':3,'B-name':4}
    model_from_scratch.cfg.dataset.data_dir = data_dir #passed using input function parameters
    model_from_scratch.update_data_dir(data_dir)
    model_from_scratch.setup_training_data()
    model_from_scratch.setup_validation_data()
    trainer = pl.Trainer(devices=1, accelerator='cpu',max_epochs=2, log_every_n_steps =1,max_steps = 32)
    model_from_scratch.set_trainer(trainer)
    trainer.fit(model_from_scratch)

**Expected behavior**

Model Should get trained with the new config changes
Changed total virtual prompt tokens
State: closed | Pull request: yes

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Fix for OOM error in 1.9.0 CI test

**Collection**: BigNLP

# Changelog 
- Changed total_virtual_tokens in example prompt learning config

# Usage
n/a

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update frozen model config with amp_o2=False:
State: closed | Pull request: yes

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Amp_O2 for prompt learning is not supported in the upcoming JoC release. So, megatron_amp_o2 needs to be overwritten and set to False in the GPT model configs that were trained with O2 in order for those models to load for prompt learning.
This will cause a time overhead as the GPT .nemo file will need to be unpacked once to get the config and overwrite it, then unpacked again when the model is restored.

**Collection**: BigNLP

# Changelog 
- Updated PromptLearningGPTModel class to over write megatron amp O2 config.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
forcing conv subsampling to 32 bit
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.
Forces 2D conv subsampling to fp32

**Collection**: [Note which collection this PR will affect]
ASR
# Changelog 
- Add specific line by line info of high level changes in this PR.
One liner change in nemo/collections/asr/modules/conformer_encoder.py
# Usage
* You can potentially add a usage example below
No changes
```python
# Add a code snippet demonstrating how to use this 
```
No changes

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
updating PretrainedModelInfo for ASR SSL models
State: closed | Pull request: yes

Signed-off-by: Krishna Puvvada <kpuvvada@nvidia.com>

# What does this PR do ?

Updating PretrainedModelInfo for ASR SSL models

**Collection**: ASR

# Changelog 
- Added conformer large, xlarge info in PretrainedModelInfo



# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Prompt Learning Pipeline Parallel
State: closed | Pull request: yes

# What does this PR do ?

1. Adds pipeline parallel capability  to GPT p-tuning and prompt tuning
2. makes prompt learning methods compataible with GPT models trained with amp O2, 
3. switches to smaller gpt model and datasets for CI test, adding tp and pp tests

**Collection**: BigNLP

# Changelog 
- Updated MegatronGPTPromptLearning class
- Updated prompt learning config
- Moved all reduce gradient method to base megatron model
- Updated prompt learning CI tests

# Usage

```
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
How to initialize the weight of each letter separately for ASR (QuartzNet and CitriNet)?
State: closed | Pull request: no
Labels: name: feature

I want to do Transfer Learning by explicitly initializing the en -> ru alphabet.
```python
dict_vocab= {
     "a": "а",
     "b": "б",
     "c": "с",
     "d": "д",
     "f": "ф",
...
}
```


Do not tell me how this can be done? I didn't find it in the tutorials.
Thanks in advance!
Tacotron2.ipynb error Hydra
State: closed | Pull request: no
Labels: name: bug

Describe the bug

When running the tutorial notebook for Tacotron2 training in Google Colab. After this step;
`!python tacotron2.py \
sample_rate=16000 \
train_dataset=tests/data/asr/an4_train.json \
validation_datasets=tests/data/asr/an4_val.json \
trainer.max_epochs=3 \
trainer.accelerator=null trainer.check_val_every_n_epoch=1 \
trainer.gpus=1`
 Hydra issue is raised:
`Traceback (most recent call last):
  File "tacotron2.py", line 44, in <module>
    main()  # noqa pylint: disable=no-value-for-parameter
  File "/usr/local/lib/python3.7/dist-packages/nemo/core/config/hydra_runner.py", line 108, in wrapper
    config_name='tacotron2.yaml',
TypeError: _run_hydra() missing 1 required positional argument: 'args'`

Maybe you also need to take a look at this warning while we are on the same step
`################################################################################
### WARNING, path does not exist: KALDI_ROOT=/mnt/matylda5/iveselyk/Tools/kaldi-trunk
###          (please add 'export KALDI_ROOT=<your_path>' in your $HOME/.profile)
###          (or run as: KALDI_ROOT=<your_path> python <your_script>.py)
################################################################################`


Steps/Code to reproduce bug

Running each cell of the Tacotron2 tutorial notebook in a new Colab session,restarting, deleting runtime...

Environment location: Collab
Method of NeMo install: As per instructions within notebook. Please specify exact commands you used to install.
My  i just use T5 in dialogue without pretrain a language model 
State: closed | Pull request: no
Labels: name: feature

I want to distill a big model to t5-base model, and the big model use bpe not sentencepiece, So the tokenizer should be load the bpe codes and the vocabs is differenct from the origin t5-base model. Therefore i am not sure about two things.
1, whether the t5 can be train in dialogue without pretrain which treat the t5 like transformer, and  i find this config (https://github.com/NVIDIA/NeMo/blob/main/examples/nlp/dialogue/conf/dialogue_config.yaml) , is it meet my demand?
2, how to set the tokenizer to use bpe codes?
Docs CSS: Update h4 tag style for the right side bar
State: closed | Pull request: yes

# What does this PR do ?

Adds style overwrite for the h4 tag located on the right side bar. Currently, the font size is too small

**Collection**: Docs

# Changelog 
- Adds style overwrite for h4 tag on the right side bar. Currently the font is too small
- Example of the issue: https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/asr/asr_language_modeling.html#hyperparameter-grid-search

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
External Vad manifest
State: closed | Pull request: no
Labels: name: feature

**Is your feature request related to a problem? Please describe.**

I have been using diarization.py example using the provided config file. The marble net vad acts really poor detecting speech. So I decided to use an External vad *( e.g pyannote vad ) to provide to the model. I could not find how to convert an external vad outputs to the manifest file required for the model

**Describe the solution you'd like**

A brief documentation on how to build the manifest file and a method to do it if possible.
[TTS] IPA CMU dict and updated heteronyms list
State: closed | Pull request: yes

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
remove redundant condition for fastpitch.
State: closed | Pull request: yes


# What does this PR do ?

remove redundant condition testing for fastpitch. `shuffle_should_be` is already known as `False` in this case, so no need to test again `not shuffle_should_be`.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Updated config to fix CI test OOM error
State: closed | Pull request: yes

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Fixes prompt learning config bug in CI test

**Collection**: BigNLP

# Changelog 
- Updated prompt learning config file

# Usage
N/A


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add Prompt Learning amp_O2 support
State: closed | Pull request: yes

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Adds amp o2 support for prompt learning for JoC release. Amp O2 support code taken from @MaximumEntropy's NMT PR. 

**Collection**: BigNLP

# Changelog 
- Updated NLP Overrides with amp o2 state dict modification 
- Updated PromptLearningGPTModel 

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[Add] automaticly resolving relative audio path
State: closed | Pull request: yes

# Add Support for Automatically Resolving Relative Audio Path

This simple modification allows using relative audio paths in the manifest, so that there is no need to change the manifest after moving the dataset to different locations. The updated `iterm_iter` function will first check if the `audio_file` exists, and skip further processing if it does exist. Then the function checks whether `audio_file` is a relative path. If it is, the function will add the parent folder of manifest to the audio file path. For tarred datasets, since the `audio_file` in manifest is not a real path, we will not perform any operation.

This feature is based on the assumption that the manifest is under the same directory as the audio directory, and the relative audio paths always begins with an audio directory (e.g., `wavs/`).

Consider the following dataset folder:
```
/home/steve/libri_speech/
----------------------------->: train_manifest.json
----------------------------->: wavs/
------------------------------------->: 11111.wav
------------------------------------->: 22222.wav
```
and the content of ` train_manifest.json` is:
```
{"audio_filepath": "wavs/11111.wav", duration: "2.3",  "text": "one one one one one"}
```

The updated function will first try to read the file `wavs/11111.wav`, but since it's a relative path it won't find it. Then the function will add the parent directory of the manifest to the audio path, which becomes `/home/steve/libri_speech/wavs/11111.wav`.
Set Save on train end to false
State: closed | Pull request: yes

# What does this PR do ?

Add documentation on setting save_on_train_end to False

**Collection**: BigNLP

# Changelog 
- Updated prompt learning docs
- Updated prompt learning config

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
pt container 1.10->1.11.0
State: closed | Pull request: yes

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
transcribe_audio
State: closed | Pull request: no
Labels: name: feature

```python
def transcribe(self, paths2audio_files: List[str], batch_size: int = 4) -> List[str]:
        """
        Takes paths to audio files and returns text transcription
        Args:
            paths2audio_files: paths to audio fragment to be transcribed
        Returns:
            transcription texts
        """
        pass
```
is there a replacement of `transcribe` that takes in the already loaded audio in numpy/torch array instead of the path ?
Fix missing validation dataset, whitelist certain keywords for datasets
State: closed | Pull request: yes

Signed-off-by: smajumdar <smajumdar@nvidia.com>

# What does this PR do ?

Fix adapters training script when there are no validation configs.

**Collection**: [ASR]

# Changelog 
- Fix validation config bug 
- Improve the handling of dataloader speeds in ASR

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
ssl fixes
State: closed | Pull request: yes

Signed-off-by: sam1373 <samuelkriman@gmail.com>

# What does this PR do ?

- fixes masked patch augmentation to round up to guarantee specified fraction of masked steps
- updates to citrinet and contextnet ssl configs and relevant decoder
- fixes tutorial to work with updated configs + minor changes
- revert rnnt decoding change since it's not needed at the moment

**Collection**: ASR

**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation
BigNLP perf regression fix
State: closed | Pull request: yes

# What does this PR do ?

Reverts a change to ModelPT's `cfg` attr.

**Collection**: All

# Changelog 
- Reverts a change to ModelPT's `cfg` attr.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Training Fastpitch on German text and phonemes and finetuning HiFi-GAN on predicted mels
State: closed | Pull request: yes

Signed-off-by: Akshit Arora <akshit.arora@colorado.edu>

# What does this PR do ?

Adds the code and tutorial to train Fastpitch model from scratch on German neutral dataset ([openslr/95](https://www.openslr.org/95)) text and phonemes, and Finetune HiFi-GAN model on predicted mels from Fastpitch. 

**Collection**: TTS

# Changelog 
- Added a tutorial to `NeMo/tutorials/tts/Fastpitch_Training_GermanTTS.ipynb`
- Added Fastpitch config for German `NeMo/examples/tts/conf/de/fastpitch_align_de.yaml`
- Added IPA symbols to GermanCharsTokenizer in `NeMo/nemo/collections/tts/torch/tts_tokenizers.py`
- Modified data.py to process `phonemes` flag and generate unique file names when the flag is set to 1 `NeMo/nemo/collections/tts/torch/data.py`

# Usage
* `Fastpitch_Training_GermanTTS.ipynb` walks through the usage.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests? 
- [ ] Did you add or update any necessary documentation? 
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* N/A
Fix epoch end for NeMo NMT
State: closed | Pull request: yes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Fixes NMT epoch end

**Collection**: NLP

# Changelog 
- NLP eval epoch end fix due to PTL bug. 

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)