renwei2024/nemo-github-issues huggingface dataset:
Add support for Apex distributed Adam optimizer with GPT-3
State: closed | Pull request: yes

# What does this PR do ?

Adds support for training GPT-3 with the [Apex implementation of the ZeRO optimizer](https://github.com/NVIDIA/apex/blob/master/apex/contrib/optimizers/distributed_fused_adam.py).

**Collection**: NLP

# Changelog 
- Add option for `distributed_fused_adam` optimizer and integrate with GPT-3 training

# Usage

Set optimizer to `distributed_fused_adam` in config file:

https://github.com/NVIDIA/NeMo/blob/23f6a95fb1479f5d2ff7111adbc5720ed3c58e48/examples/nlp/language_modeling/conf/megatron_gpt_config.yaml#L122

~Note that this optimizer is incompatible with the `megatron_amp_O2` option. It internally applies similar optimizations (bf16 params, wgrad kernel fusion), so refactoring may be warranted in the future.~ This optimizer requires the `megatron_amp_O2` option.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [x] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Depends on https://github.com/NVIDIA/apex/pull/1414.
* Depends on https://github.com/NVIDIA/apex/pull/1432
* This is mostly orthogonal to https://github.com/NVIDIA/NeMo/pull/4380. It just requires changing [this line](https://github.com/NVIDIA/NeMo/blob/5bb47b7638645b6b177638fff4e3b3a82b3fd76c/nemo/collections/nlp/models/language_modeling/megatron_gpt_model.py#L335) to:
```if self.megatron_amp_o2 or self.with_distributed_adam:```
Merge r1.10.0 main
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix for TP=2,PP=2 decoding with megatron encoder-decoder models
State: closed | Pull request: yes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Fixes decoding for TP>1, PP>1 models

**Collection**: NLP

# Changelog 
- Gather from pipeline model parallel ranks only instead of all model parallel ranks.

# Usage
* N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Problems with phone type audios 
State: closed | Pull request: no

Hi, I hv trained a new conformer model. 

For telephone audios this model performs worse than the pretrained model used for fine-tuning it. 

For Other audios its ok 

Error seen are quite odd instead of insertion or substitution all the errors deletion. It just doesn't transcribe parts of the audio that pretrain model does. 


1. Changes made for training my model 
     - have changed tokens from 128 to 512 added. Puncts. (Same tok type used unigram). 
     
     - Trained on different dataset of size ~10k  hrs. (Amond them fisher was the only telephone speech dataset)

     
     
I am unable to debug where/why this is happening  

Possible ans 
1. Turning data hd problems. 
2. Not enough telephone type training data used. Eg ( asr is not able to transcribe becz the audio is noisy )

Can anyone please pont me in the write direction.
Option to disregard document boundaries for t5, bart, ul2
State: closed | Pull request: yes

# What does this PR do ?

Adds the option to disregard document boundaries while constructing a training example for t5, bart, ul2.

This reduces the number of padding tokens in a batch for T5 from ~35% to ~4%.

**Collection**: NLP

# Changelog 
- Changes `dataset_utils.py` for T5, BART, UL2 to use `_build_samples_mapping()` from `gpt_dataset.py`
- Switches between T5 style samples mapping construction and GPT samples mapping construction based on the respect_document_boundaries flag

# Usage
* set `model.data.respect_document_boundaries=True/False` in T5, BART, UL2

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add support for ASR Adapter Auxiliary Losses
State: closed | Pull request: yes

# What does this PR do ?

Add support for auxiliary regularization losses to ASR adapters.

**Collection**: [ASR]

# Changelog 
- Add support for access mixin to all adapter modules
- Add utility methods for gathering the loss from auxiliary models
- Add tests for adapter modules

# Usage

```python

# Currently, setting the config value of `l2_lambda` for the Adapter strategy adds auxiliary losses.
python train_adapters.py \
  ... \
  model.adapter.adapter_strategy.l2_lambda=0.001
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Add kwargs to exact string match
State: closed | Pull request: yes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Exact string match fix for finetuning.

**Collection**: NLP

# Changelog 
- Exact string match fix for fine-tuning by adding kwargs.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS][DE] created the finetuning Hifigan 44100Hz recipe on HUI-Audio-Corpus-German
State: closed | Pull request: yes


    * implemented a script of generating mel-spectrograms for finetuning Hifigan using multiprocessing.
    * fixed some typos.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>


**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix exact string match metric instantiation
State: closed | Pull request: yes

# What does this PR do ?

Fixes an issue when using exact string match with general T5 finetuning

**Collection**: NLP

# Changelog 
- Fixes exact string match metric instantiation by adding **kwargs to exact string match.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Has Biomegatron been removed from huggingface.co/models? 
State: closed | Pull request: no
Labels: name: bug

name: stale

I can't seem to get nemo to recognize biomegatron as a pretrained model in the config file. For example, if I put: 
`config.model.language_model.pretrained_model_name = 'biomegatron345m_biovocab_30k_cased'` in my config file, as per (https://github.com/NVIDIA/NeMo/blob/main/tutorials/nlp/Token_Classification-BioMegatron.ipynb). I get this error: 
```
OSError: biomegatron345m_biovocab_30k_cased is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
If this is a private repository, make sure to pass a token having permission to this repo with `use_auth_token` or log in with `huggingface-cli login` and pass `use_auth_token=True`.

```

And upon scrolling through the models on huggingface.co/models, I do not see a biomegatron model listed.
Weighted bucketing
State: closed | Pull request: yes

# What does this PR do ?
- Adds to ASR bucketing config to add weights to target buckets. To be used for upsampling specific datasets.

**Collection**: [Note which collection this PR will affect]
- ASR

# Changelog 
- `collections/asr/data/audio_to_text_dataset.py` `get_tarred_dataset` function now checks if dataset config feature `bucketing_weights` is passed as a list in config. (Can be for either dataset but assumed for training in practice.)
- Added type checking to `get_tarred_dataset` to check that passed values are all ints. (Returns `ValueError` if not.)
- using `bucketing_weights`, the dataloader will append a bucket manifest x number of times, with x being the provided weight value.

# Usage
Given a set of bucketed datasets, add to a hydra config file the `bucket-weights` attribute (e.g.):
```
train_ds:
  manifest_filepath: [[bucket1manifest],[bucket1manifest],[bucket3manifest],[bucket4manifest]]
  bucketing_weights: [weight1, weight2, weight3, weight4]
```
Where `bucketing_weights` is a list of `int` of same length as number of buckets. For each weight `i`, the corresponding manifest will be appended to the dataset `i` number of times. For instance, given `bucketing_weights: [3,2,1,2]`, the resulting training set will contain 3 instance of the first bucket, 2 of the second, 1 of the third, and 2 of the final.

# Before your PR is "Ready for review"
**Pre checks**:
- [Y] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [N] Did you write any new necessary tests?
- [N] Did you add or update any necessary documentation?
- [N] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [Y] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Allow Noam lr scheduler to run for more than max_steps
State: closed | Pull request: yes

Signed-off-by: Adrian Lancucki <alancucki@users.noreply.github.com>

# What does this PR do ?

Allows to train for more than `max_steps` without lowering lr to `min_lr`.

Noam lr scheduler should decay lr indefinitely. The current implementation puts a hard limit at `max_steps` - the number of intended training epochs - after which it returns `min_lr`. This is contrary to the definition in the paper. Upon resuming from a ckpt `max_steps` are loaded from the ckpt, and if the user wishes to train for more epochs, they will unexpectedly get `min_lr`.

**Collection**: [Note which collection this PR will affect]

N/A

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Fix dataset parameter typo on tacotron2 example yaml
State: closed | Pull request: yes

# What does this PR do ?

This PR fixes dataset parameter typo on tacotron2 example yaml

**Collection**: [TTS]

# Changelog 
- Change "train_dataset" to "validation_datasets" on tacotron2.yaml conf example.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests? -> I checked that locally
- [ ] Did you add or update any necessary documentation? -> No documentation needed
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc) -> It does not
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] Bugfix

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[Fix] Relative Audio Paths
State: closed | Pull request: yes

Signed-off-by: stevehuang52 <heh@nvidia.com>

This PR fixes the second problem in this [issue 4455](https://github.com/NVIDIA/NeMo/issues/4455).

Previously, the code will fail when there are "/" in the filepaths of tarred datasets. Now we only add the manifest directory to the audio filepath if the resulting file exists, thus fixing the problem.
t5-rpe-fix targeting r1.10.0; raise exception for PP>2.
State: closed | Pull request: yes

# What does this PR do ?

Fix the RPE implementation converging slower with loss consistently higher than NeMo APE and Megatron-LM RPE implementation.

NEW: exception is raised when configured with pipelie-parallel-size > 2 and RPE.

**Collection**: NLP

# Changelog 
- Changed how RPE parameters are initialized and passed on.

# Usage
* Train with `model.position_embedding_type='relative'` config.

```python
python /home/workspace/NeMo/examples/nlp/language_modeling/megatron_t5_pretraining.py model.position_embedding_type='relative'
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] [bugfix] update indentation
State: closed | Pull request: yes

# What does this PR do ?

updated indentation

**Collection**: TTS

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Wav2vec fix
State: closed | Pull request: yes

# What does this PR do ?
- Quick fix to Jenkins CI config for Wav2Vec.
Add a one line overview of what this PR aims to accomplish.
- Halved number of conformer blocks and feedforward parameters to ease up memory requirements on Jenkins
**Collection**: [Note which collection this PR will affect]
- ASR
# Changelog 
- Halved transformer layers in wav2vec_ci (12 -> 6)
- Halved inner dimension of transformer encoder. (3072 -> 1536)
- Reduced batch size for wav2vec_ci to 8

# Before your PR is "Ready for review"
**Pre checks**:
- [Y ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ N] Did you write any new necessary tests?
- [ N] Did you add or update any necessary documentation?
- [ N] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [Y ] Bugfix
- [ ] Documentation


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Unable to train Entity Linking
State: closed | Pull request: no
Labels: name: bug

I am trying to train Entity Linking in NLP. 

It runs fine. However, while saving results after Epoch 0 it gives this error 
In automatic_optimization, when `training_step` returns a dict, the 'loss' key needs to be present

I checked and training step method does have a dictionary. What could be the reason for this?
Im running training on Google Colab (if that helps )
Add Bucketing support to TarredAudioToClassificationLabelDataset
State: closed | Pull request: yes

# What does this PR do ?

This PR adds support for bucketing to `TarredAudioToClassificationLabelDataset`.

**Collection**: [ASR]

# Changelog 
- Modify `_setup_dataloader_from_config` in `nemo.collections.asr.models.classification_models._EncDecBaseModel` to enable bucketing.
- Adds `pad_id` member to `TarredAudioToClassificationLabelDataset` class.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc) -> It does not
  - [x] Reviewer: Does the PR have correct import guards for all optional libraries?

**PR Type**:
- [x] New Feature
Memory consumption by Punctuation model
State: closed | Pull request: no

Hi,
I am using PunctuationCapitalizationModel to add punctuation marks in raw text.
When i load the pretrained model it consumes 1427 mib memory approximately. This memory never gets free. The application that i created is flask application and runs in multithreaded environment. For every new request 1427mib memory gets consumed.
Code Snippet:
model = PunctuationCapitalizationModel.from_pretrained("punctuation_en_bert")
            model = model.cpu()
            texts = [word_data]
            processed_texts = model.add_punctuation_capitalization(
                texts,
                batch_size=128,
                max_seq_length=64,
                step=8,
                margin=16,
            )
            return str(processed_texts[0])
How to free this memory?
Please help to resolve it.
[TN] Bug fix: expand serial coverage of unknown symbol, remove constraints from word graph
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] relaxed support for both CPUs and GPUs
State: closed | Pull request: yes

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix some 's' cases for IPA G2P
State: closed | Pull request: yes

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

# What does this PR do ?

Fix some handling of "s" cases in the IPA G2P class.

**Collection**: TTS

# Changelog 
- Indexing fixed for cases where there's no apostrophe
- Removed extraneous "t" added for "ts/t's"
- Removed "s" case when there's no apostrophe
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
[TTS] [bugfix] German FastPitch HiFi-GAN tutorial and lr
State: closed | Pull request: yes

# What does this PR do ?

Address QA bugs for German TTS tutorial. Correct PR instead of https://github.com/NVIDIA/NeMo/pull/4449 

**Collection**: TTS

# Changelog 
- tutorial notebook changed
- learning rate updated to 1e-3 based on https://github.com/NVIDIA/NeMo/pull/4308 

**PR Type**: Bugfix
Update megatron interface to dialogue
State: closed | Pull request: yes

# What does this PR do ?

Update interface with Megatron GPT Prompt Learning model

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
WIP Fix for #4455
State: closed | Pull request: yes

# What does this PR do ?

Proposed fix for #4455

**Collection**: asr, common

# Changelog 
- [Plumb is_tar_manifest throughout dataset code.](https://github.com/NVIDIA/NeMo/commit/b1bc3ab190cbb1e24898cd2af6ff84e2425f69d7)
- [Fix for KeyError.](https://github.com/NVIDIA/NeMo/commit/653d4d755d0132d640305017336ddbf612705aeb)
# Usage

This allow for running ASR training on tar files with keys containing "/".

is_tar_manifest needs to be plumbed through all possible parse_func's in usage today.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [X] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #4455
fix branch in link
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Unable to import nlp
State: closed | Pull request: no
Labels: name: bug

I am running this Colab Notebook : https://colab.research.google.com/github/NVIDIA/NeMo/blob/v1.0.2/tutorials/nlp/Entity_Linking_Medical.ipynb#scrollTo=2qLS4_0tyf_A

When I run second cell it gives error on this line
from nemo.collections import nlp as nemo_nlp

And the error is this:
(RuntimeError                              Traceback (most recent call last)
[<ipython-input-18-051dc782fbfe>](https://localhost:8080/#) in <module>()
     11 from tqdm import tqdm
     12 
---> 13 from nemo.collections import nlp
     14 from nemo.utils.exp_manager import exp_manager

16 frames
/usr/local/lib/python3.7/dist-packages/torch/jit/_script.py in script(obj, optimize, _frames_up, _rcb, example_inputs)

RuntimeError: 

dropout(Tensor input, float threshold, float value, bool inplace=False) -> (Tensor):
Argument threshold not provided.
:
  File "/usr/local/lib/python3.7/dist-packages/megatron/model/transformer.py", line 366
def bias_dropout_add(x, bias, residual, prob, training) :
    # type: (Tensor, Tensor, Tensor, float, bool) -> Tensor
    out = torch.nn.functional.dropout(x + bias, p=prob, training=training)
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
    out = residual + out
    return out
'bias_dropout_add' is being compiled since it was called from 'bias_dropout_add_fused_train'
  File "/usr/local/lib/python3.7/dist-packages/megatron/model/transformer.py", line 380
def bias_dropout_add_fused_train(x, bias, residual, prob) :
    # type: (Tensor, Tensor, Tensor, float) -> Tensor
    return bias_dropout_add(x, bias, residual, prob, True)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE)
[TTS] Fix typo in HiFi-GAN config's max steps
State: closed | Pull request: yes

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Co-authored-by: Eric Harper <complex451@gmail.com>

# What does this PR do ?

related to https://github.com/NVIDIA/NeMo/pull/4446, which has been merged to r1.10.0.



**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Updated notebook to fix batch configuration and precision bugs
State: closed | Pull request: yes

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Bug fixes for prompt learning notebook after pipeline parallel addition 
https://nvbugs/3681543 [ML-NeMo][A30] Multitask_Prompt_and_PTuning.ipynb branch name not 'r1.10.0' and failed with ValueError if manually changed to the r1.10.0 branch

Add a one line overview of what this PR aims to accomplish.

**Collection**: BigNLP

# Changelog 
- Updated tutorial notebook

# Usage
Run Jupiter notebook

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix typo in HiFi-GAN config's max steps
State: closed | Pull request: yes

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

# What does this PR do ?

Fix typo in the default `max_steps` for HiFi-GAN configs (see #4431).

**Collection**: TTS

# Changelog 
- Changed default steps from 25M to 2.5M

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

# Additional Information
* Raised in discussion #4431
unify intent slot dataset util functions in tutorials
State: closed | Pull request: yes

# What does this PR do ?

Unify intent slot util function file locations

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Docs: Decrease Font Size on Tables 
State: closed | Pull request: yes

# What does this PR do ?

Decrease font size on tables to fit more content

**Collection**: None

# Changelog 
- Decrease font size on tables to fit more content

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] add static method decorator.
State: closed | Pull request: yes

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Notebook bug fix: add subfolder
State: closed | Pull request: yes

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Docs: Add table overflow handling
State: closed | Pull request: yes

# What does this PR do ?

Add table overflow handling. In particular, this caused issues on the [Scores page](https://github.com/NVIDIA/NeMo/blob/r1.10.0/docs/source/asr/scores.rst)

**Collection**: None

# Changelog 
- Setup section to overflow horizontally
- Add scrolling to tables placed under `section` tags

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
T5 rpe fix 01
State: closed | Pull request: yes

# What does this PR do ?

Fix the RPE implementation converging slower with loss consistently higher than NeMo APE and Megatron-LM RPE implementation.

**Collection**: NLP

# Changelog 
- Changed how RPE parameters are initialized and passed on.

# Usage
* Train with `model.position_embedding_type='relative'` config.

```python
python /home/workspace/NeMo/examples/nlp/language_modeling/megatron_t5_pretraining.py model.position_embedding_type='relative'
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update dialogue tutorial version
State: closed | Pull request: yes

# What does this PR do ?

Update dialogue tutorial version

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)