renwei2024/nemo-github-issues huggingface dataset:
[STT] Add Esperanto (Eo) ASR Conformer-CTC and Conformer-Transducer models
State: closed | Pull request: yes
Labels: name: cherry-pick

name: ASR

# What does this PR do ?

[STT] Add Esperanto (Eo) ASR Conformer-CTC and Conformer-Transducer models

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Patch to support additional tokenizers in t5 adapter training
State: closed | Pull request: yes
Labels: name: NLP


# What does this PR do ?

patch to allow using tokenizers without additional_special_tokens_ids…

**Collection**: [NLP]

# Changelog 
- checks for `additional_special_tokens_ids` attribute in tokenizer.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[STT] Add Esperanto (Eo) ASR Conformer-CTC and Conformer-Transducer models
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

[STT] Add Esperanto (Eo) ASR Conformer-CTC and Conformer-Transducer models

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add AWS SageMaker ASR Examples
State: closed | Pull request: yes
Labels: name: feature

# What does this PR do ?

Adds AWS SageMaker example notebooks for ASR!

**Collection**: ASR

# Changelog 
- Add AWS SageMaker examples to NeMo repo.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Speaker Diarization Task: Clustering stage crashes(Process killed) for long audio files using CPU
State: closed | Pull request: no
Labels: name: stale

name: ASR

name: Speaker Tasks

name: question

Hello,

I've been testing the NeMo diarization example, using clustering inference on many different datasets and inputs using CPU only. On files up to 90 minutes in length, the diarizer reached around 25Gb RAM consumed in the clusterization stage. When I tried to diarize a 4 hour file that has 2 speakers using CPU only, the process was killed in the clusterization stage, no matter the configuration file used.

The configuration I used:
- clustering_diarizer/offline_diar_infer
- vad_multilingual_marblenet
- embeddings extraction: titanet_l
- configurations: either meeting or general ( both crashed)

The way I solved the process crashing problem was by splitting the 4 hour audio in 4 parts, and then merge the predicted rttms at the end by taking in consideration the offset between parts and checking if the speaker labels match. The issue is that the cluster labels on each file may differ from the other files, even if it is the same original audio split into 4. With only 2 speakers, it may be easy to check and switch the speakers labels so they match between all corresponding parts, but when there are more than 2 speakers, it's impossible.

My questions are: is there a way to avoid the process being killed when using long audio files(above 2 hours for example) on CPU(I can't used GPU)? If there isn't, is there a way to match the speakers from each split from the original audio? (example: speaker 1 in part 1 may be corresponding to speaker 4 in part 2) I was thinking about extracting an mean affinity matrix for each speaker in each part and then calculating the cosine distance between all speakers. 

Any solution, idea or feature that is already implemented would be greatly appreciated!
TTS Primer Tutorial throws errors in colab
State: closed | Pull request: no
Labels: name: bug

I am trying to learn from this tutorial: https://github.com/NVIDIA/NeMo/blob/main/tutorials/tts/NeMo_TTS_Primer.ipynb, but I get errors since the beginning!
In the Setup I get this:
`ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow 2.9.2 requires protobuf<3.20,>=3.9.2, but you have protobuf 3.20.3 which is incompatible.
tensorflow 2.9.2 requires tensorboard<2.10,>=2.9, but you have tensorboard 2.11.0 which is incompatible.`
And in the TN Example, this:
```
AttributeError                            Traceback (most recent call last)
[/usr/local/lib/python3.8/dist-packages/pkg_resources/__init__.py](https://localhost:8080/#) in _dep_map(self)
   3015     PKG_INFO = 'METADATA'
-> 3016     EQEQ = re.compile(r"([\(,])\s*(\d.*?)\s*([,\)])")
   3017 

24 frames
AttributeError: _DistInfoDistribution__dep_map

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
AttributeError: _pkg_info

During handling of the above exception, another exception occurred:

FileNotFoundError                         Traceback (most recent call last)
[/usr/local/lib/python3.8/dist-packages/pkg_resources/__init__.py](https://localhost:8080/#) in _get(self, path)
   1609     def _listdir(self, path):
   1610         return os.listdir(path)
-> 1611 
   1612     def get_resource_stream(self, manager, resource_name):
   1613         return open(self._fn(self.module_path, resource_name), 'rb')

FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.8/dist-packages/Pygments-2.6.1.dist-info/METADATA'
```
I haven't changed anything, just run the cells you provide in the tutorial. The environment is set with GPU.
per-micro-batch input loader
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

Per-micro-batch input loader and sampler for Megatron-based language models

# Changelog 
- `Use dataloader_iter` based step iterator API
- Update batch sampler to accommodate changes for global-batch-based sampling
- Update batch H2D copier
- Update training step hooks to reflect micro-batch to global-batch index difference

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS][DOC] add notes about automatic conversion to target sampling ra…
State: closed | Pull request: yes
Labels: name: TTS

name: cherry-pick

…tes.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Temporarily disable prompt learning CI tests
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Prompt learning CI tests are taking ~10 minutes to complete. Temporarily disabling them until they can be fixed.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Temporarily disable prompt learning tests
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

Found a few tests that are taking ~ 10 minutes to complete. Disabling them until they can be fixed.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
EMA Doesn't delete previous EMA ckpts when k > 0 for checkpointing
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

EMA saves a separate EMA based ckpt when normally saving. The problem is when saving a set amount of checkpoints (i.e k=5) the older EMA ckpts are not being deleted. This means there is an excess of EMA based ckpts.

What is expected is that older EMA ckpts are deleted.

**Proposed Solution**

Overriding the code is tricky, as the call to remove ckpts is found within functions, like such:

https://github.com/Lightning-AI/lightning/blob/master/src/pytorch_lightning/callbacks/model_checkpoint.py#L652
https://github.com/Lightning-AI/lightning/blob/master/src/pytorch_lightning/callbacks/model_checkpoint.py#L671
https://github.com/Lightning-AI/lightning/blob/master/src/pytorch_lightning/callbacks/model_checkpoint.py#L713

My suggestion would be to move this code into a separate function that can be overridden in the [NeMoModelCheckpoint](https://github.com/NVIDIA/NeMo/blob/4d3e99f986fd9354b5528a3e9c82fe01ec2df916/nemo/utils/exp_manager.py#L745) class by introducing a `delete_checkpoint` function. In this function we will also be able to delete the EMA weights.
Remove SDP (moved to separate repo) - merge to main
State: closed | Pull request: yes

# What does this PR do ?

Remove SDP code from tools/ folder and add docs page briefly describing SDP and its new location

**Collection**: tools

# Changelog 
- Remove `tools/speech_data_processor` dir
- Update docs with new SDP location

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ N/A ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ N/A ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to [PR #5574](https://github.com/NVIDIA/NeMo/pull/5574)
[Dockerfile] Remove AIS archive from docker image
State: closed | Pull request: yes
Labels: name: CI

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

# What does this PR do ?

Removes an unused archive, downloaded when preparing the AIS binary.

**Collection**: General

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
Concat tarred ds fixes, additional config parameters
State: closed | Pull request: yes
Labels: name: ASR

name: common

# What does this PR do ?

Adds ConcatDataset features to ASR configs and fixes bugs

**Collection**: [Note which collection this PR will affect]
ASR

# Changelog 
- All parameters are now optional
- One needs to pass train_ds.is_concat=True  to activate
- The datasets become a list over the possible current list
- if the concat_sampling_technique = random, one needs to specify concat_sampling_probabilities, one per dataset

# Usage
In the config file
model.train_ds.is_concat=True \
model.train_ds.concat_sampling_technique=random \.  # Default: temperature
# model.train_ds.concat_sampling_temperature=5 \.  # Default: 5
model.train_ds.concat_sampling_probabilities=[0.5,0.5] \.  # Default: none

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x ] New Feature
- [ x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fixed a missing import for gather_objects
State: closed | Pull request: yes

# What does this PR do ?

1. Added missing import for gather_objects.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fixed a missing import for gather_objects
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

This PR fixes a missing import.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS][ZH] add fastpitch and hifigan model NGC urls and update NeMo docs.
State: closed | Pull request: yes
Labels: name: TTS

name: cherry-pick

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS][DOC] add notes about automatic conversion to target sampling ra…
State: closed | Pull request: yes
Labels: name: TTS

…tes.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Enc-Dec model size reporting fixes
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Reports the model size of enc-dec models across model parallel groups similar to BERT/GPT

**Collection**: NLP

# Changelog 
- Moves param aggregation to the base model
- Implements param aggregation for enc-dec models.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fixed a missing import for gather_objects
State: closed | Pull request: yes

# What does this PR do ?

This PR fixes a missing import.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[EMA] Change success message to reduce confusion
State: closed | Pull request: yes
Labels: name: common

# What does this PR do ?

Suggestion in slack to reduce confusion. The current message made it seem like the EMA weights are loaded into the main weights which is incorrect.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Separate Training and Validation Batch Sizes
State: closed | Pull request: yes
Labels: name: NLP

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Makes training and validation batch sizes separately configurable for T5 prompt learning. Related to https://jirasw.nvidia.com/projects/NLP/issues/NLP-393?filter=allopenissues

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage

```python
 data:
    add_eos: true
    add_bos: false
    decoder_starts_with_pad: False
    add_eos_to_decoder_output: True
    add_sentinel_to_input: True
    ul2_prompt_token: null # <extra_id_s>, <extra_id_r>, <extra_id_x>

    train_ds: 
      path: ["data/squad_train.jsonl"]
      global_batch_size: 8
      micro_batch_size: 8 # micro batch size should equal global batch size when pipeline parallel = 1
      shuffle: true
      num_workers: 4
      pin_memory: true

    validation_ds: 
      path: ["data/squad_val.jsonl"]
      global_batch_size: 8
      micro_batch_size: 8 # micro batch size should equal global batch size when pipeline parallel = 1
      shuffle: true
      num_workers: 4
      pin_memory: true
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update Dockerfile to use numba==0.53.1
State: closed | Pull request: yes
Labels: name: cherry-pick

name: CI

Pinned to use `numba==0.53.1` to avoid crashing in training with `num_workers > 0`. This is just a temporary workaround, still need to fix it in the future.

Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com>
[Temp workaround] Disable test with cache_audio to unblock CI
State: closed | Pull request: yes

Temp workaround: Disable test with cache_audio=True since it is failing in CI (#5607)

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

# What does this PR do ?

Running `test_cache_datastore_manifests` with `cache_audio=True` started to fail in CI.
Here we disable it temporarily to unblock CI.

**Collection**: ASR, tests

# Changelog 
- Removed test with `cache_audio=True`

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #5607
Update Dockerfile to use numba==0.53.1
State: closed | Pull request: yes
Labels: name: CI

Pinned to use `numba==0.53.1` to avoid crashing in training with `num_workers > 0`. This is just a temporary workaround, still need to fix it in the future.

Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com>
Update Dockerfile to use numba==0.53.1
State: closed | Pull request: yes

Pinned to use `numba==0.53.1` to avoid crashing in training with `num_workers > 0`. This is just a temporary workaround, still need to fix it in the future.

Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com>
WFST Text Normalization: Address graph and OK
State: closed | Pull request: no
Labels: name: bug

name: TN/ITN

**Describe the bug**

The address graph is too permissive and always transduces "OK" to "Oklahoma" even in cases when it is not a part of an address.

Could be nice to add some test cases that evaluate the performance of the graph in cases where OK is/is not a part of an address or not, maybe also extending the address graph if possible.


**Steps/Code to reproduce bug**

```
from nemo_text_processing.text_normalization.normalize import Normalizer

print(normalizer.normalize("Ah, OK.", verbose=True, punct_post_process=True))
print(normalizer.normalize("That's OK.", verbose=True, punct_post_process=True))
```
Output:
```
tokens { name: "Ah, Oklahoma" }  tokens { name: "." }
Ah, Oklahoma.  # incorrectly transduced by the address graph
tokens { name: "That's" } tokens { name: "OK." }  
That's OK.  # expected
```

**Expected behavior**

The grammar only transforms `OK -> Oklahoma` when in the context of an address.

**Environment overview (please complete the following information)**

 - Jupyter notebook in venv on local machine
 - Method of NeMo install: pip install nemo_toolkit['all']

**Environment details**


- OS version 5.13.0-40-generic
- PyTorch version 1.13.0
- Python version 3.9.7

**Additional context**
model.inference() should take other inputs instead of only file input
State: closed | Pull request: no
Labels: name: stale

**Is your feature request related to a problem? Please describe.**

I trained a pre-trained model Bert on the Squad dataset for question_answer but to get the answer for a single question there is nothing that **model.inference** provides me as it only accepts a file. 
```
model = BERTQAModel.restore_from(config.model.nemo_path)

eval_device = [config.trainer.devices[0]] if isinstance(config.trainer.devices, list) else 1
model.trainer = pl.Trainer(
    devices=eval_device,
    accelerator=config.trainer.accelerator,
    precision=16,
    logger=False,
)

all_preds, all_nbest = model.inference(
    config.model.test_ds.file, # file_name
    num_samples=10, # setting to -1 will use all samples for inference
)
```

**Describe the solution you'd like**
So, if we can pass the question string and model infer on the question string to return us the answer, it would be great.

```
answer = model.inference("question") 
# here question can be a string on which the model can infer to give us the result or it also can be a JSON obj.
```
Update documentation and tutorials for Adapters 
State: closed | Pull request: yes
Labels: name: core

name: ASR

# What does this PR do ?

Update documentations and tutorials for adapters

**Collection**: [ASR, Core]

# Changelog 
- Update tutorial to include info about adapter changes and type checking

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [x] Documentation
Add core classes and functions for online clustering diarizer part 2
State: closed | Pull request: yes
Labels: name: ASR

Signed-off-by: Taejin Park <tango4j@gmail.com>


[[NeMo Online Speaker Diarization VIDEO DEMO LINK](
https://drive.google.com/file/d/15ro8HKmzevXh_wx9oTtfRUw9wNtj6LuJ/view?usp=share_link)]


# What does this PR do ?

Part 2 for adding core classes and functions for online clustering diarizer.
** As soon as  [PR 5526](https://github.com/NVIDIA/NeMo/pull/5526) gets merged, `Files changed` will be reduced to 2` **
Dividing the origin huge PR into two: 
(part 1: online clustering (nmesc_clustering.py) + unit tests: [PR 5526](https://github.com/NVIDIA/NeMo/pull/5526))
part 2: Segmentation + online diarizer class + unit tests

**Collection**: 

ASR

# Changelog 

the following will be included in this PR
#### `OnlineClusteringDiarizer` in online_diarizer.py (inherits ClusteringDiarizer) (will be included in part2)
online version of `ClusteringDiarizer` class in clustering_diarizer.py

#### `OnlineSegmentor` class in speaker_utils.py: (will be included in part2)
online version of speech segmentations


<img width="966" alt="image" src="https://user-images.githubusercontent.com/13211483/204661137-78d0272b-1594-4509-a486-a8720e835bc4.png">



Checkout [Help Slides Link](https://docs.google.com/presentation/d/1LDUTIGRNVuLJsy81V3GaHZlFDUpxZ9rux-fNPfgcLmw/edit?usp=sharing) for better understanding on why online diarizer needs
(1) linear sum assignment (Hungarian alg)
(2) Interval merging
(3) merge_vector function
and data flow of class and functions.

# Usage

This is core classes and functions update.
Example script will be pushed in the second round of PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.
[Online Neural Diarization of Unlimited Numbers of Speakers](https://arxiv.org/pdf/2206.02432.pdf)

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
add yaml file for stateless transducer ASR model
State: closed | Pull request: yes
Labels: name: stale

name: ASR

# What does this PR do ?

This PR adds a yaml file for the conformer transducer model with a stateless decoder.

**Collection**: 

ASR

# Changelog 

# Usage


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] fix ranges of char set for accented letters.
State: closed | Pull request: yes

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

# What does this PR do ?

Previously, the char set pattern `[À-ÿ]` includes math operators, such as `÷` and `×`. Revise the pattern to exclude these. 
```
ord("À")
Out[5]: 192
ord("ÿ")
Out[6]: 255

chr(247)
Out[8]: '÷'
chr(215)
Out[9]: '×'
```

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Remove unused TTS eval function
State: closed | Pull request: yes
Labels: name: TTS

name: cherry-pick

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

# What does this PR do ?

Removes `eval_tts_scores()`, which was unused and had `pesq` (unmaintained) and `pystoi` dependencies.

**Collection**: TTS

# Changelog 
- Removed `pesq` and `pystoi` install requirements for TTS
- Removed unused function from `helpers.py`
[TTS] Remove unused TTS eval function
State: closed | Pull request: yes
Labels: name: TTS

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

# What does this PR do ?

Removes `eval_tts_scores()`, which was unused and had `pesq` (unmaintained) and `pystoi` dependencies.

**Collection**: TTS

# Changelog 
- Removed `pesq` and `pystoi` install requirements for TTS
- Removed unused function from `helpers.py`
music_megatron port to NeMo
State: closed | Pull request: yes
Labels: name: stale

name: NLP

name: common

# What does this PR do ?

Port Music Megatron implementation onto NeMo

**Collection**: [Note which collection this PR will affect]
adding one more music tokenizer to tokenizer for megatron_gpt_pretraining.py 
# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# modify the data location in the https://github.com/NVIDIA/NeMo/tree/main/examples/nlp/language_modeling/conf/megatron_music_gpt_config.yaml file  line 167-169
python examples/nlp/language_modeling/megatron_gpt_pretraining.py \
        --config-name=megatron_music_gpt_config.yaml
        trainer.devices=1 \
        trainer.log_every_n_steps=1 \
        trainer.val_check_interval=100 \
        trainer.limit_val_batches=2 \
        trainer.accumulate_grad_batches=1 \
        trainer.max_steps=3 \
        trainer.precision=16 \
        trainer.gradient_clip_val=1.0 \
        exp_manager.exp_dir=examples/nlp/language_modeling/gpt_pretrain_results \
        model.pipeline_model_parallel_size=1 \
        model.tensor_model_parallel_size=1 \
        model.optim.name=fused_adam \
        model.optim.lr=2e-4 \
        model.optim.sched.warmup_steps=1 \
        model.optim.sched.constant_steps=1 \
        model.optim.sched.min_lr=8e-5 \
        model.max_position_embeddings=128 \
        model.encoder_seq_length=128 \
        model.data.seq_length=128 \
        model.tokenizer.tokenizer_name=music \
        model.tokenizer.tokenizer_library=music \
        model.tokenizer.vocab_file=/workspace/music_vocab.json \
        model.tokenizer.merge_file=None \
        model.num_layers=8 \
        model.hidden_size=256 \
        model.num_attention_heads=8 \
        model.data.index_mapping_dir=examples/nlp/language_modeling/gpt_index_mappings
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
  
**PR Type**:
- [x ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?
@okuchaiev @yidong72 
Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] add type hints and change varialbe names for tokenizers and g2p
State: closed | Pull request: yes
Labels: name: common

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)