renwei2024/nemo-github-issues huggingface dataset:
Tn update numbers
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Adding fisher audio conversion script from old NeMo branch
State: closed | Pull request: yes

# What does this PR do ?

Adding missing script from docs. Changed docs to point to the right scripts directory. 

**Collection**: ASR

# Changelog 
- Restored missing fisher_audio_to_wav script

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add configurable LogSoftmax support to Punctuator Model
State: closed | Pull request: yes

# What does this PR do ?

Adds a configurable parameter to apply log softmax on output logits for RIVA support

**Collection**: [Note which collection this PR will affect]
nlp/models/token_classification/punctuation_capitalization_model.py

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Notebook Bug Fixes for r1.8.0
State: closed | Pull request: yes

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Notebook bug fixes for nv-bugs: 

https://nvbugswb.nvidia.com/NVBugs5/redir.aspx?url=/3596793 [ML-NeMo][A30] Text_Classification_Sentiment_Analysis.ipynb failed with ConfigAttributeError: Key 'model' is not in struct

https://nvbugswb.nvidia.com/NVBugs5/redir.aspx?url=/3596634 [ML-NeMo][A30] Joint_Intent_and_Slot_Classification.ipynb failed with ConfigAttributeError: Key 'model' is not in struct

https://nvbugswb.nvidia.com/NVBugs5/redir.aspx?url=/3596616 [ML-NeMo][A30] Entity_Linking_Medical.ipynb failed with MisconfigurationException: `Trainer(strategy='ddp')`

**Collection**: N/A

# Changelog 
Changed `nlp_model.py` and 3 tutorial notebooks

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Tn tts e
State: closed | Pull request: yes

# What does this PR do ?
On-going TN refactoring for TTS

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
docs fixes
State: closed | Pull request: yes

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[Core] Fix type checking to be compatible with named tuples
State: closed | Pull request: yes

Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>

# What does this PR do ?

Fix inner type checking system for outputs to be compatible with named tuples.
This adds the ability to use named tuples (`typing.NamedTuple` or `collections.namedtuple`) for outputs. This is especially useful when the number of output elements is large. 
Named tuples are compatible with simple tuples, there should be no problems with mixing them.

# Usage
```python
class OutputType(NamedTuple):
    y: torch.Tensor
    z: torch.Tensor

class MultipleOutputTypesWithNamedTuple(Typing):
    @property
    def output_types(self):
        return {"y": NeuralType(('B',), ElementType()), "z": NeuralType(('B',), ElementType())}

    @typecheck()
    def __call__(self, x):
        y = x + 1
        z = x + 2
        return OutputType(y=y, z=z)

obj = MultipleOutputTypesWithNamedTuple()
result = obj(x=torch.zeros(10))
result_y, result_z = result  # can unpack or use `result.y`, `result.z`
```
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

## Who can review?

@titu1994, @okuchaiev, @ericharper
Fix NMT variable passing bug
State: closed | Pull request: yes

# What does this PR do ?
Fixes a minor NMT variable passing bug.

**Collection**: NLP

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Pass float32 array to librosa.core.resample
State: closed | Pull request: yes

Signed-off-by: rvygon <roman.vygon@gmail.com>

# What does this PR do ?

Fixes #3982, where get_samples passes int16 to librosa.core.resample instead of float32.

**Collection**: ASR

# Usage

```python
from pydub import AudioSegment
from nemo.collections.asr.parts.utils.decoder_timestamps_utils import get_samples
segment = AudioSegment.silent(duration=10000).set_frame_rate(22050).set_channels(1)
segment.export('test.wav', format='wav')

arr = get_samples('test.wav')
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #3982  (issue)
get_samples wrong dtype
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

nemo.collections.asr.parts.utils.decoder_timestamps_utils.get_samples fails if the audio file's original sample rate differs from the target sample rate.

`ParameterError: Audio data must be floating-point`

**Steps/Code to reproduce bug**

```python
from pydub import AudioSegment
from nemo.collections.asr.parts.utils.decoder_timestamps_utils import get_samples

segment = AudioSegment.silent(duration=1000).set_frame_rate(22050)
segment.export('test.wav', format='wav')
get_samples('test.wav')
```
**Expected behavior**

File is resampled successfully.

**Environment overview (please complete the following information)**

 - Method of NeMo install: pip
 - NeMo version: 1.6.2
 - librosa version: 0.8.1
Fixing issues for MegaMolBART model.decode call
State: closed | Pull request: yes

Signed-off-by: Micha Livne <mlivne@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Megatron shared YAML
State: closed | Pull request: yes

Signed-off-by: Micha Livne <mlivne@nvidia.com>

# What does this PR do ?

This PR creates a 2-hierarchy YAML configuration.
The goal is to make sure that updates to config that are shared between all megatron models are affecting all relevant models.

**Collection**: [Note which collection this PR will affect]

language nlp.language_modelling (megatron)

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix path in doc
State: closed | Pull request: yes

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

# What does this PR do ?

Fix path to creation of tarred dataset script in docs

**Collection**: docs

# Changelog 
- Path location


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation
Compatability override to load_state_dict for old TTS checkpoints
State: closed | Pull request: yes

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

# What does this PR do ?

Overrides `load_state_dict` for four TTS models (to be deprecated in 1.9.0) such that they work with old NGC checkpoints, plus minor tutorial fixes/updates.

**Collection**: TTS

# Changelog 
- Removes `forward_basis` and `inverse_basis` from the `state_dict` of existing checkpoints, which were trained before NeMo moved away from using torch_stft
- Adds `window` to `state_dict` for the preprocessor
- Semicolon added to one line in the TalkNet training tutorial to avoid printing the whole model
- Added `trainer.gpu` command line argument to Tacotron2 notebook
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Megatron NMT Restore from T5/BART and finetune
State: closed | Pull request: yes

# What does this PR do ?

Adds the ability to restore from a T5/BART model and fine-tune it for NMT. With tarred datasets only.

**Collection**: NLP

# Changelog 
- Fixes issues with rebasing to the Encoder-Decoder PP changes.
- Overrides the pre-training config with NMT-related things.
- Reconfigures the microbatch calculator on every training/eval step.

# Usage

Edit the aayn_base_megatron.yaml file with

- pretrained_model_path: /path/to/pretrained/t5.nemo
- pretrained_model_type: t5/bart

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Option to remove bias terms from Megatron transformers
State: closed | Pull request: yes

# What does this PR do ?

Adds to option to remove exclude terms from megatron transformer weight matrices.

**Collection**: NLP

# Changelog 

# Usage
* Change yaml config to

```
bias: False
bias_gelu_fusion: False
bias_dropout_add_fusion: False
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix variable name and move models to CPU in Change partition
State: closed | Pull request: yes

# What does this PR do ?
Fix tensor parallel variable name and move models to CPU in Change partition

Sidenote: Looks like Megatron models disregard the accelerator mentioned in the trainer. To work things for now, this script manually moves model to CPU

**Collection**: NLP

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Fix variable name and move models to CPU in Change partition
State: closed | Pull request: yes

# What does this PR do ?
Fix tensor parallel variable name and move models to CPU in Change partition

**Collection**: NLP

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Fixes val_check_interval, skip loading train data during eval
State: closed | Pull request: yes

# What does this PR do ?

1. Fixes the condition for verifying if val check interval is too small. 
2. Does not load train data when running trainer.validate or trainer.test
3. Fixes an error in glue_eval

**Collection**: NLP

# Changelog 

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add pinned pynini and scipy installs to TTS training tutorial
State: closed | Pull request: yes

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

# What does this PR do ?

Fixes some execution errors for the training notebook (in Colab) by adding & pinning installs.

**Collection**: TTS

# Changelog 
- Pynini pinned to version 2.1.4 (needed for training)
- Scipy pinned to version 1.7.3 (fixes comparison issue from #3961)
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

# Additional Information
* Related to #3961
[ASR] k2: Epsilon-Transducer and code imprevements
State: closed | Pull request: yes
Labels: name: core

name: ASR

# What does this PR do ?

Introduces Epsilon-Transducer and major k2 code refactoring.

**Collection**: ASR

# Note
This PR is an interim one, meant to keep the k2 core code up to date and preserve some useful prototypes like the Pruned Epsilon-Transducer. The k2-based RNNT code design is subject to upcoming refactoring following the merger of this PR and https://github.com/NVIDIA/NeMo/pull/6168.

# Changelog 
- k2 losses and decoders refactored
- Epsilon-Transducer RNNT variant added
- RNNT greedy and forced algnment added for Epsilon-Transducer
- Pruned Epsilon-Transducer proposed
- CTC and RNNT tests added

# Usage
* RNNT forced alignment: `python ./experimental/k2/align_speech_parallel.py model=stt_en_conformer_transducer_large predict_ds.manifest_filepath=./data/Librispeech/dev_clean.json predict_ds.batch_size=8 output_path=./ali_rnnt model_stride=4 aligner_args.decode_batch_size=2 aligner_args.rnnt_cfg.predictor_window_size=40 aligner_args.rnnt_cfg.predictor_step_size=4`

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

@artbataev
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
FastPitch_MixerTTS_Training tutorial not working on Colab
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

The function `pre_calculate_supplementary_data` fails when running _FastPitch_MixerTTS_Training.ipynb_ on Colab.

**Steps/Code to reproduce bug**

Open https://github.com/NVIDIA/NeMo/blob/main/tutorials/tts/FastPitch_MixerTTS_Training.ipynb on Colab, then run all the cells. It should fail here:
```
fastpitch_sup_data_path = "fastpitch_sup_data_folder"
sup_data_types = ["align_prior_matrix", "pitch"]

pitch_mean, pitch_std, pitch_min, pitch_max = pre_calculate_supplementary_data(
    fastpitch_sup_data_path, sup_data_types, text_tokenizer, text_normalizer, text_normalizer_call_kwargs
)
```
This is the output
```
TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.7/dist-packages/nemo/collections/tts/torch/data.py", line 447, in __getitem__
    align_prior_matrix = beta_binomial_prior_distribution(text_length, mel_len)
  File "/usr/local/lib/python3.7/dist-packages/nemo/collections/tts/torch/helpers.py", line 57, in beta_binomial_prior_distribution
    mel_i_prob = betabinom(phoneme_count, a, b).pmf(x)
  File "/usr/local/lib/python3.7/dist-packages/scipy/stats/_distn_infrastructure.py", line 495, in pmf
    return self.dist.pmf(k, *self.args, **self.kwds)
  File "/usr/local/lib/python3.7/dist-packages/scipy/stats/_distn_infrastructure.py", line 2997, in pmf
    cond1 = (k >= _a) & (k <= _b) & self._nonzero(k, *args)
TypeError: '<=' not supported between instances of 'numpy.ndarray' and 'Tensor'
```

**Expected behavior**

I expect it to work

**Environment overview (please complete the following information)**

 - Environment location: Google Colab

**Environment details**

- PyTorch version 1.10.0+cu111
- Python version 3.7.13
Assign a language model to Citrinet1024
State: closed | Pull request: no

Hi, Thanks for your excellent tutorial. Refer to [Citrinet paper](https://arxiv.org/abs/2104.01721), you got WER in 3 modes: without language model, with language model(6-gram and Transf). How can I assign a language model to Citrinet1024. I started fine-tuning Citrinet1024 with the Persian audio dataset.
Merge r1.8 main
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update max_epochs on megatron configs
State: closed | Pull request: yes

# What does this PR do ?

Many megatron configs had `trainer.max_epochs: null`. This resulted in PTL setting `trainer.max_epochs=-1` which means an infinite dataset but our datasets are not infinite.

This was causing problems when resuming training with GPT.

**Collection**: NLP

# Changelog 
- Update configs that have max_epochs set to null
- 
# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Exports 22.03 war
State: closed | Pull request: yes

# What does this PR do ?

Fixed fast pitch for 22.03

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix issue with Segfault in ASR models
State: closed | Pull request: yes

Signed-off-by: smajumdar <titu1994@gmail.com>

# What does this PR do ?

Fixes a segfault in nemo ASR models due to hydra / omegaconf access during DDP.

**Collection**: [ASR]

# Changelog 
- Add new flag and method to pre-allocate the value of the check
- Update skip_nan_grad code to support global DDP access.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
FastPitch FT notebook - Improving Speech Quality clarifications
State: closed | Pull request: yes

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

# What does this PR do ?
Clarifications to the FastPitch fine-tuning notebook regarding how to improve speech quality by generating synthesized mels; added missing instructions for adding more data/multiple speakers for FastPitch.

**Collection**: TTS

# Changelog 
- Moved synthesized mel code out of text block and into a separate code block, fixed typos s.t. it runs
- Added missing info for fine-tuning FastPitch on multiple speakers (need to update `sup_data_types`)
- Additional info about mixing speakers
- Other minor text updates to the notebook

**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

# Additional Information
* Related to #3678 and #3895
Hydra Error
State: closed | Pull request: no
Labels: name: bug

I'm running Tacotron2 on custom dataset and this error pops on, any idea what to do:

`[NeMo W 2022-04-07 13:27:18 optimizers:55] Apex was not found. Using the lamb or fused_adam optimizer will error out.
################################################################################
### WARNING, path does not exist: KALDI_ROOT=/mnt/matylda5/iveselyk/Tools/kaldi-trunk
###          (please add 'export KALDI_ROOT=<your_path>' in your $HOME/.profile)
###          (or run as: KALDI_ROOT=<your_path> python <your_script>.py)
################################################################################

[NeMo W 2022-04-07 13:27:19 __init__:23] `pynini` is not installed ! 
    Please run the `nemo_text_processing/setup.sh` script prior to usage of this toolkit.
[NeMo W 2022-04-07 13:27:19 experimental:28] Module <class 'nemo.collections.nlp.data.language_modeling.megatron.megatron_batch_samplers.MegatronPretrainingRandomBatchSampler'> is experimental, not ready for production and is not fully supported. Use at your own risk.
Traceback (most recent call last):
  File "tacotron2.py", line 44, in <module>
    main()  # noqa pylint: disable=no-value-for-parameter
  File "/usr/local/lib/python3.7/dist-packages/nemo/core/config/hydra_runner.py", line 108, in wrapper
    config_name=config_name,
  File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 378, in _run_hydra
    lambda: hydra.run(
  File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 214, in run_and_report
    raise ex
  File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 211, in run_and_report
    return func()
  File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 381, in <lambda>
    overrides=args.overrides,
  File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/hydra.py", line 92, in run
    run_mode=RunMode.RUN,
  File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/hydra.py", line 563, in compose_config
    from_shell=from_shell,
  File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/config_loader_impl.py", line 145, in load_configuration
    from_shell=from_shell,
  File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/config_loader_impl.py", line 231, in _load_configuration_impl
    parsed_overrides = parser.parse_overrides(overrides=overrides)
  File "/usr/local/lib/python3.7/dist-packages/hydra/core/override_parser/overrides_parser.py", line 100, in parse_overrides
    ) from e.__cause__
hydra.errors.OverrideParseException: Error parsing override 'Year'
missing EQUAL at '<EOF>'
See https://hydra.cc/docs/next/advanced/override_grammar/basic for details`
Glu activation variants
State: closed | Pull request: yes

# What does this PR do ?

Implements the ReGLU and SwiGLU activations for Megatron-based transformers from https://arxiv.org/abs/2002.05202

**Collection**: NLP

# Changelog 
- Changes `ParallelMLP` to add new activation functions.

# Usage
* Edit the `activation` in `megatron_t5_config.yaml` to `swiglu` or `reglu`.
* Or pass the activation function appropriately to `ParallelMLP`

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix issues with librosa deprecations
State: closed | Pull request: yes

Signed-off-by: smajumdar <titu1994@gmail.com>

# What does this PR do ?

Fix librosa deprecations

**Collection**: [ASR]

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Fix 22.03 fastpitch export (Workaround for 22.03 Pytorch bug)
State: closed | Pull request: yes

# What does this PR do ?
Fix 22.03 fastpitch export (WAR around PyTorch export bug).
Also, works around new export issues brought from updated Apex (will be submitting an issue report there, but the WAR is need in the meantime).

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix notebook bugs for branch r1.8.0
State: closed | Pull request: yes

# What does this PR do ?

Fixed the biomegatron model NGC link error.
`get_lm_model` can get biomegatron model through model names.
Fix the biomegatron notebooks to directly load the biomegatron from NGC.
Fix the MegatronLM ckpt conversion error. 
Fix the typos in synthetic data generation notebook.
Add rampup batch size to nemo megatron GPT
State: closed | Pull request: yes
Labels: name: stale

# What does this PR do ?

Adds rampup batch size to NeMo Megatron GPT. Scales the batch size linearly from starting batch size to global batch size by a batch size increment over a number of consumed samples.

There was a bug in the apex rampup batch size implementation that has been fixed here: https://github.com/NVIDIA/apex/commit/7d9038782b7a39c7dd05201e17586fd38b7a7c94.

Temporarily moved the needed functions over to NeMo until the above commit is in the NGC PyTorch container.

**Collection**: NLP

# Changelog 
- Add rampup batch size to the config
- Input rampup batch size to the apex num microbatch calculator
- Update the num microbatch calculator after each training step
- 
# Usage
Can be set in the config:
```bash
  # For example:
  global_batch_size: 1024
  rampup_batch_size:
    - 16
    - 8
    - 300000
  # will start with global batch size 16 and over
  # (1024 - 16) / 8 = 126 intervals will increase
  # the batch size linearly to 1024. In each interval
  # we will use approximately 300000 / 126 = 2380 samples
```
Or by the CLI:
```bash
model.rampup_batch_size=[16,8,300000]
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
remove old doc
State: closed | Pull request: yes

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X ] Bugfix
- [X ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
BERTLMModel.from_pretrained("bertlargeuncased") bug
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

running BERT_large = BERTLMModel.from_pretrained("bertlargeuncased") with error:

FileNotFoundError: src path does not exist or it is not a path in nemo file. src value I got was: /home/yzhang/data/nlp/bert/pretrained_models_google/uncased_L-24_H-1024_A-16/bert_config.json. Absolute: /home/yzhang/data/nlp/bert/pretrained_models_google/uncased_L-24_H-1024_A-16/bert_config.json

**Steps/Code to reproduce bug**

# Import BERT
from nemo.collections.nlp.models import BERTLMModel

model_large = BERTLMModel.from_pretrained("bertlargeuncased")


**Expected behavior**

like running BERT_base = BERTLMModel.from_pretrained("bertbaseuncased") successfully, should return downloaded Bert large pre-train model 

**Environment overview (please complete the following information)**

NeMo_1.7.2
Fix tarred sentence dataset length
State: closed | Pull request: yes

# What does this PR do ?

Fixes the length of TarredSentenceDataset with shard strategy as scatter.

**Collection**: NLP

# Changelog 
- Divide num batches by world size for the total length.

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
In notebook colab "TTS Inference Prosody Control" an error has been made.
State: closed | Pull request: no
Labels: name: bug

In notebook colab:
https://colab.research.google.com/github/NVIDIA/NeMo/blob/stable/tutorials/tts/Inference_DurationPitchControl.ipynb

an error has been made.

instead of:

# Load the models from NGC
fastpitch = FastPitchModel.from_pretrained("tts_en_fastpitch").eval().cuda()
hifigan = HifiGanModel.from_pretrained("tts_hifigan").eval().cuda()

Should be:

# Load the models from NGC
fastpitch = FastPitchModel.from_pretrained("tts_en_fastpitch")
hifigan = HifiGanModel.from_pretrained(model_name="tts_hifigan")
Conformer CTC dropping spoken numbers
State: closed | Pull request: no
Labels: name: bug

We have successfully trained a Conformer CTC BPE L model (non-English) with a very good WER in the lower teens mark. There is one peculiar thing that is happening though. The model is dropping numbers, interestingly not always and mostly dates. Especially if more are spoken in succession as part of a sentence. I have found a similar issue/questionon regarding the Spanish language on Riva Forums https://forums.developer.nvidia.com/t/riva-with-asr-doesnt-recognize-numbers-on-my-audio/203244, but no explanation. 

It is true that the train dataset does not cover such specific cases, but we were under the impression that in general the model learns to recognise BPE tokens from the waveform, and no "internal LM" or context dependency between BPE tokens (or at max a short one). In any case we'd expect the model to output at least something, not miss entire sections with multiple consecutive words. An example of the worst case we are seeing (translated in English) would be:
- reference: "accepted on twelfth fifth twenty twenty and modified on seventh seventh twenty twenty-one"
- transcription: "accepted and modified"  

We also get different results depending if we run transcription with `transcribe_speech.py`, `transcribe_speech_parallel.py` or buffered inference. 

Any thoughts to what might be the source of the problem, or what can we do to resolve it?