renwei2024/nemo-github-issues huggingface dataset:
Update ASR torch.stft() call to use return_complex=True
State: closed | Pull request: yes
Labels: name: ASR

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

# What does this PR do ?

Updates the remaining call to `torch.stft()` to use `return_complex=True` due to the deprecation of the `return_complex=False` option.

I've done a quick grep through the NeMo repo, this should be the only other place where `torch.stft()` (or `torch.istft()`) is called.

**Collection**: ASR

# Changelog 
- Changed the preprocessing lambda to use `return_complex=True`, which means `torch.stft()` will always return a complex tensor
- Removed the conditional for `view_as_real` since it will always need to be called now

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

# Additional Information
* See #5729 for more information (and the TTS fixes)
Update torch.stft() calls due to deprecation of return_complex=False
State: closed | Pull request: yes
Labels: name: TTS

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

# What does this PR do ?

Fixes errors due to deprecation of `return_complex=False` in `torch.stft()` in version 1.14.

Most of our calls implicitly use `return_complex=False` (was the default), which returns a tensor of shape [B, N, T, 2] where the last dimension captures the real and imaginary parts of the tensor. However, moving forward, pytorch is deprecating this in favor of the behavior of `return_complex=True`, which returns a complex tensor of shape [B, N, T].

This bugfix explicitly sets `return_complex=True` as required by v1.14, and then converts the output of `torch.stft()` back to [B, N, T, 2] using `torch.view_as_real()`.

Similarly, `torch.istft()` now wants a complex tensor input; this is fixed by calling `torch.view_as_complex()` on our original input.

**Collection**: TTS

# Changelog 
- `torch.stft()` calls explicitly set `return_complex=True` and then convert output where necessary
- Added `torch.view_as_complex()` to input of `torch.istft()`

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
ASR evaluator
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

A tool to thoroughly evaluate an ASR model.

**Collection**: ASR

# Changelog 
- Simple step to evaluate a model in all three modes currently supported by NeMo: offline, chunked, and offline_by_chunked.
- On-the-fly data augmentation (silence, noise, etc.,) for ASR robustness evaluation.
- Investigate the model's performance by detailed insertion, deletion, and substitution error rates for each and all samples.
- Evaluate models' reliability on different target groups such as gender, and audio length if metadata is presented.

# Usage

```
python asr_evaluator.py \
engine.pretrained_name="stt_en_conformer_transducer_large" \
engine.inference_mode.mode="offline" \
engine.test_ds.augmentor.noise.manifest_path=<manifest file for noise data>
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
Note because the PR is getting bigger, on-the-fly augmentation for buffered inference mode would be included in another PR soon.
[ASR] Use a subset of manifest when using `scatter` shard strategy
State: closed | Pull request: yes
Labels: name: ASR

name: common

# What does this PR do ?

When using `shard_strategy='scatter'` this PR loads only a subset of lines from the manifest file.
This may reduce the time to process a manifest file by an order of magnitude.

Opening as a draft to get feedback if there are any underlying assumptions which may be broken by this change.

**Collection**: ASR

# Changelog 
| File                      | Change |
| ---------------- | -------- |
| `manifest.py::item_iter` | Load only a subset of lines if `shard_strategy == 'scatter'` |
| `collections.py:: ASRAudioText ` | forward `shard_strategy`, `global_rank` and `world_size` to `manifest.item_iter`  |
| `collections.py:: AudioText ` | Use rank and world size to restore the original data list length for `shard_strategy='scatter'` |
|  `audio_to_text.py` | forward `shard_strategy`, `global_rank` and `world_size` to `collections.ASRAudioText` |
| `utils,py` | Added a function to get number of lines from a text file |
| `test_utils.py` | Added unit test for the utility function added above |


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Hard coded padding index for embeddings of transformer models
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

In `nemo/collections/nlp/modules/common/transformer/transformer_modules.py` a hard coded `padding_idx` is set (value 0):
https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/nlp/modules/common/transformer/transformer_modules.py#L112

According to the [documentation](https://pytorch.org/docs/stable/generated/torch.nn.Embedding.html) of `nn.Embedding` this will set the respective column to zeros on initialization. Given that the padding may be at different indices for different models (i.e. tokenizers) this should be set accordingly, and not be hard coded to index 0.

I'm also not sure if the statement from the documentation holds true for NeMo trained models:
> [...] the embedding vector at padding_idx is not updated during training, [...]

as the pertained ende model (`nmt_en_de_transformer12x2.nemo`) has non-zero weights at index 0:
```
>>> checkpoint = torch.load("model_weights.ckpt", map_location="cpu")
>>> checkpoint["decoder._embedding.token_embedding.weight"][0]
tensor([-0.0071, -0.0716,  0.0123,  ...,  0.0321,  0.0360,  0.0308])
```

I could also reproduce this with a custom trained model.

**Steps/Code to reproduce bug**

1. Observe that `padding_idx` is set to a hard coded value.
2. Observe that none of the columns of the embeddings are all zeros in the pertained MTEncDec models.

**Expected behavior**

1. `padding_idx` gets set according to the index of `<pad>` in the respective tokenizer.
2. The column at `padding_idx` does not get trained/updated during training.

**Environment overview (please complete the following information)**

 - Any

**Environment details**

- Any

**Additional context**

- Potential fix: https://github.com/NVIDIA/NeMo/pull/5543
Don't print exp_manager warning when max_steps == -1
State: closed | Pull request: yes

# What does this PR do ?

Doesn't print a warning when max_steps == -1 and val monitoring. For example, when we just provide max_epochs we don't want to see that warning.

```
[NeMo W 2023-01-03 15:47:31 exp_manager:988] The checkpoint callback was told to monitor a validation value and trainer's max_steps was set to -1. Please ensure that max_steps will run for at least 1 epochs to ensure that checkpointing will not error out.
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
Ensure EMA checkpoints are also deleted when normal checkpoints are
State: closed | Pull request: yes
Labels: name: bug

name: common

# What does this PR do ?

Closes #5631

**Collection**: Common

# Changelog 
- Delete EMA checkpoints when normal checkpoints are deleted.


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
[pre-commit.ci] pre-commit suggestions
State: closed | Pull request: yes
Labels: name: core

name: TTS

name: ASR

name: NLP

name: Speaker Tasks

name: common

<!--pre-commit.ci start-->
updates:
- [github.com/pre-commit/pre-commit-hooks: v4.3.0 → v4.4.0](https://github.com/pre-commit/pre-commit-hooks/compare/v4.3.0...v4.4.0)
- [github.com/PyCQA/isort: 4.3.21 → 5.11.4](https://github.com/PyCQA/isort/compare/4.3.21...5.11.4)
- [github.com/psf/black: 19.10b0 → 22.12.0](https://github.com/psf/black/compare/19.10b0...22.12.0)
<!--pre-commit.ci end-->
pin torchmetrics version
State: closed | Pull request: yes
Labels: name: ASR

name: CI

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

# What does this PR do ?

Fix torchmetrics version, as latest version has issues with pytorch Dataloader loading time. 

**Collection**: ASR

# Changelog 
- pin torchmetrics version to 0.10.3 
- revert accuracy API to 0.10.3 from 0.11.0 
- revert Speaker Jenkins run

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Add extra data args to support proper finetuning of HF converted T5 checkpoints
State: closed | Pull request: yes
Labels: name: NLP

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Adds args that can configure whether to add bos/eos to the encoder and to replace bos with the pad token for HF converted T5 checkpoints.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Univnet model
State: closed | Pull request: no
Labels: name: bug

name: TTS

name: stale

Hi. I have trained univnet model and I faced 2 issues. 
1. model has memory leak 
2. high response time

I attempt to convert model to onnx but I got this error:
`torch.onnx.errors.SymbolicValueError: Unsupported: ONNX export of operator Unfold, input size not accessible. Please feel free to request support or submit a pull request on PyTorch GitHub: https://github.com/pytorch/pytorch/issues [Caused by the value '381 defined in (%381 : Float(*, *, *, strides=[65600, 2050, 1], requires_grad=0, device=cuda:0) = onnx::Pad[mode="constant"](%output, %379, %380), scope: nemo.collections.tts.models.univnet.UnivNetModel::/nemo.collections.tts.modules.univnet_modules.Generator::generator/nemo.collections.tts.modules.univnet_modules.LVCBlock::res_stack.0
`

do you have any idea to fix this problem?Is fixing the memory leak problem on your plan?
@redoctopus
Hungarian TN (cleaned)
State: closed | Pull request: yes

# What does this PR do ?

As with #5715 creating to show provenance + close

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Hungarian TN
State: closed | Pull request: yes

# What does this PR do ?

As with #5715 creating to show provenance + close

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[in progress] Add Swedish TN
State: closed | Pull request: yes

# What does this PR do ?

I've just seen that the text processing module has moved; this is messy, includes pieces that ought to be in other places, and just generally needs a cleanup. The main point of making the pull request is to show that I didn't just pluck this stuff from thin air.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [X] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
stt_zh_citrinet_1024_gamma_0_25 will not be found in ASRDecoderTimeStamps
State: closed | Pull request: no
Labels: name: bug

name: stale

Hi Nemo developers,

**Describe the bug**
I tried to run tutorials/speaker_tasks/ASR_with_SpeakerDiarization.ipynb with chinese data. When I changed the model from `cfg.diarizer.asr.model_path = 'stt_en_conformer_ctc_large'` to `stt_zh_citrinet_1024_gamma_0_25`, the model could not be found correctly because of the logic in  https://github.com/NVIDIA/NeMo/blob/58d6765929d7c13ab933640d74630493957b1dfb/nemo/collections/asr/parts/utils/decoder_timestamps_utils.py#L331-L356.

According to current logic, as `stt_zh_citrinet_1024_gamma_0_25` contains citrinet it will go to 
https://github.com/NVIDIA/NeMo/blob/58d6765929d7c13ab933640d74630493957b1dfb/nemo/collections/asr/parts/utils/decoder_timestamps_utils.py#L350-L356
the `self.encdec_class` will be `EncDecCTCModelBPE`. 

However the `stt_zh_citrinet_1024_gamma_0_25` is listed here.
https://github.com/NVIDIA/NeMo/blob/58d6765929d7c13ab933640d74630493957b1dfb/nemo/collections/asr/models/ctc_models.py#L818-L822
Belongs to `EncDecCTCModel`.

I create a PR(https://github.com/NVIDIA/NeMo/pull/5712) that will not solve the issue but could express my point more clearly.

BRs,
Qxxxx
Fix zh_citrinet not found
State: closed | Pull request: yes
Labels: name: stale

name: ASR

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update libraries on Tutorial - Cython, gcc, g++
State: closed | Pull request: no

Hi.

I'm new on this field and I was going to follow [this tutorial](https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/ASR_with_NeMo.ipynb) step by step.

I just made new environment using conda on my linux server and stuck with issue below.

<img width="800" alt="image" src="https://user-images.githubusercontent.com/49643709/209797058-49fa20b2-21bc-476b-929c-a2479512ac36.png">

I installed cython and stuck again like below.

<img width="800" alt="image" src="https://user-images.githubusercontent.com/49643709/209798306-f7250507-2031-4e57-9b9d-d6f6642f7964.png">

I run `apt-get install gcc` and stuck again with issue below.

<img width="800" alt="image" src="https://user-images.githubusercontent.com/49643709/209798768-a3cedfc7-a34c-411a-a13a-e16ed907c311.png">

I run `apt-get install g++` and install just works well.

For newbie like me, it would be more great to prevent someone is getting stuck at installation.

So how about adding this commands?

```
pip install cython
apt-get install gcc
apt-get install g++
```
Can't pull docker image
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

According to the article by nvidia published [here](https://developer.nvidia.com/blog/deploying-a-1-3b-gpt-3-model-with-nvidia-nemo-megatron/), mentions the use of a couple of docker images 
```
nvcr.io/ea-bignlp/bignlp-training:22.08.01-py3
nvcr.io/ea-bignlp/bignlp-inference:22.08-py3
```
in order to use or inference on LLM. But while pulling the images I am encountering the access denied error.
**This is still the condition even after successfully logging in to nvcr.io**


**Steps/Code to reproduce bug**

docker login nvcr.io
Username: $oauthtoken
Password: <**your SECRET KEY**>
```
docker pull nvcr.io/ea-bignlp/bignlp-training:22.08.01-py3
docker pull nvcr.io/ea-bignlp/bignlp-inference:22.08-py3
```

**Expected behavior**
The docker should start pulling the images.
Merge 1.14.0 main
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Nemo with multiprocessing
State: closed | Pull request: no

Hello developers.

I have a lot of useful data and would like to retrieve embeddings and get diarization

I thought to use a multiprocessing module to speed up the process. [This module](https://docs.python.org/3/library/multiprocessing.html)
But I failed. If I make a processing pool and add tasks after a while processes stop, and it seems like processes lock themselves
I tried to set up _set_start_method('spawn')_  and it didn't work out, the exception occured

Is there a possibility to use Nemo with a multiprocessing module, has someone had such an experience?

I'll appreciate any help!
The predicted output remains empty during training. Training loss is decreasing but validation loss is constant
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

I am training stt_en_conformer_ctc_large on my custom datset+librispeech dev and test splits. Currently, half of epoch 0 has completed but the best value of WER is 1 and logs show epty in predicted. I am a beginner and do not understand the ASR process fully so it must be something that I am doing wrong. Can someone help me?. I am tokenizing using SPE with a vocab size of 15000, and am using the default training script from
```
https://github.com/NVIDIA/NeMo/blob/main/examples/asr/asr_ctc/speech_to_text_ctc_bpe.py
```
and the config from 
```
https://github.com/NVIDIA/NeMo/blob/main/examples/asr/conf/conformer/conformer_ctc_bpe.yaml
``` 
with some fields changed according to my need

**Steps/Code to reproduce bug**

Running the above training script with the config file which contains all the relevant path information.

**Expected behavior**

At least something should be predicted in logs

**Environment overview (please complete the following information)**

 - Environment location: AWS
 - Method of NeMo install: [pip install nemo_toolkit["all"]

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version Ubuntu 20.04
- PyTorch version 11.7
- Python version 3.8

**Additional context**

Add any other context about the problem here.
Tesla T4
No confidence score for megatron outputs?
State: closed | Pull request: no
Labels: name: bug

**The outputs form the megatron generations has no scoring method**

For megatron models when `model.generate() `function is called, results returned doesn't contain any scores which could give the confidence of generation.

**Here's a code snippet**

```
response = model.generate(inputs=test_examples, length_params=None)
print(response.keys()
```

Is there a way that could validate these generations from model.
ModuleNotFoundError: No module named 'nemo.utils.data_utils'
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

Whenever I try to run the file for tokenization I get the above error.

**Steps/Code to reproduce bug**

Run the command
```
python ./scripts/tokenizers/process_asr_text_tokenizer.py --manifest=../code/manifests/train_clean_5.json --data_root=tokenized --vocab_size=2048 tokenizer=bpe --log
```


**Expected behavior**

It should have tokenized the text manifest.

**Environment overview (please complete the following information)**

 - Environment location: AWS
 - Method of NeMo install: pip install nemo_toolkit['all']
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version Ubutnu 20
- PyTorch version 1.6
- Python version 3.8

**Additional context**

Add any other context about the problem here.
Example: GPU model
asr报错
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

调用stt_zh_citrinet_512模型进行音频转文字操作

**Steps/Code to reproduce bug**

asr_model = nemo_asr.models.EncDecCTCModel.from_pretrained(model_name="stt_zh_citrinet_512")
mandarin_text=asr_model.transcribe(paths2audio_files=["./test.wav"])
print(mandarin_text)**

**Expected behavior**

输出音频文本内容

**Environment overview (please complete the following information)**
安装方式为pip，命令为pip install nemo_toolkit['all']  -i https://pypi.doubanio.com/simple/
**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- torch1.10.0+cu113
- Python 3.8

**Additional context**
报错ValueError: Input signal length=2 is too small to resample from 44100->16000
![image](https://user-images.githubusercontent.com/39375578/209262117-6888b66e-d0b2-4af6-8782-451e325ddf76.png)
temp disable speaker recognision CI test
State: closed | Pull request: yes
Labels: name: CI

Signed-off-by: fayejf <fayejf07@gmail.com>

# What does this PR do ?

temporary disable speaker recognision CI test due to long duration (~20M)

**Collection**: ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Error in downloading the financial_phrase_bank dataset
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

I was trying to run "tutorials/nlp/Multitask_Prompt_and_PTuning.ipynb". But I got the following error for downloading the "financial_phrase_bank" dataset.

https://www.researchgate.net/profile/Pekka_Malo/publication/251231364_FinancialPhraseBank-v1.0/data/0c96051eee4fb1d56e000000/FinancialPhraseBank-v1.0.zip
Resolving [www.researchgate.net](http://www.researchgate.net/) ([www.researchgate.net](http://www.researchgate.net/))... 104.17.32.105, 104.17.33.105, 2606:4700::6811:2169, ...
Connecting to [www.researchgate.net](http://www.researchgate.net/) ([www.researchgate.net)|104.17.32.105|:443](http://www.researchgate.net%29%7C104.17.32.105%7C:443/)... connected.
HTTP request sent, awaiting response... 403 Forbidden
2022-12-21 22:47:57 ERROR 403: Forbidden.

**Steps/Code to reproduce bug**

I tried to download the dataset in both my local machine and Google Colab. But both cases failed. 

**Expected behavior**

I expected the dataset to be downloaded without any error.

**Environment overview (please complete the following information)**

 - Environment location: Colab, MacOS
 - Method of NeMo install: I uploaded the notebook on Colab
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version
- PyTorch version
- Python version

**Additional context**

Add any other context about the problem here.
Example: GPU model
Update to pytorch 22.12 container
State: closed | Pull request: yes
Labels: name: TTS

name: ASR

name: CI

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix for incorrect computation of batched alignment in transducers
State: closed | Pull request: yes
Labels: name: ASR

name: CI

# What does this PR do ?

The original code had a bug because of which whenever at least one batch element predicted non-blank symbol, all blank elements would duplicate the last prediction in their `.alignments` variable.

Additionally, I've unified `_greedy_decode_blank_as_pad` and `_greedy_decode_masked` into a single function with an if check. The two methods had 99% of the same code except 5 lines or so, so imo it does not make sense to have them as fully separate code paths. Please let me know if for some reason we don't want to have this change.

**Collection**: ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix missing torchelastic fixes for PTL 1.8
State: closed | Pull request: yes
Labels: name: NLP

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Fixes missing torchelastic import

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Remove cell output from tutorial
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

Signed-off-by: ericharper <complex451@gmail.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Remove cell output from tutorial
State: closed | Pull request: yes
Labels: name: NLP

Signed-off-by: ericharper <complex451@gmail.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Trainer is required in EncDecRNNTModel
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

In the constructor of the class `EncDecRNNTModel`, the PTL trainer is a kwarg with default value `null`. However, in the `self.training_step` method, the method `self.log_dict` is called (line 725 in file `collections/asr/models/rnnt_models.py`), which raises an error if no PTL trainer object is present in the `EncDecRNNTModel` object.

**Expected behavior**

If no PTL trainer is passed to the `EncDecRNNTModel`, it should not call `self.log` and just return the logs. The fix is just an if statement before the `self.log_dict`
Using nemo LM in other nemo training 
State: closed | Pull request: no
Labels: name: bug

name: stale

I want to use our own training nemo LM in other nemo training and  I found only this  reference  [https://github.com/NVIDIA/NeMo/blob/4cd9b3449cbfedc671348fbabbe8e3a55fbd659d/examples/nlp/text_classification/conf/ptune_text_classification_config.yaml](url) that have nemo_path in language model config

so I follow that(training text classification) but it doesn't work, it has to have model.tokenizer.tokenizer_name

> [NeMo I 2022-12-20 16:02:14 exp_manager:563] TensorboardLogger has been set up
> Traceback (most recent call last):
>   File "/workspace/data/bew/NeMo/examples/nlp/text_classification/text_classification_with_bert.py", line 117, in main
>     model = TextClassificationModel(cfg.model, trainer=trainer)
>   File "/opt/conda/lib/python3.8/site-packages/nemo/collections/nlp/models/text_classification/text_classification_model.py", line 54, in __init__
>     self.setup_tokenizer(cfg.tokenizer)
>   File "/opt/conda/lib/python3.8/site-packages/nemo/collections/nlp/models/nlp_model.py", line 112, in setup_tokenizer
>     tokenizer_name=cfg.tokenizer_name,
> omegaconf.errors.ConfigAttributeError: Key 'tokenizer_name' is not in struct
>         full_key: model.tokenizer.tokenizer_name
>         reference_type=Any
>         object_type=dict
> 
> Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

If I mention model.tokenizer.tokenizer_name and model.language_model.pretrained_model_name then put model.language_model.nemo_file(path to our custom nemo model), the training look ok but it seems to use the model from huggingface.

Is there a way to using custom LM model?
Add option to disable distributed parameters in distributed Adam optimizer
State: closed | Pull request: yes
Labels: name: core

name: NLP

# What does this PR do ?

Add an option to disable distributed parameters in the distributed Adam optimizer. This is similar to standard distributed data-parallelism (i.e. with gradient all-reduces), but it takes advantage of the distributed optimizer's support for overlapping communication with backward compute.

**Collection**: NLP

# Changelog 
- Add option to disabled distributed parameters in the distributed Adam optimizer

# Usage

Set the optimizer to `distributed_fused_adam` with `disable_distributed_parameters: True` in the config file:

https://github.com/NVIDIA/NeMo/blob/80cd1c135f4cf01aa04e62aa4c60fb130c889831/examples/nlp/language_modeling/conf/megatron_gpt_config.yaml#L193

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Orthogonal to https://github.com/NVIDIA/NeMo/pull/5560 and https://github.com/NVIDIA/NeMo/pull/5684.
Distributed Adam optimizer overlaps param all-gather with forward compute
State: closed | Pull request: yes
Labels: name: core

name: NLP

name: CI

# What does this PR do ?

This PR adds experimental support in GPT for the distributed Adam optimizer to overlap its parameter all-gather with the forward compute. It integrates with features added in https://github.com/NVIDIA/apex/pull/1559.

This is done with a hacky monkey-patching approach, so long-term it may be better to transition toward PyTorch's FSDP. That said, I have seen some modest speedups (~2%) training GPT-3 175B on Selene with no difference in loss values.

**Collection**: NLP

# Changelog 
- Configure dist Adam so all model parameters are views into a large contiguous buffer.
- Add GPT support for overlapping forward compute and param all-gathers

# Usage

Set the optimizer to `distributed_fused_adam` in the config file:

https://github.com/NVIDIA/NeMo/blob/80cd1c135f4cf01aa04e62aa4c60fb130c889831/examples/nlp/language_modeling/conf/megatron_gpt_config.yaml#L193

The overlapped param all-gathers are enabled by default. To disable, configure the optimizer with `overlap_param_sync: False`.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Depends on https://github.com/NVIDIA/NeMo/pull/5560
* Depends on https://github.com/NVIDIA/apex/pull/1559, which in turn depends on https://github.com/NVIDIA/apex/pull/1559.
[TN/TTS docs] TN customization, g2p docs moved to tts
State: closed | Pull request: yes
Labels: name: TTS

name: NLP

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?

- added a doc on TN/ITN customization + resources
- moved G2P to TTS
- removed text_processing folder from the docs

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Peformance Measure inference after finetuning
State: closed | Pull request: no
Labels: name: stale

I have finetuned the titanet_large model on a new dataset which is related to the work I will be performing the embedding extraction on. That dataset has been split into a test set which contains unseen speakers. I would like to see if the finetuned model is performing better on the unseen dataset as compared to the original titanet model. Any ideas in which I can compare results?

Also, is there any way I can create a trial file for my dataset?
Thanks!