renwei2024/nemo-github-issues huggingface dataset:
[TTS][zh] refine hardcoded lowercase for ASCII letters.
State: closed | Pull request: yes
Labels: name: TTS

name: common

# What does this PR do ?

1. relaxed the hardcoded lowercase for non-Chinese characters. This PR provided customization for tokenization/g2p by introducing `ascii_letter_prefix` and `ascii_letter_case` arguments. These flexible arguments helped to discriminate Chinese phonemes from ascii letters. 
2. avoided to set up cases during the initialization of tokenizer because G2P should take good care of cases, mostly because tokenizer is instantiated after g2p according to [`def _setup_tokenizer()`](https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/tts/models/fastpitch.py#L231).
3. Refined docstring for g2p and tokenizer.
4. Changes will not break pre-trained NGC models because all nightly tests are passed.
```
$ pytest --nightly tests/collections/tts/models
===================================================================================== 27 passed, 3 warnings in 330.07s (0:05:30) =====================================================================================
```

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Tensor-parallel communication overlap with userbuffer backend
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add (1) interfaces to TE and initialized (2) process group setting to support tensor-parallel communication overlap with userbuffer backend.

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
Set `ub_tp_comm_overlap` to `True`

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [*] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
add call to p2p overlap
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

add call to p2p overlap 

**Collection**: NLP
# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix Links for tutorials
State: closed | Pull request: yes
Labels: name: cherry-pick

# What does this PR do ?

Fix URL

**Collection**: [Tools]


# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Fix Links for tutorials
State: closed | Pull request: yes

# What does this PR do ?

Fix URL

**Collection**: [Tools]


# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Fix adapter tutorial r1.19.0
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Fix adapter tutorial to compatible with latest pre-traiend FastPitch checkpoint

**Collection**: [TTS]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Spec aug fix
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?
Minor fix that stops `MaskedSpecAug` from overestimating sampling populations.

Add a one line overview of what this PR aims to accomplish.
`MaskedSpecAug` was overestimating the number of available patches when sampling patches were larger than audio size. This should fix it that sampling errors aren't thrown anymore.

**Collection**: [Note which collection this PR will affect]
ASR

# Changelog 
- Altered sample estimation in `MaskedSpecAug`

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [Y ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ N] Did you write any new necessary tests?
- [ N] Did you add or update any necessary documentation?
- [ N] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [Y ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Updating label models with new langid model.
State: closed | Pull request: yes
Labels: name: stale

name: ASR

# What does this PR do ?

Updates `label_models` with new LangID model (PearlNet) for ICASSP.

**Collection**: [Note which collection this PR will affect]
ASR
# Changelog 
- Added new ngc import to `list_available_models`

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [Y ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ N] Did you write any new necessary tests?
- [ Y] Did you add or update any necessary documentation?
- [ N] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ Y] New Feature
- [ ] Bugfix
- [ Y] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
text_generation_utils memory reduction if no logprob needed
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

make text_generation_utils.py more memory efficient for inference especially for long context sequences 
* by making logprob computation optional

**Collection**: nlp

# Changelog 
- change peft eval model class for restoring to MegatronGPTSFTModel, since NLPModel does not have _build_model
- removing logprob computation in text_generation_utils since this is very memory consuming because output embedding are all gathered for this. reusing parameter compute_logprob to check if this is needed, by default false

# Usage

```python
 python examples/nlp/language_modeling/megatron_gpt_eval.py \
    ++trainer.devices=${DEVICES} \
    gpt_model_file=$CKPT \
    trainer.num_nodes=1 \
    inference.greedy=True \
    pipeline_model_parallel_size=$PP \
    tensor_model_parallel_size=$TP \
    trainer.precision=bf16 \
    inference.compute_logprob=True \

python examples/nlp/language_modeling/tuning/megatron_gpt_peft_eval.py \
 inference.compute_logprob=True ..
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Langid update
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Updates `label_models` with new ngc model (PearlNet). For ICASSP.

**Collection**: [Note which collection this PR will affect]
ASR
# Changelog 
- Added new ngc model to `label_models.list_available_models`

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [Y ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [N] Did you write any new necessary tests?
- [Y] Did you add or update any necessary documentation?
- [N] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [Y] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [Y] New Feature
- [N] Bugfix
- [N] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix checkpointed forward and add test for full activation checkpointing
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

name: CI

# What does this PR do ?

Fixes checkpointed forward and changes GPT TP test to use activation recomputation for 1 layer instead of SAR

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Bug fix to reset sequence parallelism
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?
Fixes sequence parallelism reset function to set the value to False instead of None.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
FixTextMemMapDataset index file creation in multi-node setup
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Fixes issue #6715 by creating index files on all processes with local_rank 0.

**Collection**: NLP

# Changelog 
- Update rank condition to create index files when local_rank is 0 (formerly when global_rank was 0).

# Usage
* No change in used.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to #6715
lora notebook
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix get_parameters when using main params optimizer
State: closed | Pull request: yes
Labels: name: core

name: NLP

# What does this PR do ?

When using fused_adam, self._optimizer.get_parameters() was returning an empty list.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
add support for max_total_length=4096 for 43b
State: closed | Pull request: yes
Labels: name: stale

# What does this PR do ?

add support for max_total_length=4096 for 43b customization

**Collection**: nlp

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[Bug Fix] Fix fast tokenizer
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

Currently the arguments for fast tokenizer are not properly passed through so it cannot be used

**Collection**: [NLP]

# Changelog 
This PR just properly passes the use_fast argument

# Usage

use_fast: True in the config


**PR Type**:
- [ ] Bugfix


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Lddl bert
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

This adds the ability to use LDDL dataloaders in Nemo FW

**Collection**: [NLP, NemoFW

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage

add LDDL as dataloader in config. Instructions will be added to the Megatron Launcher for data prep and use.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
-  New Feature



## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
How can I export titanet-l model to onnx?
State: closed | Pull request: no

Hello, I have exported the marblenet vad model to onnx using the https://github.com/NVIDIA/NeMo/blob/stable/tutorials/asr/Online_Offline_Microphone_VAD_Demo.ipynb tutorial. 

Unfortunately, when I try to replicate the steps for the titanet model in the _extract_embeddings function, but it doesn't work.


Is there a tutorial about exporting the speaker model to onnx, or is it possible to export ?

```
    def _extract_embeddings(self, manifest_file: str, scale_idx: int, num_scales: int):
        """
        This method extracts speaker embeddings from segments passed through manifest_file
        Optionally you may save the intermediate speaker embeddings for debugging or any use. 
        """
        self._setup_spkr_test_data(manifest_file)
        self.embeddings = {}
        self._speaker_model = self._speaker_model.to(self._device)
        self._speaker_model.eval()
        self.time_stamps = {}
        
        self.speaker_model.export('titanet_l.onnx')
        ort_session = onnxruntime.InferenceSession('titanet_l.onnx')
        
        all_embs = torch.empty([0])
        for test_batch in tqdm(self._speaker_model.test_dataloader(), desc=f'[{scale_idx+1}/{num_scales}] extract embeddings',leave=True):
            test_batch = [x.to(self._device) for x in test_batch]
            processed_signal, processed_signal_len = self.speaker_model.preprocessor(input_signal=test_batch[0],length=test_batch[1])
            with autocast():
                
                #_, embs = self._speaker_model.forward(input_signal=test_batch[0], input_signal_length=test_batch[1])
                ort_inputs = {ort_session.get_inputs()[0].name: to_numpy(processed_signal), }
                _,ologits = ort_session.run(None, ort_inputs)
                alogits = np.asarray(ologits)
                logits = torch.from_numpy(alogits[0])
                emb_shape = ologits.shape[-1]
                embs = embs.view(-1, emb_shape)
                all_embs = torch.cat((all_embs, embs.cpu().detach()), dim=0)
                
            del test_batch```

```[tasklist]
### Tasks
```
The code in the notebook is not working
State: closed | Pull request: no
Labels: name: bug

name: stale

Hi,

I tried to apply the code in your notebook called (Online_ASR_Microphone_Demo.ipynb) [link](https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/Online_ASR_Microphone_Demo.ipynb)

it works fine when using the same model in the notebook ('QuartzNet15x5Base-En')
using this code : 
`asr_model = nemo_asr.models.EncDecCTCModel.from_pretrained('QuartzNet15x5Base-En')`

when i replaced the model with another model called "Conformer-CTC-L_spe128_ar-AR_3.0.nemo" which i downloaded from your website and i used it using this code:
`asr_model = nemo_asr.models.ASRModel.restore_from(restore_path="Conformer-CTC-L_spe128_ar-AR_3.0.nemo")`

the cell called (Streaming Inference)  keep showing errors
 cell:
```
# duration of signal frame, seconds
FRAME_LEN = 1.0
# number of audio channels (expect mono signal)
CHANNELS = 1

CHUNK_SIZE = int(FRAME_LEN*SAMPLE_RATE)
asr = FrameASR(model_definition = {
                   'sample_rate': SAMPLE_RATE,
                   'AudioToMelSpectrogramPreprocessor': cfg.preprocessor,
                   'JasperEncoder': cfg.encoder,
                   'labels': cfg.decoder.vocabulary
               },
               frame_len=FRAME_LEN, frame_overlap=2, 
               offset=4)

```
error:
```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[84], line 7
      4 CHANNELS = 1
      6 CHUNK_SIZE = int(FRAME_LEN*SAMPLE_RATE)
----> 7 asr = FrameASR(model_definition = {
      8                    'sample_rate': SAMPLE_RATE,
      9                    'AudioToMelSpectrogramPreprocessor': cfg.preprocessor,
     10                    'JasperEncoder': cfg.encoder,
     11 
     12                     #'JasperEncoder': cfg.encoder.jasper,
     13                    'labels': cfg.decoder.vocabulary
     14                },
     15                frame_len=FRAME_LEN, frame_overlap=2, 
     16                offset=4)

Cell In[79], line 27, in FrameASR.__init__(self, model_definition, frame_len, frame_overlap, offset)
     25 #for block in model_definition['JasperEncoder']['jasper']:
     26 for block in model_definition:
---> 27     timestep_duration *= block['stride'][0] ** block['repeat']
     28 self.n_timesteps_overlap = int(frame_overlap / timestep_duration) - 2
     29 self.buffer = np.zeros(shape=2*self.n_frame_overlap + self.n_frame_len,
     30                        dtype=np.float32)

TypeError: string indices must be integers

```
please hep me to solve this issue
Thanks.
TitaNet validation step too slow
State: closed | Pull request: no
Labels: name: bug

Hello, I'm training TitaNet from scratch using voxceleb 1 and 2 with another private dataset, while the training is fast 6it/s on a dual 3090 setup the valitadion step takes 2200s/it. It happens using all the labels and also with a limited set of labels (1024), or even with an4 dataset.

**Steps/Code to reproduce bug**

Followed this tutorial: https://github.com/NVIDIA/NeMo/blob/stable/tutorials/speaker_tasks/Speaker_Identification_Verification.ipynb and changed the data path.


 - Environment location: Local machine AMD® Ryzen 9 3900x  80GB 2x 3090
 - Method of NeMo install: from source, using this commit https://github.com/NVIDIA/NeMo/commit/c1abc047d1d5962019f7789b20d8092c7e55b690 tested with python 3.7 and 3.9 using conda enviroments

**Environment details**

- OS version: Ubuntu 22.04.2 LTS
- PyTorch version 11.7 11.8 slow with both versions
- Python version 3.7 3.9 slow with both versions

**Additional context**

It is slow with 1 GPU and 2GPUs, while in validation step the usage jumps from 10% to 20%.

Any other network like QuartzNetor Conformer works just fine.

¿Any ideas?
Bug fix to reset sequence parallelism
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?
Fixes sequence parallelism reset function to set the value to False instead of None.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Bug fix to restore act ckpt
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?
Fixes bug to reset/restore act ckpt in GPT models.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
function in VAD utils hardcoded to only use `.wav` files
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

[This line](https://github.com/NVIDIA/NeMo/blob/372f519ddba0c57272d190d83eaa3169ff6f3488/nemo/collections/asr/parts/utils/vad_utils.py#L1077) uses only `.wav` to split filepaths to create filenames. However if the files are not `.wav`, this will still run, but with wrong file names which causes later on in the [`vad_infer.py` ](https://github.com/NVIDIA/NeMo/blob/c1abc047d1d5962019f7789b20d8092c7e55b690/examples/asr/speech_classification/vad_infer.py#L4) file to not fine the current files at the last stage where we create the final manifest. Specifically, in [here](https://github.com/NVIDIA/NeMo/blob/c1abc047d1d5962019f7789b20d8092c7e55b690/examples/asr/speech_classification/vad_infer.py#L163), if the file was `.flac`, we will be looking for `utt.txt` whereas the actual file created by the previous steps will be named `utt.flac.txt`.

**Steps/Code to reproduce bug**

- Use non wav files as data.
- run vad_infer.py

**Expected behavior**

It should handle different file formats.

**Environment overview (please complete the following information)**

N/A

**Environment details**

N/A

**Additional context**

N/A
Bug fix to restore act ckpt
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?
Fixes bug to reset/restore act ckpt in GPT models.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Bug fix for activation checkpointing restore/reset
State: closed | Pull request: yes
Labels: name: core

name: TTS

name: ASR

name: NLP

name: CI

# What does this PR do ?
Fixes activation checkpointing restore/reset parameters.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
sharded manifests docs
State: closed | Pull request: yes
Labels: name: ASR

# What does this PR do ?

Adds documentation on the use of sharded manifests

**Collection**:  ASR

# Changelog 
- `docs/source/asr/datasets.rst` changed to add the docs

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ X] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Availability of Meeting MSDD models 
State: closed | Pull request: no
Labels: name: stale

The documentation (https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/asr/speaker_diarization/results.html) states the availability of the `diar_msdd_telephonic` model, while there is a config file for a meeting Neural Diarizer at `examples/speaker_tasks/diarization/conf/inference/diar_infer_meeting.yaml`. Could some help me in finding the weights for the meeting diarizer model, 

Thank You!
Upgrade pytorch container to 23.05
State: closed | Pull request: yes
Labels: name: CI

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix checkpointed forward and add test for full activation checkpointing
State: closed | Pull request: yes
Labels: name: NLP

name: CI

# What does this PR do ?

Fixes checkpointed forward and changes GPT TP test to use activation recomputation for 1 layer instead of SAR

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
check for first or last stage
State: closed | Pull request: yes
Labels: name: cherry-pick

name: NLP

# What does this PR do ?

Fix bug when `inference.compute_logprob=True`. Only first and last pipeline stages have a response.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix fastpitch test nightly
State: closed | Pull request: yes

# What does this PR do ?

Fix multi-speaker FastPitch test nightly

**Collection**: [TTS]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fix TTS adapter tutorial
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

Fix adapter tutorial to compatible with latest pre-traiend FastPitch checkpoint

**Collection**: [TTS]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Debug Transformer Engine FP8 support with Megatron-core infrastructure
State: closed | Pull request: yes
Labels: name: NLP

# What does this PR do ?

Debugs Transformer Engine FP8 support with Megatron-core infrastructure

**Collection**: NLP

# Changelog
- Construct FP8 amax reduction group 

# Usage

Set `transformer_engine: True` and `fp8: True` in the config.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

Pinging @aklife97 @yen-shi @erhoo82 @ksivaman 

# Additional Information
* Requires https://github.com/NVIDIA/Megatron-LM/commit/f267e6186eae1d6e2055b412b00e2e545a8e896a
* Fixes bug from https://github.com/NVIDIA/NeMo/pull/6393
* Subset of https://github.com/NVIDIA/NeMo/pull/6627
* Closes https://github.com/NVIDIA/NeMo/issues/6625
Vp fixes
State: closed | Pull request: yes
Labels: name: TTS

name: ASR

name: NLP

name: Speaker Tasks

name: CI

# What does this PR do ?

VP fixes cherry picked to release branch

**Collection**: [NLP]


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation