Organizing the script folder
State: closed | Pull request: yes

Organized the scripts folder based on the task. This is an initial draft.

Suggestion to change the folder name from speaker_diarization to speaker_recognition in scripts.
I couldn't comment on unchanged files. Some links in speaker recognition notebooks point to these scripts. 
Some of them:

> in tutorials/Speaker_Recognition_Verification.ipynb

* get_hi-mia-data.py (https://github.com/NVIDIA/NeMo/blob/rir_noise_aug_2/scripts/get_hi-mia_data.py) script to download the necessary files -> get_hi-mia-data.py (https://github.com/NVIDIA/NeMo/blob/main/scripts/speaker_recognition/get_hi-mia_data.py) script to download the necessary files
* wget -P scripts https://raw.githubusercontent.com/NVIDIA/NeMo/$BRANCH/scripts/scp_to_manifest.py -> wget -P scripts https://raw.githubusercontent.com/NVIDIA/NeMo/$BRANCH/scripts/speaker_recogntion/scp_to_manifest.py

This pull request **introduces 4 alerts** when merging 26e75e512ebd029be9c09f92a61ca2ad7d0fb257 into 89a8ea037a121001e105e95b49d71906a8de29aa - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-18e7412bcf8ad035dda398a0b8f5fffef1c6d2db)

**new alerts:**

* 4 for Unused import

I've checked scripts and tutorials for speech command, VAD and speaker diarization. 

Could you please change
1.  **NeMo/scripts/** in  examples/asr/vad_infer  to **NeMo/scripts/voice_activity_detection**
2. multiple **<NEMO_ROOT>/scripts** in examples/asr/speech_to_label.py to correposinding subfolders.
3. in tutorials/asr/06 VAD: scripts/voice_activity_recognition/process_vad_data -> scripts/**dataset_processing**/process_vad_data
4. in tutorials/speaker_recognition/Speaker_Diarization_Inference.ipynb: "you can also tune the threshold on your dev set. Use this provided script" fix the link of the **script**. It would be much appreciated if you can also help change the link to https://github.com/NVIDIA/NeMo/blob/r1.0.0rc1/nemo/collections/asr/parts/vad_utils.py in the above line. 

Thanks!


> 
> 
> I've checked scripts and tutorials for speech command, VAD and speaker diarization.
> 
> Could you please change
> 
>     1. **NeMo/scripts/** in  examples/asr/vad_infer  to **NeMo/scripts/voice_activity_detection**
> 
>     2. multiple **<NEMO_ROOT>/scripts** in examples/asr/speech_to_label.py to correposinding subfolders.
> 
>     3. in tutorials/asr/06 VAD: scripts/voice_activity_recognition/process_vad_data -> scripts/**dataset_processing**/process_vad_data
> 
>     4. in tutorials/speaker_recognition/Speaker_Diarization_Inference.ipynb: "you can also tune the threshold on your dev set. Use this provided script" fix the link of the **script**. It would be much appreciated if you can also help change the link to https://github.com/NVIDIA/NeMo/blob/r1.0.0rc1/nemo/collections/asr/parts/vad_utils.py in the above line.
> 
> 
> Thanks!

Thanks for checking them out. I missed them.
1-done
2-done
3-done
4-Please check it again to make sure I put the right links. We need to change them to main after the release?

> > I've checked scripts and tutorials for speech command, VAD and speaker diarization.
> > Could you please change
> > ```
> > 1. **NeMo/scripts/** in  examples/asr/vad_infer  to **NeMo/scripts/voice_activity_detection**
> > 
> > 2. multiple **<NEMO_ROOT>/scripts** in examples/asr/speech_to_label.py to correposinding subfolders.
> > 
> > 3. in tutorials/asr/06 VAD: scripts/voice_activity_recognition/process_vad_data -> scripts/**dataset_processing**/process_vad_data
> > 
> > 4. in tutorials/speaker_recognition/Speaker_Diarization_Inference.ipynb: "you can also tune the threshold on your dev set. Use this provided script" fix the link of the **script**. It would be much appreciated if you can also help change the link to https://github.com/NVIDIA/NeMo/blob/r1.0.0rc1/nemo/collections/asr/parts/vad_utils.py in the above line.
> > ```
> > 
> > 
> > Thanks!
> 
> Thanks for checking them out. I missed them.
> 1-done
> 2-done
> 3-done
> 4-Please check it again to make sure I put the right links. We need to change them to main after the release?

The **here** one should be https://github.com/NVIDIA/NeMo/blob/r1.0.0rc1/nemo/collections/asr/parts/vad_utils.py
The **script** one should be https://github.com/NVIDIA/NeMo/blob/r1.0.0rc1/scripts/voice_activity_detection/vad_tune_threshold.py

Yes after the release. 
Thanks.

I see you are merging to rc1 branch but links are pointed to main. If we test before merging to main, notebook tutorials gonna fail 

> 
> 
> I see you are merging to rc1 branch but links are pointed to main. If we test before merging to main, notebook tutorials gonna fail

It looks tricky. I tried not to change those parts. The code should work with both actually as RC is merged into main everyday. So if it is based on main, it was going to get tested with main anyway and this PR is not going to affect the results. Should I change all to main or RC?

> > I see you are merging to rc1 branch but links are pointed to main. If we test before merging to main, notebook tutorials gonna fail
> 
> It looks tricky. I tried not to change those parts. The code should work with both actually as RC is merged into main everyday. So if it is based on main, it was going to get tested with main anyway and this PR is not going to affect the results. Should I change all to main or RC?

Actually I just see [one notebook](https://github.com/VahidooX/NeMo/blob/organize_scripts/tutorials/speaker_recognition/Speaker_Recognition_Verification.ipynb) script (get_hi-mia-data.py) to main branch, but rest of notebooks looks good. Also its path should be changed to `dataset_processing` not `speaker_recognition`. Just this rest all looks good to me. I vote main since for later releases we don't need to change again. 

> 
> 
> > > I see you are merging to rc1 branch but links are pointed to main. If we test before merging to main, notebook tutorials gonna fail
> > 
> > 
> > It looks tricky. I tried not to change those parts. The code should work with both actually as RC is merged into main everyday. So if it is based on main, it was going to get tested with main anyway and this PR is not going to affect the results. Should I change all to main or RC?
> 
> Actually I just see [one notebook](https://github.com/VahidooX/NeMo/blob/organize_scripts/tutorials/speaker_recognition/Speaker_Recognition_Verification.ipynb) script (get_hi-mia-data.py) to main branch, but rest of notebooks looks good. Also its path should be changed to `dataset_processing` not `speaker_recognition`. Just this rest all looks good to me. I vote main since for later releases we don't need to change again.

Thanks for catching them! Updated those. I also prefer main, but for now, I made them all RC as we are need to change others to master later anyway.