[WIP] Migrate SpeechLM to NeMo 2.0
State: open | Pull request: yes
Labels: name: ASR

name: NLP

name: common

name: Multi Modal

# What does this PR do ?

Migrate SpeechLM to NeMo 2.0


**Collection**: [asr,llm,slm,motimodal]

# Changelog 
- Add models, datamodules, training scripts



This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

<!-- pylint-output-strict-mode-true -->

beep boop 🤖: 🚨 The following files must be fixed before merge!

---

Your code was analyzed with PyLint. The following annotations have been identified:

```
************* Module nemo.collections.speechlm.data.audio_to_text_module
nemo/collections/speechlm/data/audio_to_text_module.py:66:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/speechlm/data/audio_to_text_module.py:70:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/speechlm/data/audio_to_text_module.py:74:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/speechlm/data/audio_to_text_module.py:78:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/speechlm/data/audio_to_text_module.py:130:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/speechlm/data/audio_to_text_module.py:33:0: W0611: Unused PromptFormatterTextProcessing imported from nemo.collections.multimodal.speech_llm.parts.utils.data_utils (unused-import)
************* Module audio_text_dataset
nemo/collections/speechlm/data/dataset/audio_text_dataset.py:500:0: C0301: Line too long (120/119) (line-too-long)
nemo/collections/speechlm/data/dataset/audio_text_dataset.py:983:0: C0301: Line too long (126/119) (line-too-long)
nemo/collections/speechlm/data/dataset/audio_text_dataset.py:445:0: C0115: Missing class docstring (missing-class-docstring)
nemo/collections/speechlm/data/dataset/audio_text_dataset.py:461:0: C0115: Missing class docstring (missing-class-docstring)
************* Module nemo.collections.speechlm.data.text_processing
nemo/collections/speechlm/data/text_processing.py:27:0: C0115: Missing class docstring (missing-class-docstring)
nemo/collections/speechlm/data/text_processing.py:55:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/speechlm/data/text_processing.py:89:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/speechlm/data/text_processing.py:93:4: C0116: Missing function or method docstring (missing-function-docstring)
************* Module nemo.collections.speechlm.recipes.pipeline
nemo/collections/speechlm/recipes/pipeline.py:68:0: C0301: Line too long (195/119) (line-too-long)
nemo/collections/speechlm/recipes/pipeline.py:35:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/speechlm/recipes/pipeline.py:107:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/speechlm/recipes/pipeline.py:167:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/speechlm/recipes/pipeline.py:16:0: W0611: Unused lightning.pytorch imported as pl (unused-import)
nemo/collections/speechlm/recipes/pipeline.py:17:0: W0611: Unused nemo_run imported as run (unused-import)
nemo/collections/speechlm/recipes/pipeline.py:23:0: W0611: Unused train imported from nemo.collections.speechlm.api (unused-import)
nemo/collections/speechlm/recipes/pipeline.py:23:0: W0611: Unused validate imported from nemo.collections.speechlm.api (unused-import)

-----------------------------------
Your code has been rated at 9.68/10
```

---

Mitigation guide:

* Add sensible and useful docstrings to functions and methods
* For trivial methods like getter/setters, consider adding `# pylint: disable=C0116` inside the function itself
* To disable multiple functions/methods at once, put a `# pylint: disable=C0116` before the first and a `# pylint: enable=C0116` after the last.

By applying these rules, we reduce the occurance of this message in future.

Thank you for improving NeMo's documentation!