Speech Regression Support
State: closed | Pull request: yes

Support for Encoder-Decoder models that learn to predict numeric values in a regression model. An example application is non-intrusive speech-quality assessment as presented here:

https://www.microsoft.com/en-us/research/uploads/prod/2019/04/preprint_ICASSP2019_MOS_estimation.pdf

Hi @titu1994. Well, this was the hard way I guess, but I think that the result is what I wanted. Please let me know how it's coming out

And error case is caught in the EncDecRegressionModel - 
```
=================================== FAILURES ===================================
_________________ TestEncDecRegressionModel.test_transcription _________________

self = <test_asr_regression_model.TestEncDecRegressionModel object at 0x7f3de006aac8>
speech_regression_model = EncDecRegressionModel(
  (preprocessor): AudioToMelSpectrogramPreprocessor(
    (featurizer): FilterbankFeatures()
  )...coder_layers): Sequential(
      (0): Linear(in_features=32, out_features=1, bias=True)
    )
  )
  (loss): MSELoss()
)
test_data_dir = '/home/mrjenkins/.jenkins/workspace/NeMo_PR-1707/tests/.data'

    @pytest.mark.unit
    def test_transcription(self, speech_regression_model, test_data_dir):
        # Ground truth labels = ["yes", "no"]
        audio_filenames = ['an22-flrp-b.wav', 'an90-fbbh-b.wav']
        audio_paths = [os.path.join(test_data_dir, "asr", "train", "an4", "wav", fp) for fp in audio_filenames]
        model = speech_regression_model.eval()

        # Test Top 1 classification transcription
        results = model.transcribe(audio_paths, batch_size=2)

>       assert len(results) == 2
E       assert 0 == 2
E         +0
E         -2
```

Hi again @titu1994. I hope that I've addressed all your concerns above

Ah and the error for this one is below.

The main cause is that we were loading up the labels and setting the len(self.labels) as the decoder num_classes inside the constructor before calling the decoder instantiation - so that users dont have to explicitly set the num classes inside the classification config - and it is infered from number of labels. Minor patch prior to `_setup_decoder()` should fix this - https://github.com/NVIDIA/NeMo/blob/f4fbe89e39047c49643a681452a08e2c15dfca15/nemo/collections/asr/models/classification_models.py#L474-L490

```
Traceback (most recent call last):
  File "examples/asr/speech_to_label.py", line 129, in <module>
    main()  # noqa pylint: disable=no-value-for-parameter
  File "/opt/conda/lib/python3.6/site-packages/nemo/core/config/hydra_runner.py", line 103, in wrapper
    strict=None,
  File "/opt/conda/lib/python3.6/site-packages/hydra/_internal/utils.py", line 347, in _run_hydra
    lambda: hydra.run(
  File "/opt/conda/lib/python3.6/site-packages/hydra/_internal/utils.py", line 201, in run_and_report
    raise ex
  File "/opt/conda/lib/python3.6/site-packages/hydra/_internal/utils.py", line 198, in run_and_report
    return func()
  File "/opt/conda/lib/python3.6/site-packages/hydra/_internal/utils.py", line 350, in <lambda>
    overrides=args.overrides,
  File "/opt/conda/lib/python3.6/site-packages/hydra/_internal/hydra.py", line 112, in run
    configure_logging=with_log_configuration,
  File "/opt/conda/lib/python3.6/site-packages/hydra/core/utils.py", line 127, in run_job
    ret.return_value = task_function(task_cfg)
  File "examples/asr/speech_to_label.py", line 117, in main
    asr_model = EncDecClassificationModel(cfg=cfg.model, trainer=trainer)
  File "/opt/conda/lib/python3.6/site-packages/nemo/collections/asr/models/classification_models.py", line 418, in __init__
    super().__init__(cfg=cfg, trainer=trainer)
  File "/opt/conda/lib/python3.6/site-packages/nemo/collections/asr/models/classification_models.py", line 70, in __init__
    self.decoder = self._setup_decoder()
  File "/opt/conda/lib/python3.6/site-packages/nemo/collections/asr/models/classification_models.py", line 430, in _setup_decoder
    return EncDecClassificationModel.from_config_dict(self._cfg.decoder)
  File "/opt/conda/lib/python3.6/site-packages/nemo/core/classes/common.py", line 263, in from_config_dict
    instance = hydra.utils.instantiate(config=config)
  File "/opt/conda/lib/python3.6/site-packages/hydra/utils.py", line 70, in call
    raise HydraException(f"Error calling '{cls}' : {e}") from e

hydra.errors.HydraException: Error calling 'nemo.collections.asr.modules.ConvASRDecoderClassification' : __init__() missing 1 required positional argument: 'num_classes'

script returned exit code 1
```

ready for review

The error is due to missing keys - following the review comments should resolve it
```
Traceback (most recent call last):

  File "examples/asr/speech_to_label.py", line 129, in <module>
    main()  # noqa pylint: disable=no-value-for-parameter
  File "/opt/conda/lib/python3.6/site-packages/nemo/core/config/hydra_runner.py", line 103, in wrapper
    strict=None,
  File "/opt/conda/lib/python3.6/site-packages/hydra/_internal/utils.py", line 347, in _run_hydra
    lambda: hydra.run(
  File "/opt/conda/lib/python3.6/site-packages/hydra/_internal/utils.py", line 201, in run_and_report
    raise ex
  File "/opt/conda/lib/python3.6/site-packages/hydra/_internal/utils.py", line 198, in run_and_report
    return func()
  File "/opt/conda/lib/python3.6/site-packages/hydra/_internal/utils.py", line 350, in <lambda>
    overrides=args.overrides,
  File "/opt/conda/lib/python3.6/site-packages/hydra/_internal/hydra.py", line 112, in run
    configure_logging=with_log_configuration,
  File "/opt/conda/lib/python3.6/site-packages/hydra/core/utils.py", line 127, in run_job
    ret.return_value = task_function(task_cfg)
  File "examples/asr/speech_to_label.py", line 117, in main
    asr_model = EncDecClassificationModel(cfg=cfg.model, trainer=trainer)
  File "/opt/conda/lib/python3.6/site-packages/nemo/collections/asr/models/classification_models.py", line 441, in __init__
    if cfg.is_regression_task:
  File "/opt/conda/lib/python3.6/site-packages/omegaconf/dictconfig.py", line 305, in __getattr__
    self._format_and_raise(key=key, value=None, cause=e)
  File "/opt/conda/lib/python3.6/site-packages/omegaconf/base.py", line 101, in _format_and_raise
    type_override=type_override,
  File "/opt/conda/lib/python3.6/site-packages/omegaconf/_utils.py", line 629, in format_and_raise
    _raise(ex, cause)
  File "/opt/conda/lib/python3.6/site-packages/omegaconf/_utils.py", line 610, in _raise
    raise ex  # set end OC_CAUSE=1 for full backtrace
  File "/opt/conda/lib/python3.6/site-packages/omegaconf/dictconfig.py", line 303, in __getattr__
    return self._get_impl(key=key, default_value=DEFAULT_VALUE_MARKER)
  File "/opt/conda/lib/python3.6/site-packages/omegaconf/dictconfig.py", line 361, in _get_impl
    node = self._get_node(key=key)
  File "/opt/conda/lib/python3.6/site-packages/omegaconf/dictconfig.py", line 383, in _get_node
    self._validate_get(key)
  File "/opt/conda/lib/python3.6/site-packages/omegaconf/dictconfig.py", line 136, in _validate_get
    key=key, value=value, cause=ConfigAttributeError(msg)
  File "/opt/conda/lib/python3.6/site-packages/omegaconf/base.py", line 101, in _format_and_raise
    type_override=type_override,
  File "/opt/conda/lib/python3.6/site-packages/omegaconf/_utils.py", line 694, in format_and_raise
    _raise(ex, cause)
  File "/opt/conda/lib/python3.6/site-packages/omegaconf/_utils.py", line 610, in _raise
    raise ex  # set end OC_CAUSE=1 for full backtrace
omegaconf.errors.ConfigAttributeError: Key 'is_regression_task' is not in struct
	full_key: model.is_regression_task
	reference_type=Any
	object_type=dict
```

Oh and there are some code formatting issues, could you run the `python setup.py style --fix --scope=nemo/`

ok, done, could you check if all is good now?

@diego-fustes Hi Thanks for this excellent contribution. Could you run `python setup.py style --fix `? There are some format issues in `tests` fails the CI tests. 

I always try to run the style checks... anyway I've done that and also renamed back the class `_EncDecBaseModel`, it looks like it was accidentally renamed in a previous merge, causing errors in the module...

Thanks a lot for your support guys! I'll keep contibuting as much as I can :)