[TTS] Fix Global Style Tokens (GSTs) implementation in FastPitch
State: closed | Pull request: yes
Labels: name: TTS

# What does this PR do ?

1. Fix bug in `nemo.collections.tts.modules.submodules.ConditionalInput.forward()`
2. Fix implementation of Global Style Tokens in FastPitch

**Collection**: TTS

# Changelog 
- When using `ConditionalInput` in `"concat"` mode, concatenate `inputs` and `conditioning` along the _feature_ dimension, not the _batch_ dimension as before
  - Despite the misleading docstring in `ConditionalInput.forward()` (corrected in this PR), `inputs` and `conditioning` have shapes `B x T x H` and `B x T x C`, respectively (to be even more precise, `conditioning` has shape `B x 1 x C` _before_ the call to `conditioning.repeat()` and `B x T x C` _after_). Consequently, concatenating the two along the batch dimension has 2 problems:
    - If `H != C`, the concatenation will fail
    - If `H == C`, the concatenation results in a `2B x T x C` tensor
       - This causes an error downstream when calling `self.concat_proj()`, which is defined as:
`self.concat_proj = torch.nn.Linear(hidden_dim + condition_dim, hidden_dim)`
       - `self.concat_proj` is meant to map `B x T x (H+C)` tensors to `B x T x H` tensors, so passing a `2B x T x (H=C)` tensor to it results in the following error:
`RuntimeError: mat1 and mat2 shapes cannot be multiplied ((2B*T)x(H=C) and (H+C)xH)` (the actual error message contains constant values, but here I have extracted variables to make my point clearer)
  - **Solution**: concatenate along the feature dimension to obtain a `B x T x (H+C)` tensor
- Fix the implementation of GSTs in FastPitch, specifically regarding the selection of the _reference audio_ for a given training example
  - The current NeMo implementation selects the reference audio for a given training example as a random example from the same speaker. Note that this also forces the user to add `speaker_id` to `sup_data_types` even when they don't want to use speaker embeddings
  - However, the [GSTs paper](https://arxiv.org/abs/1803.09017) clearly states that:
    > During training, the reference signal is ground-truth audio.
  - While this is enough reason to justify amending the current NeMo implementation, I have expressed my thoughts on what selecting the reference audio at random may lead to in [this issue](https://github.com/NVIDIA/NeMo/issues/7420) I opened over a month ago (which went totally unnoticed)

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

## Who can review?
@blisc, @okuchaiev 

# Additional Information
* Related to #7420
* I have run tests, but some failed or errored out. None of the failed tests makes use of my changes though. See [test_short_summary.txt](https://github.com/NVIDIA/NeMo/files/13063436/test_short_summary.txt) for further details