Repository: speechbrain/speechbrain

Issue:
VAD.get_speech_prob_file does not respect CUDA
Following the [colab](https://huggingface.co/speechbrain/vad-crdnn-libriparty), I try the following to enable CUDA:

```
VAD = VAD.from_hparams(source="speechbrain/vad-crdnn-libriparty", savedir="pretrained_models/vad-crdnn-libriparty", run_opts={"device":"cuda"})
audio_file = 'pretrained_model_checkpoints/example_vad.wav'
prob_chunks = VAD.get_speech_prob_file(audio_file)
```

but I get the following error:

```
/usr/local/lib/python3.7/dist-packages/speechbrain/pretrained/interfaces.py in get_speech_prob_file(self, audio_file, large_chunk_size, small_chunk_size, overlap_small_chunk)
    965 
    966             # Getting (in parallel) the frame-level speech probabilities
--> 967             small_chunks_prob = self.get_speech_prob_chunk(small_chunks)
    968             small_chunks_prob = small_chunks_prob[:, :-1, :]
    969 

/usr/local/lib/python3.7/dist-packages/speechbrain/pretrained/interfaces.py in get_speech_prob_chunk(self, wavs, wav_lens)
   1076         feats = self.modules.compute_features(wavs)
   1077         feats = self.modules.mean_var_norm(feats, wav_lens)
-> 1078         outputs = self.modules.cnn(feats)
   1079 
   1080         outputs = outputs.reshape(

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1050                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1051             return forward_call(*input, **kwargs)
   1052         # Do not call functions when jit is used
   1053         full_backward_hooks, non_full_backward_hooks = [], []

/usr/local/lib/python3.7/dist-packages/speechbrain/nnet/containers.py in forward(self, x)
    142         """
    143         for layer in self.values():
--> 144             x = layer(x)
    145             if isinstance(x, tuple):
    146                 x = x[0]

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1050                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1051             return forward_call(*input, **kwargs)
   1052         # Do not call functions when jit is used
   1053         full_backward_hooks, non_full_backward_hooks = [], []

/usr/local/lib/python3.7/dist-packages/speechbrain/nnet/normalization.py in forward(self, x)
    219             input to normalize. 3d or 4d tensors are expected.
    220         """
--> 221         return self.norm(x)
    222 
    223 

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1050                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1051             return forward_call(*input, **kwargs)
   1052         # Do not call functions when jit is used
   1053         full_backward_hooks, non_full_backward_hooks = [], []

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/normalization.py in forward(self, input)
    172     def forward(self, input: Tensor) -> Tensor:
    173         return F.layer_norm(
--> 174             input, self.normalized_shape, self.weight, self.bias, self.eps)
    175 
    176     def extra_repr(self) -> str:

/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py in layer_norm(input, normalized_shape, weight, bias, eps)
   2344             layer_norm, (input,), input, normalized_shape, weight=weight, bias=bias, eps=eps
   2345         )
-> 2346     return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)
   2347 
   2348 

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking arugment for argument weight in method wrapper_native_layer_norm)
```

This is on master.

I don't see an option in `get_speech_prob_file` to control the device, so I assume it should handle CUDA automatically.


Discussion:
Could you please give it a try to the version on the latest development
branch? We did some fixes there. On my side it was working on both cpu and
gpu

On Thu, Sep 16, 2021, 10:09 AM Joseph Turian ***@***.***>
wrote:

> Following the colab
> <https://huggingface.co/speechbrain/vad-crdnn-libriparty>, I try the
> following to enable CUDA:
>
> VAD = VAD.from_hparams(source="speechbrain/vad-crdnn-libriparty", savedir="pretrained_models/vad-crdnn-libriparty", run_opts={"device":"cuda"})
> audio_file = 'pretrained_model_checkpoints/example_vad.wav'
> prob_chunks = VAD.get_speech_prob_file(audio_file)
>
> but I get the following error:
>
> /usr/local/lib/python3.7/dist-packages/speechbrain/pretrained/interfaces.py in get_speech_prob_file(self, audio_file, large_chunk_size, small_chunk_size, overlap_small_chunk)
>     965
>     966             # Getting (in parallel) the frame-level speech probabilities
> --> 967             small_chunks_prob = self.get_speech_prob_chunk(small_chunks)
>     968             small_chunks_prob = small_chunks_prob[:, :-1, :]
>     969
>
> /usr/local/lib/python3.7/dist-packages/speechbrain/pretrained/interfaces.py in get_speech_prob_chunk(self, wavs, wav_lens)
>    1076         feats = self.modules.compute_features(wavs)
>    1077         feats = self.modules.mean_var_norm(feats, wav_lens)
> -> 1078         outputs = self.modules.cnn(feats)
>    1079
>    1080         outputs = outputs.reshape(
>
> /usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
>    1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
>    1050                 or _global_forward_hooks or _global_forward_pre_hooks):
> -> 1051             return forward_call(*input, **kwargs)
>    1052         # Do not call functions when jit is used
>    1053         full_backward_hooks, non_full_backward_hooks = [], []
>
> /usr/local/lib/python3.7/dist-packages/speechbrain/nnet/containers.py in forward(self, x)
>     142         """
>     143         for layer in self.values():
> --> 144             x = layer(x)
>     145             if isinstance(x, tuple):
>     146                 x = x[0]
>
> /usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
>    1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
>    1050                 or _global_forward_hooks or _global_forward_pre_hooks):
> -> 1051             return forward_call(*input, **kwargs)
>    1052         # Do not call functions when jit is used
>    1053         full_backward_hooks, non_full_backward_hooks = [], []
>
> /usr/local/lib/python3.7/dist-packages/speechbrain/nnet/normalization.py in forward(self, x)
>     219             input to normalize. 3d or 4d tensors are expected.
>     220         """
> --> 221         return self.norm(x)
>     222
>     223
>
> /usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
>    1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
>    1050                 or _global_forward_hooks or _global_forward_pre_hooks):
> -> 1051             return forward_call(*input, **kwargs)
>    1052         # Do not call functions when jit is used
>    1053         full_backward_hooks, non_full_backward_hooks = [], []
>
> /usr/local/lib/python3.7/dist-packages/torch/nn/modules/normalization.py in forward(self, input)
>     172     def forward(self, input: Tensor) -> Tensor:
>     173         return F.layer_norm(
> --> 174             input, self.normalized_shape, self.weight, self.bias, self.eps)
>     175
>     176     def extra_repr(self) -> str:
>
> /usr/local/lib/python3.7/dist-packages/torch/nn/functional.py in layer_norm(input, normalized_shape, weight, bias, eps)
>    2344             layer_norm, (input,), input, normalized_shape, weight=weight, bias=bias, eps=eps
>    2345         )
> -> 2346     return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)
>    2347
>    2348
>
> RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking arugment for argument weight in method wrapper_native_layer_norm)
>
> This is on master.
>
> I don't see an option in get_speech_prob_file to control the device, so I
> assume it should handle CUDA automatically.
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/speechbrain/speechbrain/issues/992>, or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AEA2ZVSYN6SODM4NHOHDRRTUCGQ2ZANCNFSM5EEFDWBA>
> .
>

To change device you should change the variable "device"in the yaml file
accordingly. You can use run_opts for doing this change (see readme in the
huggingface repo). It should work that way. Please, let us know..

On Thu, Sep 16, 2021, 10:11 AM Mirco Ravanelli ***@***.***>
wrote:

> Could you please give it a try to the version on the latest development
> branch? We did some fixes there. On my side it was working on both cpu and
> gpu
>
> On Thu, Sep 16, 2021, 10:09 AM Joseph Turian ***@***.***>
> wrote:
>
>> Following the colab
>> <https://huggingface.co/speechbrain/vad-crdnn-libriparty>, I try the
>> following to enable CUDA:
>>
>> VAD = VAD.from_hparams(source="speechbrain/vad-crdnn-libriparty", savedir="pretrained_models/vad-crdnn-libriparty", run_opts={"device":"cuda"})
>> audio_file = 'pretrained_model_checkpoints/example_vad.wav'
>> prob_chunks = VAD.get_speech_prob_file(audio_file)
>>
>> but I get the following error:
>>
>> /usr/local/lib/python3.7/dist-packages/speechbrain/pretrained/interfaces.py in get_speech_prob_file(self, audio_file, large_chunk_size, small_chunk_size, overlap_small_chunk)
>>     965
>>     966             # Getting (in parallel) the frame-level speech probabilities
>> --> 967             small_chunks_prob = self.get_speech_prob_chunk(small_chunks)
>>     968             small_chunks_prob = small_chunks_prob[:, :-1, :]
>>     969
>>
>> /usr/local/lib/python3.7/dist-packages/speechbrain/pretrained/interfaces.py in get_speech_prob_chunk(self, wavs, wav_lens)
>>    1076         feats = self.modules.compute_features(wavs)
>>    1077         feats = self.modules.mean_var_norm(feats, wav_lens)
>> -> 1078         outputs = self.modules.cnn(feats)
>>    1079
>>    1080         outputs = outputs.reshape(
>>
>> /usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
>>    1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
>>    1050                 or _global_forward_hooks or _global_forward_pre_hooks):
>> -> 1051             return forward_call(*input, **kwargs)
>>    1052         # Do not call functions when jit is used
>>    1053         full_backward_hooks, non_full_backward_hooks = [], []
>>
>> /usr/local/lib/python3.7/dist-packages/speechbrain/nnet/containers.py in forward(self, x)
>>     142         """
>>     143         for layer in self.values():
>> --> 144             x = layer(x)
>>     145             if isinstance(x, tuple):
>>     146                 x = x[0]
>>
>> /usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
>>    1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
>>    1050                 or _global_forward_hooks or _global_forward_pre_hooks):
>> -> 1051             return forward_call(*input, **kwargs)
>>    1052         # Do not call functions when jit is used
>>    1053         full_backward_hooks, non_full_backward_hooks = [], []
>>
>> /usr/local/lib/python3.7/dist-packages/speechbrain/nnet/normalization.py in forward(self, x)
>>     219             input to normalize. 3d or 4d tensors are expected.
>>     220         """
>> --> 221         return self.norm(x)
>>     222
>>     223
>>
>> /usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
>>    1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
>>    1050                 or _global_forward_hooks or _global_forward_pre_hooks):
>> -> 1051             return forward_call(*input, **kwargs)
>>    1052         # Do not call functions when jit is used
>>    1053         full_backward_hooks, non_full_backward_hooks = [], []
>>
>> /usr/local/lib/python3.7/dist-packages/torch/nn/modules/normalization.py in forward(self, input)
>>     172     def forward(self, input: Tensor) -> Tensor:
>>     173         return F.layer_norm(
>> --> 174             input, self.normalized_shape, self.weight, self.bias, self.eps)
>>     175
>>     176     def extra_repr(self) -> str:
>>
>> /usr/local/lib/python3.7/dist-packages/torch/nn/functional.py in layer_norm(input, normalized_shape, weight, bias, eps)
>>    2344             layer_norm, (input,), input, normalized_shape, weight=weight, bias=bias, eps=eps
>>    2345         )
>> -> 2346     return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)
>>    2347
>>    2348
>>
>> RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking arugment for argument weight in method wrapper_native_layer_norm)
>>
>> This is on master.
>>
>> I don't see an option in get_speech_prob_file to control the device, so
>> I assume it should handle CUDA automatically.
>>
>> —
>> You are receiving this because you are subscribed to this thread.
>> Reply to this email directly, view it on GitHub
>> <https://github.com/speechbrain/speechbrain/issues/992>, or unsubscribe
>> <https://github.com/notifications/unsubscribe-auth/AEA2ZVSYN6SODM4NHOHDRRTUCGQ2ZANCNFSM5EEFDWBA>
>> .
>>
>

Okay. I install the develop branch:

```
#!pip3 install speechbrain
!pip3 install git+https://github.com/speechbrain/speechbrain.git
```

I add run_opts to the hparams load as in huggingface (I also did this above):

```
VAD = VAD.from_hparams(source="speechbrain/vad-crdnn-libriparty", savedir="pretrained_models/vad-crdnn-libriparty", run_opts={"device":"cuda"})
```

Unfortunately I still get the same error :(
Ok...weird..so there should still be a tensor to put on the right
device...I can double check it as soon as I can

On Thu, Sep 16, 2021, 10:24 AM Joseph Turian ***@***.***>
wrote:

> Okay. I install the develop branch:
>
> #!pip3 install speechbrain
> !pip3 install git+https://github.com/speechbrain/speechbrain.git
>
> I add run_opts to the hparams load as in huggingface (I also did this
> above):
>
> VAD = VAD.from_hparams(source="speechbrain/vad-crdnn-libriparty", savedir="pretrained_models/vad-crdnn-libriparty", run_opts={"device":"cuda"})
>
> Unfortunately I still get the same error :(
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/speechbrain/speechbrain/issues/992#issuecomment-920695785>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AEA2ZVUFOUX7X5ZUQSVDECLUCGSTRANCNFSM5EEFDWBA>
> .
>

Thank you @mravanelli ! This is perhaps also a good unit test. But github doesn't allow GPUs. I have seen pytorch-lightning use Azure GPU cloud for GPU tests.
Yes, unit tests with the gpu part would be great indeed. But switching to
some CI solutions that are not free adds an extra cost that an
academically-driven project like SpeechBrain might not be able to always
sustain (in our case we are mainly a large community of volunteers and only
sometimes we can rely on the help of sponsors).

On Thu, Sep 16, 2021, 11:27 AM Joseph Turian ***@***.***>
wrote:

> Thank you @mravanelli <https://github.com/mravanelli> ! This is perhaps
> also a good unit test. But github doesn't allow GPUs. I have seen
> pytorch-lightning use Azure GPU cloud for GPU tests.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/speechbrain/speechbrain/issues/992#issuecomment-920738753>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AEA2ZVSS3ICLMDL4AHDO3O3UCGZ63ANCNFSM5EEFDWBA>
> .
>

True. Azure might give open source grants. (Travis does)
Same problem here ! Any update on this ? Thanks a lot for this wonderful work !
Hello, 

Indeed, when we use `run_opts={"device":"cuda"}` we get an error. Instead, you need to modify the `hyperparams.yaml` file by replacing `device='cpu'` to `device='cuda:0'`.  Now, the VAD will run on the GPU, and you will not have any errors thrown! 🙂

The .yaml file is located in the savedir. 

Best.

> Hello,
> 
> Indeed, when we use `run_opts={"device":"cuda"}` we get an error. Instead, you need to modify the `hyperparams.yaml` file by replacing `device='cpu'` to `device='cuda:0'`. Now, the VAD will run on the GPU, and you will not have any errors thrown! 🙂
> 
> The .yaml file is located in the savedir.
> 
> Best.

But would be great if "run_opts" will be fixed, extra yaml file only for one param it's weird :confounded:
Any news about this? I just encountered the same problem.
Hey @TechInterMezzo, as I mentioned, one way to solve this issue is by modifying the `hyperparams.yaml` file stored in your `savedir` location, and change `device` to be `cuda`. 

Hey @lapers, I will have a look. This is weird indeed. 

Resolving patch:
diff --git a/speechbrain/pretrained/interfaces.py b/speechbrain/pretrained/interfaces.py
--- a/speechbrain/pretrained/interfaces.py
+++ b/speechbrain/pretrained/interfaces.py
@@ -1243,7 +1243,6 @@ def __init__(self, *args, **kwargs):
         super().__init__(*args, **kwargs)
         self.time_resolution = self.hparams.time_resolution
         self.sample_rate = self.hparams.sample_rate
-        self.device = self.hparams.device
 
     def get_speech_prob_file(
         self,


Test patch: