PoeticTTS / Layers /Swish.py
Florian Lux
add initial infrastructure
b3fa29f
raw
history blame
470 Bytes
# Copyright 2020 Johns Hopkins University (Shinji Watanabe)
# Northwestern Polytechnical University (Pengcheng Guo)
# Apache 2.0 (http://www.apache.org/licenses/LICENSE-2.0)
# Adapted by Florian Lux 2021
import torch
class Swish(torch.nn.Module):
"""
Construct an Swish activation function for Conformer.
"""
def forward(self, x):
"""
Return Swish activation function.
"""
return x * torch.sigmoid(x)