Inverse MelSpectrogram

username: neovand
cooked: <p>Hi I’m trying to make an autoencoder for speech data. The network’s input and output are Mel spectrograms. How can I obtain the audio waveform from the generated mel spectrogram?</p>

username: JuanFMontesinos
cooked: <p><a href="https://librosa.org/doc/latest/generated/librosa.feature.inverse.mel_to_audio.html?highlight=mel%20spectrogram" class="onebox" target="_blank" rel="nofollow noopener">https://librosa.org/doc/latest/generated/librosa.feature.inverse.mel_to_audio.html?highlight=mel%20spectrogram</a></p>

username: TPU
cooked: <aside class="quote no-group" data-username="neovand" data-post="1" data-topic="93827" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/neovand/48/28081_2.png" class="avatar"> neovand:</div>
<blockquote>
<p>How can I obtain the audio waveform from the generated mel spectrogram?</p>
</blockquote>
</aside>
<p><a href="https://librosa.org/doc/latest/generated/librosa.istft.html" rel="noopener nofollow ugc"><code>librosa.istft()</code></a> should also work, I think.</p>
<p>(from Wave-U-Net utils: <a href="https://github.com/f90/Wave-U-Net/blob/master/Utils.py" rel="noopener nofollow ugc">https://github.com/f90/Wave-U-Net/blob/master/Utils.py</a>)</p>

username: knoriy
cooked: <p>As <a href="https://discuss.pytorch.org/u/JuanFMontesinos">JuanFMontesinos</a> said <a href="https://librosa.org/doc/main/generated/librosa.feature.inverse.mel_to_audio.html" rel="noopener nofollow ugc">Librosa</a> is a great and easy way of converting Mel-spectrograms.</p>
<p>I’ve also used WaveGlow in the past to do the same task, you can use the Nvidia implementation <a href="https://github.com/NVIDIA/waveglow" rel="noopener nofollow ugc"><strong>here</strong></a></p>

username: TPU
cooked: <aside class="quote no-group" data-username="neovand" data-post="1" data-topic="93827" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/neovand/48/28081_2.png" class="avatar"> neovand:</div>
<blockquote>
<p>Hi I’m trying to make an autoencoder for speech data. The network’s input and output are Mel spectrograms. How can I obtain the audio waveform from the generated mel spectrogram?</p>
</blockquote>
</aside>
<p>Here’s a small example using <code>librosa.istft</code> from this <a href="http://www.eecs.qmul.ac.uk/~ewerts/publications/2020_StollerEwertDixon_FactorGAN_ICLR.pdf" rel="noopener nofollow ugc">FactorGAN</a> implementation:</p>
<pre data-code-wrap="py"><code class="lang-py">def spectrogramToAudioFile(magnitude, fftWindowSize, hopSize, phaseIterations=10, phase=None, length=None):
    '''
    Computes an audio signal from the given magnitude spectrogram, and optionally an initial phase.
    Griffin-Lim is executed to recover/refine the given the phase from the magnitude spectrogram.
    :param magnitude: Magnitudes to be converted to audio
    :param fftWindowSize: Size of FFT window used to create magnitudes
    :param hopSize: Hop size in frames used to create magnitudes
    :param phaseIterations: Number of Griffin-Lim iterations to recover phase
    :param phase: If given, starts ISTFT with this particular phase matrix
    :param length: If given, audio signal is clipped/padded to this number of frames
    :return:
    '''
    if phase is not None:
        if phaseIterations &gt; 0:
            # Refine audio given initial phase with a number of iterations
            return reconPhase(magnitude, fftWindowSize, hopSize, phaseIterations, phase, length)
        # reconstructing the new complex matrix
        stftMatrix = magnitude * np.exp(phase * 1j) # magnitude * e^(j*phase)
        audio = librosa.istft(stftMatrix, hop_length=hopSize, length=length)
    else:
        audio = reconPhase(magnitude, fftWindowSize, hopSize, phaseIterations)
    return audio
</code></pre>
<p>Link: <a href="https://github.com/f90/FactorGAN/blob/ae57301195984092ee40742273e1034f3ae27e32/Utils.py" rel="noopener nofollow ugc">https://github.com/f90/FactorGAN/blob/ae57301195984092ee40742273e1034f3ae27e32/Utils.py</a></p>