Patent ID: 11915714
Assignee: NORTHWESTERN UNIVERSITY
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 10:
11. A system comprising:
a feature extraction subsystem configured to:
receive a transcript of a speaker and audio data from an edit region of an audio recording of the speaker as distinct from an unedited portion of the audio recording
apply a phoneme duration model to the audio data to predict phoneme durations;
apply, using the phoneme durations, a pitch generation model to the audio data to provide a target prosody for the audio data, the target prosody including a target pitch contour; and
compute acoustic features representing samples, wherein respective acoustic features for a sample of the samples comprise (i) a pitch feature representing at least one of the target pitch contour or the audio data and (ii) a periodicity feature and cepstral coefficients representing the audio data;

a neural vocoder configured to:
input the acoustic features for the sample;
generate a probability distribution of possible excitation values for the sample based on the acoustic features; and
determine a synthesized output sample by sampling the probability distribution using a constant sampling temperature; and

an operations subsystem configured to provide output audio data including the synthesized output sample, the output audio data configured to be combined with the unedited portion of the audio recording.