Patent ID: 11900947
Assignee: FTR LABS PTY LTD
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 0:
1. A method of developing models in an automatic diarisation system comprising a system controller, a digital signal processor and a memory, the method comprising the steps of:
a) retrieving from the memory one or more annotated audio recordings by the controller, each annotated audio recording comprising an audio file and associated annotation data identifying timing in the file for changes between speakers, changes from active speech to non-speech, and differentiation of individual speakers;
b) performing, using the digital signal processor, feature extraction on the one or more annotated audio recordings, wherein the feature extraction is performed based on frames of samples, processed to extract speech feature components compatible for reliable modelling using Gaussian mixture modelling (GMM);
c) using the controller, the controller using the annotation data for accumulating a set of all non-speech features for each recording;
d) using the controller, the controller forming from the annotation data a set comprising speech features for all speakers for each recording;
e) developing a universal background model (UBM) for each recording using the set comprising speech features for all speakers for each recording, wherein the universal background model (UBM) is developed as an extensive Gaussian mixture model using an Expectation-Maximisation (EM) algorithm;
f) storing the UBM for input to automated diarisation processing;
g) performing speech Gaussian mixture modelling (GMM) for each recording using sampled data from the set comprising speech features for all speakers for the recording and storing the speech GMM for each recording;
h) performing GMM for each recording using sampled data from the set comprising all non-speech features for the recording and storing the non-speech GMM for each recording, wherein the speech GMMs, non-speech GMMs are stored for input to voice activity detection in an automated diarisation process;
i) using the controller, the controller forming from the annotation data for each recording a plurality of sets of speech features for each speaker, each set representing speech features for a set short time duration, and computing statistics for each set;
j) performing joint factor analysis (JFA) hyperparameter training for each speaker using the plurality of sets of speech features for the speaker, the computed statistics for the speaker and the UBM to determine characterising speaker JFA hyperparameters for the speaker, and storing the speaker JFA hyperparameters for each speaker;
k) computing statistics for non-speech features for each set of all non-speech features for each recording;
l) performing JFA hyperparameter training for non-speech using the set of all non-speech features for each recording, the computed statistics for the non-speech and the UBM to determine characterising non-speech JFA hyperparameters for non-speech, and storing the non-speech JFA hyperparameters for each recording;
whereby the speaker JFA Hyperparameters and non-speech JFA hyperparameters are input to speaker identification in the automated diarisation process.