Patent ID: 11869484
Assignee: HONG KONG APPLIED SCIENCE AND TECHNOLOGY RESEARCH INSTITUTE CO., LTD
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 10:
11. A method for automatic generation and update of a knowledge graph from multi-modal sources, the method comprising:
clustering, by a speaker diarization module, an input audio stream by:
partitioning the input audio stream into audio segments;
classifying speakers of the audio segments as agent or customer; and
clustering the audio segments based on speaker classification;

transcribing, by an audio transcription module, the clustered audio segments to transcripts based on an acoustic model;
labelling, by a speech parsing module, the transcripts by:
extracting entities of interest and schema of relations from the transcripts; and
labelling words of the transcripts corresponding to the extracted entities of interest with a plurality of pre-defined tags from a domain-specific language model;

generating, by a conversation parsing module, pairs of question and answer by:
updating a dynamic information word set VD with the labelled words of the transcripts and a static information word set VS based on the extracted schema of relations from the transcripts;
retrieving one or more sentence patterns from the domain-specific language model; and
generating the pairs of question and answer based on the dynamic information word set VD, the static information word set VS and the one or more sentence patterns;

updating, by a knowledge graph container, a knowledge graph by:
receiving the extracted entities of interest and schema of relations;
representing, by a knowledge graph container, the extracted entities of interest as nodes in the knowledge graph; and
representing, by a knowledge graph container, the extracted schema of relations as labels and edges between nodes in the knowledge graph.