Patent ID: 11942093
Assignee: SYNCWORDS LLC
Field: Audio-visual technology (Electrical engineering)
Classification: CPC G  H | IPC G

Claim 5:
6. A method for automatically dubbing a video program comprising video image frames and synchronized audio speech by one or more speakers recorded in a source language into a plurality of target languages simultaneously using speech-to-text transcriptions and language translation, said method comprising:
a. capturing the video program;
b. transmitting the video program to a transcription service;
c. converting the synchronized audio speech to text strings, wherein each text string further comprises a plurality of words;
d. determining the temporal start and end points for each of the plurality of words;
e. from the temporal start and end points for each of the plurality of words, determining the timing of pauses between each of the plurality of words;
f. from the temporal start and end points for each of the plurality of words, determining which words in each text string form phrases and which words in each text string form sentences;
g. assigning temporal anchors to each word, phrase and sentence;
h. assigning parameters to each word, phrase and sentence, wherein said parameters determine:
a speaker identifier;
a gender of the speaker;
whether the speaker is an adult or a child;
an inflection and emphasis of each word in the phrase;
a volume of each word in the phrase;
a tonality of each word in the phrase;
a raspness of each word in the phrase; and
an emotional indicator for the phrase,
wherein the speaker identifier and the emotional indicator are each determined using artificial intelligence;

i. synchronizing the assigned parameters of each word, phrase and sentence using the temporal anchors within each text string;
j. translating the text strings in the source language to produce a plurality of text scripts in the various target languages, wherein each of plurality of text scripts contains a series of concatenated text strings along with associated inflection, tonality, emphasis, raspness, emotional indication, and volume indicators as well as timing and speaker identifiers for each word, phrase, and sentence that is derived from the synchronized audio speech recorded in the source language;
k. from the translated text strings, dubbing translated audio strings in the various target languages that are time synchronized to their source language audio strings by utilizing the temporal anchors;
l. analyzing the placement and superposition of subtitles comprising the text strings onto the video program using artificial intelligence; and
m. transmitting the video program containing the translated audio strings and translated text strings for final processing.