Speech processing computer system forming collaborative dialog data structures

A collaborative speech processing computer receives packets of sampled audio streams. The sampled audio streams are forwarded to a speech-to-text conversion server via a data network. Packets are received via the data network that contain text strings converted from the sampled audio steams by the speech-to-text conversion server. Speakers are identified who are associated with the text strings contained in the data packets. The text strings and the identifiers of the associated speakers are added to a dialog data structure in a repository memory. Content of at least a portion of the dialog data structure is displayed on a display device.

TECHNICAL FIELD

The present disclosure is related to speech processing computer systems and more particularly to voice recognition servers.

BACKGROUND

In software development and some other product development environments, team members huddle together each morning for a stand-up meeting where they review progress and essentially re-plan the project. During the daily meetings, which are called “scrums,” a scrum master asks the team members these three questions that can include: what did you do yesterday; what will you do today; are there any impediments in your way. The scrum master functions to, for example: help the team to reach consensus for what can be achieved during a specific period of time; help the team to reach consensus during the daily scrum; help the team to stay focused and follow the agreed-upon rules for daily scrums; remove obstacles that are impeding the team's progress; and protect the team from outside distractions.

Tracking progress toward completing project tasks, task issues raised by team members, and the contributions by individual team members toward those tasks can be a time consuming process and which can interfere with the ongoing collaboration among team members and impede the free flowing discussions that are considered important to providing a supportive project environment.

SUMMARY

Some embodiments disclosed herein are directed to methods by a collaborative speech processing computer. Data packets of sampled audio streams are obtained. The sampled audio streams are forwarded to a speech-to-text conversion server via a data network. Data packets are received, via the data network, which contain text strings converted from the sampled audio steams by the speech-to-text conversion server. Speakers are identified who are associated with the text strings contained in the data packets. The text strings and the identifiers of the associated speakers are added to a dialog data structure in a repository memory. Content of at least a portion of the dialog data structure is output to a display device for display

Some other related embodiments disclosed herein are directed to a collaborative speech processing computer. A network interface is configured to communicate with a speech-to-text conversion server. A processor is connected to receive the data packets from the network interface. A memory stores program instructions that are executable by the processor to perform operations. The operations include obtaining data packets of sampled audio streams. The sampled audio streams are forwarded to the speech-to-text conversion server via the network interface. Data packets containing text strings converted from the sampled audio steams by the speech-to-text conversion server, are received via the network interface. A project task is selected from among a plurality of project tasks defined in a project database based on a closest matching of words in the one of the text strings to a set of keywords for the project task that is among sets of keywords that have been defined for the plurality of project tasks. A group of candidate speakers are identified who are defined in the project database as being associated with the project task selected. Spectral characteristics of a voice contained in the sampled audio stream that was converted to the one of the text strings is compared to known spectral characteristics that are defined for the candidate speakers in the group. One person is selected as the speaker from among the candidate speakers in the group, based on a relatively closeness of the comparisons of spectral characteristics.

In a further embodiment, at least one wireless user terminal is selected that is determined to have been a source of one of the sampled audio streams and that is determined to not be licensed to one of the candidate speakers in the group. A microphone mute command is then sent to the at least one wireless user terminal selected to trigger temporary muting of a microphone of the at least one wireless user terminal.

Some other related embodiments disclosed herein are directed to another collaborative speech processing computer that performs operation that include obtaining data packets of sampled audio streams, and forwarding the sampled audio streams to the speech-to-text conversion server via the network interface. Data packets containing text strings converted from the sampled audio steams by the speech-to-text conversion server, are received via the network interface. A project task is selected from among a plurality of project tasks defined in a project database based on a closest matching of words in the one of the text strings to a set of keywords for the project task that is among sets of keywords that have been defined for the plurality of project tasks. A speaker is selected who is defined in the project database as being associated with the project task selected. A microphone is selected from among a plurality of microphones that is determined to more strongly sense voice of the speaker identified relative to the other microphones. A gain is increased that is applied to audio signals from the microphone selected.

It is noted that aspects described with respect to one embodiment disclosed herein may be incorporated in different embodiments although not specifically described relative thereto. That is, all embodiments and/or features of any embodiments can be combined in any way and/or combination. Moreover, methods, systems, and/or computer program products according to embodiments will be or become apparent to one with skill in the art upon review of the following drawings and detailed description. It is intended that all such additional methods, systems, and/or computer program products be included within this description and protected by the accompanying claims.

DETAILED DESCRIPTION

Various embodiments will be described more fully hereinafter with reference to the accompanying drawings. Other embodiments may take many different forms and should not be construed as limited to the embodiments set forth herein. Like numbers refer to like elements throughout.

According to various embodiments of the present disclosure, a collaborative speech processing computer receives packets of sampled audio streams. The sampled audio streams are forwarded to a speech-to-text conversion server via a data network. Packets are received via the data network that contain text strings converted from the sampled audio steams by the speech-to-text conversion server. Speakers are identified who are associated with the text strings contained in the packets. The text strings and the identifiers of the associated speakers are added to a dialog data structure in a repository memory. Content of at least a portion of the dialog data structure is displayed on a display device.

The collaborative speech processing computer may be part of a virtual scrum master system. For example, in some embodiments a virtual scrum master is provided as an electronic tool (e.g., server) that facilitates the textual recordation and organization of spoken conversations by scrum meeting attendees. The virtual scrum master tool listens to spoken conversations by scrum meeting attendees, converts a stream of audio samples of the spoken conversations to a data packets containing digital samples of the audio stream, dynamically identifies speakers during the conversations, and associates identifiers for the speakers to the converted text strings. The virtual scrum master tool can then organize the converted text strings with the associated speaker identifiers into a scrum knowledgebase. The scrum knowledgebase can be mined for project planning, tracking progress attributable to individual team members, identifying risks with individual project deliverables, etc.

FIG. 1is a block diagram of a computer system that includes a collaborative speech processing computer100that operationally interfaces with a project database102and a natural language speech-to-text server130in accordance with some embodiments. The collaborative speech processing computer100may form a virtual scrum master tool in accordance with some embodiments. The collaborative speech processing computer100may communicate through a data network124, e.g., private network and/or public network (Internet), with the natural language speech-to-text server130.

One approach includes having scrum meeting attendees set their wireless terminals110, e.g., mobile phones, tablet computers, etc., on a table nearest their seated/standing position. The collaborative speech processing computer100forms a communication link through a radio access network120(e.g., Bluetooth, WiFi, and/or cellular radio interface) with the wireless terminals110. Scrum applications200on the wireless terminals110generate data packets of sampled audio streams which are sent to the collaborative speech processing computer100with identifiers of the wireless terminals110and/or the user names that have been registered in the scrum applications200and/or as user information registered in the wireless terminal settings. The collaborative speech processing computer100correlates mobile phone identifiers to scrum meeting attendees' names. The collaborative speech processing computer100sends the sampled audio streams to the remote networked natural language speech-to-text server130, e.g., such as to APIs of natural language speech-to-text servers provided by Google, Apple, and/or Microsoft. The collaborative speech processing computer100receives responsive conversation text strings from the natural language speech-to-text server130, and associates speaker identifiers with the conversation text. The conversation text strings are stored in a project database102or, more generally, in a dialog data structure in a repository memory, with speaker identifiers.

The radio access network120may be connected to the collaborative speech processing computer100through a data network122, which may be part of the data network124. In some other embodiments described below, instead of using wireless user terminals to sense voices, one or more microphones can be positioned among the users to provide audio streams that are sampled to generate the data packets provided to the collaborative speech processing computer100. Although illustrated separately for ease of reference, one or more of the functions disclosed herein for the collaborative speech processing computer100, the project database102, and/or the natural language speech-to-text server130may be combined within a computer system10.

FIG. 2is a combined data flow diagram and flowchart of operations that may be performed by wireless user terminals110, the collaborative speech processing computer100, and the natural language speech-to-text server130ofFIG. 1in accordance with some embodiments. Although single user terminal110is illustrated inFIG. 2, is be understood that the functionality illustrated therein may be replicated across a plurality of computer terminals201.

Referring toFIG. 2, the wireless user terminal110executes a scrum application200that performs illustrated operations, which include generating202a sample audio stream from output of a microphone that may be part of the terminal or operationally interconnected thereto, e.g., Bluetooth headset. Data packets are generated that contain the sample audio stream and may further contain an identifier for the user terminal110and/or a registered user or other subscriber. The data packets are communicated (e.g. streamed)204to the collaborative speech processing computer110, such as by packet radio communications through the radio access network120which are forwarded through the network122.

The collaborative speech processing computer100receives the data packets containing the sampled audio streams from the wireless user terminal110, and processes206the sampled audio streams for improved speech-to-text recognition. The processing can include filtering the sampled audio streams to remove noise and/or to increase the voice to background noise ratio for improved speech-to-text recognition capability. The data packets are forwarded208to the natural language speech-to-text server130via the data network124. The forwarding208may include sending messages to the server130that provide the sample audio streams to a speech recognition application programming interface (API) of a speech recognition application executed by the server130, e.g., such as to APIs of natural language speech recognition applications hosted by Google, Apple, and/or Microsoft.

The natural language speech-to-text server130recognizes and converts210speech in the sampled audio streams to text, and sends212data packets containing text strings, which are converted from the sampled audio streams, through the data network124to the collaborative speech processing computer100.

The collaborative speech processing computer100receives the data packets containing the text strings, and identifies214speakers who are associated with the text strings contained in the data packets. For example, the collaborative speech processing computer100may process the text string contained in a receive data packet using one or more of operations described below regardingFIGS. 3 and/or 4to identify the person who's voice is captured in the sampled audio stream that became converted to that text string. As will be described in further detail below, various further embodiments are directed to operations for dynamically identifying speakers during meetings, such as during scrum group discussions, and correlating the speaker identifiers with the text segments that are later returned by the natural language speech-to-text server performing natural language speech-to-text conversion.

One or more commands can be provided 216 to the scrum application200to perform operations that are further described below, which can include: 1) generating an interruption warning message to one of the speakers who is identified as having interrupted one or more other speakers at least a threshold number of times; 2) sending a command to increase amplification gain (e.g., amplify amplitude of voice) or decrease (e.g., decrease amplitude of voice) applied to audio signals from the microphone of the wireless user terminal110; 3) sending a microphone mute command to the wireless user terminal110to trigger temporary muting of a microphone of the wireless user terminal110; 4) sending a stream pause command to the wireless user terminal110to trigger the wireless user terminal110to cease sending further data packets containing a sampled audio stream; and/or 5) sending a stream resumption command to the wireless user terminal110to trigger the wireless user terminal110to resume sending further data packets containing a sampled audio stream. The wireless user terminal110can respond to one or more such commands by controlling222it's audio sampling characteristics, such as by controlling an amplification gain applied to audio signals from the microphone, muting the microphone signal, ceasing streaming of data packets of a sampled audio stream, and/or resuming streaming of data packets of a sampled audio stream. The wireless user terminal110may generate224a notification to a user, such as by displaying a notification message, outputting an audible notification through speaker, and/or generating a vibrational notification.

The collaborative speech processing computer100adds218the text strings and the identifiers of the associated speakers to a dialog data structure in a repository memory, which may reside in the project database102. Content of at least a portion of the dialog data structure may be output to a display device for display220.

As will be explained in further detail below, the collaborative speech processing computer100can rely on the results of querying the project database102(FIGS. 1, 3, and 4) to identify persons who are likely to have spoken the converted text strings. Identifying the speakers can be particularly advantageous for enabling tracking of progress toward completing project tasks, task issues raised by individual team members, and the contributions by individual team members toward those tasks. Moreover, audio signals from the identified task associated speakers may be handled operationally differently than those from other speakers.

In one embodiment, the operations for identifying one of the speakers associated with one of the text strings contained in the data packets, includes selecting a project task from among a plurality of project tasks that are defined in the project database102based on a closest matching of words in the one of the text strings to a set of keywords for the project task that is among sets of keywords that have been defined for the plurality of project tasks. The project database102can define a set of keywords that are associated with each project task (e.g., keywords corresponding to task descriptions, milestones, dates, product interfaces, supplier names, customer names, etc), and therefore different project tasks typically have different sets of keywords. The project database102can include a list of persons who are members of a project, can identify persons who are responsible for which tasks of a project, and can identify which sub-tasks each person is responsible for handling. The project database102may identify functional reporting structures, including which who is responsible for managing a project, task, and/or sub-task and overseeing progress by certain identified other persons. A person can therefore be identified as the speaker or as a candidate speaker from among a defined group of possible speakers who is/are associated by the project database102with the project task that was selected.

The operation for identifying214as the speaker a person who is defined in the project database102as being associated with the project task that is selected, can include comparing spectral characteristics of a voice contained in the sampled audio stream, which was converted to the one of the text strings, to spectral characteristics that are defined for a plurality of persons who are identified by the project database102as being associated with the project task selected, and then selecting one person as the speaker from among the plurality of persons, who are identified by the project database102as being associated with the project task selected, based on a relatively closeness of the comparisons of spectral characteristics.

In a further embodiment, the comparison of the spectral characteristics of the voice contained in the sampled audio stream to spectral characteristics that are defined for the persons who are defined by the project database102as being associated with the project task that was selected, can include comparing a spoken rate of words that are contained in the sampled audio stream, which was converted to the one of the text strings, to spoken rates of words that are defined for the plurality of persons who are defined by the project database102as being associated with the project task selected. For example, different persons can often be characterized by different speech rates (e.g., number of words spoken of a defined time period), and the characteristic speech rate for a defined person can be learned by the collaborative speech processing computer100and/or another system component and stored as information associated with that person's identifier in the project database102.

In another further embodiment, the comparison of the spectral characteristics of the voice contained in the sampled audio stream to spectral characteristics that are defined for the persons who are defined by the project database102as being associated with the project task that was selected, can include comparing a frequency spectrum waveform in the sampled audio stream, which was converted to the one of the text strings, to frequency spectrum waveforms that are defined for the plurality of persons who are identified by the project database102as being associated with the project task that was. For example, different persons can often be characterized by different voice frequency spectrum waveforms (e.g., voice pitch and frequency waveform for various defined spoken words), and the characteristic voice frequency spectrum waveform for a defined person can be learned by the collaborative speech processing computer100and/or another system component and stored as information associated with that person's identifier in the project database102.

More generally, the operation for identifying214a speaker can include comparing spectral characteristics of a voice contained in the sampled audio stream, which was converted to the one of the text strings, to spectral characteristics that are defined for a plurality of persons who are defined by the project database102, and selecting one person as the speaker from among the plurality of persons who are defined by the project database102based on a relatively closeness of the comparisons of spectral characteristics.

The operation for adding218the text strings and the identifiers of the associated speakers to the dialog data structure in a repository memory, can include storing the one of the text strings and an identifier of the person selected to a dialog data structure in the project database102with a defined association to the project task that was selected.

Various other operations are directed to providing operational commands216to the user terminal110responsive to identification214of the speaker.

In one embodiment, the collaborative speech processing computer100identifies time over-lapping voices in the sample audio streams. Responsive to the identification of the time over-lapping voices, the computer100tracks which of the speakers has begun talking before another one of the speakers has stopped talking, and generates an interruption warning message that is provided to one six to the wireless user terminal110operated by the speaker who is identified as having interrupted one or more other speakers at least a threshold number of times. That's wireless user terminal110can generate224an associated notification to the user, such as by displaying a warning message to the user that cautions against interruption of other scrum meeting members and/or other persons involved in a discussion. These operations may be advantageous in that they can improve the speech recognition capability of the natural language speech-to-text server130by avoiding occurrence of combined overlapping voices in the sampled audio streams that could otherwise impede or prevent speech recognition operation.

In another embodiment, the collaborative speech processing computer100selects a project task from among a plurality of project tasks defined in the project database102based on closeness of matching (e.g., selecting the closest match) between words in one of the text strings to keywords defined for the project task, and identifies a plurality of persons who are defined in the project database102as being associated with the project task that was selected. The computer100can then select at least one wireless user terminal110that has been a source of one of the sampled audio streams and that is determined to not be licensed to one of the persons identified, and responsively send a microphone mute command to the at least one wireless user terminal110that was selected in order to trigger temporary muting of a microphone of the at least one wireless user terminal110. In this manner, the collaborative speech processing computer100can control which wireless user terminals110are operating to stream voices or other audio. Wireless user terminals that are being operated by persons who have been defined in the project database102as being responsible for a particular task that determined as being discussed can have their voices streamed through their respective wireless user terminals110, while other persons who are not defined in the project database102as being responsible for the particular task being discussed can have their voices muted by their respective wireless user terminals110.

In another embodiment, the collaborative speech processing computer100selects a project task from among a plurality of project tasks defined in the project database102based on closeness of matching (e.g., selecting the closest match) between words in one of the text strings to keywords defined for the project task, and identifies a plurality of persons who are defined in the project database102as being associated with the project task selected. The computer100then selects at least one wireless user terminal110that has been a source of one of the sampled audio streams and that is determined to not be licensed to one of the persons identified, and sends a stream pause command to the at least one wireless user terminal110selected in order to trigger the at least one wireless user terminal110to cease sending further data packets containing a sampled audio stream. Accordingly, a wireless user terminal that is being operated by a person who is not defined in the project database102as being responsible for a particular task that determined as being discussed can be controlled to stop sending data packets containing sampled audio streams, which can reduce the operational processing by the collaborative speech processing computer100and by the natural language speech-to-text server130, decrease the data communication utilization of the network124and the network122, decrease the wireless communication resources that are provided by the radio access network120and being utilized by the wireless terminals110, and decrease the possible erroneous speech recognition of background noise that may otherwise be provided by the wireless user terminal110operated by that person.

In a further embodiment, responsive to occurrence of a defined event after sending of the stream pause command to the selected wireless user terminal110, the collaborative speech processing computer100sends216a stream resumption command to the at least one wireless user terminal110that was selected in order to trigger the at least one wireless user terminal110to resume sending further data packets containing a sampled audio stream.

In another further embodiment, the operation for detecting occurrence of the defined event after sending of the stream pause command is based on determining that another project task has been selected from among the plurality of project tasks defined in the project database based on closeness of matching (e.g., selecting the closest match) between words in another one of the text strings received after sending of the stream pause command to keywords defined for the project task. Thus, when the converted text strings are compared to the keywords in the project database102and result in determination that the conversation is now concerning another project task, the collaborative speech processing computer100can responsively send the stream resumption command to the wireless user terminal(s)110, which were earlier commanded to cease sending sampled audio stream data packets, to trigger those wireless user terminal(s)110to resume sending further data packets containing sampled audio streams.

In another further embodiment, when the packets of sampled audio streams are received from wireless user terminals110, the collaborative speech processing computer100parses the packets to determine terminal identifiers of the wireless user terminals110. Names of the speakers are determined based on the terminal identifiers, such as by using the terminal identifiers to look up subscriber names in a subscriber database. The subscriber database may be a cellular home subscriber registry that is queried using the mobile identifier for a cellular phone. The names of the speakers are embedded as metadata in files of the sampled audio streams forwarded to the speech-to-text conversion server130, and the speakers who are associated with the text strings contained in the packets that are received are identified based on metadata returned by the speech-to-text conversion server130.

FIG. 3is a combined data flow diagram and flowchart of some other operations that may be performed by user terminals110, the collaborative speech processing computer100, the natural language speech-to-text server130, and the project database102ofFIG. 1in accordance with some other embodiments.

Referring toFIG. 3, the collaborative speech processing computer100and the natural language speech-to-text server130can perform the operations206-212described above forFIG. 2. The collaborative speech processing computer100can correlate310text contained in the text strings converted by the server130to keywords that are associated with project tasks, project timelines, and/or project members. The collaborative speech processing computer100may query312the project database102using text that is parsed from the converted text strings to identify matches between the text and various keywords have been defined for the various project tasks, project timelines, and/or project members. When the correlation satisfies314one or more defined rules, the collaborative speech processing computer100can trigger responsive actions. The actions can include sending questions and/or notification messages to the wireless user terminals110for display316through a display device to associated users. The wireless user terminal110can responsively relay spoken and/or enter user responses to the collaborative speech processing computer100for processing and possible further actions. The actions can alternatively or additionally include operational commands that are sent to the wireless user terminals1102trigger operations it can include, as described above, adjusting gain that is applied to microphone signals, ceasing their sending of data packets containing sampled audio streams, and/or resuming their sending of data packets containing sampled audio streams.

The collaborative speech processing computer100adds318text and speaker identifiers to a dialogue data structure, e.g., scrum dialog data structure, in the project database102. The collaborative speech processing computer100may alternatively or additionally update320tracking of project progress, individual project member contributions to the discussion of project tasks, and/or update an assessment of project risks based on the converted speech from the member discussions.

FIG. 4is a combined data flow diagram and flowchart of some other operations that may be performed by an array of microphones400, the project database102, the collaborative speech processing computer100, and the natural language speech-to-text server130ofFIG. 1in accordance with some other embodiments.

Referring toFIG. 4, the microphones400may be spaced apart along a conference table to be adjacent to one or more meeting participants, and can be configured to have directional sensitivity aimed in a direction to sense one or more participant speakers in that direction while not sensing, or substantially attenuating, any speakers who are outside the adjacent cone-shaped region extending in that direction. Microphones400output an audio stream that may be digitally sampled by the microphones400or by other equipment, such as by a microphone input interface of the collaborative speech processing computer100.

The collaborative speech processing computer100processes the sampled audio stream for improved speech-to-text recognition. The processing can include filtering the sampled audio streams to remove noise and/or to increase the voice to background noise ratio for improved speech-to-text recognition capability. The data packets are forwarded208to the natural language speech-to-text server130via the data network124. The forwarding208may include sending messages to the server130that provide the sample audio streams to a speech recognition application programming interface (API) of a speech recognition application executed by the server130. The APIs may correspond to those provided by natural language speech-to-text servers operated by Google, Apple, and/or Microsoft.

The natural language speech-to-text server130recognizes and converts210speech in one of the sampled audio streams to text, and sends data packets containing text strings, which are converted from the sampled audio stream, through the data network124to the collaborative speech processing computer100.

The collaborative speech processing computer100receives the data packets containing the text strings, and identifies420speakers who's voice is captured in the sampled audio stream that became converted to a text string. For example, the collaborative speech processing computer100may dynamically identify speakers during meetings, such as during scrum group discussions, and correlate the speaker identifiers with the text segments that are later returned by the natural language speech-to-text server performing natural language speech-to-text conversion.

The collaborative speech processing computer100can rely on the results of querying the project database102to identify persons who are likely to have spoken the converted text strings. Identifying the speakers can be particularly advantageous for enabling tracking of progress toward completing project tasks, task issues raised by individual team members, and the contributions by individual team members toward those tasks.

In one embodiment, the operations for identifying one of the speakers associated with one of the text strings contained in the data packets, includes selecting424a project task from among a plurality of project tasks defined in the project database102based on a closest matching of words in the one of the text strings to a set of keywords for the project task that is among sets of keywords that have been defined for the plurality of project tasks. The computer100may identify unique words or combinations of words and the converted text string, which are then used to query426the project database102to select one of the project task defined therein that has, for example, a greatest level of matching between the queried words and the set of keywords to find for that project task. A person is then identified as the speaker who is defined in the project database102as being associated with the project task that was selected.

The operation for identifying420as the speaker a person who is defined in the project database102as being associated with the project task that is selected, can include comparing (correlating)422spectral characteristics of a voice contained in the sampled audio stream, which was converted to the one of the text strings, to spectral characteristics that are defined for a plurality of persons who are identified by the project database102as being associated with the project task selected, and then selecting one person as the speaker from among the plurality of persons, who are identified by the project database102as being associated with the project task selected, based on a relatively closeness of the comparisons of spectral characteristics.

In a further embodiment, the comparison422of the spectral characteristics of the voice contained in the sampled audio stream to spectral characteristics that are defined for the persons who are defined by the project database102as being associated with the project task that was selected, can include comparing a spoken rate of words that are contained in the sampled audio stream, which was converted to the one of the text strings, to spoken rates of words that are defined for the plurality of persons who are defined by the project database102as being associated with the project task selected. For example, different persons can often be characterized by different speech rates (e.g., number of words spoken of a defined time period), and the characteristic speech rate for a defined person can be learned by the collaborative speech processing computer100and/or another system component and stored as information associated with that person's identifier in the project database102.

In another further embodiment, the comparison422of the spectral characteristics of the voice contained in the sampled audio stream to spectral characteristics that are defined for the persons who are defined by the project database102as being associated with the project task that was selected, can include comparing a frequency spectrum waveform in the sampled audio stream, which was converted to the one of the text strings, to frequency spectrum waveforms that are defined for the plurality of persons who are identified by the project database102as being associated with the project task that was. For example, different persons can often be characterized by different voice frequency spectrum waveforms (e.g., voice pitch and frequency waveform for various defined spoken words), and the characteristic voice frequency spectrum waveform for a defined person can be learned by the collaborative speech processing computer100and/or another system component and stored as information associated with that person's identifier in the project database102.

More generally, the operation for identifying214a speaker can include comparing422spectral characteristics of a voice contained in the sampled audio stream, which was converted to the one of the text strings, to spectral characteristics that are defined for a plurality of persons who are defined by the project database102, and selecting one person as the speaker from among the plurality of persons who are defined by the project database102based on a relatively closeness of the comparisons of spectral characteristics.

The collaborative speech processing computer100may operate to control430selection and/or filtering of sampled audio streams based on associated persons identify project roles.

In one embodiment, the collaborative speech processing computer100identifies time over-lapping voices in the sample audio streams. Responsive to the identification of the time over-lapping voices, the computer100tracks which of the speakers has begun talking before another one of the speakers has stopped talking. The computer100selects a microphone from among the array of microphones400which output audio signals contained in respective ones of the sampled audio streams, based on the microphone sensing voice from one of the speakers who is identified as having interrupted one or more other speakers at least a threshold number of times. The computer100responsively outputs430a control signal and/or filtering command signal to the selected microphone400that causes it to decrease amplification gain, (e.g., mute or decrease amplitude) applied to audio signals.

In another embodiment, the collaborative speech processing computer100selects a project task from among a plurality of project tasks defined in the project database102based on closeness of matching between words in one of the text strings to keywords defined for the project task, and identifies a person who is identified by the project database as being associated with the project task selected. The computer100then selects a microphone from among the array of microphones400that is determined to more strongly sense voice of the person identified relative to the other microphones, and responsively outputs430a control signal and/or filtering command signal to the selected microphone400that causes it to increase gain, (e.g., increasing amplitude and/or cease muting) applied to audio signals.

In a further operational embodiment, the computer100identifies persons who are defined in the project database as being associated with the project task selected. The computer100then selects a subgroup of the microphones that are determined to more strongly sense voices of those identified persons relative to the other ones of the microphones400. The computer100controls430filtering of sampled audio streams from those selected microphones. The computer100may output a control signal and/or filtering command signal that causes the selected subgroup of microphones to increase the amplification gain applied to audio signals they are sensing for output as a streaming audio signal.

In some embodiments, the collaborative speech processing computer100operational controls430selection and/or filtering of sample audio streams by controlling operation for 10 the processing of the sample audio streams for speech-to-text recognition. For example, the computer100may choose to ignore, e.g., not further process for sending to the natural language speech-to-text server130, audio streams from some microphones which are determined to be associated with sensing voices of persons who are not defined by the project database102as being associated with a project task that is currently being discussed. Alternatively, the computer100may send all audio streams to the natural language speech-to-text server130for conversion, but may only select text strings for addition to the dialogue data structure in the project planning database102that are determined to correspond to persons who are defined in the project database102as being associated with the project task that is being discussed. Selecting among which audio streams are processed and sent to the natural language speech-to-text server130for processing and/or selecting among which text strings are to be added to the dialogue data structure can reduce the operational processing by the collaborative speech processing computer100and by the natural language speech-to-text server130, decrease the data communication utilization of the network124and the network122, and decrease the possible erroneous speech recognition of background noise that may otherwise be created by persons don't have responsibility for the project task is presently being discussed.

The collaborative speech processing computer100adds318the text strings and the identifiers of the associated speakers to a dialog data structure in a repository memory, which may be a scrum meeting dialog data structure that resides in the project database102.

The collaborative speech processing computer100may update320tracking of project progress, individual project member contributions to the discussion of project tasks, and/or update an assessment of project risks based on the converted speech from the member discussions. The dialogue data structure in the project database wanted to can thereby be mined for project planning, tracking progress attributable to individual team members, identifying risks with individual project deliverables, etc.

In one embodiment, the collaborative speech processing computer100selects a project task from among a plurality of project tasks defined in the project database102based on closeness of matching between words in one of the text strings to keywords defined for the project task, and identifies a plurality of persons who are defined in the project database102as being associated with the project task that was selected. The computer100can then select at least one wireless user terminal110that has been a source of one of the sampled audio streams and that is determined to not be licensed to one of the persons identified, and responsively send430a microphone mute command to the at least one wireless user terminal110that was selected in order to trigger temporary muting of a microphone of the at least one wireless user terminal110. In this manner, the collaborative speech processing computer100can control which wireless user terminals110are operating to stream voices or other audio. Wireless user terminals that are being operated by persons who have been defined in the project database102as being responsible for a particular task that determined as being discussed can have their voices streamed through their respective wireless user terminals110, while other persons who are not defined in the project database102as being responsible for the particular task being discussed can have their voices muted by their respective wireless user terminals110.

In another embodiment, the collaborative speech processing computer100selects a project task from among a plurality of project tasks defined in the project database102based on closeness of matching between words in one of the text strings to keywords defined for the project task, and identifies a plurality of persons who are defined in the project database102as being associated with the project task selected. The computer100then selects at least one wireless user terminal110that has been a source of one of the sampled audio streams and that is determined to not be licensed to one of the persons identified, and send a stream pause command to the at least one wireless user terminal110selected in order to trigger the at least one wireless user terminal110to cease sending further data packets containing a sampled audio stream. Accordingly, a wireless user terminal that is being operated by a person who is not defined in the project database102as being responsible for a particular task that determined as being discussed can be controlled to stop sending data packets containing sampled audio streams, which can reduce the operational processing by the collaborative speech processing computer100and by the natural language speech-to-text server130, decrease the data communication utilization of the network124and the network122, decrease the wireless communication resources that are provided by the radio access network120and being utilized by the wireless terminals110, and decrease the possible erroneous speech recognition of background noise that may otherwise be provided by the wireless user terminal110operated by that person.

In a further embodiment, responsive to occurrence of a defined event after sending of the stream pause command to the selected wireless user terminal110, the collaborative speech processing computer100sends216a stream resumption command to the at least one wireless user terminal110that was selected in order to trigger the at least one wireless user terminal110to resume sending further data packets containing a sampled audio stream.

In another further embodiment, the operation for detecting occurrence of the defined event after sending of the stream pause command is based on determining that another project task has been selected from among the plurality of project tasks defined in the project database based on closeness of matching between words in another one of the text strings received after sending of the stream pause command to keywords defined for the project task. Thus, when the converted text strings are compared to the keywords in the project database102and result in determination that the conversation is now concerning another project task, the collaborative speech processing computer100can responsively send the stream resumption command to the wireless user terminal(s)110, which were earlier commanded to cease sending sampled audio stream data packets, to trigger those wireless user terminal(s)110to resume sending further data packets containing sampled audio streams.

FIG. 5is a block diagram of a speech processing computer system10that can be configured to perform operations in accordance with some embodiments. The system10can include the collaborative speech processing computer100, the project database102, and/or other system components configured to operate according one or more embodiments herein. Referring toFIG. 5, the system10can include network interface circuitry530which communicates via the one or more data networks122and/or124with the radio access network120, the project database102, the natural language speech-to-text server130, and/or other components of the system10. The system10includes processor circuitry510(hereinafter “processor”) and memory circuitry530(hereinafter “memory”) that contains computer program code nine twenty-two which performs various operations disclosed herein when executed by the processor510. The processor510may include one or more data processing circuits, such as a general purpose and/or special purpose processor (e.g., microprocessor and/or digital signal processor), which may be collocated or distributed across one or more data networks (e.g., network(s)124and/or122). The processor510is configured to execute computer program instructions among the program code522in the memory520, described below as a computer readable medium, to perform some or all of the operations and methods for one or more of the embodiments disclosed herein.

FIG. 6is a block diagram of a wireless user terminal110that can be configured to perform operations in accordance with some embodiments. Referring toFIG. 6, the terminal110can include a RF transceiver circuit630which use RF signaling according to one or more wireless communication protocols to communicate with the radio access network120. The wireless communication protocols can include, but are not limited to, wireless local area network (e.g., IEEE 802.11), Bluetooth, and/or one or more 3GPP cellular communication protocols such as 4G, 5G, etc. The terminal110includes processor circuitry610(hereinafter “processor”) and memory circuitry620(hereinafter “memory”) that contains computer program code622which performs various operations disclosed herein when executed by the processor610. Program code622can include the scrum application200described herein. The processor610may include one or more data processing circuits, such as a general purpose and/or special purpose processor (e.g., microprocessor and/or digital signal processor), which may be collocated or distributed across one or more data networks (e.g., network(s)124and/or122). The processor610is configured to execute computer program instructions among the program code622in the memory620, described below as a computer readable medium, to perform some or all of the operations and methods for one or more of the embodiments disclosed herein.

Further Definitions and Embodiments: