Queue management for visual interruption symbols in a virtual meeting

A system and method for identifying the next speaker from multiple meeting participants requesting to speak in a virtual meeting and persistently displaying a visual interruption symbol for the next speaker is disclosed. The system is configured to maintain a display of the visual interruption symbol along with a visual representation of the next speaker at a position of each meeting interface associated with meeting participants while changing displays of other visual representations of other participants at other positions of each meeting interface. The system is also configured to suppress displays of other visual interruption symbols from other participants of the virtual meeting.

TECHNICAL FIELD

The present disclosure relates to systems and methods for automatically adapting meeting interfaces of an application providing a virtual meeting in response to identifying the next speaker from multiple meeting participants requesting to speak in the virtual meeting, and persistently displaying a visual interruption symbol for that next speaker.

BACKGROUND

Virtual conferencing/meeting allows two or more people at multiple places to communicate with each other through video, audio, and text transmissions in an online meeting, which is particularly useful when a face-to-face meeting is unavailable or burdensome. To avoid interruption of another's presentation in a virtual meeting, an attendee/participant can raise a hand to indicate his/her desire to speak. Sometimes multiple participants may raise their hands to request speaking at approximately the same time, but only one participant can take the floor to talk at a certain time. Current virtual meeting systems can categorize and order requesting meeting participants, and highlight each participant in the category for displaying to participants. However, displaying each requesting participant as the current virtual meeting is underway tends to create sensory distractions. When user interactions such as sharing, chatting, talking occur in the virtual meeting while a participant is waiting to talk, a “raised hand” indication for this participant can easily be hidden in the interactions and thus be ignored. To timely identify and notify the next speaker to take over the meeting floor and talk, a meeting organizer may have to keep track of the “raised hand” indications, thereby impairing user experience. Another technical problem is that, when a requesting participant uses an electronic device with a small screen (e.g., a smartphone) to join the virtual meeting, the display of multiple “raised hand” indications may not fit in the small screen, and the requesting participant would miss his/her turn to talk unless he/she scrolls down the view to see all the “raised hand” indications.

Hence, there is a need for a system and method for queuing meeting participants requesting to speak in a virtual meeting and persistently highlighting one or more meeting participants in the next position(s) of the queue.

SUMMARY

In one aspect, a raising-hand queue management system for automatically adapting meeting interfaces of an application providing a virtual meeting in response to receiving interruption signals includes a processor, and a memory, coupled to the processor, configured to store executable instructions. The instructions, when executed by the processor, cause the processor to receive a plurality of interruption signals, from meeting interfaces of the application providing the virtual meeting, wherein the meeting interfaces are among a set of meeting interfaces associated with the application providing the virtual meeting on a respective set of client devices for a respective set of participants of the virtual meeting; based on a criterion associated with the virtual meeting: determine an order for surfacing the interruption signals, and identify, from the interruption signals, a first subset of interruption signals associated with a first subset of participants, wherein the first subset of interruption signals includes one or more of the received interruption signals but less than a total number of the received interruption signals; responsive to identifying the first subset of interruption signals, send a first interruption symbol raising signal over a communication network to each meeting interface of the set of meeting interfaces, the interruption symbol raising signal being configured to turn on a first subset of visual interruption symbols associated with the first subset of participants for displaying on each meeting interface, wherein the displayed first subset of visual interruption symbols includes one or more visual interruption symbols but less than a total number of the received interruption signals.

The instructions, when executed by the processor, also cause the processor to, responsive to detecting an interruption symbol lowering signal to turn off one of the first subset of visual interruption symbols associated with one of the first subset of participants on each meeting interface: update each meeting interface to turn off the one of the first subset of visual interruption symbols associated with the one of the first subset of participants; subsequent to turning off the one of the first subset of visual interruption symbols, automatically identify a next interruption signal for surfacing from the plurality of interruption signals other than the first subset of interruption signals according to the determined order; and send a next interruption symbol raising signal corresponding to the next interruption signal over the communication network to each meeting interface of the set of meeting interfaces to turn on a next visual interruption symbol associated with a next participant for displaying on each meeting interface.

In another aspect, a method of automatically adapting meeting interfaces of an application providing a virtual meeting in response to receiving interruption signals includes: receiving a plurality of interruption signals, from meeting interfaces of the application providing the virtual meeting, wherein the meeting interfaces are among a set of meeting interfaces associated with the application providing the virtual meeting on a respective set of client devices for a respective set of participants of the virtual meeting; based on a criterion associated with the virtual meeting: determining an order for surfacing the interruption signals, and identifying, from the interruption signals, a first subset of interruption signals associated with a first subset of participants, wherein the first subset of interruption signals includes one or more of the received interruption signals but less than a total number of the received interruption signals; responsive to identifying the first subset of interruption signals, sending a first interruption symbol raising signal over a communication network to each meeting interface of the set of meeting interfaces, the interruption symbol raising signal being configured to turn on a first subset of visual interruption symbols associated with the first subset of participants for displaying on each meeting interface, wherein the displayed first subset of visual interruption symbols includes one or more visual interruption symbols but less than a total number of the received interruption signals.

The method also includes, responsive to detecting an interruption symbol lowering signal to turn off one of the first subset of visual interruption symbols associated with one of the first subset of participants on each meeting interface: updating each meeting interface to turn off the one of the first subset of visual interruption symbols associated with the one of the first subset of participants; subsequent to turning off the one of the first subset of visual interruption symbols, automatically identifying a next interruption signal for surfacing from the plurality of interruption signals other than the first subset of interruption signals according to the determined order; and sending a next interruption symbol raising signal corresponding to the next interruption signal over the communication network to each meeting interface of the set of meeting interfaces to turn on a next visual interruption symbol associated with a next participant for displaying on each meeting interface.

DETAILED DESCRIPTION

When multiple meeting participants request speaking in a virtual meeting and only one participant can speak at a time, the existing virtual meeting systems categorize these requesting participants and turn on a visual interruption symbol for each participant in the category so that the participants displayed with the visual interruption symbols can speak in an order. The visual interruption symbol represents each participant's request to contribute to the virtual meeting. In one specific example, the visual interruption symbol corresponds to a raised hand symbol in Microsoft Teams® or Zoom® application. A technical problem herein is that the display of visual interruption symbols for each requesting participant may be unfocused by other interactions, thereby causing participants to spend extra time and resources on tracking the next participant to talk. For example, while a first requesting participant is waiting for his/her turn to speak in a meeting, other participants may be talking verbally, chatting with textual messages, transmitting files, sharing whiteboards, i.e., acting actively. Usually, these active participants and active interactions will appear in the main views of meeting interfaces associated with each meeting participant. This can easily occupy the relatively inactive display of the visual interruption symbol of the first requesting participant. The first participant needs to find a way to see the hidden display of the visual interruption symbol and speak at an appropriate time. Even if the display of the visual interruption symbols does not become inattentive/distracted by other interactions (e.g., in a virtual class where students merely ask questions), there is still another technical problem. An electronic device used for joining the virtual meeting has a limited display space, which may be insufficient to display every visual interruption symbol with every requesting participant. Therefore, it is very likely that the participant(s) in a lower order or priority may not know at what time he/she should talk unless he/she navigates to another view or menu of the meeting interface to obtain such information.

To address the above technical problems, the technical solution described in the present application persistently displays a visual interruption symbol only for the participant that is next in queue to speak and deliberately suppresses the display of other visual interruption symbols for other requesting participants until it is their turn to speak in the meeting. In one implementation, the technical solution includes server(s) and client devices. The server(s) may receive multiple interruption signals from multiple participants via multiple meeting interfaces associated with multiple client devices for requesting permission to speak in a virtual meeting. Upon receiving the multiple interruption signals, the server(s) may identify a first interruption signal received from a first participant based on a criterion associated with the virtual meeting, and send a first interruption symbol raising signal to each client device to turn on a first visual interruption symbol from the first participant for persistently displaying on each meeting interface. The first visual interruption symbol may be a raised hand symbol indicating that the first participant raises a virtual hand in the meeting via a first meeting interface.

By persistently displaying the first visual interruption symbol, the technical solution maintains a display of the first visual interruption symbol along with a visual representation of the first participant at a position of the meeting interfaces of meeting participants while changing displays of visual representations of other participants at other positions of the meeting interfaces. In other words, regardless of the frequent/continuous display changes of other participants with active interactions, the next participant in the queue, i.e., the next speaker, will stay in the upfront view of the meeting interfaces until this participant is no longer the next speaker, that is, he/she has started talking or has completed talking. By displaying only the first visual interruption symbol, the technical solution suppresses displays of other visual interruption symbols from other requesting participants. Therefore, one glance at the meeting interface would allow every meeting participant to know who the next speaker is. Accordingly, the technical benefits of the technical solution described herein include saving computing resources such as processing time, bandwidth, etc., otherwise spent on searching and finding the next speaker in the line. Especially when participants use small screen devices for joining the meeting, no navigation to layers of views/menus is needed, which significantly improves user experience.

The technical solution queues the received multiple interruption signals based on a criterion associated with the virtual meeting. The criterion includes at least one of timestamps associated with the interruption signals, user roles associated with participants of the virtual meeting, relevancy of the participants to the virtual meeting, organizer preferences, and other meeting policies associated with the virtual meeting. For example, the technical solution may recognize an important meeting participant and include a persistent visual representation for this important participant in the meeting interface when he/she signals to talk instead of queuing this participant based on the time of arrival. Therefore, important messages from the important participant can always be delivered in the limited time duration of the virtual meeting. In another example, if a participant frequently raises his/her hand and interrupts the meeting, the technical solution may move this participant towards the end of the queue such that the limited bandwidth and resource would be balanced to other participants. Therefore, ordering the requesting participants based on different factors relating to the virtual meeting and meeting participants improves the efficiency and effectiveness of the virtual meeting, thereby reducing the cost of computing resources used in the virtual meeting.

FIG.1is a block diagram of an example raising-hand queue management system100. As shown, the system100includes a virtual meeting server101and one or more client devices103a-103ncoupled for electronic communication via a network109. It should be understood that the system100depicted inFIG.1is provided by way of example and the system100and/or further systems contemplated by this present disclosure may include additional and/or fewer components, may combine components and/or divide one or more of the components into additional components, etc. For example, the system100may include any number of virtual meeting servers101, client devices103a-103n, or networks109.

The network109may be a conventional type, wired and/or wireless, and may have numerous different configurations including a star configuration, token ring configuration, or other configurations. For instance, the network109may include one or more local area networks (LAN), wide area networks (WAN) (e.g., the Internet), public networks, private networks, virtual networks, mesh networks, peer-to-peer networks, and/or other interconnected data paths across which multiple devices may communicate. The network109may also be coupled to or include portions of a telecommunications network for sending data in a variety of different communication protocols. In one implementation, the network109includes Bluetooth® communication networks or a cellular communications network for sending and receiving data including via short messaging service (SMS), multimedia messaging service (MMS), hypertext transfer protocol (HTTP), direct data connection, WAP, email, etc.

The client devices103a-103n(or collectively client device103) may include virtual or physical computer processors, memor(ies), communication interface(s)/device(s), etc., which, along with other components of the client device103, are coupled to the network109via signal lines113a-113nfor communication with other entities of the system100. In one implementation, the client device103a-103n, accessed by users125a-125nvia signal lines115a-115nrespectively, may send and receive data to and from other client device(s)103and/or the virtual meeting server101, and may further analyze and process the data. For example, the client devices103a-103nmay communicate with the virtual meeting server101to generate and update a meeting interface for display on each of the client devices103a-103n. Non-limiting examples of client device103may include a laptop computer, a desktop computer, a tablet computer, a mobile telephone, a personal digital assistant (PDA), a mobile email device, or any other electronic devices capable of processing information and accessing the network109.

The virtual meeting server101is a computing device that communicates with one or more client devices103to host an online virtual meeting and to process the data exchanged with the one or more client devices103. In one implementation, the virtual meeting server101receives a request from the user125a(e.g., a meeting organizer) via the client device103ato establish a virtual space for an online meeting. The virtual meeting server101supports chat service, calling service, and other services so that multiple participants can hear, see, chat, and share data with each other in the virtual space. The virtual meeting server101may also track and analyze the communication messages/signals used in the virtual meeting to instruct one or more client devices103to automatically update the meeting interfaces with information of turning on/off a visual interruption symbol for a next speaker and switching to display another visual interruption symbol for a subsequent next speaker. In one implementation, the virtual meeting server101may include a server-side application to communicate with client-side applications residing on the client devices103, and to provide the virtual meeting to participants via meeting interfaces. The application may be Microsoft Teams® application, Zoom® application, etc. The virtual meeting server101is communicatively coupled to the network109via signal line111.

As depicted, the virtual meeting server101includes a raising-hand queuing component105. In one implementation, the raising-hand queuing component105receives multiple interruption signals from multiple participants who want to talk in a virtual meeting. Instead of showing multiple requesting participants altogether on each meeting interface, the raising-hand queuing component105may display the requesting participants one by one based on a queue that orders the multiple requesting participants. For example, the raising-hand queuing component105may identify a first interruption signal from a first participant (e.g., the next speaker in the queue) and turn on only a first visual interruption symbol (e.g., a raised hand symbol) from the first participant on each meeting interface. This first visual interruption symbol will be persistent in the display of each meeting interface until the raising-hand queuing component105detects an interruption symbol lowering signal to turn off the first visual interruption symbol, e.g., after the first participant starts to talk or has completed talking in the meeting. Responsive to detecting the interruption symbol lowering signal, the raising-hand queuing component105may retrieve a next interruption signal associated with a next participant from the queue of the multiple interruption signals and turn on a visual interruption symbol from the next participant for persistently displaying on each meeting interface. The raising-hand queuing component105will be described in more detail below with reference toFIG.2.

Although only one raising-hand queuing component105is depicted inFIG.1, it is possible that one or more instances of raising-hand queuing component105also reside on the one or more client devices103. For example, an instance of the raising-hand queuing component105may be part of the serve-side application providing the virtual meeting on the virtual meeting server101while other instances of the raising-hand queuing component105may be included in corresponding client-side applications installed on the client devices103a-103n. Each instance may be configured to perform certain functionalities depending on where the instance resides. In one implementation, the virtual meeting server101may be a cloud server that possesses larger computing capabilities and computing resources than the client device103a, and therefore may perform more complex computation than the client device103acan. In the present disclosure, the raising-hand queuing component105mainly prioritizes the interruption signals collected from different meeting participants in a queue and performs subsequent operations based on the queue, which is typically implemented by the virtual server101. While the raising-hand queuing component105on the client side may perform some operations such as detecting, receiving, and sending signals that support the main functionalities performed at the server side, for simplicity and clarity,FIG.1does not include the instances of raising-hand queuing component105on the client side. The description below will be based on the system structure depicted inFIG.1.

In one implementation, the virtual meeting server101also includes a data store107. The data store107stores data related to different types of signals such as interruption signals, interruption symbol raising signals, interruption symbol lower signals, and acoustic signals. The data may include timestamps of signals (e.g., when a hand was raised), source of the signals (e.g., from which meeting interface the signal was sent), intensity levels of signals, etc. The data store107also stores any other information related to the operations described herein, such as interactions occurring during the time that a next participant is waiting for talk, information of participants requesting to speak, user roles of the participants, etc.

FIG.2is a block diagram of example components of a raising-hand queuing component105. The raising-hand queuing component105includes hardware and/or software logic for receiving multiple interruption signals representing multiple participants' requests to contribute to a virtual meeting, identifying, from the multiple participants, a small subset of participant(s) that will contribute/speak next in the meeting, and displaying only a visual interruption symbol for this identified participant(s) on each participant's meeting interface. By not displaying every visual interruption symbol for every requesting participant, the raising-hand queuing component105reduces the amount of signals/data exchanged between the client devices103and the virtual meeting server101, thereby improving user navigation experience and increasing operational efficiency.

In one implementation, the raising-hand queuing component105includes a detection module201, a queue engine203, an action engine205, and a user interface module207. The detection module201receives multiple interruption signals from multiple participants of a virtual meeting and, in response, transmits the received interruption signals along with associated metadata to notify the queue engine203to identify a predetermined number of interruption signal(s). The multiple interruption signals represent the multiple participants' requests to speak in the virtual meeting. The predetermined number of interruption signal(s) are associated with a predetermined number of participant(s) who will speak next in the meeting since the multiple participants cannot speak simultaneously in the meeting. The identification of the predetermined number of interruption signal(s) will be described below in more detail with reference to the queue engine203.

In one implementation, the multiple participants may send the multiple interruption signals via multiple meeting interfaces to request to speak in the meeting without interrupting the meeting. The multiple meeting interfaces are among a set of meeting interfaces associated with the virtual meeting that are displayed on the client devices103for each meeting participant. For example, one participant may signal that he/she wishes to speak or contribute to the meeting by selecting/tapping/touching a “Raise Hand” option included in a meeting interface as depicted inFIG.5A. In one implementation, each interruption signal is associated with a timestamp. The timestamp shows the time that the one participant requested permission to speak. In another implementation, each interruption signal is also associated with other metadata such as a source, times of the interruption signal was received from the source within a certain period of time, etc. The detection module201may send the received interruption signals associated with the metadata to the queue engine203for identifying the predetermined number of interruption signal(s) or a first subset of interruption signals.

Once receiving the identified predetermined number of interruption signal(s) or the first subset of interruption signals, e.g., a first interruption signal from a first participant that will talk next in the meeting, the detection module201may generate and send a first interruption symbol raising signal over a communication network (e.g., the network109) to each of the client devices103to turn on a first visual interruption symbol from the first participant for persistently displaying on each meeting interface. It should be noted a first signal, first symbol or first participant may be more often used in the below description, these terms could also mean first subset of signals, symbols, or participants. The first visual interruption symbol for the first participant will maintain an “on” status, i.e., visible to every participant, until the first participant is no longer a next speaker. A visual interruption symbol may be an icon, a flag, a button, or other types of graphical symbol that identifies a unique “raised hand” status, that is, the first participant is requesting permission to speak or contribute. For example, the visual interruption symbol is a raised hand symbol as depicted inFIG.5B.

After the transmission of the first interruption symbol raising signal, the detection module201detects whether there is an interruption symbol lowering signal from the first participant. The interruption symbol lowering signal indicates that the first participant is talking or has completed talking, and thus is no longer a next speaker. In one implementation, the detection module201may detect the interruption symbol lowering signal for turning off the first visual interruption symbol responsive to receiving a user operation from at least one of the first participant or a meeting organizer. For example, after selecting a “Raise Hand” option to trigger the first visual interruption symbol to appear in the meeting interface, the first participant can select this option again to make the symbol disappear. In another example, the meeting organizer may have control to turn off the first visual interruption symbol by selecting a different interface element.

However, user operations are unpredictable. In the case that both the first participant and meeting organizer fail to timely turn off or lower down the first visual interruption symbol, in one implementation, the detection module201may notify the queue engine203to identify the next speaker for persistently display. In the meantime, the detection module201may also communicate with the action engine205to turn off the symbol for the first participant. In another implementation, after a threshold amount of time has passed without receiving the user operation, the detection module201may enable a prompt to be generated to the first participant. The prompt reminds the first participant to lower down the first visual interruption symbol/hand within a second threshold amount of time. If still no user operation is timely received, the detection module201again notifies the action engine205to turn off the hand for the first participant.

In another implementation, the detection module201may automatically generate the interruption symbol lowering signal responsive to detecting an acoustic signal from the first participant. The acoustic signal includes a voice of the first participant indicating that he/she has started talking for a threshold amount of time (e.g., five seconds) or has completed talking in the virtual meeting. For example, the detection module201may communicate with a recognition engine (not shown) to identify the voice of the first participant based on the intensity level of the voice, the specific client device (e.g., client device103a) that the first participant uses for accessing the virtual meeting, etc.

If the detection module201detects the interruption symbol lowering signal from the first participant, it will trigger the first visual interruption symbol from the first participant to be turned off on each meeting interface. More importantly, responsive to detecting this interruption symbol lowering signal, the module(s)/engine(s) of the raising-hand queuing component105may cooperate with each other to start a new cycle for identifying the next participant who should talk after the first participant and displaying a new/next visual interruption symbol corresponding to this next participant on each meeting interface associated with the meeting participants. In other words, the next participant will replace the first participant for displaying persistently.

The queue engine203receives the multiple interruption signals and associated metadata from the detection module201, and identifies a predetermined number of interruption signals from the participant(s) who will speak next in the virtual meeting. The predetermined number is an integer less than the number of the multiple signals/participants. Typically, the predetermined number is one. In one implementation, a participant (e.g., a meeting organizer) may configure the predetermined number via a meeting interface. This number is usually kept small (e.g., one or two) such that the limited display space of meeting interfaces associated with the client device103will not be crowded by highlighted participants and thus lose the meaningful highlight of a next speaker. In the description hereafter, the predetermined number will be considered as one unless specified otherwise.

In one implementation, the queue engine203may configure and manage a queue of the multiple interruption signals received from the detection module201. The queue engine203may prioritize the multiple interruption signals in the queue according to a criterion and identify the interruption signal from a next speaker, i.e., “next speaker signal,” based on the queue. For example, the queue engine203may identify the interruption signal at the beginning or the first position of the queue as the next speaker signal. The criterion may include at least one of timestamps associated with the multiple interruption signals, user roles associated with participants of the virtual meeting, relevancy of the participants to the virtual meeting, organizer preferences, or any combination thereof.

In one implementation, the queue engine203may manage the queue based on timestamps associated with the multiple interruption signals (i.e., timestamp criterion). A timestamp indicates the time that a corresponding signal was received. In such a case, the queue engine203implements the queue as a first-in-first-out (FIFO) queue in which an interruption signal with a most recent timestamp, i.e., the latest signal, is placed at the end or last position of the queue. The queue engine203may also identify the interruption signal that entered the queue at the earliest time as the next speaker signal.

In one implementation, the queue engine203may also manage the queue based on user roles associated with participants of the virtual meeting (i.e., user role criterion). A user role may include a job role, a meeting role, etc. A job role is the function that a meeting participant fills within a corporation, such as a chief executive officer (CEO), a department manager, an engineer, etc. In one implementation, the queue engine203may query a corporation directory to obtain the job role of each meeting participant and apply a policy criterion relating to job roles to the ordering of the queue. For example, if a CEO raises his/her hand in the virtual meeting, the queue engine203may place the corresponding interruption signal in the beginning of the queue even if another meeting participant has raised hand before the CEO. This queue arrangement can guarantee that the interruption signal from the CEO is the next speaker signal and the CEO is the next speaker, and thus assure that important messages from participants with important job roles be delivered.

On the other hand, the meeting role of a participant may include an organizer, a presenter, an audience, etc. The queue engine203also orders the queue using the meeting roles. In one implementation, the queue engine203may associate/create an interruption signal to the meeting organizer and leave a position of the queue to this organizer interruption signal such that the meeting organizer can take over the floor to speak in the meeting when needed. For example, when a participant starts to talk in the meeting and the queue of interruption signals is empty (i.e., no other participants are waiting to talk), the queue engine203may move the organizer interruption signal towards the top of the queue so that the meeting organizer can host the meeting after the current participant finishes talking. In another example of a classroom meeting where a professor is a presenter, the queue engine203may arrange the queue in a way that the professor is always the next speaker when he/she wants to talk in the class.

In one implementation, the queue engine203may also manage the queue based on relevancy of the participants to the virtual meeting (i.e., relevancy criterion). The queue engine203may determine the relevancy based on a user role of the participant and/or the characteristics of the meeting. The meeting characteristics may include type, subject, content, scheduled time of the meeting, scheduled conference room of the meeting, etc. For example, the queue engine203may determine that a department manager is more relevant to a department meeting of his/her department than an executive officer who oversees every department meeting. Accordingly, when both the department manager and executive officer intent to talk in the meeting, the queue engine203may assign a position closer to the beginning of the queue to the interruption signal of the department manager rather than to the interruption signal of the executive officer. In another example, for a patent disclosure meeting, the queue engine203may determine that a patent counsel's opinion is critical and thus assign a higher priority to the counsel over other meeting participants.

The queue engine203is also configured to adjust the relevancy determination and adjust the order of signals in the queue accordingly. For example, based on the scheduled time or agenda of the meeting, the queue engine203may determine that the virtual meeting will discuss a first department during a first time period and discuss a second department during a second time period. Responsive to receiving the requests to speak from first and second department managers at the first and second time periods, the queue engine203may prioritize the first department manager over the second department manager during the first time period while prioritizing the second department manager over the first department manager during the second time period. In another example, if a virtual class is moved from a first classroom to a second classroom (e.g., a simulated courtroom), the queue engine203may modify the queue of interruption signals because the professor is the meeting presenter in the first classroom while the students are presenters in the second classroom.

In one implementation, the queue engine203may also manage the queue based on organizer preferences (i.e., organizer criterion). In this scenario, the next speaker identified by the meeting organizer will be moved to the beginning of the queue. The meeting organizer may set up his/her preference rules or select a next speaker from a list of requesting participants via a configuration interface as depicted inFIG.5F. For example, if a participant raised his/her hand more than a threshold number of times during a specific time period (e.g., more than two times within 10 minutes), the queue engine203may move the position of the interruption signal associated with this participant to the end of the queue according to the organizer preferences. This can prevent a few audiences from repeatedly raising their hands and distracting the presenter. In another example, the queue engine203may allow the meeting organizer to select a participant who barely talked in the meeting but eventually raised his/her hand as the next speaker.

It should be noted that the queue engine203may further manage the queue based on other meeting policies associated with the virtual meeting. For example, the queue engine203may cooperate with the action engine205and the user interface module207to display at least two persistent images: one for the next speaker, and another for the CEO. Therefore, whenever the CEO wants to talk, he/she can talk in the meeting right after the current speaker is done talking. According to a different corporate policy, however, the queue engine203may determine that the CEO's hand-raising does not supersede another participant's hand-rising if they are in a casual meeting discussing, for example, where to picnic.

As described above, due to the application of the criterion such as user role criterion, relevancy criterion, or organizer criterion, the queue is not necessarily a FIFO queue determined based on the time of arrivals of the interruption signals. In such a case, there will be a queue delay for some participants. For example, the participant(s) who raised hand(s) and triggered interruption signal(s) before the CEO raised his/her hand will be delayed because the queue engine203moves the CEO to the first position of the queue according to the user role criterion. In one implementation, the queue engine203may set up a delay cap policy to include a maximum amount of time or a maximum number of times that can be delayed. Therefore, after identifying an interruption signal as a next interruption signal by overriding the timestamp criterion and applying a different criterion, the queue engine203will determine the other interruption signal, i.e., the overridden signal, as a next interruption signal before a threshold amount of time elapses or threshold numbers of delay runs out. In other words, a deferred participant will be able to talk within a maximum time or number of delays. This inclusive cap policy allows each participant's activity to be considered and conducted, thereby improving user experience in the virtual meeting.

The queue engine203arranges the queue of the multiple interruption signals received from the detection module201in an order determined based on a criterion, which corresponds to an ordered set of participants scheduled to speak in a virtual meeting. In one implementation, the queue engine203may identify a predetermined number of interruption signals from a predetermined number of positions in the beginning of the queue, and correspond each of the identified interruption signals to a specific participant. The position of an interruption signal in the queue reflects the order that the corresponding participant will talk in the meeting. For example, if only one interruption signal is identified, the corresponding participant would be the next speaker. If two interruption signals in the first and second positions are identified, the participant corresponding to the signal in the first position will talk and then the participant corresponding to the signal in the second position will talk. The predetermined number is usually one or two.

In one implementation, the queue engine203may transmit information of the identified participant(s), e.g., the next speaker or the speaker subsequent to the next speaker, to the action engine205for further processing. The queue engine203may then remove the identified predetermined number of interruption signals from the queue and re-order the queue. The queue engine203may also store the information of the queue in the data store107.

Responsive to receiving the information of identified participant(s) from the queue engine203, the action engine205may send an interruption symbol raising signal to each meeting interface to turn on a visual interruption symbol from the identified participant(s) (e.g., the first participant) for persistently displaying on each meeting interface associated with meeting participants. In one implementation, the action engine205may instruct the interface module207to display a persistent image for the first participant on each meeting interface. The persistent image includes the visual interruption symbol (e.g., a “raised hand” symbol) posed on a visual representation of the first participant (e.g., a picture, an avatar). In contrast, other un-identified participants are only displayed on each meeting interface with corresponding visual representations. In the cases where more than one participant, e.g., the first participant (the next speaker) and a second participant (the next speaker after the first participant), are identified, the action engine205may also include a number that reflects the speaking order of the first and second participants in each persistent image. In other cases, the action engine205may also change the color, shape, or frame of the persistent image to distinguish the first participant from other requesting participant(s), i.e., awaiting speaker(s), and non-requesting meeting participants on each meeting interface. A person with ordinary skill may recognize there are other ways to display the next speaker(s) differently from other participants.

The action engine205operates to display the identified next speaker(s), i.e., the first participant, in a persistent way. In one implementation, the action engine205maintains a display of the visual interruption symbol along with the visual representation of the first participant at a position of the meeting interface while changing the displays of other visual representations of other participants at other positions of the meeting interface. For example, a meeting interface may include a specified number (e.g., nine) of grids or tiles. Each grid includes a visual representation of a participant. When more than the specified number of participants are in the meeting, only visual representations of partial participants will be displayed in the grids. The first participant may or may not be included in the grids. However, when the first participant is identified as the next speaker, the action engine205will include the persistent image of the first participant in one of the grids. This persistent image, in particular, the visual interruption symbol or the “raised hand” symbol associated with the visual representation of the first participant, will stay at that position of the grids until the first participant is no longer the next speaker. The first participant becomes a current or past speaker after the first participant starts talking or has completed talking in the meeting. Once the visual interruption signal of the first participant disappears, the visual interruption signal of the next participant after the first participant will show up if the queue of interruption signal is not empty. During the time period that the action engine205keeps the persistent image of the first participant unchanged in that position, the display of other grids can be freely changed. For example, two participants that are actively exchanging chat messages may appear in the grids to replace two other inactive participants. Therefore, regardless of the interactions among other participants and consequent display changes of other visual representations of other participants, the visual representation and visual interruption symbol of the first participant remains untouched in a position of the meeting interface. The position may be any position of the grid view, not necessarily the center position.

When displaying the persistent image of the first participant, the action engine205also suppresses displays of other visual interruption symbols from other participants of the virtual meeting. The action engine205extracts a predetermined number of next speaker(s) based on the queue of interruption signals and highlights only the extracted speakers on the meeting interface. The action engine205does not distinguish other requesting participants in the display although these participants are waiting to speak in an order indicated by corresponding positions in the queue of interruption signals. Therefore, the action engine205does not need to give the participant associated with the beginning position/index of the queue, i.e., the next speaker or the first participant, the center of attention by placing the first participant in a central grid of the meeting interface. The next speaker(s) stand out when only one or two of the participants are highlighted and highlighted persistently.

In one implementation, the action engine205may turn off/lower down the visual interruption symbol by the first participant after the first participant has contributed to the meeting and is no longer a next speaker. In one implementation, the action engine205automatically generates an interruption symbol lowering signal and sends the interruption symbol lowering signal to each of the client devices103such that the client devices103can automatically update the display of each meeting interface and turn off the visual interruption symbol. For example, the action engine205may automatically remove a “raised hand” flag or icon of the first participant from a position of the grid of each meeting interface associated with meeting participants as a result of turning off the visual interruption symbol. In another implementation, the action engine205receives an interruption symbol lowering signal from the detection module201and notifies the client devices103to lower down the visual interruption symbol associated with the first participant.

In one implementation, the action engine205may automatically turn off the visual interruption symbol or lower the first participant's hand in two scenarios: (1) after detecting that the first participant starts talking or (2) after detecting that the first participant stops talking. In the first scenario, the action engine205may generate the interruption symbol lowering signal to lower the “raised hand” after a specific amount of time (e.g., five seconds) of receiving the acoustic signal including the first participant's voice. Therefore, the first participant's hand may be lowered when he/she is still talking. In the second scenario, the action engine205may determine the received acoustic signal no longer include the voice from the first participant and turn off the visual interruption signal. In other words, the action engine205lowers the first participant's hand after he/she has done talking in the meeting. In another implementation, the action engine205may turn off the visual interruption symbol of the first participant when the first participant or the meeting organizer acts to manually lower the hand. Once the visual interruption symbol of the first participant is turned off, the action engine205cooperates with other module(s) and/or engine(s) of the raising-hand queuing component105to identify the next speaker after the first participant for persistently display. The overall procedure for identifying and displaying a next speaker will be described below with reference toFIGS.3and4.

In one implementation, if the first participant or the meeting organizer fails to timely lower down the hand of the first participant, the action engine205may cooperate with the user interface module207to generate a prompt to remind the first participant to lower down the hand so that another cycle of identifying a next speaker can start. The action engine205may also communicate with the user interface module207to provide, on the first meeting interface, the first participant an option to configure whether to manually or automatically lower his/her hand. The action engine205may further instruct the user interface module207to generate other notifications, for example, allow a meeting organizer to select a value of the predetermined number to control how many “next speakers” can be concurrently displayed on the meeting interface.

The user interface module207receives instruction(s) from the action engine205to send graphical data to the client device103, causing the client device103to present a user interface to a user or a participant. Example user interfaces are shown inFIGS.5A-5G. In one implementation, the user interface module207generates the graphical data for providing a user interface that presents a selectable “Raise Hand” option to a participant or displays a visual interruption symbol for the participant once the participant selects the “Raise Hand” option. In another implementation, the user interface module207may generate a notification or a prompt for display on the meeting interface associated with one or more users. In yet another implementation, the user interface module207may generate a meeting interface that allows a participant to provide input for configuring specific functionalities. The user interface module207may generate graphical data for providing other user interfaces to meeting participants.

FIG.3is a block diagram of an example procedure of identifying and displaying a next speaker from multiple meeting participants requesting to speak in a virtual meeting. A set of participants including participants X, Y, and Z use the client devices103a-103nto join the virtual meeting. The three participants X, Y, and Z are also separately depicted in the middle ofFIG.3to explain the overall queuing and display process. At the approximate same time, each of the participants X, Y, and Z respectively selects a “Raise Hand” option via a meeting interface respectively displayed on the client device103a,103band103c. X's selection triggers an interruption signal x to be sent at301ato a virtual meeting server101. Y's selection triggers an interruption signal y to be sent at301bto the virtual meeting server101. Z's selection triggers an interruption signal z to be sent at301cto the virtual meeting server101. These signals indicate that X, Y, and Z are requesting permission to speak in the meeting.

The virtual meeting server101creates a queue to include the three interruption signals x, y, and z. The virtual meeting server101may use the queue of signals x, y, and z, which are respectively received from participants X, Y, and Z, to identify which of X, Y, and Z is a next speaker. Depending on the different criteria used for arranging the signals in the queue, the queue can be different. For example, if the virtual meeting server101uses timestamps associated with the three signals as the criterion, the signals will be ordered as in the first queue351. However, if the virtual meeting server101uses another criterion such as the user role criterion to form the queue, the signals will be ordered as in the second queue353. Although X is the first participant requesting to speak, Y, is an important participant that prioritizes over X when the user role criterion is applied. Therefore, in the second queue353, signal y from participant Y is in the first position while signal x from participant X is in the second position. In other words, the next speaker will be X if the first queue351is used while the next speaker will be Y if the second queue353is used. Suppose the virtual meeting server101applies the user role criterion to queue the signals, the result would be identifying interruption signal y from a next speaker Y at303.

Responsive to the identification of interruption signal y, at305, the virtual meeting server101communicates with each client device103to update the meeting interface associated with each participant in the virtual meeting and turn on a visual interruption symbol associated with Y on each meeting interface. Although three participants X, Y, and Z have requested to speak, only speaker Y is identified as the next speaker and is highlighted with the visual interruption symbol to every meeting participant.

Once Y is displayed as the next speaker, the virtual meeting server101may remove the corresponding signal y from the queue353. As a result, the queue353is changed to queue355. The queue355includes only signals x and z, where the positions of x and z indicate that the next speaker after Y is X. In the meantime, Y is waiting for his/her turn to talk. If Y starts to talk in the meeting (e.g., for a threshold amount of time) or has completed talking, Y may manually lower down his/her raised hand, or a meeting organizer may manually lower down the raised hand for Y, or the virtual meeting server101may automatically lower down the raised hand for Y. This causes an interruption symbol lowering signal to be generated and sent to the virtual meeting server101at307.

Responsive to receiving the interruption symbol lowering signal, at309, the virtual meeting server101communicates with each client device103to update the meeting interface associated with each participant in the virtual meeting and turn off the visual interruption symbol associated with Y on each meeting interface. It should be noted that during the entire time that Y was identified as a next speaker until Y was no longer the next speaker, Y's visual interruption symbol is persistently displayed to every meeting participant no matter how many changes affected the display of other participants.

The virtual meeting server101now starts a new cycle to find a next speaker from the rest of the requesting participants. At311, based on the queue355, the virtual meeting server101identifies the interruption signal x from a next speaker X. Then, at313, the virtual meeting server101communicates with the client device103to update each meeting interface to turn on a visual interruption symbol associated with X on each meeting interface. With the display of the next speaker X to every meeting participant, the queue355becomes the queue357. The queue357includes only signal z indicating that the last speaker or the next speaker after X is Z. At315, the virtual meeting server101receives the interruption symbol lowering signal from X to release X's persistent display. As a result, the virtual meeting server101may cooperate with the client device103to start another cycle to let Z take the floor to talk in the meeting. The steps performed in this cycle are similar and thus skipped inFIG.3.

FIG.4is a flowchart of an example method400for automatically adapting meeting interfaces of an application providing a virtual meeting in response to receiving interruption signals by a raising-hand queuing component105of the virtual meeting server101. At block401, the raising-hand queuing component105receives multiple interruption signals, from multiple meeting interfaces of the application providing the virtual meeting. An interruption signal represents a participant's intention/request to contribute to the virtual meeting. The multiple meeting interfaces are among a set of meeting interfaces associated with the application providing the virtual meeting on a respective set of client devices for a respective set of participants of the virtual meeting.

At block403, based on a criterion associated with the virtual meeting, the raising-hand queuing component105identifies, from the multiple interruption signals, a first subset of interruption signals associated with a first subset of participants. For example, the raising-hand queuing component105may determine an order for surfacing the interruption signals based on the criterion and identify a first subset of interruption signals based on the order. The first subset of interruption signals includes one or more of the received interruption signals or (e.g., a predetermined number of interruption signals) but less than a total number of the received interruption signals. A predetermined number is a small number like one or two. When this number is one, a single next speaker, e.g., the first participant, is identified. When this number is two, two speakers in the first and second positions of the queue are identified. The criterion may include at least one of timestamps associated with the multiple interruption signals, user roles associated with participants of the virtual meeting, relevancy of the participants to the virtual meeting, organizer preferences, other meeting policies associated with the virtual meeting, or any combination thereof.

At block405, responsive to identifying the first subset of interruption signals, the raising-hand queuing component105sends a first interruption symbol raising signal over a communication network to each meeting interface of the set of meeting interfaces. The interruption symbol raising signal is configured to turn on a first subset of visual interruption symbols associated with the first subset of participants for displaying on each meeting interface. The displayed first subset of visual interruption symbols includes one or more visual interruption symbols but less than a total number of the received interruption signals. The visual interruption symbol may be an icon, a flag, a button, or other types of graphical symbol that identifies a unique “raised hand” status, that is, the first participant is requesting permission to speak. The display may be persistent, that is, the raising-hand queuing component105may maintain a display of each of the first subset of visual interruption symbols along with a visual representation of a corresponding participant of the first subset of participants at a position of each meeting interface while changing displays of one or more visual representations of one or more other participants at one or more other positions of each meeting interface. The persistent display also means that the raising-hand queuing component105may suppress displays of other visual interruption symbols associated with other participants of the virtual meeting.

Responsive to detecting an interruption symbol lowering signal to turn off one of the first subset of visual interruption symbols associated with one of the first subset of participants on each meeting interface, at block407, the raising-hand queuing component105updates each meeting interface to turn off the one of the first subset of visual interruption symbols associated with the one of the first subset of participants. Once this first visual interruption symbol is turned off or lowered down, a next speaker needs to be surfaced to show on the display. At block409, subsequent to turning off the one of the first subset of visual interruption symbols, the raising-hand queuing component105automatically identifies a next interruption signal for surfacing from the plurality of interruption signals other than the first subset of interruption signals. At block411, the raising-hand queuing component105sends a next interruption symbol raising signal corresponding to the next interruption signal over the communication network to each meeting interface of the set of meeting interfaces to turn on a next visual interruption symbol associated with a next participant for displaying on each meeting interface. The identification of the next participant is according to the order determined by queuing the interruption signals based on the criterion as described in block403.

FIG.5Aillustrates an example graphical user interface500presenting a “Raise Hand” option. The user interface500is a meeting interface associated with an online virtual meeting that is displayed for a specific meeting participant (i.e., user CD at501). The meeting interface500includes an area503for displaying visual representations of participants. As depicted, area503includes nine grids, and a visual representation504of user CD is displayed in grid505. The virtual meeting has a total number of 11 participants as shown in505, where only nine participants are displayed in area503due to the limited spaces. In addition to the number indicator506, an action bar507also includes a time indicator showing the progress of the virtual meeting and other options, for example, a “Raise Hand” option in509. Multiple meeting attendees or participants, who intend to speak, can simply click the “Raise Hand” option509to raise a virtual hand.

FIG.5Billustrates an example graphical user interface510displaying a next speaker. User CD is one of the multiple participants who selected the “Raise Hand” option shown inFIG.5A. Based on an ordered queue, CD is identified as the next speaker. The user interface510is the meeting interface of user DS511, which shows that CD is highlighted with a persistent image. The persistent image of CD in grid513includes the visual representation504and a visual interruption symbol515. The visual interruption symbol515shows that CD has raised a virtual hand and the persistent image of CD shows that CD is the next speaker in the queue. At this first time point, there are still 11 participants in the meeting as shown in516. The visual representations517,518, and519around the persistent image of CD respectively represent users AB, GM, and UV.

FIG.5Cillustrates an example graphical user interface520showing the persistent display of the next speaker inFIG.5B. The user interface520is the meeting interface of user DS511at a second time point after the first time point ofFIG.5B. At the second time point, CD still has not got permission to talk, and therefore, the display of CD is persistent in grid513. Neither the visual representation504nor the visual interruption symbol515are changed. However, the meeting participants drop from11to8as shown in523. Therefore, only eight visual representations are shown in nine grids, which leaves the previously represented user UV in529to become blank. Also, the participant represented in grid525is changed from AB to RST, and the participant represented in grid527is changed from GM to DS. ComparedFIGS.5B and5C, CD is displayed persistently as long as CD is still the next speaker.

FIG.5Dillustrates an example graphical user interface530showing a different persistent image. Instead of having a color different from other grids, the persistent image in grid513has a different frame. A person of ordinary skill in this art would realize there are many other ways to display a persistent image.

FIG.5Eillustrates an example graphical user interface540showing two persistent images concurrently. A predetermined number of next speaker(s) can be identified responsive to multiple meeting participants raising hands to request to speak. When the number is one, only one user, CD, is identified as shown inFIGS.5B-5D. When the number is changed to two as shown in this figure, another user XY is also identified. CD still has a persistent image in grid513, but a number 1 is added to show that he/she will be the next speaker. The newly identified XY has a persistent image in grid543, which includes number 2 in543to show that he/she would talk after CD.

FIG.5Fillustrates an example graphical user interface550showing a configuration interface presented to a meeting organizer JD in551. As depicted, when JD selects the visual interruption symbol515included in the persistent image of the next speaker CD, a menu can be populated for JD to perform some configurations. For example, JD may select the “Highlight Number” option in553to set an integer value in555. This value is the predetermined number that defines how many next speakers can be identified and displayed concurrently on the meeting interface. InFIG.5B-5D, this value is one. InFIG.5E, this value is two. The meeting organizer JD may also select “Organizer Preference Configuration” in557to set up organizer preferences or an organizer criterion. For example, JD can configure preferences using different rules as shown in558, or choose a next speaker using the “Select Highlighted” option in559. When options558and/or559are selected, other interface elements will show up to help JD to finish the configuration task.

FIG.5Gillustrates an example graphical user interface560showing a lowering-hand notification. The user interface560is the meeting interface of participant CD at561after CD has talked in the meeting in response to his/her raising hand request. At this point, the persistent image513should be removed as CD is no longer a next speaker. To allow the speaker next in line to be identified and displayed, CD's visual interruption symbol or hand in515needs to be lowered down. However, if a threshold amount of time passed and the hand in515is still on, a reminder563will be generated and presented to CD. CD can choose “Yes” in565or “No” in567to determine whether to lower down the hand within 10 minutes. If not, the visual interruption symbol515will be automatically cleared.

FIG.6is a functional block diagram of an example computer system600upon which aspects of this disclosure may be implemented. It will be understood that the logic blocks illustrated inFIG.6represent functions, and do not necessarily correspond to particular hardware on a one-to-one basis. The computer system600can include a data processor604, coupled by a bus602to an instruction memory or main memory606, a read-only memory608, and a storage device610. The instruction memory606can include a tangible medium retrievably storing computer-readable instructions, that when executed by the data processor604cause the processor to perform functions, processes, and operations described herein, for example, in reference toFIGS.1-4.

The computer system600can also include a display612, a user interface or other input device614, and a cursor control616, either as separate devices or combined, for example, as a touchscreen display. The computer system600can also include a communications interface618, configured to interface with a local network622by a bus620for accessing a local host server624, and to communicate through an ISP626to the Internet628, and access a remote server630.

Some implementations comprise a computer-readable storage media including executable instructions, that when executed on at least one processor, cause the processor to perform any of the above-described methods.