TECHNIQUES FOR RICH INTERACTION IN REMOTE LIVE PRESENTATION AND ACCURATE SUGGESTION FOR REHEARSAL THROUGH AUDIENCE VIDEO ANALYSIS

Techniques performed by a data processing system for facilitating an online presentation session include establishing the session for a first computing device of a presenter and a plurality of second computing devices of a plurality of participants, receiving a set of first media streams comprising presentation content from the first computing device, sending a set of second media streams to the plurality of second computing devices, receiving a set of third media streams from the computing devices of a first subset of the plurality of participants including video content of first subset of the participants captured by the respective computing devices of the first subset of participants, analyzing the set of third media streams to identify a set of first reactions by the first subset participants to obtain first reaction information, determining first graphical representation information representing the first reaction information, and sending a fourth media stream to cause the first computing device to display the first graphical representation information while the presentation content is being provided via the set of first media streams.

BACKGROUND

Many workplaces, schools, universities, and other organizations which may traditionally conduct in-person meetings, classes, and/or presentations have had to quickly adapt to remote presentations. Workplaces may conduct meetings and/or presentations with colleagues and/or clients via remote videoconferencing and/or collaboration platforms. Teachers and professors may conduct classes using similar technologies which allow the teachers and professors to present lectures and/or interact with their students via a virtual classroom setting provided by a remote videoconferencing and/or collaboration platform.

With an in-person meeting, the presenter can readily interact with audience members to ask questions, answer questions, and/or receive other user feedback. However, with a remote presentation and remote learning, the presenter may have a more difficult time engaging with the audience due to the lack of direct interaction with the audience. Hence, there is a need for improved systems and methods of remote audience interaction for improving audience engagement.

SUMMARY

An example data processing system according to the disclosure may include a processor and a computer-readable medium storing executable instructions. The instructions when executed cause the processor to perform operations including establishing an online presentation session for a first computing device of a presenter and a plurality of second computing devices of a plurality of participants, receiving, via a network connection, a set of first media streams comprising presentation content from the first computing device of the presenter, sending, via the network connection, a set of second media streams to the plurality of second computing devices of the plurality participants, wherein content of the set of second media streams is based on content the set of first media streams, receiving, via the network connection, a set of third media streams from the second computing devices of a first subset of the plurality of participants, the set of third media streams including video content of first subset of the plurality of participants captured by the respective second computing devices of the first subset of the plurality of participants, analyzing the set of third media streams to identify a set of first reactions by the first subset of the plurality of participants to obtain first reaction information, determining first graphical representation information representing the first reaction information, and sending, via the network connection, a fourth media stream to the first computing device that includes the first graphical representation information to cause the first computing device to display the first graphical representation information on a display of the first computing device while the presentation content is being provided via the set of first media streams.

An example method implemented in a data processing system for facilitating an online presentation session includes establishing an online presentation session for a first computing device of a presenter and a plurality of second computing devices of a plurality of participants, receiving, via a network connection, a set of first media streams comprising presentation content from the first computing device of the presenter, sending, via the network connection, a set of second media streams to the plurality of second computing devices of the plurality participants, wherein content of the set of second media streams is based on content the set of first media streams, receiving, via the network connection, a set of third media streams from the second computing devices of a first subset of the plurality of participants, the set of third media streams including video content of first subset of the plurality of participants captured by the respective second computing devices of the first subset of the plurality of participants, analyzing the set of third media streams to identify a set of first reactions by the first subset of the plurality of participants to obtain first reaction information, determining first graphical representation information representing the first reaction information, and sending, via the network connection, a fourth media stream to the first computing device that includes the first graphical representation information to cause the first computing device to display the first graphical representation information on a display of the first computing device while the presentation content is being provided via the set of first media streams.

An example computer-readable storage medium on which are stored instructions. The instructions when executed cause a processor of a programmable device to perform functions of establishing an online presentation session for a first computing device of a presenter and a plurality of second computing devices of a plurality of participants, receiving, via a network connection, a set of first media streams comprising presentation content from the first computing device of the presenter, sending, via the network connection, a set of second media streams to the plurality of second computing devices of the plurality participants, wherein content of the set of second media streams is based on content the set of first media streams, receiving, via the network connection, a set of third media streams from the second computing devices of a first subset of the plurality of participants, the set of third media streams including video content of first subset of the plurality of participants captured by the respective second computing devices of the first subset of the plurality of participants, analyzing the set of third media streams to identify a set of first reactions by the first subset of the plurality of participants to obtain first reaction information, determining first graphical representation information representing the first reaction information, and sending, via the network connection, a fourth media stream to the first computing device that includes the first graphical representation information to cause the first computing device to display the first graphical representation information on a display of the first computing device while the presentation content is being provided via the set of first media streams.

DETAILED DESCRIPTION

Techniques for improving audience engagement and for rich interactive feedback for online presentations and/or communications sessions are described. These techniques provide a technical solution for solving the technical problem of how to improve audience engagement for online presentations and/or communications sessions. The participants of such an online presentation and/or communication session are located at different locations than that of the presenter and must interact with the presenter through their computing devices. Techniques are provided herein to facilitate express and/or implicit live user feedback from the participants to the presenter during the presentation or communications session. The participants may provide feedback by selecting a reaction icon or emoji representing the participants' reactions to the presentation. The participants may also convey reactions to the presentation content by making certain gestures or performing certain actions. The participants computing devices may capture and transmit video content of the participant that may be analyzed using one or more machine learning models that are configured to recognize gestures, poses, and/or other actions by the participants. The feedback information may be provided to the presenter in real time during the presentation so that the presenter may assess audience engagement in real time. The presenter may determine whether to make changes to the presentation or to ask the audience whether there are any questions. The feedback information may also be summarized into a report at the end of the presentation. The report provides a technical benefit of mapping feedback information to a particular time within the presentation so that the presenter has information as to how the audience reacted to each slide, topic, or other portion of the presentation. This information may be used to improve the content included in the presentation.

Techniques for providing feedback for improving presenter skills are also provided. These techniques provide a technical solution for the problem of providing useful feedback to presenters to improve their presentation skills. The technical solution utilizes one or more machine learning models to configured to analyze audio and/or video content of the presentation to identify aspects of the presentation that the presenter may be able to improve and to highlight aspects of the presentation which the presenter performed well. Critiques of various aspects of the presentation may be provided, such as, but not limited to pacing, vocal pattern, language usage, excessive wordiness, overly complex vocabulary, distracting behaviors, and/or other aspects of the presentation may be assessed using machine learning models trained to identify aspects of the presentation that may be improved or aspects of the presentation for which the presenter performed well. The presentation content, such as slides or documents, may also be analyzed by one or more machine learning models to provide feedback on these materials that may be used to improve the presentation. These techniques may be used to analyze a presentation that has been given to a live audience. Additionally, these techniques may also be used to rehearse a presentation and to obtain constructive feedback that may be used to improve the presentation skills of the presenter and/or the presentation content prior to providing the presentation to a live audience. These and other technical benefits of the techniques disclosed herein will be evident from the discussion of the example implementations that follow.

The following terminology is used in the description. A “presentation” or “online presentation” as used herein refers to content that is be shared by a presenter with one or more participants. The online presentation content may include a slide show, document, video, images, and/or other content. The online presentation content may also include an audio discussion that accompanies the presentation content. The online presentation may be a standalone online presentation or may be part of an online communications session. A “presenter” as used herein refers to a user of a client device that is sharing an online presentation content with at least one participant. The presenter may be participant of an online communications session with other participants and may assume the role of presenter for at least a portion of the online communications session. A “participant” as used herein refers to a user who is part of the audience of the online presentation being shared by the presenter. An online presentation may include multiple participants, and the participants may be located remotely from the presenter. The participants may receive the online presentation content over a network connection at a client device with audiovisual capabilities for outputting the online presentation content to the participants.

FIG. 1is a diagram showing an example computing environment100in which the techniques disclosed herein for a presentation and communications platform may be implemented. The computing environment100may include a presentation and communications platform110. The example computing environment may also include a plurality of client devices, such as client devices105a,105b,105c, and105d. The client devices105a,105b,105c, and105dand the presentation and communications platform110may communicate via the network120. Additional details of the presentation and communications platform110and client devices105a,105b,105c, and105dare discussed in greater detail with respect toFIG. 2.

The presentation and communications platform110may be implemented as a cloud-based service or set of services. The presentation and communications platform110may be configured to schedule and host online presentations, virtual meetings, video conferences, online collaboration sessions, and/or other online communications sessions in which at least a portion of the participants are located remotely from the presenter. The presentation and communications platform110may be used by companies, schools, universities, and other organizations which may traditionally conduct in-person meetings, classes, and/or presentations but must adapt to rapidly changing requirements in which many are working or attending school from home. The presentation and communications platform110provides services that enable the presenter to present content to remote participants and/or to facilitate a meeting that includes the remote participants. The presentation and communications platform110may also facilitate the collecting of feedback and response information from the participants of a presentation or communication session that may help the presenter to improve the content presented and/or the presenter's presentation techniques.

The presentation and communications platform110may receive live feedback from participants during an online presentation from the participants using the client devices105b,105c, and105dto participate in the online presentation. As will be discussed in the examples that follow, the feedback may be express reactions or implicit reactions derived from user actions or behavior. The express reactions may be provided through user interface elements provided by the applications on the client devices105b,105c, and105dused by the participants to receive and consume the presentation and/or communication session contents. The user interface elements may permit the participants to select reactions to be sent to the client device105aof the presenter of the online presentation. The presentation and communications platform110may also be configured to recognize participant gestures and actions in audio and/or video streams captured by the client devices105b,105c, and105dof the participants and sent to the presentation and communications platform110.

The presentation and communications platform110may be implemented by a presentation platform, such as Microsoft PowerPoint Live, which enables a presenter to present a presentation online and to invite users to view the presentation on their own devices. The presentation and communications platform110may be implemented by a communications platform, such as Microsoft Teams, which provides an online hub for team collaboration including chat and video conferencing. A presenter may utilize such a communications platform to conduct a meeting, a lecture, conference, or other such event online in which participants may be able to communicate with the presenter as well as other participants via chat and audio and/or video conferencing. In such an online communications platform, a participant may serve as a presenter for part of an online communications session, while another participant may serve as a presenter for another part of the online communications session.

The client devices105a,105b,105c, and105dare computing devices that may be implemented as a portable electronic device, such as a mobile phone, a tablet computer, a laptop computer, a portable digital assistant device, a portable game console, and/or other such devices. The client devices105a-105dmay also be implemented in computing devices having other form factors, such as a vehicle onboard computing system, a video game console, a desktop computer, and/or other types of computing devices. Each of the client devices105a-105dmay have different capabilities based on the hardware and/or software configuration of the respective client device. While the example implementation illustrated inFIG. 1includes four client devices, other implementations may include a different number of client devices.

FIG. 2is a diagram showing additional details of the presentation and communications platform and client devices of the computing environment shown inFIG. 1. The presentation and communications platform110may include a content creation and editor unit205, a scheduling and participant invitation unit210, a stream processing unit215, a feedback and reporting unit225, a presentation coaching unit230, and a presentation hosting unit240.

The presentation and communications platform110includes a hosting element provided by the presentation hosting unit240for hosting an online presentation in which participants may provide live feedback to the presenter during the presentation. The presentation and communications platform110also includes a coaching element providing by the presentation coaching unit230which may analyze the presentation provided by the presenter and provide feedback to the presenter for improving various aspects of the presentation. The presentation coaching unit230may also be used to rehearse the presentation without an audience to help the presenter hone their presentation skills and improve the presentation content prior to presenting to an audience. The presentation and communications platform110implements an architecture for efficiently analyzing audio, video, and/or multimodal media streams and/or presentation content. A technical benefit of this architecture is the media streams and/or presentation content may be analyzed to extract feature information for processing by the various models, and the high-level feature information output by the models may then be utilized by both the presentation coaching unit230and the presentation hosting unit240. This approach provides a more efficient use of memory and processing resources on the data processing system hosting the presentation and communications platform110by eliminating the need to analyze content separately for the presentation coaching unit230and the presentation hosting unit240.

The presentation hosting unit240may be configured to facilitate hosting of an online presentation by a presenter. The presentation hosting unit240may be configured to permit the presenter to share a presentation content with a plurality of participants. The presentation hosting unit240may be configured to engage with the audience by providing the audience with the ability to send reaction icons or emojis. Emojis are graphic symbols that represent an idea or concept that are used in a variety of messaging applications. Emojis may serve as a shortcut for conveying an idea in graphic form and are commonly used to react to a message. The presentation hosting unit240is configured to enable participants to an online presentation to send emoji feedback to the presenter during the online presentation and to present this feedback to the presenter in real time. The presentation hosting unit240may provide the presenter with configuration settings in which the presenter may control whether the feedback is visible only to the presenter or is visible to all the participants of the online presentation.

The presentation hosting unit240provides means for the participants to expressly or to implicitly generate emoji feedback to the presenter. A participant may expressly generate reactions to the presentation by clicking on or otherwise activating a reaction icon or emoji representing the participant's reaction to the presentation. However, clicking on or otherwise activating a reaction icon is not the most natural way for participants to engage with the presenter. The presentation and communications platform110provides an additional means for the participants to engage with the presenter. The participants may engage with the presenter by providing more natural reactions to the presentation content, such as a thumbs up or thumbs down gesture, smiling, laughing, shaking their head or nodding, yawning, and/or other actions in response to the presentation content. The client devices105b,105c, and105dof the participants may be configured to capture audio and/or video streams of the participants while the presentation is underway. The presentation and communications platform110may receive and analyze these streams using machine learning models to identify these user actions and to map these to reaction icons or emojis that may automatically be shown to the presenter during the online presentation. In some implementations, the reaction icons or emojis may be rendered over the presentation content being shared by the presenter by the client device105aof the presenter and/or by the client devices105b,105c, and105dof the participants if the presenter has chosen to share the reactions with participants. The reaction icons or emojis may be rendered over the presentation content or otherwise rendered on a display of the client device. In some implementations, the reaction icons or emojis may appear as an animation that appears briefly before fading away. Using this latter method of analyzing the participant actions to generate reactions to the online presentation may promote user engagement by providing a more natural means for interacting with the online presentation. This approach may also provide more information that the presenter may be able to use to better understand audience engagement than may otherwise be available. Participants may not be inclined to react to the online presentation if they must select an appropriate reaction from a list of available reactions and click on that reaction. The techniques disclosed herein may provide valuable additional reaction information to the presenter in real time by automatically generating such reaction information based on the participants actions.

The presentation coaching unit230is configured to provide a non-biased and safe environment for presenters to practice and improve their public speaking skills. The presentation coaching unit230may also be useful for presenters who do not have anyone available with whom they can practice their presentation. The presentation coaching unit230may analyze audio, video, and presentation content with machine learning models trained to identify aspects of the presenter's presentation skills and the presentation content are good and those that may benefit from improvement. The presentation coaching unit230may provide feedback critiques on aspects of the presentation skills, such as but not limited to pacing, vocal pattern, volume, whether the presenter is speaking in monotone, and/or language usage. The language usage aspect may include identifying use of filler words, informal speech, slang, euphemisms, culturally sensitive terms, obscene or vulgar language, usage of vocabulary that is unusual or may be confusing or unnecessarily complicated. The presentation coaching unit230may also detect when the presenter is being overly wordy. The presentation coaching unit230may also detect where the presenter is simply reading text on a slide or other presentation content. The presentation coaching unit230may also provide feedback on presentation content, such as the layout of slides or other content and language usage in the slides or other content.

While the example implementation shown inFIG. 2discusses the use of the techniques disclosed herein with an online presentation, the techniques for automatically generating reaction information for a presenter may be extended to online communications sessions or online meetings where one participant may at least temporarily assume the role of a presenter by speaking to the other participants of the online communications session about some topic. The presentation and communications platform110may analyze the audio and/or video streams captured by the client devices105of the other participants and automatically generate reactions as discussed above. The reactions may be presented to just the participant that is currently acting as a presenter or to all the participants of the online communications session.

The content creation and editor unit205may provide an application that allows a presenter to create and/or edit content to be presented during an online presentation and/or during an online communications session. The presenter may create the presentation context on their client device105or another computing device and import the presentation content to the presentation and communications platform110to host the online presentation. The content creation and editor unit205may provide the presenter with another option for creating and/or editing the presentation content via a web-based application. The content creation and editor unit205may provide a user interface that may be accessed via the browser application255bof the client device105aof the presenter that allows the presenter to create and/or edit the content of the presentation online. The presentation and communications platform110may also be configured to store the presentation content for the presenter and/or to enable the presenter to store the presentation in a cloud-based file hosting service, such as but not limited to Microsoft OneDrive or Google Drive.

The stream processing unit215may be configured to process the media streams received from the client devices105and to analyze the contents of the media streams to automatically identify participant reaction information and/or to generate feedback that may be used to help the presenter improve their presentation skills. The stream processing unit215may use or more machine learning models to analyze the media stream content and to provide high-level feature information that may be used by one or more downstream components to provide various features to the presenter and/or the participants of the online presentation. Additional features of the stream processing unit215are provided in the examples that follow.

The feedback and reporting unit225may be configured to receive high-level feature information generated by the stream processing unit215and reactions information provided by the participants and to generate one or more summary reports provide participant reaction information and recommendations for how the presenter may improve their presentation skills and/or presentation content. The reporting aspect of the feedback and reporting unit225may be triggered automatically at the end of an online presentation to provide the summary reports to the presenter. The feedback aspect of the feedback and reporting unit225may include providing to the presenter live feedback received from participants during the presentation. The examples which follow provide additional details of how such live feedback may be generated based on the machine learning models identifying reactions based on express and/or implicit reactions information provided by the participants. The feedback may be presented to the presenter and/or shared with the participants of the online presentation session. The feedback may also be summarized in the reactions information in the summary reports provided to the presenter upon completion of the online presentation session.

The presentation hosting unit240may permit the presenter to schedule the online presentation or communication session in which the online presentation is to be presented. The scheduling and participant invitation unit210may provide a user interface that allows the presenter to schedule the online presentation or communication session in which the online presentation is to be presented. The scheduling and participant invitation unit210may send invitations to participants to participate in an online presentation. The invitations may include a link to the online presentation and/or a Quick Response (QR) code that the participant may scan in order to connect to the online presentation or to accept the invitation to participate in the online presentation. The scheduling and participant invitation unit210may add a reminder to the calendar of the participants for the date and time for which the online presentation is scheduled.

In the example shown inFIG. 2, the client device105ais being used by the presenter to control an online presentation or to facilitate an online communications session, and the client device105bis being used by a participant of the online presentation to receive and consume the online presentation content. The client device105amay include a native application250a, a browser application255a, a stream processing unit260a, and a content capture unit265a, and the client device105bmay include a native application250b, a browser application255b, a stream processing unit260b, and a content capture unit265b. Client devices105cand105dhave been omitted fromFIG. 2as a matter of clarity. Each of the client devices may include the same elements or may include a different combination of elements. The client devices105of the presenter and the participants need not be identical.

The native applications250aand250bmay be an application developed for use on the client device105. The native applications250aand250bmay be a presentation application that may communicate with the presentation and communications platform110to provide a user interface for creating, modifying, participating in, and/or conducting online presentations. The native applications250aand250bmay also be a communications platform application, such as but not limited to Microsoft Teams, which may permit a presenter to share an online presentation with participants as part of an online communications session. The native application250aand250bmay be the same application or a different application in some implementations. For example, the presenter may present an online presentation using a first native application250awhile a participant may view and/or participate in the online presentation using a second native application250b.

The browser applications255aand255bmay be an application for accessing and viewing web-based content. The browser applications255aand255bmay be the same application or may be different applications. In some implementation, the presentation and communications platform110may provide a web application for conducting and/or participating in an online presentation and/or communication session. The presenter or the participants may access the web application and render a user interface for interacting with the presentation and communications platform110in the browser applications255aand255b. In some implementations, the presentation and communications platform110may support both the native application250aand255band the web application, and the presenter and participants may choose which approach best suites them for conducting and/or participating in an online presentation and/or communications session.

The client device250amay also include a stream processing unit260a, and the client device250bmay include a stream processing unit260b, which may be configured to generate one or more media streams to be transmitted to the presentation and communications platform110. Some examples of the media streams that may be transmitted between the presentation and communications platform110and the client devices105are described in greater detail with respect toFIG. 5.

The content capture units265aand265bmay be configured to capture audio content and/or video content using the microphone and camera of the client device105aand105b, respectively. The content capture units265aand265bmay be configured to interface with these hardware elements to capture the audio content and video content that may be provided to the stream processing unit s260aand265bof the respective client devices105aand105b. The stream processing units260aand265bmay be configured to process the audio content and/or the video content obtained by the content capture units265aand265b, respectively, and process that audio content and/or video content into one or more media streams that may be transmitted to the presentation and communications platform110.

FIG. 3is a diagram showing examples of data exchanged between the presentation and communications platform110and the client devices105a,105b,105c, and105d. As discussed in the preceding examples, the presentation and communications platform110may transmit one or more presentation media streams305to the each of the client devices105over the network120. The one or more presentation media streams305may include one or more audio media streams, one or more video media streams, and/or other media streams. The one or more presentation media streams may include an audio component of the presentation where the presenter is discussing presentation content being shared with the participants. The presentation content may include a set of slides, a document, or other content that may be discussed during presentation. The presentation content may be provided to the client devices of the participants by the presentation and communications platform110before or during the online presentation. A copy of the presentation content may be provided to the computing devices105of the participants to permit the participants to navigate independently through the presentation content during the online presentation. The presentation media streams305may include navigation signals that may be used by the computing devices105of the participants to display a particular portion of the presentation content that the presenter is currently discussing in the online presentation. A participant may override these navigation signals and independently navigate to a different slide or portion of the presentation content during the presentation. Such navigation overrides may be reported to the presenter via the reactions data315to permit the presenter to identify portions of the presentation that may be unclear or for which the presenter should have spent more time discussing. The reactions data315received from the participants may be anonymized by the presentation and communications platform110to remove any personalized information that may identify that a particular reaction has originated from a particular participant. The anonymized data may be presented to the presenter during the online presentation and/or included in one or more summary reports generated for the presenter at the conclusion of the online presentation.

The client devices105b,105c, and105dof the participants of the presentation may send one or more participant media streams310b,310c, and310dto the presentation and communications platform110. The presentation and communications platform110may analyze the participant media streams310b,310c, and310d, as will be discussed in the examples that follow, to identify reactions by the participants. The presentation and communications platform110may also aggregate the participant media streams310b,310c, and310dinto the participant media stream310awhich is sent to the client device105aof the presenter. The client device105aof the presenter may present the participant media stream310ato the presenter so that the presenter. The participant media stream310amay include audio and/or video content of the participants of the online presentation. The presenter may wish to be presented with this content so the presenter may hear questions and/or see the participants of the online presentation to better engage with the audience. The client devices105b,105c, and105dmay also transmit reactions data315to the presentation and communications platform110. The reactions data315may be generated by the client device105of the participants in response to the participant selecting a reaction icon or emoji representing the participants' reactions to the presentation.

FIG. 4is a diagram showing additional details of the stream processing unit shown as215inFIG. 2. The steam processing unit may include a stream and reaction data receiver unit405, a frame and filtering preprocessing unit410, and a video-based, audio-based, and multi-modal analyzers unit415(also referred to herein as “analyzers unit415”).

The stream and reaction data receiver unit405may be configured to receive the presentation media streams305afrom the client device105aof the presenter, and the participant media streams310b,310c, and310dand the reactions data315b,315c, and315dfrom the client devices105b,105c, and105dof the participants to the online presentation. The stream and reaction data receiver unit405may output the received streams as an input to the frame and filtering preprocessing unit410.

The frame and filtering preprocessing unit410may be configured to convert the media streams and/or reaction data received by the stream and reaction data receiver unit405into a format or formats that the machine learning models of the analyzers unit415may utilize. The frame and filter preprocessing unit410may be configured to perform feature extraction on the media streams and/or reaction data. The particular features that may be extracted depend on the types of machine learning models are implemented by the analyzers unit415. In some implementations, the models may be grouped into categories of models where each of the categories of model may share the same preprocessing feature extraction steps. This approach provides a technical benefit of reducing the processing resources required to preprocess the media streams and/or reaction data by performing the feature extraction for a particular category of model once and providing those features as an input to each of the models of that category.

The output from the stream processing unit215may be provided to one or more downstream consumers420. The downstream consumers420may include the feedback and reporting unit225and the presentation coaching unit230of the presentation and communications platform110. Other downstream consumer420may also be configured to receive the outputs of the stream processing unit215. The output from the stream processing unit215may include high-level feature information. The high-level feature information may include information such as gestures being made by the presenter and/or the participants, language usage by the presenter, language pattern of the presenter, emotional state of the presenter and/or the participants, eye contact and/or gaze direction of the presenter, body pose of the presenter and/or participants, and/or other information about the presenter and/or the participants. The high-level feature information may be generated by the machine learning models of the analyzer unit415. These models will be described in greater detail with respect toFIG. 6.

FIG. 5is a diagram showing an example of video streams505that may be received at the presentation and communications platform and the client devices.FIG. 5shows that the video streams may be intermittent, may be received without audio, may be received with audio, or may be received as audio-only. The intermittent nature of the video streams may be a result of network issues and/or the streams being interrupted at the client device105. For example, a participant at a client device105may turn on or off the video camera and/or the microphone of the client device105. Some participants may enable the microphone and disable the video camera of their client devices105, while other participants may enable the video camera and disable the microphone. As a result, the client devices105of the participants may generate audio media stream, video media streams, no media streams, or intermittently generate different types of media streams as the participants change the settings of the respective computing devices105during the online presentation.

The frame and filtering preprocessing unit410may be configured to handle the changing conditions of the stream content. The frame and filtering preprocessing unit410may be configured to determine whether a particular media stream contains audio, video, or both at a particular time and to process the stream using to convert the media stream into an appropriate format to serve as an input to the machine learning models for analyzing that type of content. As the type of content changes over time, the frame and filtering preprocessing unit410may adapt to the change in content type. For example, the stream vi shown inFIG. 5initially includes a vide stream without audio but later includes an audio component. Initially, the frame and filtering preprocessing unit410may process the stream vi to generate an input or inputs for models that process features from video content. Later, the frame and filtering preprocessing unit410may process the stream vi to generate an input or inputs for models that may process features from video content, audio content, or multimodal content. The examples shown inFIG. 5illustrate the concepts disclosed herein and do not limit the media streams to the specific configuration shown therein. In other implementations, separate audio streams and video streams may be generated by the client devices105during the online presentation, and the frame and filtering preprocessing unit410may process each of the media streams separately.

FIG. 6is a diagram showing additional details of the video-based, audio-based, and multi-modal analyzers unit415shown inFIG. 4. The client devices105of the participants and the presenter may include a microphone for capturing audio content of the presenter and a camera for capturing video content of the presenter. The analyzers unit415may include one or more machine learning models trained to analyze audio-based content, video-based content, or multimodal content. Multimodal content may comprise audiovisual content which has both audio and video components.

The models may be local to the presentation and communications platform110, such as those of the analyzers unit415. At least a portion of the models may be implemented by a remote server or cloud-based services. In such implementations, the analyzers unit415may be configured to send the feature information expected by the model as an input to the remote server or services and to receive high-level feature information output by the remote model from the server or service. In some implementations the analyzers unit415may utilize the Microsoft Azure Application Programming Interface (API) for creating an interface between the analyzers unit415and one or more remote models. The models may be implemented using various machine learning architectures such as deep neural networks (DNNs), recurrent neural networks (RNNs), convolutional neural networks (CNNs), and/or other types of neural networks. The particular architecture selected for a model may be based on the type of analysis to be performed by the model. In some implementations, the models may be custom developed for a analyzing a particular aspect of a presentation. For example, a model may be trained to detect specific gestures that participants of an online presentation and/or communication session are expected to perform. Other models may be a more general-purpose model that is used to analyze a particular input and is not specifically tailored for use for analyzing content associated with online presentations. For example, a model for identifying language usage issues, such as obscenity or vulgar language may be a general-purpose model for identifying such language in audio or video content.

The models may be configured to receive feature data extracted from the presentation media streams305, the participant media streams310, and/or the reactions data315. As discussed with respect toFIG. 4, the models may be grouped into categories of model based on what type of analysis the model is trained to perform and/or based on the inputs that the model is configured to received. The example shown inFIG. 6includes a pose detection model605, a gesture detection model610, an emotion detection model615, a language usage detection model620, and a language pattern detection model625. Other implementations of the analyzers unit415may include other models in addition to or instead of one or more of the models shown inFIG. 6. The models may be machine learning models trained to provide an output that includes high-level feature information based an output based on features included in the inputs. The types of high-level feature information that may be provided by a particular model depends upon the type of model being used and the types of participant or presenter behavior the model is configured to identify.

A technical benefit provided by the analyzers unit415is that machine learning models may analyze audio content, video content, and/or multi-modal content captured by the client devices105of both the presenters and the participants to automatically identify actions by the participants indicative of audience engagement and to automatically identify actions by the presenter that may impact audience engagement. The actions taken by the participants may be used to provide reactions information indicative of audience engagement to the presenter in real time during the presentation. The actions taken by the presenter may be used to identify presentation skills which the presenter may improve as well as presentation skills that the presenter has done well. The reactions information and presentations skills information may be compiled into a summary report, such as those shown inFIGS. 9 and 11, that may be provided to the presenter at the end of the presentation. These summary reports provide information that may be used by the presenter to understand how the audience perceived the presentation as a whole, suggestions for how the presenter may improve the presentation and/or their presentation skills, and a summary of aspects of the presentation that done well. The feature data associated with the presenter's actions may be provided to the presentation coaching unit230which may be configured to generate suggestions that the presenter may use to improve their presentation skills. The presentation coaching unit230may provide real-time tutorials to guide the presenter through a rehearsal of a presentation and provide critiques and feedback during the rehearsal that may help the presenter to improve their presentation skills. The presentation coaching unit230may also provide suggestions and feedback to the feedback and reporting unit225for inclusion of the suggestions and feedback in the summary reports that may be provided to the presenter after a presentation or rehearsal.

The pose detection model605may be configured to analyze features extracted from video content of the presenter or a participant to identify a pose of that person and to output high-level features information that represents the identified pose. The model may determine that the person is standing, sitting upright, slouched down, or some other position. The pose information may be indicative of engagement of a presenter or participant. For example, if the presenter is slouched down during the presentation, they may appear to be disinterested to the participants, and if the participant is slouched down, the participant may be bored or confused by the presentation content. The presentation coaching unit230may be configured to analyze the high-level features obtained from the pose detection model605to identify a pose of the presenter during a presentation or a rehearsal that may be distracting to the audience and may provide suggestions to the presenter for eliminating such poses.

The gesture detection model610may be configured to analyze features extracted from video content of the presenter or a participant to identify a gesture made by that person and to output high-level features information that represents the identified gesture. The gesture information may be output as high-level features and provided as an input to the feedback and reporting unit225. The feedback and reporting unit225may be configured to identify certain gestures made by a participant as being a reaction that may be sent to the client device105aof the presenter to help the presenter to gain an understanding of the audience engagement in near real time during the presentation. A technical benefit of this approach is that participants may make certain gestures to cause reactions to a presentation to be automatically generated without having to activate a button or icon for that reaction on the user interface. For example, the participant may clap, provide a thumbs up or thumbs down gesture, shrug, nod or shake their head, or perform actions that may be identified by the gesture detection model610and mapped to a reaction by the feedback and reporting unit225. This approach may increase audience engagement with an online presentation.

The presentation coaching unit230may be configured to analyze the high-level features obtained from the gesture detection model610to identify a gesture made by the presenter during a presentation or a rehearsal that may be distracting to the audience and may provide suggestions to the presenter for eliminating such gestures. For example, the presenter may unconsciously touch their face or cover their mouth with their hand while presenting. Such behavior may be distracting to the audience, and the presentation coach may provide an indication to the presenter that the gesture should be avoided.

The emotion detection model615may be configured to analyze features extracted from video content of the presenter or a participant to identify an emotional state of that person and to output high-level features information that represents the identified emotional state. The emotion information may be output as high-level feature and provided as an input to the feedback and reporting unit225. The feedback and reporting unit225may be configured to identify certain emotional states of a participant as being a reaction that may be sent to the client device105aof the presenter to help the presenter to gain an understanding of the audience engagement in near real time during the presentation. Furthermore, the emotion information may be determined for the presenter, and this information may be used by the presentation coaching unit230to provide suggestions to the presenter if the presenter appears to be unhappy, anxious, angry, stressed, or exhibit other emotions that may distract from the presentation or otherwise impact the presenter's performance. The presentation coaching unit230may provide suggestions to the presenter for dealing with stress or anxiety related to public speaking. These suggestions may include techniques for dealing with stress or anxiety related to public speaking.

The language usage detection model620may be configured to analyze features extracted from video content of the presenter or a participant to identify language usage of that person and to output high-level features information that represents the language usage. With respect to the participants, the feedback and reporting unit225may be configured to identify certain language usage of a participant as being a reaction that may be sent to the client device105aof the presenter to help the presenter to gain an understanding of the audience engagement in near real time during the presentation. For example, a participant may utter the word “what?” or utterance “huh?” during the presentation if they do not understand something that is being presented. The feedback and reporting unit225may be configured to maps this reaction to a “confused” reaction that may be sent to the client device105aof the presenter to help the presenter to gain an understanding that at least some of the participants may be confused by a portion of the presentation. With respect to the presenter, the presentation coaching unit230may be configured to identify certain language usage of a presenter during a presentation or rehearsal that may detract from the presentation. For example, the feedback and reporting unit225may be configured to identify the usage of obscenities or vulgar language, slang, filler words, difficult words, and/or other language usage that the presenter should avoid. The presentation coaching unit230may provide suggestions for alternative language and/or language to be avoided during a presentation. These suggestions may be included in the summary report or reports that may be provided to the presenter at the end of the presentation.

The language pattern detection model625may be configured to analyze features extracted from video content of the presenter to output high-level features information that identifies the language pattern issues in the presentation. The language pattern detection model625may be trained to identify issues such as pacing, volume, pauses, and/or other issues related to the speech pattern of the presenter. For example, the language pattern detection model625may detect that the presenter may be speaking to quickly or too slowly, may be speaking too quietly or too loudly, or may be pausing too often or for too long during the presentation. The presentation coaching unit230may provide suggestions for improving the pacing, volume, and/or other aspects of the language patterns used by the presenter during the presentation. These suggestions may be included in the summary report or reports that may be provided to the presenter at the end of the presentation.

FIG. 7is a diagram showing an example user interface705for conducting an online presentation from the client device105of a presenter. The user interface705may be generated by the presentation hosting unit240and may be rendered in the browser application255aor the native application250aof the client device105aof the presenter in such implementations. In other implementations, the native application250aof the client device105aof the presenter may be a presentation application that is configured to provide a user interface for creating, modifying, conducting, and participating in online presentations and/or communication sessions. The native application250amay communicate with the presentation and communications platform110in such implementations to provide the various services described in the preceding examples.

The user interface705includes a content pane710that may be used to display a presentation or other content that the presenter is sharing with the participants to the online presentation or online communications session. The content pane710shown inFIG. 7is displaying a slide show that is being presented to the participants. The content pane710may be used to display content received from the client devices105of the participants, such as video of the participants themselves or other content shared by the participants.

The user interface705may include a presentation summary725that may be used to present information about the online presentation and/or communication session to the presenter. A transcript715of the audio portion of the online presentation and/or communication session may be generated by the stream processing unit215by analyzing the spoken content provided by the presenter and the participants. The language in which the transcript is presented to the presenter and/or each of the participants may be configurable. In some implementations, the presenter may select the transcript language. In other implementations, the presentation and communications platform110may provide a user interface that enables the presenter and/or the participants to each set language preferences for the transcript. The presentation and communications platform110may be configured to automatically translate the transcript to the preferred language if supported or may be configured to obtain a translation of the transcript from an external translation service in substantially real time and to display the appropriate translation to the presenter and/or the participants. Thus, the presenter and/or participants may be able to participant in the online presentation and/or communication session in one language but may obtain a transcript in a second language with which the presenter and/or participants are more comfortable.

The reactions of participants720may also be displayed in the presentation summary725. As discussed in the preceding examples, participants may provide user reactions to the online presentation and/or communication session from their respective client devices105. The reactions data may be transmitted from the client devices105of the participants to the presentation and communications platform110in the reactions data315. The reactions data315may include an indication that the participant has selected a reaction icon or emoji representing the participant's reactions to the presentation. The feedback and reporting unit225may receive the reactions data315from the client devices of the participants and combine that reach data315into the reactions data315atransmitted from the presentation and communications platform110to the client device105aof the presenter. As discussed in the preceding examples, the stream processing unit215may also be configured to recognize reactions included in the audio and/or video media streams of the participants captured by the participants' respective client device105. The client devices105of the participants may transmit one or more participant media streams310that may be analyzed by the stream processing unit215to recognize gestures made by the participants. For example, a participant may make certain gestures that are captured in a video stream captured by client device105of the participant. These gestures may be recognized by the machine learning models used by the presentation and communications platform110to identify such gestures. The gestures may then be mapped by the feedback and reporting unit225to a reaction icon or emoji which provide a graphical representation of the reaction. The reaction icon or emoji be transmitted to the client device105aof the presenter in the reactions data315a.

The reactions of the participants720may display a representation of the reaction icon or emoji of each of the reactions received and a total indicating the number of reactions received. In some implementations, the reactions may also be displayed as an animation that may be overlaid on the contents of the content pane710as they are received. The presenter can use this reaction information as means for measuring audience engagement with the online presentation and/or communication session. The presenter may use this information to make changes to the online presentation and/or communication session. For example, if the reactions

The presentation and communications platform110may also provide an option that allows the present to selectively enable or disable the sharing of the reaction information with other users. The presentation and communications platform110may allow the presenter to enable or disable the sharing of the reaction information at any time during the presentation. In other implementations, the presentation and communications platform110may allow the presenter to selectively enable or disable reactions for specific presentations and/or online communications session or to enable or disable reactions by default for all presentations and/or online communications being hosted by the presenter. The presentation and communications platform110may allow the presenter to override these presentation-specific and/or defaults settings to selectively enable or disable the sharing of the reaction information with the participants. The client devices105of the participants may display these reactions as will be discussed with respect to the user interface shown inFIG. 8.

The layout of the user interface705is an example of one possible layout of the user interface that may be provided by the presentation and communications platform110and/or the native application250. Other implementations may utilize a different layout and may omit one or more of the features shown inFIG. 7and/or include one or more additional features not shown in example ofFIG. 7. For example, the user interface705may include one or more control elements that are not shown that allows the presenter to configure one or more aspects of the online presentation and/or communication session. The user interface705may include controls for enabling and/or disabling sharing of reaction information with participants, for enabling and/or disabling the microphone and/or the video camera of the client device105aof the presenter, for setting the transcript language and/or for enabling or disabling the display of the transcript on the user interface705.

FIG. 8is a diagram showing an example user interface805for participating in an online presentation from the client device105of a participant. The user interface805may have a layout that is similar to the user interface705shown on the client device105of the presenter. The user interface805may be generated by the presentation hosting unit240and may be rendered in the browser application255or the native application250of the client device105of the participant in such implementations. In other implementations, the native application250of the client device105of the participant may be a presentation application that is configured to provide a user interface for creating, modifying, conducting, and participating in online presentations and/or communication sessions. The native application250may communicate with the presentation and communications platform110in such implementations to provide the various services described in the preceding examples.

The user interface805may include a content pane810that is similar to the content pane710of the user interface705. The content pane810may be used to display presentation content being presented by the presenter and/or video content of the presenter and/or other participants. The presentation and communications platform110may associate presentation content uploaded by the presenter with the presentation and/or online session. The presentation and communications platform110may send a copy of the presentation content to the client device105of the participants as the participants join the online presentation and/or communication session. The presentation content may be a set of slides created by a presentation application, such as a Microsoft PowerPoint, Google Slides, or Prezi. The presentation content may comprise a document, such as a Microsoft word document, a Google Docs document, or other type of word processing document. The presentation content may also include other types of content, such as video content, web-based content, images, video, and/or other types of content.

The client device105aof the presenter may transmit navigation signals in the presentation media streams305awhich indicate a position within the presentation content which the presenter is currently discussing. The navigation signals may be detected in the presentation media streams305received by the client devices105of the participants and used to synchronize the display of the presentation content in the content pane810of the user interface805with the location being discussed by the presenter. The user interface805may be configured to allow the user to override the automatic navigation to independently navigate to a different portion of the presentation content than the presenter is currently discussing. For example, a participant may navigate back to a previous slide in a presentation to refer to content included therein. The user may navigate using a keyboard, mouse, touchscreen, or other navigational tools available on the client device105. The user interface805may be configured to detect such an override of the automatic navigation and to report details of such manual navigation in the reaction data315. For example, the manual navigation information may include information as to which portions of the presentation content to which the participant navigated, at which point in the presentation the user navigated to these portions, and how long the user remained on these portions of the presentation. The manual navigation information may be collected and reported back to the presentation and communications platform110. The presentation and communications platform110may analyze this information to determine whether the certain portions of the presentation may not have been clear and may benefit from additional details.

The user interface805may include a presentation summary825that is similar to the presentation summary725shown in the user interface705used by the presenter. The transcript815may be similar to the transcript715of the user interface705. The presentation summary825shown to the participants may be slightly different from that shown on the user interface705. For example, the user interface805may include reactions820. The reactions820includes a set of reaction icons or emojis providing a graphical representation of various reactions to the presentation content. The user may click on or otherwise activate a reaction icon or emoji to cause the user interface805to send an identifier for the reaction icon or emoji to the presenter. The identifier for the reaction icon or emoji may be added to the reactions data315sent by client device105of the participant to the presentation and communications platform110. As discussed in the preceding examples, the presentation and communications platform110may add the aggregate the reactions data315from each of the participants and send the aggregated data to the client device105aof the presenter for display. In some implementations, the aggregated reactions data may be provided to the client device of each of the participants and may be displayed to the participants.

FIG. 9is an example of an example presentation summary report910that may be provided to the presenter upon completion of the presentation or online communications session. The presentation summary report may be shown to the presenter in the user interface905of the application. As can been seen inFIG. 9, the summary report910may replace the presentation content shown in the preceding examples with the summary report910automatically upon completion of the presentation. The feedback and reporting unit225may be configured to provide a summary of participant feedback to the presenter at the end of the presentation or online communications session. The presentation summary report910may include audience reaction information as well as presentation critiques and highlights information. The presentation summary report910may include information provided by the presentation coaching unit235based on the analysis of the presentation media streams305which may capture audio and/or video content of the presenter. The analyzer unit415of the stream processing unit215may analyze audio content, video content, or both provided the presenter during the online presentation or communications session. As discussed in the preceding examples, the analyzer unit415may output high-level features information output by the machine learning models. The feedback and reporting unit225may be configured to analyze these high-level features to identify presentation critiques and presentation highlights. The presentation critiques may provide information for aspects of the presenter's presentation skills that may be subject to improvement. The feedback and reporting unit225may also include presentation highlights which include aspects of the presenter's presentation skills which the presenter did very well. Other types of critiques, such as those described in the other examples provided herein, may also be included in the presentation summary report910. The presentation summary report910may include a summary of audience reactions received during the online presentation and/or the orientation. The presentation summary report910may also include a live feedback score that is based on participant feedback obtained at the end of the online presentation. The feedback may be obtained by presenting the participants with user interface similar to the live polls shown inFIGS. 12A-12Cwhich may include a series of questions asking the participant to rate various aspects of the presentation. The feedback and reporting unit225may be configured to collate the responses from the participants to generate the live feedback score.

FIG. 10is an example of another presentation summary report1005that may be provided to the presenter upon completion of the presentation or online communications session. The presentation summary report1005may include similar content as the presentation summary report910. The presentation summary report1005may be sent to the presenter via email upon completion of the online presentation or communications session. The feedback and reporting unit225may be configured to generate the presentation summary report1005and to email the presentation summary report to an email address associated with the presenter. In some implementations, the feedback and reporting unit225may be configured to generate both the presentation summary report910and the presentation summary report1005. The presentation summary report910may be rendered on a display of the client device105of the presenter upon completion of the online presentation and the presentation summary report1005may be emailed to the presenter.

FIG. 11is an example of an example presentation summary report1110that may be provided to the presenter upon completion of the presentation or online communications session. The summary report1110is similar to the summary report905but it includes an option1115the allows the user to open their presentation in a slide designer application that can help improve the layout of the slides. The summary report1110also includes an option1120that allows the user to open their presentation in the presentation coach application to work on their presentation skills. The presentation coach application load the audio, video, slides, and/or other content and provide the presenter with feedback on those elements as well as walk the presenter through one or more tutorials that for improving their presentation skills. These tutorials may include capturing audio and/or video of the presenter and providing feedback in substantially real time.

FIG. 12Ais an example of a user interface1205for creating a live poll that may be rendered on a display of the client devices105of participants of a presentation or online communications session. The content creation and editor unit205of the presentation and communications platform110may provide a user interface in which a presenter may create a live poll that may be presented to participants during an online presentation. The user interface1205may be rendered in the browser application255aor the native application250aof the client device105aof the presenter in such implementations. The poll may also be created using an application or service that is external to the content creation and editor unit205and be imported into the content creation and editor unit205. The poll may be created using a cloud-based service, such as but not limited to Microsoft Forms, which may be accessed by the browser application255aor the native application250aof the client device105aof the presenter. The native application250aof the client device105aof the presenter may also be configured to implement a live poll.

Live polls may be used to obtain feedback regarding the presentation or communication session and/or regarding content thereof. The polls may include a question and a set of two or more answers the user may select in response to the question. Some polls may be configured to allow the user to select multiple answers. The presenter may create the poll in advance and the presentation and communications platform110may provide a means for launching the poll during the presentation or communication session. The content creation and editor unit205may be configured to allow the presenter to create new polls during the presentation or communication session. A technical benefit of this approach to polling is that it allows the presenter to engage with the participants by creating polls on the fly during the presentation without interrupting the presentation or communication session.

FIG. 12Bis an example of a user interface1210for presenting a live poll to participants of a presentation or online communications session. The poll created by the presenter using the user interface1205may be included in the presentation content transmitted to the client devices105of the participants in the presentation media streams305. The browser application255bor the native application250bof the client device105bof the participant may render the user interface1210on a display of the client device105bof the participant. The participant may select an answer or answers to the poll and submit the response. The client device105bmay transmit the poll response to the presentation and communications platform110in the reactions data315.

FIG. 12Cis an example of a user interface1215for displaying results of a live poll that may be rendered on a display of the client device105of the presenter. The browser application255or the native application250of the client device105of the presenter's client device105may display the user interface1215in response to the presenter launching the live poll. The poll results provided by the participants may be collated by the presentation and communications platform110and the results sent in the reactions data stream315afrom the presentation and communications platform110to the client device105aof the presenter. The presentation and communications platform110may update the poll results as additional responses are received from the participants. The poll results may also be provided to the feedback and reporting unit225of the presentation and communications platform110, and the feedback and reporting unit225may include the poll results in the presentation summary report or reports generated at the end of the presentation and sent to the presenter.

FIG. 13is a flow chart of an example process1300for hosting an online presentation. The process1300may be implemented by the presentation and communications platform110.

The process1300may include an operation1310of establishing an online presentation session for a first computing device of a presenter and a plurality of second computing devices of a plurality of participants. As discussed in the preceding examples, the presentation hosting unit240of the presentation and communications platform110may receive a request from the client device105aof the presenter to establish the online presentation session. The presenter may optionally schedule the online presentation for a future day and time or may request that the online presentation be established immediately.

The process1300may include an operation1320of receiving, via network connection, a set for first media streams comprising presentation content from the first computing device of the presenter. The client device105aof the presenter may transmit the presentation media streams305ato the presentation and communications platform110.

The process1300may include an operation1330of sending, via the network connection, a set of second media streams to the plurality of second computing devices of the plurality of participants. The second media streams may be the presentation media streams305b,305c, and305dsent to the client devices105b,105c, and105dof the participants. The content of the second set of media streams is based on content of the set of first media streams. The presentation and communications platform110may send the content of the presentation media streams305ato the client devices105b,105c, and105dof the participants. The presentation and communications platform110may preprocess the stream content before sending the content to the client devices105b,105c, and105dof the participants. For example, the presentation and communications platform110may preprocess the media streams sent to the client devices105b,105c, and105dbased on the capabilities of the client devices105b,105c, and105d. The video encoding format and/or other parameters may be adjusted based on the capabilities of the client devices105b,105c, and105d. Thus, the presentation media streams305b,305c, and305dsent to each of the client devices105b,105c, and105dmay be slightly different.

The process1300may include an operation1340of receiving, via the network connection, a set of third media streams from the computing devices of a first subset of the plurality of participants. The set of third media streams include video content of the first subset of the plurality of participants captured by the respective computing devices of the first subset of the plurality of participants. The third media streams may be the participant media streams310send by the client devices105of the participants to the presentation and communications platform110. The third media streams may include video and/or audio content of the participants captured by the client devices105of the participants.

The process1300may include an operation1350of analyzing the set of third media streams to identify a set of first reactions by the first subset of the plurality of participants to obtain first reaction information. The stream processing unit215of the presentation and communications platform110may analyze the third set of media streams using one or more machine learning models, as discussed with respect to the examples shown inFIGS. 4 and 6. The machine learning models may output high-level feature information identified in the third media streams.

The process1300may include an operation1360of determining first graphical representation information representing the first reaction information. The high-level feature information may identify a gesture made by the participant, a pose of the participant, and/or other actions by the participant that may be mapped to a reaction. The high-level feature information may be mapped to a reaction by the feedback and reporting unit225.

The process1300may include an operation1370of sending, via the network connection, a fourth media stream to the first computing device that includes the first graphical representation information to cause the first computing device to display the first graphical representation on a display of the first computing device while the presentation content is being provided via the set of first media streams. The feedback and reporting unit225may aggregate the reactions identified in the participant media streams310with the reactions included in the reactions data315b,315c, and315d. The aggregated reactions data may be provided to the client device105aof the presenter as the reactions data315. The client device may present the reactions to the presenter during the presentation as discussed in the preceding examples.

The detailed examples of systems, devices, and techniques described in connection withFIGS. 1-13are presented herein for illustration of the disclosure and its benefits. Such examples of use should not be construed to be limitations on the logical process embodiments of the disclosure, nor should variations of user interface methods from those described herein be considered outside the scope of the present disclosure. It is understood that references to displaying or presenting an item (such as, but not limited to, presenting an image on a display device, presenting audio via one or more loudspeakers, and/or vibrating a device) include issuing instructions, commands, and/or signals causing, or reasonably expected to cause, a device or system to display or present the item. In some embodiments, various features described inFIGS. 1-13are implemented in respective modules, which may also be referred to as, and/or include, logic, components, units, and/or mechanisms. Modules may constitute either software modules (for example, code embodied on a machine-readable medium) or hardware modules.

In some examples, a hardware module may be implemented mechanically, electronically, or with any suitable combination thereof. For example, a hardware module may include dedicated circuitry or logic that is configured to perform certain operations. For example, a hardware module may include a special-purpose processor, such as a field-programmable gate array (FPGA) or an Application Specific Integrated Circuit (ASIC). A hardware module may also include programmable logic or circuitry that is temporarily configured by software to perform certain operations and may include a portion of machine-readable medium data and/or instructions for such configuration. For example, a hardware module may include software encompassed within a programmable processor configured to execute a set of software instructions. It will be appreciated that the decision to implement a hardware module mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry (for example, configured by software) may be driven by cost, time, support, and engineering considerations.

Accordingly, the phrase “hardware module” should be understood to encompass a tangible entity capable of performing certain operations and may be configured or arranged in a certain physical manner, be that an entity that is physically constructed, permanently configured (for example, hardwired), and/or temporarily configured (for example, programmed) to operate in a certain manner or to perform certain operations described herein. As used herein, “hardware-implemented module” refers to a hardware module. Considering examples in which hardware modules are temporarily configured (for example, programmed), each of the hardware modules need not be configured or instantiated at any one instance in time. For example, where a hardware module includes a programmable processor configured by software to become a special-purpose processor, the programmable processor may be configured as respectively different special-purpose processors (for example, including different hardware modules) at different times. Software may accordingly configure a processor or processors, for example, to constitute a particular hardware module at one instance of time and to constitute a different hardware module at a different instance of time. A hardware module implemented using one or more processors may be referred to as being “processor implemented” or “computer implemented.”

Hardware modules can provide information to, and receive information from, other hardware modules. Accordingly, the described hardware modules may be regarded as being communicatively coupled. Where multiple hardware modules exist contemporaneously, communications may be achieved through signal transmission (for example, over appropriate circuits and buses) between or among two or more of the hardware modules. In embodiments in which multiple hardware modules are configured or instantiated at different times, communications between such hardware modules may be achieved, for example, through the storage and retrieval of information in memory devices to which the multiple hardware modules have access. For example, one hardware module may perform an operation and store the output in a memory device, and another hardware module may then access the memory device to retrieve and process the stored output.

In some examples, at least some of the operations of a method may be performed by one or more processors or processor-implemented modules. Moreover, the one or more processors may also operate to support performance of the relevant operations in a “cloud computing” environment or as a “software as a service” (SaaS). For example, at least some of the operations may be performed by, and/or among, multiple computers (as examples of machines including processors), with these operations being accessible via a network (for example, the Internet) and/or via one or more software interfaces (for example, an application program interface (API)). The performance of certain of the operations may be distributed among the processors, not only residing within a single machine, but deployed across several machines. Processors or processor-implemented modules may be in a single geographic location (for example, within a home or office environment, or a server farm), or may be distributed across multiple geographic locations.

FIG. 14is a block diagram1400illustrating an example software architecture1402, various portions of which may be used in conjunction with various hardware architectures herein described, which may implement any of the above-described features.FIG. 14is a non-limiting example of a software architecture and it will be appreciated that many other architectures may be implemented to facilitate the functionality described herein. The software architecture1402may execute on hardware such as a machine1500ofFIG. 15that includes, among other things, processors1510, memory1530, and input/output (I/O) components1550. A representative hardware layer1404is illustrated and can represent, for example, the machine1500ofFIG. 15. The representative hardware layer1404includes a processing unit1406and associated executable instructions1408. The executable instructions1408represent executable instructions of the software architecture1402, including implementation of the methods, modules and so forth described herein. The hardware layer1404also includes a memory/storage1410, which also includes the executable instructions1408and accompanying data. The hardware layer1404may also include other hardware modules1412. Instructions1408held by processing unit1406may be portions of instructions1408held by the memory/storage1410.

The example software architecture1402may be conceptualized as layers, each providing various functionality. For example, the software architecture1402may include layers and components such as an operating system (OS)1414, libraries1416, frameworks1418, applications1420, and a presentation layer1444. Operationally, the applications1420and/or other components within the layers may invoke API calls1424to other layers and receive corresponding results1426. The layers illustrated are representative in nature and other software architectures may include additional or different layers. For example, some mobile or special purpose operating systems may not provide the frameworks/middleware1418.

The OS1414may manage hardware resources and provide common services. The OS1414may include, for example, a kernel1428, services1430, and drivers1432. The kernel1428may act as an abstraction layer between the hardware layer1404and other software layers. For example, the kernel1428may be responsible for memory management, processor management (for example, scheduling), component management, networking, security settings, and so on. The services1430may provide other common services for the other software layers. The drivers1432may be responsible for controlling or interfacing with the underlying hardware layer1404. For instance, the drivers1432may include display drivers, camera drivers, memory/storage drivers, peripheral device drivers (for example, via Universal Serial Bus (USB)), network and/or wireless communication drivers, audio drivers, and so forth depending on the hardware and/or software configuration.

The libraries1416may provide a common infrastructure that may be used by the applications1420and/or other components and/or layers. The libraries1416typically provide functionality for use by other software modules to perform tasks, rather than rather than interacting directly with the OS1414. The libraries1416may include system libraries1434(for example, C standard library) that may provide functions such as memory allocation, string manipulation, file operations. In addition, the libraries1416may include API libraries1436such as media libraries (for example, supporting presentation and manipulation of image, sound, and/or video data formats), graphics libraries (for example, an OpenGL library for rendering 2D and 3D graphics on a display), database libraries (for example, SQLite or other relational database functions), and web libraries (for example, WebKit that may provide web browsing functionality). The libraries1416may also include a wide variety of other libraries1438to provide many functions for applications1420and other software modules.

The frameworks1418(also sometimes referred to as middleware) provide a higher-level common infrastructure that may be used by the applications1420and/or other software modules. For example, the frameworks1418may provide various graphic user interface (GUI) functions, high-level resource management, or high-level location services. The frameworks1418may provide a broad spectrum of other APIs for applications1420and/or other software modules.

The applications1420include built-in applications1440and/or third-party applications1442. Examples of built-in applications1440may include, but are not limited to, a contacts application, a browser application, a location application, a media application, a messaging application, and/or a game application. Third-party applications1442may include any applications developed by an entity other than the vendor of the particular platform. The applications1420may use functions available via OS1414, libraries1416, frameworks1418, and presentation layer1444to create user interfaces to interact with users.

Some software architectures use virtual machines, as illustrated by a virtual machine1448. The virtual machine1448provides an execution environment where applications/modules can execute as if they were executing on a hardware machine (such as the machine1500ofFIG. 15, for example). The virtual machine1448may be hosted by a host OS (for example, OS1414) or hypervisor, and may have a virtual machine monitor1446which manages operation of the virtual machine1448and interoperation with the host operating system. A software architecture, which may be different from software architecture1402outside of the virtual machine, executes within the virtual machine1448such as an OS1450, libraries1452, frameworks1454, applications1456, and/or a presentation layer1458.

FIG. 15is a block diagram illustrating components of an example machine1500configured to read instructions from a machine-readable medium (for example, a machine-readable storage medium) and perform any of the features described herein. The example machine1500is in a form of a computer system, within which instructions1516(for example, in the form of software components) for causing the machine1500to perform any of the features described herein may be executed. As such, the instructions1516may be used to implement modules or components described herein. The instructions1516cause unprogrammed and/or unconfigured machine1500to operate as a particular machine configured to carry out the described features. The machine1500may be configured to operate as a standalone device or may be coupled (for example, networked) to other machines. In a networked deployment, the machine1500may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a node in a peer-to-peer or distributed network environment. Machine1500may be embodied as, for example, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a gaming and/or entertainment system, a smart phone, a mobile device, a wearable device (for example, a smart watch), and an Internet of Things (IoT) device. Further, although only a single machine1500is illustrated, the term “machine” includes a collection of machines that individually or jointly execute the instructions1516.

The machine1500may include processors1510, memory1530, and I/O components1550, which may be communicatively coupled via, for example, a bus1502. The bus1502may include multiple buses coupling various elements of machine1500via various bus technologies and protocols. In an example, the processors1510(including, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an ASIC, or a suitable combination thereof) may include one or more processors1512ato1512nthat may execute the instructions1516and process data. In some examples, one or more processors1510may execute instructions provided or identified by one or more other processors1510. The term “processor” includes a multi-core processor including cores that may execute instructions contemporaneously. AlthoughFIG. 15shows multiple processors, the machine1500may include a single processor with a single core, a single processor with multiple cores (for example, a multi-core processor), multiple processors each with a single core, multiple processors each with multiple cores, or any combination thereof. In some examples, the machine1500may include multiple processors distributed among multiple machines.

The memory/storage1530may include a main memory1532, a static memory1534, or other memory, and a storage unit1536, both accessible to the processors1510such as via the bus1502. The storage unit1536and memory1532,1534store instructions1516embodying any one or more of the functions described herein. The memory/storage1530may also store temporary, intermediate, and/or long-term data for processors1510. The instructions1516may also reside, completely or partially, within the memory1532,1534, within the storage unit1536, within at least one of the processors1510(for example, within a command buffer or cache memory), within memory at least one of I/O components1550, or any suitable combination thereof, during execution thereof. Accordingly, the memory1532,1534, the storage unit1536, memory in processors1510, and memory in I/O components1550are examples of machine-readable media.

As used herein, “machine-readable medium” refers to a device able to temporarily or permanently store instructions and data that cause machine1500to operate in a specific fashion, and may include, but is not limited to, random-access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical storage media, magnetic storage media and devices, cache memory, network-accessible or cloud storage, other types of storage and/or any suitable combination thereof. The term “machine-readable medium” applies to a single medium, or combination of multiple media, used to store instructions (for example, instructions1516) for execution by a machine1500such that the instructions, when executed by one or more processors1510of the machine1500, cause the machine1500to perform and one or more of the features described herein. Accordingly, a “machine-readable medium” may refer to a single storage device, as well as “cloud-based” storage systems or storage networks that include multiple storage apparatus or devices. The term “machine-readable medium” excludes signals per se.

The I/O components1550may include a wide variety of hardware components adapted to receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so on. The specific I/O components1550included in a particular machine will depend on the type and/or function of the machine. For example, mobile devices such as mobile phones may include a touch input device, whereas a headless server or IoT device may not include such a touch input device. The particular examples of I/O components illustrated inFIG. 15are in no way limiting, and other types of components may be included in machine1500. The grouping of I/O components1550are merely for simplifying this discussion, and the grouping is in no way limiting. In various examples, the I/O components1550may include user output components1552and user input components1554. User output components1552may include, for example, display components for displaying information (for example, a liquid crystal display (LCD) or a projector), acoustic components (for example, speakers), haptic components (for example, a vibratory motor or force-feedback device), and/or other signal generators. User input components1554may include, for example, alphanumeric input components (for example, a keyboard or a touch screen), pointing components (for example, a mouse device, a touchpad, or another pointing instrument), and/or tactile input components (for example, a physical button or a touch screen that provides location and/or force of touches or touch gestures) configured for receiving various user inputs, such as user commands and/or selections.

In some examples, the I/O components1550may include biometric components1556, motion components1558, environmental components1560, and/or position components1562, among a wide array of other physical sensor components. The biometric components1556may include, for example, components to detect body expressions (for example, facial expressions, vocal expressions, hand or body gestures, or eye tracking), measure biosignals (for example, heart rate or brain waves), and identify a person (for example, via voice-, retina-, fingerprint-, and/or facial-based identification). The motion components1558may include, for example, acceleration sensors (for example, an accelerometer) and rotation sensors (for example, a gyroscope). The environmental components1560may include, for example, illumination sensors, temperature sensors, humidity sensors, pressure sensors (for example, a barometer), acoustic sensors (for example, a microphone used to detect ambient noise), proximity sensors (for example, infrared sensing of nearby objects), and/or other components that may provide indications, measurements, or signals corresponding to a surrounding physical environment. The position components1562may include, for example, location sensors (for example, a Global Position System (GPS) receiver), altitude sensors (for example, an air pressure sensor from which altitude may be derived), and/or orientation sensors (for example, magnetometers).

The I/O components1550may include communication components1564, implementing a wide variety of technologies operable to couple the machine1500to network(s)1570and/or device(s)1580via respective communicative couplings1572and1582. The communication components1564may include one or more network interface components or other suitable devices to interface with the network(s)1570. The communication components1564may include, for example, components adapted to provide wired communication, wireless communication, cellular communication, Near Field Communication (NFC), Bluetooth communication, Wi-Fi, and/or communication via other modalities. The device(s)1580may include other machines or various peripheral devices (for example, coupled via USB).