Virtual Presentation Rehearsal

In one implementation, a method of assisting in the rehearsal of a presentation is performed at a device including a display, one or more processors, and non-transitory memory. The method includes obtaining a difficulty level for a rehearsal of a presentation. The method includes displaying, on the display, one or more slides of the presentation. The method includes displaying, on the display in association with a volumetric environment, one or more virtual objects based on the difficulty level.

TECHNICAL FIELD

The present disclosure generally relates to systems, methods, and devices for assisting a user rehearse a presentation.

BACKGROUND

Many people experience anxiety or “stage fright” in anticipation or during performance of a presentation before an audience. Repeated practice performing the presentation can lessen this anxiety.

SUMMARY

Various implementations disclosed herein include devices, systems, and methods for assisting in the rehearsal of a presentation. In various implementations, the method is performed by a device including a display, one or more processors, and non-transitory memory. The method includes obtaining a difficulty level for a rehearsal of a presentation based on a difficulty level for a prior rehearsal of the presentation and a rating of the prior rehearsal of the presentation. The method includes displaying, on the display, one or more slides of the presentation. The method includes displaying, on the display in association with a volumetric environment, one or more virtual objects based on the difficulty level.

Various implementations disclosed herein include devices, systems, and methods for providing feedback on a performance of a presentation. In various implementations, the method is performed by a device including a display, one or more processors, and non-transitory memory. The method includes displaying, on the display, one or more slides of a presentation. The method includes recording audio of a user during a performance of the presentation. The method includes generating feedback based on comparing the audio of the user to previously recorded audio of the user during a prior performance of the presentation. The method includes providing the feedback to the user.

DESCRIPTION

People may sense or interact with a physical environment or world without using an electronic device. Physical features, such as a physical object or surface, may be included within a physical environment. For instance, a physical environment may correspond to a physical city having physical buildings, roads, and vehicles. People may directly sense or interact with a physical environment through various means, such as smell, sight, taste, hearing, and touch. This can be in contrast to an extended reality (XR) environment that may refer to a partially or wholly simulated environment that people may sense or interact with using an electronic device. The XR environment may include virtual reality (VR) content, mixed reality (MR) content, augmented reality (AR) content, or the like. Using an XR system, a portion of a person's physical motions, or representations thereof, may be tracked and, in response, properties of virtual objects in the XR environment may be changed in a way that complies with at least one law of nature. For example, the XR system may detect a user's head movement and adjust auditory and graphical content presented to the user in a way that simulates how sounds and views would change in a physical environment. In other examples, the XR system may detect movement of an electronic device (e.g., a laptop, tablet, mobile phone, or the like) presenting the XR environment. Accordingly, the XR system may adjust auditory and graphical content presented to the user in a way that simulates how sounds and views would change in a physical environment. In some instances, other inputs, such as a representation of physical motion (e.g., a voice command), may cause the XR system to adjust properties of graphical content.

Numerous types of electronic systems may allow a user to sense or interact with an XR environment. A non-exhaustive list of examples includes lenses having integrated display capability to be placed on a user's eyes (e.g., contact lenses), heads-up displays (HUDs), projection-based systems, head mountable systems, windows or windshields having integrated display technology, headphones/earphones, input systems with or without haptic feedback (e.g., handheld or wearable controllers), smartphones, tablets, desktop/laptop computers, and speaker arrays. Head mountable systems may include an opaque display and one or more speakers. Other head mountable systems may be configured to receive an opaque external display, such as that of a smartphone. Head mountable systems may capture images/video of the physical environment using one or more image sensors or capture audio of the physical environment using one or more microphones. Instead of an opaque display, some head mountable systems may include a transparent or translucent display. Transparent or translucent displays may direct light representative of images to a user's eyes through a medium, such as a hologram medium, optical waveguide, an optical combiner, optical reflector, other similar technologies, or combinations thereof. Various display technologies, such as liquid crystal on silicon, LEDs, uLEDs, OLEDs, laser scanning light source, digital light projection, or combinations thereof, may be used. In some examples, the transparent or translucent display may be selectively controlled to become opaque. Projection-based systems may utilize retinal projection technology that projects images onto a user's retina or may project virtual content into the physical environment, such as onto a physical surface or as a hologram.

As noted above, it is common for a person to feel anxiety or “stage fright” in anticipation or during performance of a presentation before other people. Repeated rehearsal increases a user's confidence in his or her ability to perform the presentation and can lessen this anxiety. Further, rehearsal under various different circumstances, such as in different locations or in front of different people, can best reduce this anxiety as the performer becomes more confident that the presentation will be well-given under any set of circumstances. Accordingly, in various implementations, an electronic device provides various XR environments, with various levels of difficulty, in which a user can rehearse a presentation.

In various implementations, the electronic device further provides feedback to the user regarding performance of the presentation, either during a rehearsal or a live presentation. Such feedback may be provided during the performance of the presentation or when the performance of the presentation has concluded. Feedback further boosts user confidence and reduces anxiety.

FIG.1is a block diagram of an example operating environment100in accordance with some implementations. While pertinent features are shown, those of ordinary skill in the art will appreciate from the present disclosure that various other features have not been illustrated for the sake of brevity and so as not to obscure more pertinent aspects of the example implementations disclosed herein. To that end, as a non-limiting example, the operating environment100includes a controller110and an electronic device120.

In some implementations, the controller110is configured to manage and coordinate an XR experience for the user. In some implementations, the controller110includes a suitable combination of software, firmware, and/or hardware. The controller110is described in greater detail below with respect toFIG.2. In some implementations, the controller110is a computing device that is local or remote relative to the physical environment105. For example, the controller110is a local server located within the physical environment105. In another example, the controller110is a remote server located outside of the physical environment105(e.g., a cloud server, central server, etc.). In some implementations, the controller110is communicatively coupled with the electronic device120via one or more wired or wireless communication channels144(e.g., BLUETOOTH, IEEE 802.11x, IEEE 802.16x, IEEE 802.3x, etc.). In another example, the controller110is included within the enclosure of the electronic device120. In some implementations, the functionalities of the controller110are provided by and/or combined with the electronic device120.

In some implementations, the electronic device120is configured to provide the XR experience to the user. In some implementations, the electronic device120includes a suitable combination of software, firmware, and/or hardware. According to some implementations, the electronic device120presents, via a display122, XR content to the user while the user is physically present within the physical environment105that includes a table107within the field-of-view111of the electronic device120. As such, in some implementations, the user holds the electronic device120in his/her hand(s). In some implementations, while providing XR content, the electronic device120is configured to display an XR object (e.g., an XR cylinder109) and to enable video pass-through of the physical environment105(e.g., including a representation117of the table107) on a display122. The electronic device120is described in greater detail below with respect toFIG.3.

According to some implementations, the electronic device120provides an XR experience to the user while the user is virtually and/or physically present within the physical environment105.

In some implementations, the user wears the electronic device120on his/her head. For example, in some implementations, the electronic device includes a head-mounted system (HMS), head-mounted device (HMD), or head-mounted enclosure (HME). As such, the electronic device120includes one or more XR displays provided to display the XR content. For example, in various implementations, the electronic device120encloses the field-of-view of the user. In some implementations, the electronic device120is a handheld device (such as a smartphone or tablet) configured to present XR content, and rather than wearing the electronic device120, the user holds the device with a display directed towards the field-of-view of the user and a camera directed towards the physical environment105. In some implementations, the handheld device can be placed within an enclosure that can be worn on the head of the user. In some implementations, the electronic device120is replaced with an XR chamber, enclosure, or room configured to present XR content in which the user does not wear or hold the electronic device120.

The memory220includes high-speed random-access memory, such as dynamic random-access memory (DRAM), static random-access memory (SRAM), double-data-rate random-access memory (DDR RAM), or other random-access solid-state memory devices. In some implementations, the memory220includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory220optionally includes one or more storage devices remotely located from the one or more processing units202. The memory220comprises a non-transitory computer readable storage medium. In some implementations, the memory220or the non-transitory computer readable storage medium of the memory220stores the following programs, modules and data structures, or a subset thereof including an optional operating system230and an XR experience module240.

The operating system230includes procedures for handling various basic system services and for performing hardware dependent tasks. In some implementations, the XR experience module240is configured to manage and coordinate one or more XR experiences for one or more users (e.g., a single XR experience for one or more users, or multiple XR experiences for respective groups of one or more users). To that end, in various implementations, the XR experience module240includes a data obtaining unit242, a tracking unit244, a coordination unit246, and a data transmitting unit248.

In some implementations, the data obtaining unit242is configured to obtain data (e.g., presentation data, interaction data, sensor data, location data, etc.) from at least the electronic device120ofFIG.1. To that end, in various implementations, the data obtaining unit242includes instructions and/or logic therefor, and heuristics and metadata therefor.

In some implementations, the tracking unit244is configured to map the physical environment105and to track the position/location of at least the electronic device120with respect to the physical environment105ofFIG.1. To that end, in various implementations, the tracking unit244includes instructions and/or logic therefor, and heuristics and metadata therefor.

In some implementations, the coordination unit246is configured to manage and coordinate the XR experience presented to the user by the electronic device120. To that end, in various implementations, the coordination unit246includes instructions and/or logic therefor, and heuristics and metadata therefor.

In some implementations, the data transmitting unit248is configured to transmit data (e.g., presentation data, location data, etc.) to at least the electronic device120. To that end, in various implementations, the data transmitting unit248includes instructions and/or logic therefor, and heuristics and metadata therefor.

Although the data obtaining unit242, the tracking unit244, the coordination unit246, and the data transmitting unit248are shown as residing on a single device (e.g., the controller110), it should be understood that in other implementations, any combination of the data obtaining unit242, the tracking unit244, the coordination unit246, and the data transmitting unit248may be located in separate computing devices.

In some implementations, the one or more XR displays312are configured to provide the XR experience to the user. In some implementations, the one or more XR displays312correspond to holographic, digital light processing (DLP), liquid-crystal display (LCD), liquid-crystal on silicon (LCoS), organic light-emitting field-effect transitory (OLET), organic light-emitting diode (OLED), surface-conduction electron-emitter display (SED), field-emission display (FED), quantum-dot light-emitting diode (QD-LED), micro-electro-mechanical system (MEMS), and/or the like display types. In some implementations, the one or more XR displays312correspond to diffractive, reflective, polarized, holographic, etc. waveguide displays. For example, the electronic device120includes a single XR display. In another example, the electronic device includes an XR display for each eye of the user. In some implementations, the one or more XR displays312are capable of presenting MR and VR content.

In some implementations, the one or more image sensors314are configured to obtain image data that corresponds to at least a portion of the face of the user that includes the eyes of the user (and may be referred to as an eye-tracking camera). In some implementations, the one or more image sensors314are configured to be forward-facing so as to obtain image data that corresponds to the scene as would be viewed by the user if the electronic device120was not present (and may be referred to as a scene camera). The one or more optional image sensors314can include one or more RGB cameras (e.g., with a complimentary metal-oxide-semiconductor (CMOS) image sensor or a charge-coupled device (CCD) image sensor), one or more infrared (IR) cameras, one or more event-based cameras, and/or the like.

The memory320includes high-speed random-access memory, such as DRAM, SRAM, DDR RAM, or other random-access solid-state memory devices. In some implementations, the memory320includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory320optionally includes one or more storage devices remotely located from the one or more processing units302. The memory320comprises a non-transitory computer readable storage medium. In some implementations, the memory320or the non-transitory computer readable storage medium of the memory320stores the following programs, modules and data structures, or a subset thereof including an optional operating system330and an XR presentation module340.

The operating system330includes procedures for handling various basic system services and for performing hardware dependent tasks. In some implementations, the XR presentation module340is configured to present XR content to the user via the one or more XR displays312. To that end, in various implementations, the XR presentation module340includes a data obtaining unit342, a rehearsal unit344, a feedback unit345, an XR presenting unit346, and a data transmitting unit348.

In some implementations, the data obtaining unit342is configured to obtain data (e.g., presentation data, interaction data, sensor data, location data, etc.) from at least the controller110ofFIG.1. To that end, in various implementations, the data obtaining unit342includes instructions and/or logic therefor, and heuristics and metadata therefor.

In some implementations, the rehearsal unit344is configured to obtain XR content based on a difficulty level of a presentation rehearsal. To that end, in various implementations, the rehearsal unit344includes instructions and/or logic therefor, and heuristics and metadata therefor.

In some implementations, the feedback unit345is configured to generate feedback by comparing a current presentation (either a rehearsal or a live presentation) to previously recorded rehearsals of the presentation. To that end, in various implementations, the feedback unit345includes instructions and/or logic therefor, and heuristics and metadata therefor.

In some implementations, the XR presenting unit346is configured to present XR content via the one or more XR displays312, such as a representation of the selected text input field at a location proximate to the text input device. To that end, in various implementations, the XR presenting unit346includes instructions and/or logic therefor, and heuristics and metadata therefor.

In some implementations, the data transmitting unit348is configured to transmit data (e.g., presentation data, location data, etc.) to at least the controller110. In some implementations, the data transmitting unit348is configured to transmit authentication credentials to the electronic device. To that end, in various implementations, the data transmitting unit348includes instructions and/or logic therefor, and heuristics and metadata therefor.

Although the data obtaining unit342, the rehearsal unit344, the feedback unit345, the XR presenting unit346, and the data transmitting unit348are shown as residing on a single device (e.g., the electronic device120), it should be understood that in other implementations, any combination of the data obtaining unit342the rehearsal unit344, the feedback unit345, the XR presenting unit346, and the data transmitting unit348may be located in separate computing devices.

FIGS.4A-4D,5A-5D, and6A-6Gillustrate various XR environments displayed by an electronic device as a user performs a number of rehearsals of a presentation as various difficulty levels. In particular,FIGS.4A-4Dillustrate a first XR environment400as a user performs a first rehearsal of the presentation at a first difficulty level,FIGS.5A-5Dillustrate a second XR environment500as a user performs a second rehearsal of the presentation at a second difficulty level greater (e.g., more difficult) than the first difficulty level, andFIGS.6A-6Gillustrate a third XR environment600as a user performs a third rehearsal of the presentation at a third difficulty level greater (e.g., more difficult) than the second difficulty level. In various implementations, the various XR environments include various virtual objects based on the corresponding difficulty level.

FIGS.4A-4Dillustrate the first XR environment400displayed, at least in part, by a display of the electronic device. The first XR environment400is based on a physical environment of a living room in which the electronic device is present.FIGS.4A-4Dillustrate the first XR environment400during a series of time periods. In various implementations, each time period is an instant, a fraction of a second, a few seconds, a few hours, a few days, or any length of time.

The first XR environment400includes a plurality of objects, including one or more physical objects (e.g., a picture401and a couch402) of the physical environment and one or more virtual objects (e.g., virtual animals411A-411B and a slide window412). In various implementations, certain objects (such as the physical objects401and402and the virtual animals411A-411B) are displayed at a location in the first XR environment400, e.g., at a location defined by three coordinates in a three-dimensional (3D) XR coordinate system. Accordingly, when the electronic device moves in the first XR environment400(e.g., changes either position and/or orientation), the objects are moved on the display of the electronic device, but retain their location in the first XR environment400. Such virtual objects that, in response to motion of the electronic device, move on the display, but retain their position in the XR environment are referred to as world-locked objects. In various implementations, certain virtual objects (such as the slide window412) are displayed at locations on the display such that when the electronic device moves in the first XR environment400, the objects are stationary on the display on the electronic device. Such virtual objects that, in response to motion of the electronic device, retain their location on the display are referred to as head-locked objects or display-locked objects.

FIG.4Aillustrates the first XR environment400during a first time period. During the first time period, the electronic device displays the virtual animals411A-411B on the couch402. During the first time period, the electronic device displays the slide window412displaying a first slide of a presentation. In various implementations, the slide window412is displayed at a fixed location on the display. In various implementations, the size and/or the location of the slide window412is manipulable by a user of the electronic device. For example, in various implementations, the user provides user input to move the slide window412to a different location on the display. As another example, in various implementations, the user provides user input to change a size of the slide window412. In various implementations, the slide window412is at least partially transparent.

During the first time period, the user provides a next-slide user input to change the slide window412to display the next slide of the presentation. In various implementations, the user input is a hand gesture or a vocal command.

FIG.4Billustrates the first XR environment400during a second time period subsequent to the first time period. During the second time period, in response to the next-slide user input, the slide window412displays a second slide of the presentation. During the second time period, the user provides a next-slide user input.

FIG.4Cillustrates the first XR environment400during a third time period subsequent to the second time period. During the third time period, in response to the next-slide user input, the slide window412displays a third slide of the presentation. During the third time period, the user provides a next-slide user input.

During the first time period, second time period, and third time period (referred to collectively as the rehearsal time period) the user performs a rehearsal of the presentation by speaking. During the rehearsal time period, the electronic device records audio of the user performing the rehearsal of the presentation (e.g., audio of the user speaking). The electronic device generates feedback based on the audio of the user. In various implementations, the electronic device generates feedback based on other data collected during the rehearsal time period, such as eye gaze tracking data or biometric data.

FIG.4Dillustrates the first XR environment400during a fourth time period subsequent to the third time period. During the fourth time period, in response to the next-slide user input and in accordance with a determination that the third slide was the final slide of the presentation, the slide window412is replaced with a feedback window415. The feedback window415is a display-locked virtual object that includes the feedback generated based on the audio of the user and/or other data collected during the rehearsal time period.

For example, inFIG.4D, the feedback window415indicates that the user seemed nervous, e.g., by analyzing vocal characteristics of the audio of the user, gaze, posture, movement, and/or biometrics such as heart rate, skin temperature and/or resistance, or pupil dilation. The feedback window415indicates that the user spoke at a decent pace, but could speak more clearly, e.g., by analyzing the speed of speech and performing speech recognition. The feedback window415indicates that the user mentioned most of the points in the presentation, e.g., by performing speech recognition and comparing the recognized speech to the text of the slides.

The feedback window415includes an objective score based on the audio of the user and/or other data collected during the rehearsal time period. Further, the feedback window415requests a subjective score from the user to indicate how the user would score their performance of the rehearsal of the presentation. In various implementations, the user provides the subjective score via user input, such as a vocal response.

For the rehearsal of the presentation, the electronic device stores the audio of the user and/or other data collected during the rehearsal time period and further stores the objective score and the subjective score.

FIGS.5A-5Dillustrate the second XR environment500displayed, at least in part, by a display of the electronic device. The second XR environment500is a virtual environment of a classroom.FIGS.5A-5Dillustrate the second XR environment500during a series of time periods. In various implementations, each time period is an instant, a fraction of a second, a few seconds, a few hours, a few days, or any length of time.

The second XR environment500includes a plurality of virtual objects including a virtual chair521, a virtual desk522, virtual students511A-511C, and a slide window512. In various implementations, certain objects (such as the virtual chair521, the virtual desk522, and the virtual students511A-511C) are displayed at a location in the second XR environment500, e.g., at a location defined by three coordinates in a three-dimensional (3D) XR coordinate system. Accordingly, when the electronic device moves in the second XR environment500(e.g., changes either position and/or orientation), the objects are moved on the display of the electronic device, but retain their location in the second XR environment500. In various implementations, certain virtual objects (such as the slide window512) are displayed at locations on the display such that when the electronic device moves in the second XR environment500, the objects are stationary on the display on the electronic device.

FIG.5Aillustrates the second XR environment500during a first time period. During the first time period, the electronic device displays the virtual students511A-511C sitting in virtual chairs behind virtual desks (such as the virtual student511C sitting in the virtual chair521behind the virtual desk522). During the first time period, the electronic device displays the slide window512displaying the first slide of the presentation. Further, the slide window512displays an eye tracking indicator530over a portion of the first slide of the presentation. The eye tracking indicator530indicates a portion of the slide at which audience members are gazing based on audience eye tracking data obtained by the electronic device. InFIG.5A, because the second XR environment500includes no physical audience members, the electronic device generates (rather than receives) the audience eye tracking data.

During the first time period, the user provides a next-slide user input to change the slide window512to display the next slide of the presentation.

FIG.5Billustrates the second XR environment500during a second time period subsequent to the first time period. During the second time period, in response to the next-slide user input, the slide window512displays a second slide of the presentation.

During the second time period, the second XR environment500includes a first feedback notification525A. During the first time period, the electronic device records audio of the user performing a first portion of rehearsal of the presentation (e.g., audio of the user speaking). Based on comparing the audio of the user to previously recorded audio of the user and/or other data (such as the text of the slides), the electronic device generates and displays feedback during the rehearsal. Thus, in comparing the audio of the user recording during the first time period to the text of the slides, the first feedback notification525A indicates that the user has failed to mention a point of the first slide.

During the second time period, the virtual student511C stands up from the virtual chair521behind the virtual desk522and begins to exit the classroom. During the second time period, the user provides a next-slide user input.

FIG.5Cillustrates the second XR environment500during a third time period subsequent to the second time period. During the third time period, in response to the next-slide user input, the slide window512displays a third slide of the presentation.

During the third time period, the second XR environment500includes a second feedback notification525B. During the first time period and second time period, the electronic device records audio of the user performing a first portion and second portion of a rehearsal of the presentation (e.g., audio of the user speaking). Based on comparing the audio of the user to previously recorded audio of the user and/or other data (such as the text of the slides), the electronic device generates and displays feedback during the rehearsal. Thus, in comparing the audio of the user recording during the first time period and second time period (e.g., the amount of time of the first time period and second time period) to previously recorded rehearsals (e.g., the corresponding amount of time of previously rehearsals), the second feedback notification525B indicates that the user should slow down. Further, to indicate the user should slow down, the slide window512is dimmed.

During the third time period, the virtual student511C has exited the classroom. During the third time period, the user provides a next-slide user input.

During the first time period, second time period, and third time period (referred to collectively as the rehearsal time period) the user performs a rehearsal of the presentation by speaking. During the rehearsal time period, the electronic device records audio of the user performing the rehearsal of the presentation (e.g., audio of the user speaking). The electronic device generates feedback based on the audio of the user. In various implementations, the electronic device generates feedback based on other data collected during the rehearsal time period, such as eye gaze tracking data or biometric data.

FIG.5Dillustrates the second XR environment500during a fourth time period subsequent to the third time period. During the fourth time period, in response to the next-slide user input and in accordance with a determination that the third slide was the final slide of the presentation, the slide window512is replaced with a feedback window515.

FIGS.6A-6Gillustrate the third XR environment600displayed, at least in part, by a display of the electronic device. The third XR environment600is a virtual environment of an auditorium.FIGS.6A-6Gillustrate the third XR environment600during a series of time periods. In various implementations, each time period is an instant, a fraction of a second, a few seconds, a few hours, a few days, or any length of time.

The third XR environment600includes a plurality of virtual objects including a virtual chair631, a virtual refreshment table621, a virtual microphone622, virtual audience members611A-611E, and a slide window612. In various implementations, certain objects (such as the virtual chair631, the virtual refreshment table621, the virtual microphone622, and the virtual audience members611A-611E) are displayed at a location in the third XR environment600, e.g., at a location defined by three coordinates in a three-dimensional (3D) XR coordinate system. Accordingly, when the electronic device moves in the third XR environment600(e.g., changes either position and/or orientation), the objects are moved on the display of the electronic device, but retain their location in the third XR environment600. In various implementations, certain virtual objects (such as the slide window612) are displayed at locations on the display such that when the electronic device moves in the third XR environment600, the objects are stationary on the display on the electronic device.

FIG.6Aillustrates the third XR environment600during a first time period. During the first time period, the electronic device displays the virtual audience members611A-611E sitting in virtual chairs (such as the virtual audience member611E sitting in the virtual chair631). During the first time period, the electronic device displays the slide window612displaying the first slide of the presentation.

FIG.6Billustrates the third XR environment600during a second time period subsequent to the first time period. During the second time period, the virtual audience member611E has left the virtual chair631and moved toward the virtual refreshment table621.

FIG.6Cillustrates the third XR environment600during a third time period subsequent to the second time period. During the third time period, the virtual audience member611E has returned to the virtual chair631.

During the third time period, the third XR environment600includes a first feedback notification625A. During the first time period and second time period, the electronic device records audio of the user performing a first portion of rehearsal of the presentation (e.g., audio of the user speaking). Based on comparing the audio of the user to previously recorded audio of the user and/or other data (such as the text of the slides), the electronic device generates and displays feedback during the rehearsal. Thus, in comparing the audio of the user recording during the first time period and the second time period to a previous recording of audio of the user performing a rehearsal of the presentation and/or the text of the slides, the first feedback notification625A indicates that the user may have missed a slide transition, e.g., began speaking about a point on a subsequent slide without performing a next-slide gesture.

During the third time period, the user provides a next-slide user input to change the slide window612to display the next slide of the presentation.

FIG.6Dillustrates the third XR environment600during a fourth time period subsequent to the third time period. During the fourth time period, the slide window612displays the second slide of the presentation. Further, during the fourth time period, the third XR environment600includes virtual balls642falling from the ceiling. Such virtual objects, such as virtual balloons, virtual lights, or virtual confetti function as a distraction for the user. Rehearsal with such distractors increases a user's confidence in presenting under any circumstances.

FIG.6Eillustrates the third XR environment600during a fifth time period subsequent to the fourth time period. During the fifth time period, the third XR environment600includes a second feedback notification625B indicating that the user has spent too much time on the second slide of the presentation, e.g., in comparing the audio of the user recording during earlier time periods to a previous recording of audio of the user performing a rehearsal of the presentation.

During the fifth time period, the user provides a next-slide user input to change the slide window612to display the next slide of the presentation.

FIG.6Fillustrates the third XR environment600during a sixth time period subsequent to the fifth time period. During the sixth time period, the slide window612displays the third slide of the presentation. Further, during the sixth time period, the virtual audience member611A has stood up to ask a question of the user. In various implementations, the question presented (e.g., via a speaker) is based on the audio of the user and/or other data collected during the rehearsal period, such as the text of the slides.

During the sixth time period, the user provides a next-slide user input to change the slide window612to display the next slide of the presentation.

During the first time period, second time period, third time period, fourth time period, fifth time period, and sixth time period (referred to collectively as the rehearsal time period) the user performs a rehearsal of the presentation by speaking. During the rehearsal time period, the electronic device records audio of the user performing the rehearsal of the presentation (e.g., audio of the user speaking). The electronic device generates feedback based on the audio of the user. In various implementations, the electronic device generates feedback based on other data collected during the rehearsal time period, such as eye gaze tracking data or biometric data.

FIG.6Gillustrates the third XR environment600during a seventh time period subsequent to the sixth time period. During the seventh time period, in response to the next-slide user input and in accordance with a determination that the third slide was the final slide of the presentation, the slide window612is replaced with a feedback window615.

FIG.7is a flowchart representation of a method700of assisting in the rehearsal of a presentation in accordance with some implementations. In various implementations, the method700is performed by a device including a display, one or more processors, and non-transitory memory (e.g., the electronic device120ofFIG.3). In some implementations, the method700is performed by processing logic, including hardware, firmware, software, or a combination thereof. In some implementations, the method700is performed by a processor executing instructions (e.g., code) stored in a non-transitory computer-readable medium (e.g., a memory).

The method700begins, in block710, with the device obtaining a difficulty level for a rehearsal of a presentation based on a difficulty level of a prior rehearsal of the presentation and a rating of the prior rehearsal of the presentation. For example, in various implementations, upon completing a rehearsal of the presentation at a particular difficulty level and receiving a rating of the rehearsal of the presentation above a threshold, the difficulty level is increased. In various implementations, the difficulty level is based additionally or alternatively on a user input.

In various implementations, the difficulty level is a single value selected from a plurality of predefined values. For example, the difficulty level may be “easy”, “moderate”, “hard”, or “extremely hard”. As another example, the difficulty level may be a natural number between 1 and 10.

In various implementations, the difficulty level includes a plurality of different values corresponding to different virtual object sets. For example, the difficulty level can include a first value corresponding to one of a plurality of different virtual venues (e.g., “home”, “school”, or “classroom”), a second value corresponding to a number of virtual audience members (e.g., “few”, “many”, or “full house”), and a third value corresponding to a level of distraction (e.g., “none”, “mild”, “strong”, or “extreme”).

In various implementations, the difficulty level is a single value that maps to a plurality of different values corresponding to different virtual object sets. For example, a difficulty level of 1 maps to “home”, “few”, and “none”; a difficulty level of 3 maps to “school”, “few”, and “mild”; and a difficulty level of 5 maps to “school”; “many”; and “strong”.

The method700continues, in block720, with the device displaying, on the display, one or more slides of the presentation. For example, inFIGS.4A-4C, the electronic device displays the slide window412which displays, at various times, the first, second, and third slides of the presentation. In various implementations, the device displays the one or more slides of the presentation in a slide window. In various implementations, the size and/or the location of the slide window is manipulable by a user of the device. For example, in various implementations, the user provides user input to move the slide window to a different location on the display. As another example, in various implementations, the user provides user input to change a size of the slide window. In various implementations, the slide window is at least partially transparent.

The method700continues, in block730, with the device displaying, on the display in association with a volumetric environment, one or more virtual objects based on the difficulty level. For example, inFIG.4A, the electronic device displays the virtual animals411A-411B. As another example, inFIG.5A, the electronic device displays the virtual students511A-511C and the walls, floor, and other objects (e.g., the virtual chair521and the virtual desk522) of a virtual classroom. As another example, inFIG.6A, the electronic device displays the virtual audience members611A-611E, and the wall, floors and other objects (e.g., the virtual chair631, virtual refreshment table621, and virtual microphone622) of a virtual auditorium.

In various implementations, displaying the one or more virtual objects based on the difficulty level includes displaying a virtual venue based on the difficulty level. For example, inFIG.5A, the electronic device displays a virtual classroom based on the second difficulty level and, inFIG.6A, the electronic device displays a virtual auditorium based on the third difficulty level. InFIG.4A, based on the first difficulty level, the electronic device does not display a virtual venue, but displays other virtual objects within the physical environment of the electronic device. Thus, in various implementations, at higher difficulty levels, the virtual venue is larger.

In various implementations, displaying the one or more virtual objects based on the difficulty level includes displaying one or more virtual audience members based on the difficulty level. For example, inFIG.4A, the electronic device displays the virtual animals411A-411B based on the first difficulty level, inFIG.5A, the electronic device displays the virtual students511A-511C based on the second difficulty level, and, inFIG.6A, the electronic device displays the virtual audience member611A-611E based on the third difficulty level.

In various implementations, a number of the virtual audience members is based on the difficulty level. In various implementations, at higher difficulty levels, the number of virtual audience members is larger. For example, inFIG.4A, the number of virtual audience members (e.g., the virtual animals411A-411B) is two based on the first difficulty level, inFIG.5A, the number of virtual audience members (e.g., the virtual students511A-511C) is six based on the second difficulty level, and, inFIG.6A, the number of virtual audience members (e.g., the virtual audience members611A-611E) is eighteen based on the third difficulty level.

In various implementations, an identity of a particular virtual audience member of the one or more virtual audience members is based on the difficulty level. In various implementations, at a first difficulty level, a particular virtual audience member is a stuffed animal or a cartoon avatar. In various implementations, at a second difficulty level higher than the first difficulty level, a particular virtual audience member is a person or photorealistic avatar. In various implementations, contacts of the user (as derived from contact information stored on the electronic device) are represented as virtual audience members. At different difficulty levels, different contacts of the user are represented as virtual audience members. For example, at a first difficulty level, contacts of the user identified as friends and/or family are represented as virtual audience members and, at a second difficulty level, contacts of the user identified as co-workers or professional contacts are represented as virtual audience members.

In various implementations, a behavior of a particular virtual audience member of the one or more virtual audience members is based on the difficulty level. For example, inFIG.5B, based on the second difficulty level, the virtual student511C stands up from the virtual chair521behind the virtual desk522and begins to exit the classroom. As another example, inFIG.6B, based on the third difficulty level, the virtual audience member611E has left the virtual chair631and moved toward the virtual refreshment table621. As another example, inFIG.6F, based on the third difficulty level, the virtual audience member611A has stood up to ask a question of the user.

In various implementations, based on the difficulty level, the virtual audience members exhibit various behaviors to distract the user during the rehearsal of the presentation. In various implementations, at a low difficulty level, a virtual audience member is displayed as engaged with (e.g., paying attention to) the presentation. However, at a higher difficulty level, the virtual audience member is displayed as bored, uninterested, or distracted. The virtual audience member may be displayed in such states via their facial expression. Thus, in various implementations, a facial expression of a particular virtual audience member of the one or more virtual audience members is based on the difficulty level.

In various implementations, a virtual audience member is displayed as moving within the XR environment, e.g., to leave the XR environment, get refreshments, change seats, or ask a question of the user. In various implementations, the virtual audience members ask questions of the user based on the difficulty level. For example, at a low difficulty level, a virtual audience member may ask a straightforward question based on the slides, audio of the user during the rehearsal, or previously recorded audio of the user during a previous rehearsal. At a higher difficulty level, the virtual audience member may ask a more difficult, less relevant, or more non-sequitur question. In various implementations, at a low difficulty level, a virtual audience member asks a question of the user at the end of the presentations. At a higher difficulty level, the virtual audience member interrupts the presentation to ask the question.

In various implementations, displaying the one or more virtual objects based on the difficulty level includes displaying one or more virtual distractors based on the difficulty level. For example, inFIG.6D, the electronic device displays virtual balls642falling from the ceiling. In various implementations, the visual distractors may include virtual lights or virtual confetti. In various implementations, at a first difficulty level, visual distractors are not displayed and, at a second difficulty level higher than the first difficulty level, visual distractors are displayed.

In various implementations, displaying the one or more virtual objects based on the difficulty level includes displaying one or more virtual objects based on the one or more slides of the presentation. For example, as noted above, in various implementations, a virtual audience member asks a question based on the content of the slides of the presentation. In various implementations, during a performance of a presentation, audience feedback is displayed by the electronic device, such as an eye tracking indicator indicating a portion of the slide at which audience members are gazing based on audience eye tracking data received by the electronic device, an engagement bar indicating a level at which audience members are engaged with the presentation, or search notifications indicating web search queries performed by audience members during the presentation. In various implementations, based on the slides of the presentation and the difficulty level, the electronic device displays fake audience feedback. For example, at a first difficulty level, the electronic device displays an eye tracking indicator indicating a portion of the slide that the user is discussing and, at a second difficulty level higher than the first difficulty level, the electronic displays an eye tracking indicator over an irrelevant portion of the slide. As another example, at a first difficulty level, the electronic device displays an engagement bar indicating that the audience is engaged and, at a second difficulty level higher than the first difficulty level, the electronic device displays an engagement bar indicating that the audience is bored or confused.

FIG.8is a flowchart representation of a method800of providing feedback on a performance of a presentation in accordance with some implementations. In various implementations, the method800is performed by a device including a display, one or more processors, and non-transitory memory (e.g., the electronic device120ofFIG.3). In some implementations, the method800is performed by processing logic, including hardware, firmware, software, or a combination thereof. In some implementations, the method800is performed by a processor executing instructions (e.g., code) stored in a non-transitory computer-readable medium (e.g., a memory).

In various implementations, a performance of a presentation is a rehearsal of a presentation (e.g., to virtual audience members). In various implementations, a performance of a presentation is a live performance of a presentation (e.g., to physical audience members).

The method800begins, in block810, with the device displaying, on the display, one or more slides of a presentation. For example, inFIGS.4A-4C, the electronic device displays the slide window412which displays, at various times, the first, second, and third slides of the presentation. In various implementations, the device displays on the one or more slides of the presentation in a slide window. In various implementations, the size and/or the location of the slide window is manipulable by a user of the device. For example, in various implementations, the user provides user input to move the slide window to a different location on the display. As another example, in various implementations, the user provides user input to change a size of the slide window. In various implementations, the slide window is at least partially transparent.

The method800continues, in block820, with the device recording audio of a user during a performance of the presentation. In various implementations, the audio of the user includes speech of the user. In various implementations, the device performs speech recognition on the audio of the user to generate text of the performance of the presentation. In various implementations, portions of text of the performance of the presentation are time-stamped. In various implementations, the device records slide transition times of the performance of the presentation.

In various implementations, the device stores previously recorded audio of the user during prior performance of the presentation. In various implementations, the previously recorded audio of the user includes speech of the user. In various implementations, the device performs speech recognition on the previously recorded audio of the user to generate text of the prior performance of the presentation. In various implementations, portions of the text of the prior performance of the presentation are time-stamped. In various implementations, the device stores slide transition times of the prior performance of the presentation.

The method800continues, in block830, with the device generating feedback based on comparing the audio of the user to the previously recorded audio of the user during a prior performance of the presentation. In various implementations, the audio of the user is compared to multiple sets of previously recorded audio during prior performances of the presentations. In various implementations, the audio of the user is compared to the previously recorded audio of the user during one or more prior performances that have a subjective and/or objective rating greater than a threshold. Thus, the current performance is compared to a well-rated prior performance or desired performance.

In various implementations, generating the feedback includes comparing content of the audio of the user (e.g., speech of the user, possibility as text of the performance of the presentation) to content of the previously recorded audio of the user (e.g., speech of the user, possibility as text of the prior performance of the presentation). For example, in various implementations, the feedback indicates that the user has failed to discuss a topic, make a point, or say a particular keyword in the performance of the presentation that the user did discuss, make, or say in the prior performance. For example, inFIG.5B, the first feedback notification525A indicates that the user failed to make “POINT 1B” from the slides.

In various implementations, generating the feedback includes comparing a time of the audio of the user to a corresponding time of the previously recorded audio of the user. For example, in various implementations, an amount of time spent discussing a particular topic or presenting a particular slide is compared to the corresponding amount of time spent in a prior performance of the presentation. For example, inFIG.6E, the second feedback notification625B indicates that the user has spent too much time on the current slide (e.g., more than a prior performance).

In various implementations, generating the feedback includes comparing an intelligibility of speech of the audio of the user to an intelligibility of speech of the previously recorded audio of the user. Various measures of intelligibility can be used including speech recognition metrics. For example, inFIG.5C, the second feedback notification525B indicates that the user is speaking too fast (e.g., faster than a prior performance).

In various implementations, generating the feedback is further based on comparing slide transition times during the performance of the presentation to previously recorded slide transition times during the prior performance of the presentation. For example, such comparison can be used to determine the amount of time spent presenting each slide. As another example, such comparison can be used to determine what speech corresponds which each slide so as to determine whether a user missed a slide transition. For example, inFIG.6C, the first feedback notification625A indicates that the user has possibly missed a slide transition (e.g., because the user has spent more time on the first slide than in a prior performance or because the user is discussing content that was discussed in a prior performance while the second slide was displayed).

In various implementations, generating the feedback is further based on comparing the audio of the user to the one or more slides of the presentation. In various implementations, the feedback is based on comparing the audio of the user to content of the one or more slides of the presentation. For example, by comparing the text of the presentation to text of the slides, the electronic device can determine that certain topics were not discussed. For example, inFIG.5B, the first feedback notification525A indicates that the user failed to make “POINT 1B” from the slides.

In various implementations, generating the feedback is further based on a biometric of the user during the performance of the presentation. Such biometrics include heart rate, skin temperature and/or resistance, or pupil dilation. For example, the electronic device can determine whether the user is calm or nervous and provide corresponding feedback. For example, inFIG.4D, the feedback window415indicates that the user “seemed nervous.”

In various implementations, generating the feedback is based on movement of the user during the performance of the presentation. For example, in various implementations, feedback is based on minimal or excessive hand movement, minimal or excessing head movement (indicating the gaze of the user, e.g., where the user is looking and/or making eye contact), or posture.

As noted above, in various implementations, the electronic device displays one or more virtual distractors. In various implementations, generating the feedback is based on the presence of virtual distractors. For example, inFIG.6G, the feedback window615indicates that the user “handled distractions well.” For example, in various implementations, when the pace of slide transitions, amount of movement of the user, or audio of the user (e.g., pace, tone, clarity) is steady in the presence of virtual distractors as compared to the absence of virtual distractors, feedback is generated that the user successfully handled the distractions.

In various implementations, the device determines a time-varying stress level based on the biometrics. In various implementations, the stress level is determined using a machine learning algorithm. For example, in various implementations, the biometrics are fed into a neural network to generate the stress level at various times during performance of the presentation. Thus, in various implementations, the feedback includes a stress level of the user.

The method800continues, in block840, with the device providing the feedback to the user. In various implementations, providing the feedback is performed during the performance of the presentation. In various implementations, providing the feedback to the user includes displaying, on the display, a feedback notification. For example, inFIG.5B, the electronic device displays the first feedback notification525A during the second rehearsal of the presentation. In various implementations, providing the feedback to the user includes altering display of the one or more slides of the presentation. As another example, inFIG.5C, the electronic device displays the second feedback notification525B and dims the slide window512during the second rehearsal of the presentation.

In various implementations, providing the feedback to the user is performed after performance of the presentation. For example, inFIG.5D, the electronic device displays the feedback window515including feedback on the performance of the presentation.

As noted above, in various implementations, the device stores previously recorded audio of the user during prior performance of the presentation. In various implementations, the device stores additional data regarding the prior performance of the presentation, such as slide transition times, eye tracking gaze information of the user, or body tracking information of the user.

In various implementations, based on the data regarding the prior performance of the presentation, the device generates an XR environment representation of the prior performance of the presentation. For example, in various implementations, the device displays an XR environment while playing the previously recorded audio of the user during prior performance of the presentation. In various implementations, the XR environment includes a screen displaying a current slide of the presentation based on previously recorded slide transition times. In various implementations, the XR environment includes a representation of the user based on previously recorded eye tracking gaze information of the user or body tracking information of the user. Thus, in various implementations, the user (or another user) can view a representation of the prior performance of the presentation from any location in the XR environment.

For example, in various implementations, the third XR environment600ofFIGS.6A-6Gincludes a virtual screen and a user at the location of the virtual chair631(or any other location in the third XR environment600) can experience a representation of the prior performance of the presentation by looking at the virtual screen, either with a representation of the presenter or without a representation of the presenter.