System and method for controlling presentations and videoconferences using hand motions

A system and method are disclosed for controlling presentations and videoconference using hand motions. A camera captures video of an area relative to content displayed on a display device from a content source. A control unit is communicatively coupled to the content source, the display device, and the camera. The control unit receives captured video from the camera. The control unit detects a hand motion by a presenter that occurs within the captured video and determines the location within the captured video of at least one control for controlling the presentation or videoconference. The control unit determines if the detected hand motion occurs within the determined location of the at least one control, and the control unit controls the content source based on the determined control.

FIELD OF THE DISCLOSURE

The subject matter of the present disclosure relates to a system and method for controlling presentations using hand or other physical motions by the presenter relative to the displayed presentation content.

BACKGROUND OF THE DISCLOSURE

Speakers often use content, such as PowerPoint slides, Excel spreadsheets, etc., during a presentation or videoconference. Often, the speakers must control the content themselves or have a second person control the content for them during the presentation or videoconference. These ways of controlling content can cause distractions. For example, having to call out instructions to another person to flip the slides of a presentation forward or backward can be distracting or not understood. During a presentation, for example, the audience may ask questions that often require jumping to random slides or pages. If a second person is controlling the content, the speaker has to relay instructions to the second person to move to the correct slide.

SUMMARY OF THE DISCLOSURE

A system and method are disclosed for controlling presentations and videoconference using hand motions. In one embodiment, the system includes a content source, a display, a camera, and a control unit. The content source can be a computer, a videoconferencing system, a video camera, or other device that provides content. The content can be moving video, images, presentation slides, spreadsheets, live computer screen shots, or other displayable subject matter. The camera captures video of an area relative to the content being displayed on the display device from the content source. The control unit is communicatively coupled to the content source, the display device, and the camera. The control unit receives captured video from the camera. The control unit detects a hand motion by a presenter that occurs within the captured video and determines the location within the captured video of at least one control for controlling the presentation or videoconference. The control unit determines if the detected hand motion has occurred within the determined location of the control and controls the content source based on the control triggered by the hand motion.

The at least one control can be a small icon included in the displayed content. In this way, the system allows natural hand motions to control the content of a presentation or videoconference by providing the small icon in the displayed content. To change content or control aspects of the presentation or videoconference, the speaker or presenter needs only to move a hand relative to the icon so that the camera captures the hand motion and the control unit detects that the control of the icon has been selected.

The control icons can be implemented as an overlay on top of the content video, or the control icons can be included as part of the content in the form of an image incorporated into a slide presentation. In another alternative, the control icons can be a physical image placed on the wall behind the presenter or speaker in the view angle of the camera.

The camera is used to capture motions of the speaker regardless of which of the above type of icon is used. A particular icon is activated when motion vectors in the captured video reach a predetermined threshold in the area or location of the icon. To place icons within the content stream, the content is preferably displayed as a background image using a chroma key technique, and an image pattern matching algorithm is preferably used to find the placement of the icon. If the icon is overlaid on top of the camera video after the camera has captured the video of the speaker, then the placement or location of the icon will be already known in advance so that the control unit will not need to perform an image pattern matching algorithm to locate the icon.

In one benefit of the system, speakers or presenters using the system can naturally control a presentation or videoconference without requiring a second person to change presentation slides, change content, or perform any other various types of control.

While the subject matter of the present disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawings and are herein described in detail. The figures and written description are not intended to limit the scope of the inventive concepts in any manner. Rather, the figures and written description are provided to illustrate the inventive concepts to a person skilled in the art by reference to particular embodiments, as required by 35 U.S.C. §112.

DETAILED DESCRIPTION

Referring toFIG. 1, an embodiment of a presentation system10according to certain teachings of the present disclosure is illustrated. The presentation system10includes a control unit12, a camera14, and one or more content devices16and18. In the present embodiment, the control unit12is shown as a computer, and the camera14is shown as a separate video camera. In an alternative embodiment, the control unit12and the camera14can be incorporated into a single videoconferencing unit. In addition, the present embodiment shows the content devices as a projector16and screen18. In alternative embodiments, the one or more content devices can include a television screen or a display coupled to a videoconferencing unit, a computer, or the like.

The presentation system10allows the presenter to use physical motions or movements to control the presentation and the content. As described below, the presenter can use hand motions relative to a video applet, displayed icon, or area to control the playing of video, to change slides in a presentation, and to perform other related tasks associated with a presentation. For example, the control unit12includes presentation software for presenting content, such as a PowerPoint® presentation. The control unit12provides the content to the projector16, which then projects the content on the screen18. In one embodiment, one or more video applets or visual icons are overlaid on the content presented on the screen. As the presenter conducts the presentation, the camera14captures video of motion made relative to the displayed icon on the screen18. This captured video is provided to the control unit12. In turn, the control unit12determines from the captured video whether the presenter has made a selection of a control on the displayed icon. If so, the control unit12controls the presentation of the content by performing the control selected by the presenter. In general, the video applets or visual icons can be placed as visual elements over captured video, can be placed as a physical object that is then captured in video, or can be incorporated into a content stream, such as being a visual button in Power point slide.

As noted above, one or more visual icons can overlay content being presented. InFIG. 2A, an example of a visual icon30is shown overlaying content20displayed on the screen18. In one implementation, the icon30is incorporated into the presentation content. For example, the icon30can be added as a graphical element to a slide of a PowerPoint presentation. In another implementation, the icon30can be overlaid or transposed onto the content of the presentation. Either way, the camera (14;FIG. 1) is directed at the screen18or at least at the area of the icon30. During the presentation, the camera (14) captures video of the area of the icon30in the event that the presenter makes any motions or movements over the icon30that would initiate a control.

In another example,FIG. 2Bshows a physical icon32placed adjacent the content20being displayed on the screen18. For example, the physical icon32can be a plaque or card positioned on a wall next to the screen18. The camera (14;FIG. 1) directed at the icon32captures video of the area of the icon32in the event that the presenter makes a motion over one of the controls of the icon32.

Referring toFIG. 3, another embodiment of a presentation system50according to certain teachings of the present disclosure is illustrated. In this embodiment, the presentation system50includes a videoconferencing unit52having an integral camera54. The videoconferencing unit52is connected to a video display or television56. The videoconferencing unit52is also connected to a network for videoconferencing using techniques known to those skilled in the art. The display56shows content60of a videoconference. In the present embodiment, the content60includes presentation material62, such as presentation slides, video from the connected camera54, video from a remote camera of another videoconferencing unit, video from a separate document camera, video from a computer, etc. The content60also includes video of a presenter64superimposed over the presentation material62. In addition, an icon34is shown in the content60on the display56.

As discussed above, there are several ways to include the icon34into the presentation system50. The icon34can be incorporated as a visual element into the presentation material62, whereby the incorporated icon34is presented on the display56as part of the presentation material62. Alternatively, the icon34can be a visual element generated by the videoconferencing unit52, connected computer, or the like and superimposed on the video of the presentation material62and/or the video of the presenter64. In yet another alternative, the icon34can be a physical object having video of it captured by the camera54in conjunction with the video of the presenter64and superimposed over the presentation material62.

Again, the presentation system50allows the presenter64to use physical motions or movements to control the presentation and the content60. For example, the presenter64, who is able to view herself superimposed on presentation material62on the display56, can use hand motions relative to the displayed icon34to control the playing of video, to change slides in a presentation, and to perform other related tasks associated with a presentation.

As discussed above, the icon34can be incorporated as a visual element in the presentation material62shown on the display56. For example, the icon34can be visual buttons added to slides of a PowerPoint presentation. Because the icon34is incorporated into the presentation material62, the icon34will likely have a fixed or know location. The camera54captures video of the presenter64who in turn is able to see her own hand superimposed on the presentation materials62when she makes a hand motion within the area of the incorporated icon34. The video from the camera54is analyzed to detect if a hand motion occurs within the known or fixed location of the icon34. For example, the analysis determines motion vectors that occur within the video stream of the camera54and determine if those motion vectors exceed some predetermined threshold within an area of the icon34. If the hand motion is detected, then the videoconferencing unit50determines what control has been invoked by the hand motion and configures an appropriate command, such as instructing to move to the next slide in a PowerPoint presentation, etc.

As discussed above, the icon34can be a visual element added to the video of the presenter64captured by the camera54. The added icon34is shown on the display56along with the video of the presenter64. Therefore, the presenter64is able to see her own hand when she makes a motion relative to the added icon34. The video from the camera54is analyzed to detect if a hand motion occurs within the known or fixed location of the added icon34, and the videoconferencing unit50determines which control has been invoked by the hand motion.

As discussed above, the icon34can be a physical element placed next to the presenter64(e.g., located on the wall behind the presenter64). The location of the physically placed icon64can be determined from the video captured by the camera54. The presenter64can make a hand motion relative to the physically placed icon34, and the camera54can capture the video of the presenter's hand relative to the icon34. The captured video can then be analyzed to detect if a hand motion occurs within the area of the icon34, and the videoconferencing unit50can determine which control has been invoke by the hand motion.

In the embodiments ofFIGS. 2A-2Band3, the icons30,32, and34can have any of a number of potential controls for controlling a presentation. Each control can be displayed as a part of a separate area of the icons30,32, and34so that the presenter can move her hand or other object in the separate area to implement the desired control. For example, changing to the next slide in a PowerPoint presentation can simply require that the presenter move her hand over a graphical element of the icons30,32, and34corresponding to advancing to the next slide. Which controls are used on the icons30,32, and34as well as their size and placement can be user-defined and can depend on the particular implementation. In addition to controlling a presentation (e.g., moving to next slide, moving back a slide, etc.), embodiments of the disclosed system100can be used to control a mouse pointer in a desktop environment, to control camera movements of a videoconference, to control volume, contrast, brightness levels, and to control other aspects of a presentation or videoconference with hand motions.

Given the above description, we now turn to a more detailed discussion of a presentation system according to certain teachings of the present disclosure. Referring toFIG. 4, an embodiment of a presentation system100according to certain teachings of the present disclosure is schematically illustrated. In the discussion that follows, some components of the presentation system100are discussed in terms of modules. It will be appreciated that these modules can be implemented as hardware, firmware, software, and any combination thereof. In addition, it will be appreciated that the components of the presentation system100can be incorporated into a single device, such as a videoconferencing unit or a control unit, or can be implemented across a plurality of separate devices coupled together, such as a computer, camera, and projector.

To capture video images relative to an icon, the presentation system100includes a camera110and a video capture module120. To handle content, the presentation system100includes a content source140and a content capture module150. To handle controls, the presentation system100includes an icon motion trigger module170and a content control module180. Depending on how the icon is superimposed, incorporated, or added, the presentation system100uses either an icon location detection module160or an icon overlay module190.

During operation, the camera110captures video and provides a video feed112to the video capture module120. For videoconferencing, the camera110is typically directed at the presenter. In one embodiment, the icon (not shown) to be used by the presenter to control the presentation can be overlaid on or added to the video captured by the camera110. Accordingly, the location of the icon and its various controls can be known, fixed, or readily determined by the system100. In this embodiment, the video capture module120provides camera video via a path129to the icon overlay module190. At the icon overlay module190, the icon is overlaid on or added to video that is provided to the preview display192. In this way, the presenter can see herself on the preview display192and can see the location of her hand relative to the icon that has been added to the original video from the camera110. Because the location of the added icon is known or fixed, the icon overlay module190provides a static location197of the icon to the icon motion trigger module170that performs operation discussed later.

In another embodiment, the icon may not be overlaid on or added to the video from the camera110. Instead, the icon may be a physical element placed at a random location within the field of view of the camera110. In this embodiment, the location of the icon and its various controls must first be determined by the system100. In this case, the video capture module120sends video to the icon location detection module160. In turn, this module160determines the dynamic icon location. For example, the icon location detection module160can use an image pattern-matching algorithm known in the art to find the location of the icon and its various controls in the video from the camera110. For example, the image pattern-matching algorithm can compare expected pattern or patterns of the icon and controls to portions of the video content captured with the camera110to determine matches. Once the location of the icon and its controls are determined, the module160provides the location162to the icon motion trigger module170.

In another embodiment, the icon may be incorporated as a visual element in the content from the content source140. For example, the icon may be a tool bar added to screens or slides of a presentation from the content source140. In this embodiment, the content capture module150receives a content video feed from the content source140and sends captured content video to the icon location detection module160. One embodiment of the disclosed system100can uses a chroma key technique and pattern-matching to detect the location of the icon. Because the icon is incorporated as a visual element within the content stream, the content can be displayed as a background image using a chroma key technique. The background image of the content can then be sampled, and the video pixels from the camera110that fall within the chroma range of the background pixels are placed in a background map. The edges can then be filtered to reduce edge effects. The icon location detection module160can then use an image pattern-matching algorithm to determine the location of the icon and the various controls in the content stream. Once determined, the module160provides the location162to the icon motion trigger module170. Other algorithms known in the art can be used that can provide better chroma key edges and can reduce noise, but one skilled in the art will appreciate that computing costs must be considered for a particular implementation.

While the static or dynamic location of the icon is determined as discussed above, the video capture module120also provides video information to the motion estimation and threshold module130. This module130determines vectors or values of motion (“motion vector data”) occurring within the provided video content from the camera110and provides motion vector data to the trigger module170. To determine motion vector data, the motion estimation and threshold module130can use algorithms known in the art for detecting motion within video. For example, the algorithm may be used to place boundaries around the determined icon or screen location and to then identify motion occurring within that boundary.

In one embodiment, the module130can determine motion vector data for the entire field of the video obtained by the video capture module120. In this way, the motion estimation and threshold module130can ignore anomalies in the motion occurring in the captured video. For example, the module130could ignore data obtained when a substantial portion of the entire field has motion (e.g., when someone passes by the camera110during a presentation). In such a situation, it is preferred that the motion occurring in the captured video not trigger any of the controls of the icon even though motion has been detected in the area of the icon.

In alternative embodiments, the motion estimation and threshold module130can determine motion vector data for only predetermined portions of the video obtained by the video capture module120. For example, the module130can focus on calculating motion vector data in only a predetermined quadrant of the video field where the icon would preferably be located. Such a focused analysis by the module130can be made initially or can even be made after first determining data over the entire field in order to detect any chance of an anomaly as discussed above.

Continuing with the discussion, the trigger module170has received information on the location of the icon—either the static location197from the icon overlay module190or the dynamic location162from the icon location detection module160. In addition, the trigger module170has received information on the motion vector data from the motion estimation and threshold module130. Using the received information, the trigger module170determines whether the presenter has selected a particular control of the icon. For example, the trigger module170determines if the motion vector data within areas of the controls in the icon meet or exceed a threshold. When a control is triggered, the trigger module170sends icon trigger information178to a content control module180. In turn, the content control module180sends control commands to the content source140via a communications channel184.

The foregoing description of preferred and other embodiments is not intended to limit or restrict the scope or applicability of the inventive concepts conceived of by the Applicants. For example, the embodiment of the presentation system100ofFIG. 4has been described as having both an icon overlay module190and an icon location detection module160. It will be appreciated that the presentation system100can include only one or the other of these modules160and190as well as including both. In another example, embodiments of the presentation system50ofFIG. 3 and 100ofFIG. 4have been described in the context of videoconferencing. However, with the benefit of the present disclosure, it will be appreciated that the disclosed system and associated methods can be used in other implementations, such as PowerPoint presentations, closed circuit video presentations, video games, etc. Moreover, a content source for the disclosed system can be a computer, a videoconferencing system, a video camera, or other device that provides content. The content for the disclosed system can be moving video, still images, presentation slides, live views of a computer screen, or any other displayable subject matter.