Abstract:
A virtual tour host server decomposes a virtual tour into a plurality of video faces which are stored. When a user desires to begin a virtual tour using a mobile device, the mobile device detects and reports a field of view to the host server. The host server then associates a subset of the plurality of video faces corresponding to the detected field of view, and sends a video stream of the virtual tour recomposed from the subset of video faces, enabling the mobile device to display the virtual tour video stream.

Description:
CROSS REFERENCE TO RELATED APPLICATION 
       [0001]    This non-provisional application claims the benefit of provisional application No. 61/590,682 filed on Jan. 25, 2012, entitled “Systems and Methods for Panoramic Video Streaming of Virtual Tours”, which application is incorporated herein in its entirety by this reference. 
     
    
     BACKGROUND 
       [0002]    The present invention relates to systems and methods for streaming of virtual tours. More particularly, the present invention relates to efficient streaming of panoramic virtual tours from a host to a mobile device. 
         [0003]    Conventionally, panoramic video is converted to a series of, for example, six cube-face video files which can be displayed as adjacent surfaces within a virtual cube. Users typically select their viewing angle within this cubic environment, thus determining which of the cube-face video files are displayed within their field of view. Given that real-time panoramic video requires any portion of these six cube-faces be available (including that which is not currently being viewed), conventional methods of panoramic video streaming typically require inefficiently high amounts of data be streamed and buffered so that it can be delivered upon request. 
         [0004]    It is therefore apparent that an urgent need exists for highly efficient systems and methods of streaming panoramic video which fulfills viewer expectations for real-time applications such as virtual tours. 
       SUMMARY 
       [0005]    To achieve the foregoing and in accordance with the present invention, systems and methods for streaming virtual tours are provided. In particular, the systems and methods efficiently stream panoramic virtual tours from a host server to a mobile device. 
         [0006]    In one embodiment, a host server decomposes a virtual tour into a plurality of video faces which are stored in the host server. When a user desires to begin a virtual tour using a mobile device, a virtual tour viewer is launched on the mobile device. 
         [0007]    The mobile device detects the user&#39;s field of view on the mobile device and reports the field of view to the virtual tour host server. The host server then associates a subset of the plurality of video faces which corresponds to the detected field of view of the mobile device, and sends a video stream of the virtual tour recomposed from the subset of the plurality of video faces. The mobile device then seamlessly displays the video stream using the virtual tour viewer executing in the mobile device. 
         [0008]    In some embodiments, the mobile device also sends the user&#39;s navigational intentions to the host server, enabling the host server to anticipate the next video face likely to be needed by the mobile device. This additional video face can then be streamed to and buffered by the mobile device. 
         [0009]    Note that the various features of the present invention described above may be practiced alone or in combination. These and other features of the present invention will be described in more detail below in the detailed description of the invention and in conjunction with the following figures. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0010]    In order that the present invention may be more clearly ascertained, some embodiments will now be described, by way of example, with reference to the accompanying drawings, in which: 
           [0011]      FIG. 1  is a perspective view of a mobile device capable of determining motion change(s) of the mobile device caused by a user for navigating virtual tours in accordance with one embodiment of the present invention; 
           [0012]      FIGS. 2A and 2B  illustrate six quadrilateral video faces of a cubic panoramic video environment for the embodiment of  FIG. 1 ; 
           [0013]      FIGS. 3A and 3B  illustrate exemplary multi quadrilateral video faces of a cylindrical panoramic video environment for the embodiment of  FIG. 1 ; 
           [0014]      FIGS. 4A and 4B  illustrate exemplary multi-faceted polygonal video faces of a spherical panoramic video environment for the embodiment of  FIG. 1 ; and 
           [0015]      FIG. 5  is a flow diagram illustrating the decomposition, recomposition and streaming of panoramic videos for the embodiment of  FIG. 1 . 
       
    
    
     DETAILED DESCRIPTION 
       [0016]    The present invention will now be described in detail with reference to several embodiments thereof as illustrated in the accompanying drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the present invention. It will be apparent, however, to one skilled in the art, that embodiments may be practiced without some or all of these specific details. In other instances, well known process steps and/or structures have not been described in detail in order to not unnecessarily obscure the present invention. The features and advantages of embodiments may be better understood with reference to the drawings and discussions that follow. 
         [0017]    The present invention relates to systems and methods for efficiently decomposing, storing, recomposing and streaming of panoramic videos for virtual tours (“VT”) executing on mobile devices. Note that the term “mobile device” is intended to include all portable electronic devices including cellular phones, computerized tablets, cameras, and hand-held gaming devices. 
         [0018]    To facilitate discussion,  FIG. 1  illustrates an exemplary mobile device capable of determining motion change(s) of the mobile device  100  caused by a user for navigating virtual tours in accordance with one embodiment of the present invention.  FIGS. 2A and 2B  illustrate a six-sided cubic panoramic video environment for mobile device  100 .  FIGS. 3A and 3B  illustrate another exemplary multi-sided cylindrical panoramic video environment for mobile device  100 , while  FIGS. 4A and 4B  illustrate yet another exemplary multi-faceted spherical panoramic video environment for mobile device  100 . 
         [0019]    As illustrated by  FIGS. 2A ,  2 B and the exemplary flow diagram of  FIG. 5 , in step  510 , the panoramic VT is decomposed into six quadrilateral video faces of a cubic panoramic video environment, e.g., video-faces  210 ,  220 ,  230 ,  240 , and stored in a VT host server (not shown). These six video faces are now available to be streamed on-demand from the host server to a mobile device  100  via, for example, a wide area network such as the Internet. 
         [0020]    When a user, e.g., a VT visitor, desires to commence a virtual tour, a VT viewer is launched on mobile device  100  (step  520 ). In order to provide a realistic real-time viewing experience for the visitor of the virtual tour, mobile device  100  provides the current field-of-view (FOV) to the VT server (step  530 ). 
         [0021]    In this embodiment, as shown in  FIG. 2B , the FOV of mobile device  100  can include up to three of the six video faces, e.g., video faces  210 ,  220 ,  230 . Accordingly, the VT server associates the FOV with a corresponding subset of video face(s) comprising up to three of the required video faces (step  540 ). This cubic video environment is well-suited for VTs of “surround-locations” such as amusement park rides, scuba diving spots and action movie sets. The VT server re-composed a panoramic VT derived from this selected subset of up to three video face(s) and efficiently streams the VT to the mobile device  100  for display (steps  550 ,  560 ). 
         [0022]    Similarly, as exemplified by  FIGS. 3A ,  3 B and the flow diagram by  FIG. 5 , in step  510 , the panoramic VT is decomposed into multiple quadrilateral video-faces of a cylindrical panoramic video environment, e.g., eight video faces  310 ,  320 ,  330 ,  340 ,  350 ,  360 ,  370 ,  380 , and stored in a VT host server (not shown). These eight video faces are now available to be streamed on-demand from the host server to mobile device  100 . 
         [0023]    The VT visitor commences a virtual tour by launching a VT viewer on mobile device  100  and provides the current field-of-view (FOV) to the VT server (steps  520 ,  530 ). As shown in  FIG. 3B , the FOV of mobile device  100  can include up to three of the eight video faces, e.g., faces  380 ,  310 ,  320 . Accordingly, the VT server associates the FOV with a current subset of video face(s) comprising up to three of the required video-faces (step  540 ). The VT server re-composed a panoramic VT derived from this selected subset of up to three video face(s) and efficiently streams the VT to the mobile device  100  for display (steps  550 ,  560 ). 
         [0024]    Referring now to  FIGS. 4A ,  4 B and the flow diagram by  FIG. 5 , in step  510 , the panoramic VT is decomposed into multiple polygonal video faces of a multi-faceted spherical panoramic video environment, e.g., hexagonal video faces  410 ,  420 ,  430 ,  440 ,  450 , and stored in a VT host server (not shown). In this example, video-faces  410 ,  420  . . .  450  are similar to the polygonal, e.g., hexagonal and/or pentagonal, leather panels for constructing a soccer ball or volleyball. This spherical video environment is also suited for VTs of “surround-locations” such as amusement park rides, scuba diving spots and action movie sets. These video faces are now available to be streamed on-demand from the host server to mobile device  100 . 
         [0025]    The VT visitor commences a virtual tour by launching a VT viewer on mobile device  100  and provides the current field-of-view (FOV) to the VT server (steps  520 ,  530 ). As exemplified by  FIG. 4B , the FOV of mobile device  100  includes the three video faces  410 ,  420 ,  430 . Accordingly, the VT server associates the FOV with a current subset of video face(s) comprising up to three of the required video faces (step  540 ). The VT server re-composed a panoramic VT derived from this selected subset of up to three video face(s) and efficiently streams the VT to the mobile device  100  for display (steps  550 ,  560 ). 
         [0026]    It is contemplated that the VT viewer of the mobile device  100  may be capable of zooming in and out thereby contracting and expanding the field-of-view experienced by the VT visitor. Accordingly, depending on the zoom level selected, the field-of-view may include up to “N” video faces, wherein “N” can be any integer from one and up to the total number of video faces minus one; provided the total number of video faces is equal to or greater than four. 
         [0027]    Many modifications and additions to the embodiments of the present invention are also possible. For example, mobile device  100  may also provide the user&#39;s VT navigational intentions to the VT server, enabling the VT server to anticipate which video face(s) may be needed soon, and thus enabling the VT server to begin streaming one or more additional video-face(s) to be buffered by the mobile device  100 , thereby reducing jitter caused by transitional delays. 
         [0028]    It should also be appreciated that the decomposition  510  and recomposition  550  of virtual tours into video faces can be implemented at the single level described above, or as nested levels. For example, video face  210  may be further decomposed into two or more sub-faces (not shown). Accordingly, the anticipatory streaming of additional video face(s) may optionally be implemented at the sub-face(s) level. 
         [0029]    In sum, the present invention provides systems and methods for efficiently decomposing, storing, recomposing and streaming of panoramic videos for virtual tours executing on mobile devices. The advantages of such systems and methods include lower bandwidth requirements, higher resolution for the same bandwidth, a superior virtual tour visitor experience, lower power consumption by the mobile devices, lower memory requirement for the mobile devices 
         [0030]    While this invention has been described in terms of several embodiments, there are alterations, modifications, permutations, and substitute equivalents, which fall within the scope of this invention. It should also be noted that there are many alternative ways of implementing the methods and apparatuses of the present invention. It is therefore intended that the following appended claims be interpreted as including all such alterations, modifications, permutations, and substitute equivalents as fall within the true spirit and scope of the present invention.