Abstract:
The present invention provides a system and method for combining asymmetrical camera views from a front racing and a back facing camera. Resizing and quality enhancement techniques are used to bring both front and back camera to same quality. Further, a panoramic mode of from camera and back camera are utilized to create a uniform stitching

Description:
[0001]    The present application claims priority from U.S. Provisional Application No. 61/747,769, filed Dec. 31, 2012, the entire disclosure of which is incorporated herein by reference. 
     
    
     BACKGROUND 
       [0002]    In some conventional systems for displaying a 360-degree view of a space, multiple video or image capture devices are used to record pictures or videos of the space. Those pictures or videos are then stitched together using a conventional stitching manner in order to create a 360-degree view of the space. 
         [0003]    In these conventional systems, the images or videos captured are typically of the same quality (resolution, brightness, image/video size, etc.) such that the stitching process is relatively straightforward. The stitching program will match the images or videos together, taking into consideration any slight changes in viewing height, angle, etc. and create a seamless stitched image or video. 
         [0004]    Conventional systems do not address issues that may arise when attempting to stitch together images or videos captured by devices that do not have the same quality (resolution, brightness, image/video size, etc.). 
         [0005]    There exists a need in the market to stitch images or video captured by devices with differing quality to create a seamless stitched video or image. 
       SUMMARY OF INVENTION 
       [0006]    Aspects of the present invention provide a system and method for stitching images or video captured by devices with differing quality to create a seamless stitched video or image. 
         [0007]    An embodiment of the present invention is drawn to a video device including a first video camera, a second video camera, a view synthesizer, a controller and an encoder. The first video camera is arranged to record a first video image of a first field of view and is operable to output a first stream of video data based on the first video image. The second video camera is arranged to record a second video image of a second field of view and is operable to output a second stream of video data based on the second video image. The view synthesizer is operable to perform one of generating a stitched video image stream, generating a first picture-in-picture video image stream and generating a second picture-in-picture video image stream. The generating of a stitched video image stream comprises stitching an adjusted first video image stream and an adjusted second video image stream. The adjusted first video image stream is based on the first video image stream. The adjusted second video image stream is based on the second video image stream. The stitched video image stream has a third field of view. The generating of a first picture-in-picture video image stream comprises replacing a first portion of the adjusted first video image stream corresponding to a first portion of an adjusted first video image with a second portion of the adjusted second video image stream corresponding to a second portion of an adjusted second video image. The generating of a second picture-in-picture video image stream comprises replacing a third portion of the adjusted second video image stream corresponding to a third portion of the adjusted second video image with a fourth portion of the adjusted first video image stream corresponding to a fourth portion of the adjusted first video image. The controller is operable to instruct the view synthesizer to output a raw video signal as one of the adjusted first video image stream, the adjusted second video image stream, the stitched video image stream, the first picture-in-picture video image stream and the second picture-in-picture video image stream. The encoder is operable to encode the raw video signal into an encoded signal. 
     
    
     
       BRIEF SUMMARY OF THE DRAWINGS 
         [0008]    The accompanying drawings, which are incorporated in and form a part of the specification, illustrate example embodiments and, together with the description, serve to explain the principles of the invention. In the drawings: 
           [0009]      FIG. 1  illustrates a video device in accordance with embodiments of the present invention; 
           [0010]      FIGS. 2   a - d  illustrate video recording devices capturing video of subjects in accordance with embodiments of the present invention; 
           [0011]      FIG. 3  illustrates an exploded view of the controller of the video device of  FIG. 1 ; 
           [0012]      FIGS. 4   a - b  illustrates a video display device in accordance with embodiments of the present invention; 
           [0013]    FIG. S illustrates an exploded view of the controller of the video display device of  FIG. 4 ; 
           [0014]      FIGS. 6   a - b  illustrate embodiments of various options of viewing decoded video streams in accordance with embodiments of the present invention; and 
           [0015]      FIGS. 6   c - d  illustrate additional embodiments of various options of viewing decoded video streams in accordance with embodiments of the present invention. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0016]    The present invention discloses a system and method to provide a 360-degree view of a setting. The 360-degree view is created by using two cameras to capture videos of the same setting, with the lenses of the cameras pointed 180-degrees from each other. The two views created by the cameras can then be stitched together to create a single video in which the perspective of the setting to be viewed can be chosen by the user. 
         [0017]    Stitching videos together from two cameras is a simple task when the cameras are the same or have similar recording characteristics. For example, when two cameras are used that record the video using the same resolution, angle, ambient lighting, etc., stitching the two videos together is easy because neither video needs to he modified to match the qualify of the other. While playing the stitched video, the user would be able to choose how to view the setting. The user may choose to view the setting through the first camera, the second camera, or even a picture-in-picture version of both cameras at the same time. In addition, if the user is viewing the setting on a smartphone, it may be possible to pan across the setting by tilting the smartphone to see different views of the setting. 
         [0018]    When two cameras are used that have different recording characteristics, though, the stitching process becomes more complicated. At least one of the videos must be modified to match the quality of the other; otherwise the stitched video will look distorted when the user views it. 
         [0019]    To overcome this issue, the present invention includes two aspects. First, the quality Of the videos may be adjusted during the recording and encoding portion of the process such that the quality of the videos is equivalent and provides for easier stitching. Second, the quality of the videos may be adjusted during the playing and decoding portion of the process such that the quality of the videos is equivalent and provides for easier stitching. In many cases, the quality adjustment will only need to occur during one of the recording/encoding or playing/decoding portions of the process, but there may also be instances where the quality adjustment would need to occur during both processes.  FIGS. 1-3  are drawn to the recording/encoding of the video and  FIGS. 4-6   d  are drawn to playing/decoding the video. 
         [0020]    Example embodiments of the present invention will now be described with reference to  FIGS. 1-6   d.    
         [0021]      FIG. 1  illustrates a video device in accordance with embodiments of the present invention. 
         [0022]    As shown in the FIG., video device  100  includes a video camera  102 , a video camera  104 , a quality adjuster  106 , a view synthesizer  108 , an encoder  110  and a controller  112 . 
         [0023]    Quality adjuster  106  is arranged to receive a video stream  114  from video camera  102  and to receive a video stream  116  from video camera  104 . View synthesizer  108  is arranged to receive an adjusted video stream  118  from quality adjuster  106  and to send/receive instructions  122  to/from controller  112 . Encoder  110  is arranged to receive a stitched video stream  120  from view synthesizer  108  and output an encoded video stream  124 . 
         [0024]    In this example, video camera  102 , video camera  104 , quality adjuster  106 , view synthesizer  108 , encoder  110  and controller  112  are distinct elements. However, in some embodiments, at least two of video camera  102 , video camera  104 , quality adjuster  106 , view synthesizer  108 , encoder  110  and controller  112  may be combined as a unitary device. In other embodiments, at least one video camera  102 , video camera  104 , quality adjuster  106 , view synthesizer  108 , encoder  110  and controller  112  may be implemented as a computer having stored therein tangible, non-transitory, computer-readable media for carrying or having computer-executable instructions or data structures stored thereon. Such tangible, non-transitory, computer-readable media can be any available media that can be accessed by a general purpose or special purpose computer. Non-limiting examples of tangible, non-transitory, computer-readable media include physical storage and/or memory media such as RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store desired program code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless) to a computer, the computer properly views the connection as a computer-readable medium. Thus, any such connection is properly termed a tangible, non-transitory, computer-readable medium. Combinations of the above should also be included within the scope of tangible, non-transitory, computer-readable media. 
         [0025]    Video cameras  102  and  104  record live action that occurs in front of the camera lens. Video cameras  102  and  104  may be standalone (i.e., camcorders or pocket sized video cameras), or they may be integrated into a device with multiple functions (i.e., a cellular phone or MP3 player). Video cameras  102  and  104  may be equivalent devices in that the recording capabilities are equivalent, or they may not be equivalent devices in that the recording capabilities are not equivalent (i.e., there are differences in image resolution, brightness, size, etc.). In example embodiments, the lenses used in video cameras  102  and  104  may be a type of ƒ-θ lens, which allows each video camera to capture video in a 180-degree space. 
         [0026]    Video cameras  102  and  104  produce video streams  114  and  116 , respectively, and send them to quality adjuster  106 . Video streams  114  and  116  are not altered from their native formats as recorded on video cameras  102  and  104 . Video streams  114  and  116  may be in any of the following forms: 3GP, ASF, AVI, DVR-MS, FLV, F4V, IFF, MKV, JPEG 2000, QuickTime, MPEG, MPEG-2, MP4, RM, OGG, NUT, MXF, GXF, ratDVD, SVI, VOB, DivX, or any other conventional video format that can be recorded on a video camera. 
         [0027]    Quality adjuster  106  is operable to adjust the quality of one or more incoming video streams such that, after the quality adjustment is complete, all video streams are of the same quality. Parameters that may be adjusted by quality adjuster  106  include video resolution, brightness, contrast, color, distortion, size, and any other parameter known to those of ordinary skill in the art of adjusting the quality of video streams. 
         [0028]    Quality adjuster  106  generates adjusted video stream  118 , based on video streams  114  and  116 . Quality adjuster  106  is operable to create parity between video stream  114  and video stream  116  by adjusting a parameter of at least one of video stream  114  and video stream  116 . Accordingly, all video streams included in adjusted video stream  118  will have similar resolution, brightness, contrast, color, distortion, size, and other parameters known to those of ordinary skill in the art. 
         [0029]    View synthesizer  108  is arranged to receive video stream  118 , to generate a single, stitched video stream  120  based on the user&#39;s input, and to bidirectionally communicate with controller  112 . View synthesizer  108  may include any commercially available software for stitching two or more video streams into a single video stream. Some non-limiting examples of software that may be used in view synthesizer  108  include MindTree™, SceneTracker™, and any other software known to those of ordinary skill in the art of video stitching. 
         [0030]    Stitched video stream  120  includes all versions of the stitched video streams that are created by view synthesizer  108 . Stitched video stream  120  may be in any of the forms noted above with reference to video streams  114  and  116 . 
         [0031]    Encoder  110  is operable to encode a stitched video stream such that viewing the stitched video stream is convenient for the user. Encoder  110  may encode the stitched video stream in order to standardize the file format, transmission speed, security, file size, or any other parameter known by those of ordinary skill in the art of video encoding. In an example embodiment, encoder  110  encodes in a Multiview Video Coding (MVC), which is an amendment to H.264/MPEG-4 AVC video compression standard developed with joint efforts by MPEG/VCEG that enables efficient encoding of sequences captured simultaneously from multiple cameras using a single video stream. 
         [0032]    Controller  112  is operable to bidirectionally communicate with view synthesizer  108  to send/receive instructions  122 . Controller  112  provides a user access to view synthesizer  108  in order to control the views created by view synthesizer  108 . Controller  112  will be further described with reference to  FIG. 3 . 
         [0033]    In operation, video device  100  is turned on by the user and video cameras  102  and  104  begin to record videos of different subjects. 
         [0034]      FIGS. 2   a - b  illustrate an example scenario of video recording devices capturing video of subjects in accordance with embodiments of the present invention. 
         [0035]    As shown in the figures, video of subject  202  is being captured by video camera  102 , and video of subject  204  is being captured by video camera  104 . Video camera  102  can capture video within capture window  203 , and video camera  104  can capture video within capture window  205 . 
         [0036]    With reference to  FIG. 1 , the videos recorded, adjusted for quality, synthesized and encoded may be videos of subjects  202  and  204 . As discussed above, video cameras  102  and  104  may be housed within the same device (where subjects  202  and  204  are in the same room), or they may be separate devices. In the case where video cameras  102  and  104  are separate devices, it is not necessary for subjects  202  and  204  to be in the same room. In fact, they may be very far apart, but the video streams generated from video cameras  102  and  104  may be sent (via wired or wireless manner) to quality adjuster  106  for further processing and subsequent synthesis. This type of situation may be of importance during an interview for a job, where the interviewer and interviewee may not be able to meet face to face, but instead conduct the interview via a webcam, Skype™, FaceTime™, or other communication mechanism in which the subjects can see and hear each other. In reviewing the interview, it may be beneficial to record video streams from both subjects and view the interview from the perspective of each subject. 
         [0037]      FIGS. 2   c - d  illustrate another example scenario of video recording devices capturing video of subjects in accordance with additional embodiments of the present invention. 
         [0038]    As shown in  FIG. 2   c , subjects  210  and  212  are speaking with each other, and the conversation is being recorded by video device  100 . The portion of subject  210  that is recorded by video device  100  is shown by capture window  206 . The portion of subject  212  that is recorded by video device  100  is shown by capture window  208 . 
         [0039]      FIG. 2   d  depicts an overhead view of the situation shown in  FIG. 2   c . As shown in the figure, capture window  206  encompasses the entire space 180 degrees in front of video device  100 , facing subject  210 . In addition, capture window  208  encompasses the entire space 180 degrees in front of video device  100 , facing the opposite direction toward subject  212 . In order for capture windows  206  and  208  to capture video of a 180-degree space, it may be necessary for each camera within video device  100  to employ an ƒ-θ lens, also known as a fisheye lens. 
         [0040]    For purposes of explanation, the situation depicted in  FIGS. 2   c - d  is an interview where subject  210  is interviewing subject  212  for a job opportunity. Subject  210  may want to record the interview session so it could be reviewed at another time by other individuals that may help to determine whether or not subject  212  is suitable for the position. Subject  210  would then place video device  100  on the table in between subjects  210  and  212  and begin recording. Video device  100  will record everything within capture windows  210  and  212  such that it can be viewed at a later time. 
         [0041]    Regardless of whether the subjects are in different locations, as in the scenario depicted with reference to  FIGS. 2   a - b , or in the same location, as in the scenario depicted with reference to  FIGS. 2   c - d , the video streams will be processed the same way. 
         [0042]    Returning to  FIG. 1 , as video camera  102  records, video stream  114  flows from video camera  102  to quality adjuster  106 . As video camera  104  records, video stream  116  flows from video camera  104  to quality adjuster  106 . For purposes of explanation, the resolution of video camera  102  is higher than that of video camera  104 . 
         [0043]    Quality adjuster  106  receives video streams  106  and  108  and compares the streams to determine if the quality of the streams needs to be adjusted prior to sending the streams to view synthesizer  108 . if the quality needs to be adjusted, quality adjuster  106  will adjust the quality of video streams  106  and  108 . 
         [0044]    Quality adjuster  106  compares video streams  106  and  108  to determine any differences between the two streams. In this case, quality adjuster  106  would determine that video stream  114  is a higher resolution than video stream  116 , and thus further processing is required to create a seamless view of the scene. Next, quality adjuster  106  adjusts the resolution of video stream  114 , video stream  116 , or both in order to create adjusted video stream  118  that contains the adjusted versions of video streams  106  and  108  such that the resolution of the video streams looks identical and there will not be a noticeable difference between the two streams when they are viewed by a user. 
         [0045]    The output from quality adjuster  106  is adjusted video stream  118  that flows from quality adjuster  106  to view synthesizer  108 . View synthesizer  108  receives instructions  122  from controller  112  that instructs view synthesizer  108  to generate a certain view from adjusted video stream  118 . The process of generating commands for view synthesizer  108  will be discussed in further detail with reference to  FIG. 3 . 
         [0046]    After receiving instructions  122  from controller  112 , view synthesizer  108  creates the desired stitched video stream  120  from adjusted video stream  118  by blending the video streams contained within adjusted video stream  118  together to create a seamless view of the scene being recorded. Stitched video stream  120  is then sent to encoder  110  for further processing. 
         [0047]    Encoder  110  receives stitched video stream  120  and encodes stitched video stream  120  to convert it to encoded video stream  124 . The encoding process is a driven by a codec (a device or computer program that encodes a data stream or signal for transmission), and the encoding process may modify stitched video stream  120  to compress the video data into a smaller size, change the video format, increase the security of the video transmission, etc, such that transmitting the encoded video stream  124  is efficient. Again, in an example embodiment, encoder  110  encodes in a MVC codec. 
         [0048]      FIG. 3  illustrates an exploded view of the controller of the video device of  FIG. 1 . 
         [0049]    As shown in the figure, controller  112  includes graphic user interface  302  and accelerometer  304 . 
         [0050]    Graphic user interface  302  is a way for the user to communicate with view synthesizer  108  from  FIG. 1  and inform view synthesizer  108  which view to create. Graphic user interface  302  may include a touchscreen, where the user can physically touch a screen to provide input, or it may include a screen where options may be highlighted using another system and method (e.g., a mouse, trackpad, etc.) to provide input. 
         [0051]    Accelerometer  304  is a device that communicates with view synthesizer  108  from  FIG. 1  to inform view synthesizer  108  of updates to be made within the current view being displayed. Accelerometer  304  is operable to determine the orientation of video device  100  as it is moved and rotated. 
         [0052]    In operation, a user may use graphic user interface  302  to enter a preference of how to record a certain video. That preference is relayed back to view synthesizer  108  of  FIG. 1 , and view synthesizer  108  creates the desired view as described with reference to  FIG. 1  above. In addition, accelerometer  304  may be used to enter additional preferences to be relayed back to view synthesizer  108 . These preferences include recording angle, rotating, panning, or any other preferences that may be relayed by the use of an accelerometer and the associated motion thereof. The preferences would be determined by changing the physical position or orientation of video device  100 . 
         [0053]      FIG. 4   a  illustrates a video display device in accordance with embodiments of the present invention. 
         [0054]    As shown in the figure, video display device  400  includes receiver  402 , view synthesizer  406 , display  408  and controller  410 . Receiver  402  includes decoding portions  412  and  414 . 
         [0055]    Receiver  402  is operable to receive encoded video stream  124 . Encoded video stream  124  may include multiple individual encoded video streams, and receiver  402  is operable to separate the single encoded video stream  124  into multiple encoded video streams. The number of encoded video streams generated will be equivalent to the original number of video streams recorded. In this case, encoded video stream  124  is separated in to encoded video streams  416  and  418 . Receiver  402  may be referred to as a digital media receiver (DMR), media extender, media streamer, digital media hub, digital media adapter, or any other common name by which a receiver is known by those of ordinary skill in the art. 
         [0056]    Encoded video streams  416  and  418  are video data that have been encoded from then native file formats in order to either compress the video data into a smaller size, change the video format, increase the security of the video transmission, etc, such that viewing encoded video steams  416  and  418  is simple for the user. 
         [0057]    Decoding portions  412  and  414  are arranged to receive encoded video streams  416  and  418  and decode them to create decoded video streams  420  and  422  to prepare them for display or viewing. Decoding portions  412  and  414  may include a codec, which would perform the decoding operation on the encoded video streams. In an example embodiment, decoding portions  412  and  414  are able to decode streams that have been encoded with an MVC codec. 
         [0058]    Decoded video streams  420  and  422  are sent from decoding portions  412  and  414  to view synthesizer  406 . Decoded video streams  420  and  422  may be of any of the video formats discussed above with reference to video streams  106  and  108 . 
         [0059]    View synthesizer  406  is arranged to receive decoded video streams  420  and  422  and generate a single, stitched video steam  426  based on the user&#39;s input, and to send/receive instructions  428  with controller  410 . View synthesizer  406  may include any commercially available software for stitching two or more video streams into a single video stream. Some non-limiting examples of software that may be used in view synthesizer  420  include MindTree™, SceneTracker™, and any other software known to those of ordinary skill in the art of video stitching. 
         [0060]    Display  408  may include any device capable of receiving stitched video stream  426  and showing the video, including a television, computer screen, smartphone, combination MP3 player, or any other display known by those of ordinary skill in the art that may be able to receive a video stream and show the video. 
         [0061]    Controller  410  is operable to bidirectionally communicate with view synthesizer  406  to send/receive instructions  428 . Controller  410  provides a user access to view synthesizer  406  in order to control the video output displayed to the user. Controller  410  will be further described with reference to  FIG. 5 . 
         [0062]    In this example, receiver  402 , view synthesizer  406 , display  408 , controller  410 , decoding portion  412  and decoding portion  414  are distinct elements. However, in some embodiments, at least two of receiver  402 , view synthesizer  406 , display  408 , controller  410 , decoding portion  412  and decoding portion  414  may be implemented as a computer having stored therein tangible, non-transitory, computer-readable media for carrying or having computer-executable instructions or data structures stored thereon. 
         [0063]    In an example scenario, an individual not present at the interview discussed with reference to  FIGS. 2   c - d  would like to view video of the interview. To do so, the individual would turn on video display device  400 , and receiver  402  would receive encoded video stream  124 . Receiver  402  would split encoded video stream  124  into separate encoded video streams  416  and  418 , as discussed above. In this example, encoded video stream  124  is split into two separate video streams because the original video streams originated from two separate video capture devices. If the original video streams originated from three separate video capture devices, then the encoded video stream would be split into three separate video streams, etc. 
         [0064]    Encoded video streams  416  and  418  are sent to decoding portions  412  and  414 , respectively, where video streams  416  and  418  are decoded to prepare them for viewing. As described above, the decoding operation may be accomplished by employing a codec known by those of ordinary skill in the art. 
         [0065]    After decoding, decoded video streams  420  and  422  are sent to view synthesizer  406 , where they will be stitched together and the view will be prepared according to instructions  428  received from controller  410 . 
         [0066]    After receiving instructions  428  from controller  410 , view synthesizer  406  creates the desired stitched video stream  426  from decoded video streams  420  and  422  by blending the video streams together to create a seamless view of the scene being viewed. Stitched video stream  426  is then sent to display  408 . 
         [0067]    Display  408  receives the stitched video stream and shows stitched video stream  426  such that a user can view stitched video stream  426 . 
         [0068]    With reference to  FIGS. 6   a - d , some example viewing scenarios are shown according to aspects of the present invention. 
         [0069]    As shown in  FIG. 6   a , video display device  400  includes an image of subject  206  on display  408  and toggle buttons  602  and  604 . 
         [0070]    As shown in  FIG. 6   b , video display device  400  includes an image of subject  208  on display  424  and toggle buttons  602  and  604 . 
         [0071]    In one instance, and with reference to  FIGS. 6   a - b , the user would like to see subjects  206  or  208  on the full screen of display  424 , and this preference would be entered into video display device  400  via controller  410 . 
         [0072]    Referring now to  FIG. 5 , controller  410  includes graphic user interface  502  and accelerometer  504 . 
         [0073]    Graphic user interface  502  is a way for the user to communicate with view synthesizer  406  from  FIG. 4   a  and inform view synthesizer  406  which view to create for viewing. Graphic user interface  502  may include a touchscreen, where the user can physically touch a screen to provide input, or it may include a screen where options may be highlighted using another method (e.g., a mouse, trackpad, etc.) to provide input. 
         [0074]    Accelerometer  504  is a device that communicates with view synthesizer  406  from  FIG. 4   a  to inform view synthesizer  406  of updates to be made within the current view being displayed. Accelerometer  504  is operable to determine the orientation of video display device  400  as it is moved and rotated. 
         [0075]    Referring back to  FIGS. 4   a  and  5 , the user inputs the preference to view the video of one of subjects  206  or  208  via graphic user interface  502 . Controller  410  relays the instructions  428  to view synthesizer  406 , which generates the desired view and sends the stitched video stream  426  to display  408 . 
         [0076]    Referring back to  FIGS. 6   a - b , the user is presented with an option to view video of either subject  206  or  208  via the use of toggle buttons  602  and  604 . Toggle buttons  602  and  604  allow the user to toggle the view between subject  206  and subject  208 . For instance, pressing toggle button  604  may prompt display  408  to show video of subject  208 , and pressing toggle button  602  may prompt display  408  to show video of subject  206 . In the example of the interview from  FIG. 2   c - d , the user may want to watch subject  206  ask a question and then subsequently watch subject  208  answer the question. In this case, the user may press toggle button  602  to show the video of subject  206  asking the question on display  408 , and then press toggle button  604  after the question is asked so that the video of subject  208  answering the question is shown on display  408 . 
         [0077]    Referring again to  FIGS. 4   a  and  5 , while the user is viewing either subject  206  or  208  on display  408 , it may be desirable to view other parts of the room. For example, the viewer may want to see if subject  208  is fidgeting with his hands while answering a question, which may indicate that he is nervous or potentially fabricating his response. In this case, the user would simply tilt video display device  400  down, which would trigger accelerometer  504  to notify view synthesizer  406  that the desired view has changed and thus needs to be modified. View synthesizer would then generate the updated view as desired and send the new stitched video stream  426  to display  408 . The user would then see the hands of subject  208  to determine whether or not he was fidgeting while answering a question. 
         [0078]    In another instance, and with reference to  FIGS. 6   c - d , the user would like to see subjects  206  and  208  on display  408  at the same time via picture-in-picture. 
         [0079]    As shown in  FIG. 6   c , video display device  400  includes images of subjects  206  and  208  on display  408 , and toggle buttons  606  and  608 . 
         [0080]    As shown in  FIG. 6   d , video display device  400  includes images of subjects  206  and  208  on display  424 , and toggle buttons  606  and  608 . 
         [0081]    Referring back to  FIGS. 4   a  and  5 , the user inputs the preference to view picture-in-picture videos of subjects  206  and  208  via graphic user interface  502 . Controller  410  relays instructions  428  to view synthesizer  406 , which generates the desired view and sends the stitched video stream  426  to display  408 . 
         [0082]    Referring back to  FIG. 6   c - d , the user is presented with an option to view a picture-in-picture video of either subject  206  or  208  as the larger picture via the use of toggle buttons  606  and  608 . Toggle buttons  606  and  608  allow the user to toggle the views to show subject  206  or subject  208  as the larger video. For instance, pressing toggle button  606  may prompt display  408  to show a large video of subject  208  and a smaller video of subject  206  within the large video, and pressing toggle button  608  may prompt display  408  to show a large video of subject  206  and a smaller video of subject  208  within the large video, in the example of the interview from  FIG. 2   c - d , the user may want to watch both subjects  206  and  208  during the interview, but may want to focus on different subjects during different times. For instance, the user may want to focus on subject  206  asking a question and then subsequently focus on subject  208  answering the question. In this case, the user may press toggle button  608  to show the larger video of subject  206  asking the question with the smaller video of subject  208  within the large video on display  408 , and then press toggle button  606  after the question is asked so that the video of subject  208  answering the question becomes the large video shown on display  408 , with the video of subject  206  becoming the smaller video within the large video. 
         [0083]    Referring again to  FIGS. 4   a  and  5 , while the user is viewing either of the views from  FIGS. 6   c - d  on display  408 , it may be desirable to view other parts of the room. For example, the viewer may want to see if subject  208  is fidgeting with his hands while answering a question, which may indicate that he is nervous or potentially fabricating his response. In this case, the user would simply tilt video display device  400  down, which would trigger accelerometer  504  to notify view synthesizer  406  that the desired view has changed and thus needs to he modified. View synthesizer would then generate fee updated view as desired and send the new stitched video stream  426  to display  408 . The user would then see the hands of subject  208  to determine whether or not he was fidgeting while answering a question. 
         [0084]    With the picture-in-picture views discussed in  FIGS. 6   c - d , and in one embodiment of tilting video display device  400  to alter the view on display  424 , video display device  400  may only alter the view of the larger video in response to video display device  400  being tilted. In another embodiment of tilting video display device  400  to alter the view on display  424 , video display device  400  may only alter the view of the smaller video in response to video display device  400  being tilted. In yet another embodiment of tilting video display device  400  to alter the view on display  424 , video display device  400  may alter both the larger and smaller videos in response to video display device  400  being tilted. 
         [0085]      FIG. 4   b  illustrates an alternate embodiment of a video display device in accordance with aspects of the present invention. 
         [0086]    As shown in the figure, video display device  450  includes all elements included within video display device  400 , with the addition of quality adjuster  404 . 
         [0087]    Quality adjuster  404  is operable to adjust the quality of one or more decoded incoming video streams such that, after the quality adjustment is complete, all decoded video streams are of the same quality. Parameters that may be adjusted by quality adjuster  404  include video resolution, brightness, contrast, color, distortion, size, and any other parameter known to those of ordinary skill in the art of adjusting the quality of video streams. 
         [0088]    Quality adjuster  404  is arranged to receive decoded video streams  420  and  422  and adjust the quality of the streams and output adjusted video stream  424  that is sent to view synthesizer  406 . 
         [0089]    In operation, encoded video stream  124  would be received and decoded in a manner equivalent to that described with reference to  FIG. 4 . In this case, however, the quality adjuster is present to verify that the quality of the video streams has not been degraded by the encoding/decoding processes. 
         [0090]    Quality adjuster  404  receives decoded video streams  420  and  422  and compares the streams to determine if the quality of the streams needs to be adjusted prior to sending the streams to view synthesizer  406 . The quality of the streams may have been affected by the encoding/decoding process, and quality adjuster  404  checks to see if any such losses have occurred. If the quality needs to be adjusted, quality adjuster  404  will adjust the quality of video streams  420  and  422 . 
         [0091]    Quality adjuster  404  compares decoded video streams  420  and  422  to determine any differences between the two streams. In this ease, and as a non-limiting example, quality adjuster  404  may determine that decoded video stream  420  is a higher resolution than decoded video stream  422 , and thus further processing is required to create a seamless view of the scene. Next, qualify adjuster  404  adjusts the resolution of decoded video stream  420 , decoded video stream  422 , or both in order to create adjusted video stream  424  that contains the adjusted versions of decoded video streams  420  and  422  such that the resolution of the Video streams looks identical and there will not be a noticeable difference between the two streams when they are viewed by a user. 
         [0092]    The output from quality adjuster  404  is adjusted video stream  424  that flows from quality adjuster  404  to view synthesizer  406 . View synthesizer  406  receives instructions  428  from controller  410  that instructs view synthesizer  406  to generate a certain view from adjusted video stream  424 . The process of generating commands for view synthesizer  406  has been previously described with reference to  FIGS. 6   a - d.    
         [0093]    Quality adjuster  404  may only be necessary within video display device  450  if there is no quality adjustment when the videos are recorded by one or more video devices, however it may be desirable to have an additional quality adjuster to assure that the video streams are not corrupted during the encoding/decoding process. The options for including a quality adjuster include: quality adjusters included on both video recording and display devices, a quality adjuster included only on the video recording device (as shown in  FIG. 1 ) or a quality adjuster included only on the video display device. 
         [0094]    The foregoing description of various preferred embodiments have been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed, and obviously many modifications and variations are possible in light of the above teaching. The example embodiments, as described above, were chosen and described in order to best explain the principles of the invention and its practical application to thereby enable others skilled in the art to best utilize the invention in various embodiments and with various modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the claims appended hereto.