Abstract:
A video object detecting apparatus enables a user to not only simply watch the video but also to explore a video using a motion and a posture of the user. In addition, the user may manipulate the video using body motion instead of general input units such as a mouse and a touch system. Also, a video object deforming apparatus enables the user to recognize a difference in motions between an interactive video and the user, and provides the user with a video reflecting the difference.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
       [0001]    This application claims the priority benefit of Korean Patent Application No. 10-2011-0037877, filed on Apr. 22, 2011, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference. 
       BACKGROUND 
       [0002]    1. Field 
         [0003]    Example embodiments of the following description relate to a video object detecting apparatus and a video object deforming apparatus, and more particularly, to apparatuses to detect and deform a video object of an interactive video. 
         [0004]    2. Description of the Related Art 
         [0005]    Research and development related to technology for controlling a user interface (UI) by sensing a body motion of a user is rapidly increasing. However, in general, current motion sensing technologies are limited to a graphic-based game input device or UI operator substituting a remote controller of electronic devices. 
         [0006]    In 2007, the worldwide movie industry market measured 27,403 million dollars in terms of theater sales, 55,837 million dollars in terms of home-video sales, and 2,664 million dollars in terms of online movie sales, for a combined total sales of 85,904 million dollars. By location, the sales measured 33,717 million dollars in the U.S.A. and 22,238 million dollars in Western Europe. 
         [0007]    The movie industry market, in terms of sales, was almost equivalent to the worldwide game market in 2007 which had total sales of 86,418 million dollars, including 35,837 million dollars in terms of arcade games, 3,042 million dollars in terms of PC games, 37,415 million dollars in terms of console games, 7,155 million dollars in terms of online games; and 2,969 million dollars in terms of mobile games. In consideration of this, it is expected that UI technology based on body motion will be actively used in controlling an interactive video as well as the graphic-based game input device. In the event that the application field expands to the music video and music broadcasting market or the health video market, the value of an interactive video control technology will further increase. 
         [0008]    Since the graphic-based technology that represents a 2-dimensional/3-dimensional (2D/3D) object by a real-time rendering engine is convenient to produce interactive content using a new modality, such as motion sensing, motion recognition technology has been rapidly spread. However, as to video contents currently used which are implemented as scenes photographed by a 2D/3D camera and sequentially played back through post-processing such as adjusting or editing, for example, an innovative scheme for producing and appreciating an interactive video is needed to utilize video contents having modality such as a natural motion of a user. 
         [0009]    Accordingly, the following embodiments will describe a new video object detecting and deforming apparatus for utilizing an interactive video. 
       SUMMARY 
       [0010]    The example embodiments may include a video object detecting apparatus that may include an object retrieval unit to retrieve a sensing object from a sensing image related to a user, and also retrieve a video object from an interactive video, a calculation unit to calculate similarity between the sensing object and the video object, and a frame extraction unit to extract a frame having a highest similarity from the interactive video. 
         [0011]    The video object detecting apparatus may include an object retrieval unit to retrieve a video object input by a user from an interactive video, a tracking unit to track a position of a control point for controlling the video object, a calculation unit to calculate a difference of positions between a moment of the video object and the control point, based on an identification (ID) of the video object and the position of the control point, and a frame extraction unit to extract a frame, having a lowest similarity, from the interactive video. 
         [0012]    The example embodiments may also include a video object deforming apparatus that may include an object retrieval unit to retrieve a sensing object from a sensing image related to a user, and also retrieve a video object from an interactive video, a calculation unit to calculate a difference between an object shape related to the sensing object and a depth image related to the video object, and a generation unit to generate a difference map based on the calculated difference. 
         [0013]    The example embodiments may also include a video object detecting method that may include retrieving a sensing object from a sensing image related to a user and also retrieving a video object from an interactive video, calculating similarity between the sensing object and the video object, and extracting a frame, having a lowest similarity, from the interactive video. 
         [0014]    The example embodiments may also include a video object detecting method that may include retrieving a video object input by a user from an interactive video, tracking a position of a control for controlling the video object, calculating a difference of positions between a moment of the video object and the control point, based on an identification of the video object and the position of the control point, and extracting a frame having a lowest similarity from the interactive video. 
         [0015]    The example embodiments may also include a video object deforming method that may include retrieving a sensing object from a sensing image related to a user and also retrieving a video object from an interactive video, calculating a difference between an object shape related to the sensing object and a depth image related to the video object, and generating a difference map based on the calculated difference. 
         [0016]    Additional aspects, features, and/or advantages of example embodiments will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the disclosure. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0017]    These and/or other aspects and advantages will become apparent and more readily appreciated from the following description of the example embodiments, taken in conjunction with the accompanying drawings of which: 
           [0018]      FIG. 1  illustrates an interactive video processing system according to example embodiments; 
           [0019]      FIG. 2  illustrates an interactive video according to example embodiments; 
           [0020]      FIG. 3  illustrates a structure of a video object detecting apparatus according to example embodiments; 
           [0021]      FIG. 4  illustrates a structure of a video object detecting apparatus according to other example embodiments; 
           [0022]      FIG. 5  illustrates a structure of a video object deforming apparatus according to example embodiments; 
           [0023]      FIG. 6  illustrates a video object detecting method according to example embodiments; 
           [0024]      FIG. 7  illustrates a video object detecting method according to other example embodiments; and 
           [0025]      FIG. 8  illustrates a video object deforming method according to example embodiments. 
       
    
    
     DETAILED DESCRIPTION 
       [0026]    Reference will now be made in detail to example embodiments, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout. Example embodiments are described below to explain the present disclosure by referring to the figures. 
         [0027]      FIG. 1  illustrates an interactive video processing system according to example embodiments. 
         [0028]    Referring to  FIG. 1 , the interactive video processing system includes a video object detecting apparatus  110  and a video object deforming apparatus  120 . 
         [0029]    A sensing image  101  produced by sensing a user and an interactive video  102  may be input to the video object detecting apparatus  110 , and the video object detecting apparatus  110  may explore a frame including a most similar video object to a motion or posture of the user. For example, when the user is playing an interactive video about yoga and the user assumes a particular yoga posture, the video object detecting apparatus  110  according to the example embodiments may explore video frames including an object, such as a person, in a similar posture as the particular yoga posture of the user, and output the searched video frames to a video reproducing apparatus  130 . Accordingly, the user may be able to search and reproduce a part from the interactive video about yoga, related to a desired motion or posture X. 
         [0030]    As an example, the sensing image  101  may be a video image about a motion or posture of the user, taken by a sensor such as a camera and a motion sensor. Here, the sensing object may be an object representing the user in the sensing image  101 . 
         [0031]    As another example, the interactive video  102  may be an image of the object, taken by at least one of a color camera capable of taking a color image and a depth camera capable of taking a depth image. Here, the video object may be an object such as a video image of a person and a thing, included in the interactive video  102 . 
         [0032]    The interactive video  102  will be described in further detail later with reference to  FIG. 2 . 
         [0033]    The video object deforming apparatus  120  according to the example embodiments may be input with the sensing image  101  and the interactive video  102 , thereby generating a difference between the user and the video object or deforming the video object according to the difference. For example, when the user plays an interactive video about yoga, the video object deforming apparatus  120  may provide the user with a difference between a yoga posture of the user and a yoga posture being imitated from the interactive video, through the video reproducing apparatus  130 . Therefore, the user may recognize the difference and correct his or her posture. In addition, the video object deforming apparatus  120  may deform the video object using the generated difference and provide the user with the deformed video object through the video reproducing apparatus  130 . 
         [0034]    The video object detecting apparatus  110  and the video object deforming apparatus  120  may be provided independently or compatibly. 
         [0035]      FIG. 2  illustrates an interactive video according to example embodiments. 
         [0036]    Referring to  FIG. 2 , the interactive video includes at least one color image. 
         [0037]    For example, the interactive video may include an n-number of color images  210  including a video object, such as a person, assuming a yoga posture in a frame t  201 . 
         [0038]    According to one aspect, the interactive video may further include at least one depth image  220 . 
         [0039]    According to another aspect, the interactive video may further include at least one video object skeleton  230  representing the video object in a skeleton form. 
         [0040]    According to yet another aspect, the interactive video may further include at least one video object shape  240  representing a shape of the video object. 
         [0041]      FIG. 3  illustrates a structure of a video object detecting apparatus  300  according to example embodiments. 
         [0042]    Referring to  FIG. 3 , the video object detecting apparatus  300  may include an object retrieval unit  310 , a calculation unit  320 , and a frame extraction unit  330 . 
         [0043]    The object retrieval unit  310  may retrieve a sensing object from a sensing image  302  related to a user  301 . That is, the object retrieval unit  310  may retrieve an object representing the user  301  from the sensing unit  302 . 
         [0044]    Depending on embodiments, the object retrieval unit  310  may retrieve the video object from the sensing image  302  by a model-based tracking method. 
         [0045]    Depending on embodiments, the object retrieval unit  310  may retrieve the video object from the sensing image  302  as a video object skeleton in a skeleton form. Alternatively, the object retrieval unit  310  may retrieve the video object from the sensing image  302  as a sensing object shape representing a shape of the video object. 
         [0046]    The object retrieval unit  310  may retrieve a video object from an interactive video  303 . That is, the object retrieval unit  310  may retrieve an object such as a video image of a person or a thing from the interactive video  303 . 
         [0047]    Depending on embodiments, the object retrieval unit  310  may retrieve the video object from the interactive video  303  by the model-based tracking method. 
         [0048]    Depending on embodiments, the object retrieval unit  310  may retrieve the video object as the video object skeleton in the skeleton form, from the interactive video  303 . Alternatively, the object retrieval unit  310  may retrieve the video object as the video object shape representing the shape of the video object, from the interactive video  303 . 
         [0049]    The calculation unit  320  may calculate similarity between the sensing object and the video object. 
         [0050]    The frame extraction unit  330  may extract a highest-similarity frame  304  having a highest similarity from the interactive video  303 . That is, the frame extraction unit  330  may extract the highest-similarity frame  304  containing a video object most similar to the video image of the user  301  from the interactive video  303 . 
         [0051]    According to certain embodiments, the frame extraction unit  330  may output the extracted highest-similarity frame  304  to a video reproducing apparatus and the video reproducing apparatus may reproduce the extracted highest-similarity frame  304 . 
         [0052]    Hereinafter, an operation of the video object detecting apparatus  300  will be described in detail about a case where the object retrieval unit  310  retrieves the sensing object as a sensing object skeleton and retrieves the video object as a video object skeleton. 
         [0053]    The object retrieval unit  310  may retrieve the sensing object as the sensing object skeleton from the sensing image  302 . 
         [0054]    The sensing object skeleton being retrieved may include an n-number of joints located in positions satisfying Equation 1 shown below. 
         [0000]      J k,1≦k≦n    Equation 1
 
         [0055]    Additionally, the video retrieval unit  310  may retrieve the video object from the interactive video  303 , as the video object skeleton. 
         [0056]    Here, each frame t belonging to a total duration T of the interactive video  303  may be expressed as Equation 2 shown below. 
         [0000]      tεT(S k )   Equation 2
 
         [0057]    Here, the video object skeleton stored in each frame t of the interactive video  303  may include an n-number of joints located in positions satisfying Equation 3 shown below. 
         [0000]      S k,1≦k≦n (t)   Equation 3
 
         [0058]    According to the example embodiments, the calculation unit  320  may calculate similarity between the sensing object and the video object using Equation 4 shown below. 
         [0000]    
       
         
           
             
               
                 
                   
                     D 
                      
                     
                       ( 
                       t 
                       ) 
                     
                   
                   = 
                   
                     
                       ∑ 
                       
                         joint 
                         = 
                         1 
                       
                       n 
                     
                      
                     
                        
                       
                         ( 
                         
                           
                             J 
                             n 
                           
                           - 
                           
                             
                               S 
                               n 
                             
                              
                             
                               ( 
                               t 
                               ) 
                             
                           
                         
                         ) 
                       
                        
                     
                   
                 
               
               
                 
                   Equation 
                    
                   
                       
                   
                    
                   4 
                 
               
             
           
         
       
     
         [0059]    According to Equation 5 shown below, the frame extraction unit  330  may extract a frame that minimizes a value of D(t) as shown in Equation 4 to be the highest-similarity frame  304 . 
         [0000]        t *=argmin {εT(s)}   D ( t )   Equation 5
 
         [0060]      FIG. 4  illustrates a structure of a video object detecting apparatus  400  according to other example embodiments. 
         [0061]    Referring to  FIG. 4 , the video object detecting apparatus  400  may include an object retrieval unit  410 , a tracking unit  420 , a calculation unit  430 , and a frame extraction unit  440 . 
         [0062]    The object retrieval unit  410  may retrieve a video object  403  from an interactive video  404 . Depending on embodiments, each video object  403  may include its own identification (ID). 
         [0063]    The tracking unit  420  may track a position of a control point  402  for controlling the video object  403  input by a user  401 . 
         [0064]    Parts of a human body, such as a hand, a face, or a foot, for example, may be set as the control point  402 . 
         [0065]    The calculation unit  430  may calculate a difference between a moment of the video object  403  and a position of the control point  402 , based on the identification of the video object  403  and the position of the control point  402 . 
         [0066]    The frame extraction unit  440  may extract a frame having the smallest difference calculated from the interactive video  404 . 
         [0067]    That is, when the user  401  wants to move the video object  402  of the interactive video  404  using the control point  402 , the video object detecting apparatus  400  may retrieve a frame containing the most similar object to the video object  402 . 
         [0068]    Hereinafter, an operation of the video object detecting apparatus  400  will be described in reference to a case where the object retrieval unit  410  retrieves the video object as the video object skeleton. 
         [0069]    The object retrieval unit  410  may retrieve the video object as the video object skeleton from the interactive video  404 . 
         [0070]    The tracking unit  420  may retrieve a joint corresponding to a hand of the user  401  from the video object skeleton. In addition, the tracking unit  420  may track the retrieved joint to the position of the control point  402 . 
         [0071]    The moment of the video object skeleton may be expressed by Equation 6 as follows. 
         [0000]    
       
         
           
             
               
                 
                   
                     F 
                      
                     
                       ( 
                       
                         t 
                         , 
                         i 
                       
                       ) 
                     
                   
                   = 
                   
                     
                       1 
                       n 
                     
                      
                     
                       
                         ∑ 
                         
                           joint 
                           = 
                           1 
                         
                         n 
                       
                        
                       
                         
                           S 
                           n 
                         
                          
                         
                           ( 
                           
                             t 
                             , 
                             i 
                           
                           ) 
                         
                       
                     
                   
                 
               
               
                 
                   Equation 
                    
                   
                       
                   
                    
                   6 
                 
               
             
           
         
       
     
         [0072]    The calculation unit  430  may calculate the difference between the moment of the video object skeleton and the position of the control point  402 , based on the identification of the video object skeleton and the position of the control point  402 . 
         [0073]    The frame extraction unit  440  may extract the frame having the smallest difference calculated from the interactive video  404  by Equation 7 shown below. 
         [0000]        t*= argmin {tεT(S)} || p−F ( t,i )||  Equation 7
 
         [0074]    In Equation 7 above, p denotes the position of the control point  402 . 
         [0075]      FIG. 5  illustrates a structure of a video object deforming apparatus  500  according to example embodiments. 
         [0076]    Referring to  FIG. 5 , the video object deforming apparatus  500  includes an object retrieval unit  510 , a calculation unit  520 , and a generation unit  530 . 
         [0077]    The object retrieval unit  510  may retrieve a sensing object from a sensing image  502  related to a user  501 . That is, the object retrieval unit  510  may retrieve the sensing object form representing the user  501  from the sensing image  502 . 
         [0078]    Depending on embodiments, the object retrieval unit  510  may retrieve the sensing object from the sensing image  502  by a model-based tracking method. 
         [0079]    Depending on embodiments, the object retrieval unit  510  may retrieve the video object from the sensing image  502  as a video object skeleton in a skeleton form. Alternatively, the object retrieval unit  510  may retrieve the video object from the sensing image  502  as a sensing object shape representing a shape of the video object. 
         [0080]    The object retrieval unit  510  may retrieve a video object from an interactive video  505 . That is, the object retrieval unit  510  may retrieve an object such as a video image of a person or a thing from the interactive video  505 . 
         [0081]    Depending on embodiments, the object retrieval unit  510  may retrieve the video object from the interactive video  505  by the model-based tracking method. 
         [0082]    Depending on embodiments, the object retrieval unit  510  may retrieve the video object as the video object skeleton in the skeleton form, from the interactive video  505 . Alternatively, the object retrieval unit  510  may retrieve the video object as the video object shape representing a shape of the video object, from the interactive video  505 . 
         [0083]    The calculation, unit  520  may calculate a difference between an object shape related to the sensing object and a depth image related to the video object. 
         [0084]    The generation unit  530  may generate a difference map based on the calculated difference. That is, the difference map may express a difference between the video object and the sensing object. 
         [0085]    According to example embodiments, the video object deforming apparatus  500  may further include a segmentation unit  540 , a database (DB)  550 , and a deformation unit  560 . 
         [0086]    The segmentation unit  540  may segment a color video object from a color image related to the video object, based on the depth image related to the video object. 
         [0087]    The DB  550  may store a 3D morphable object model. 
         [0088]    The deformation unit  560  may deform the video object based on the generated difference map, the segmented color video object, and the 3D morphable object model stored in the DB  550 . 
         [0089]    The deformation unit  560  may deform the color video object with respect to the 3D morphable object model stored in the DB  550 . In addition, the deformation unit  560  may perform 3D model deformation with respect to the textured 3D morphable object model, based on the generated difference map. Furthermore, the deformation unit  560  may deform the video object by performing 2D projection with respect to the 3D morphable object model passed through the 3D model deformation. 
         [0090]    The video object deforming apparatus  500  may further include a synthesis unit  570 . 
         [0091]    The synthesis unit  570  may synthesize the deformed video object with the interactive video  505 . Accordingly, the user  501  may be provided with not only an original video object  503  before the deformation, but also a deformed video object  504  to which the difference between the video image of the user  501  and the original video object  503  is reflected. 
         [0092]      FIG. 6  illustrates a video object detecting method according to example embodiments. 
         [0093]    Referring to  FIG. 6 , in operation  610 , the video object detecting method may retrieve a sensing object from a sensing image and also retrieve a video object from an interactive video. 
         [0094]    According to example embodiments, the video object detecting method may retrieve the sensing object from the sensing image related to a user. That is, the video object detecting method may retrieve an object representing the user from the sensing image. 
         [0095]    Depending on embodiments, the video object detecting method may retrieve the sensing object from the sensing image by a model-based tracking method. 
         [0096]    Depending on embodiments, the video object detecting method may retrieve the sensing object as a sensing object skeleton having a skeleton form from the sensing image. Depending on embodiments, the video object detecting method may retrieve the sensing object as a sensing object shape representing a shape of the video object, from the sensing image. 
         [0097]    The video object detecting method may retrieve the video object from the interactive video. More specifically, the video object detecting method may retrieve an object such as a video image of a person or a thing from the interactive video. 
         [0098]    Depending on embodiments, the video object detecting method may retrieve the video object from the interactive video by the model-based tracking method. 
         [0099]    Depending on embodiments, the video object detecting method may retrieve the video object as the video object skeleton in the skeleton form, from the interactive video. Alternatively, the video object detecting method may retrieve the video object as the video object shape representing a shape of the video object, from the interactive video. 
         [0100]    In operation  620 , the video object detecting method may calculate similarity between the sensing object and the video object. 
         [0101]    In operation  630 , the video object detecting method may extract a frame, having a highest similarity, from the interactive video. Accordingly, the frame containing an object most similar to the video image of the user may be extracted from the interactive video. 
         [0102]    As an example, the video object detecting method may output the extracted frame. Further to the example, a video reproducing apparatus may reproduce the extracted frame. 
         [0103]    Hereinafter, an operation according to the video object detecting method will be described about a case where the sensing object is retrieved as a sensing object skeleton and the video object is retrieved as a video object skeleton. 
         [0104]    The video object detecting method may retrieve the sensing object as the sensing object skeleton from the sensing image. 
         [0105]    Here, the sensing object skeleton may include an n-number of joints located in positions satisfying Equation 1-2 shown below. 
         [0000]      J k,1≦k≦n    Equation 1-2
 
         [0106]    In addition, the video object detecting method may retrieve the video object as the video object skeleton from the interactive video 
         [0107]    Here, each frame t belonging to a total duration T of the interactive video may be expressed by Equation 2-2 as follows. 
         [0000]      tεT(S k )   Equation 2-2
 
         [0108]    Here, the video object skeleton stored in the frame t of the interactive video may include an n-number of joints located in positions satisfying Equation 3-2 below. 
         [0000]      S k,1≦k≦n (t)   Equation 3-2
 
         [0109]    The video object detecting method may calculate similarity between the sensing object and the video object using Equation 4-2 below. 
         [0000]    
       
         
           
             
               
                 
                   
                     D 
                      
                     
                       ( 
                       t 
                       ) 
                     
                   
                   = 
                   
                     
                       ∑ 
                       
                         joint 
                         = 
                         1 
                       
                       n 
                     
                      
                     
                        
                       
                         ( 
                         
                           
                             J 
                             n 
                           
                           - 
                           
                             
                               S 
                               n 
                             
                              
                             
                               ( 
                               t 
                               ) 
                             
                           
                         
                         ) 
                       
                        
                     
                   
                 
               
               
                 
                   Equation 
                    
                   
                       
                   
                    
                   4 
                    
                   
                     - 
                   
                    
                   2 
                 
               
             
           
         
       
     
         [0110]    The video object detecting method may extract a frame that minimizes D(t) in Equation 4-2 to have a highest-similarity frame using Equation 5-2 below. 
         [0000]        t *=argmin {6εT(S)}   D ( t )   Equation 5-2
 
         [0111]      FIG. 7  illustrates a video object detecting method according to other example embodiments. 
         [0112]    Referring to  FIG. 7 , the video object detecting method may retrieve a video object from an interactive video in operation  710 . Depending on embodiments, the video object may include its own ID. 
         [0113]    In operation  720 , the video object detecting method may track a position of a control point for controlling the video object input by a user. 
         [0114]    Parts of a human body, such as a hand, a face, or a foot, for example, may be set as the control point. 
         [0115]    In operation  730 , the video object detecting method may calculate a difference between a moment of the video object and the position of the control point, based on an identification of the video object and the position of the video object. 
         [0116]    In operation  740 , the video object detecting method may extract a frame having the smallest difference calculated from the interactive video. 
         [0117]    Therefore, when the user wants to move the video object of the interactive video using the control point, the video object detecting method may retrieve a frame containing a most similar object to the video object. 
         [0118]    Hereinafter, an operation according to the video object detecting method will be described in reference to a case where the video object is retrieved as the video object skeleton. 
         [0119]    The video object detecting method may retrieve the video object as the video object skeleton from the interactive video. 
         [0120]    For example, the video object detecting method may retrieve joints corresponding to a hand of the user from the video object skeleton. In addition, the video object detecting method may track the retrieved joint to the position of the control point. 
         [0121]    The moment of the video object skeleton may be expressed by Equation 6-2 as follows. 
         [0000]    
       
         
           
             
               
                 
                   
                     F 
                      
                     
                       ( 
                       
                         t 
                         , 
                         i 
                       
                       ) 
                     
                   
                   = 
                   
                     
                       1 
                       n 
                     
                      
                     
                       
                         ∑ 
                         
                           joint 
                           = 
                           1 
                         
                         n 
                       
                        
                       
                         
                           S 
                           n 
                         
                          
                         
                           ( 
                           
                             t 
                             , 
                             i 
                           
                           ) 
                         
                       
                     
                   
                 
               
               
                 
                   Equation 
                    
                   
                       
                   
                    
                   6 
                    
                   
                     - 
                   
                    
                   2 
                 
               
             
           
         
       
     
         [0122]    The video object detecting method may calculate the difference between the moment of the video object skeleton and the position of the control point, based on the identification of the video object skeleton and the position of the control point. 
         [0123]    The video object detecting method may extract the frame having the smallest difference calculated from the interactive video by Equation 7-2 below. 
         [0000]        t *=argmin {tεT(S)}   ||p−F ( t,i )||  Equation 7-2
 
         [0124]    In Equation 7-2 above, p denotes the position of the control point. 
         [0125]      FIG. 8  illustrates a flowchart showing a video object deforming method according to example embodiments. 
         [0126]    Referring to  FIG. 8 , the video object deforming method may retrieve a sensing object from a sensing image and retrieve a video object from an interactive video in operation  810 . 
         [0127]    The video object deforming method may retrieve the sensing object from the sensing image related to a user. More specifically, the video object deforming method may retrieve an object representing a user from the sensing image. 
         [0128]    Depending on embodiments, the video object deforming method may retrieve the sensing object from the sensing image by a model-based tracking method. 
         [0129]    Depending on embodiments, the video object deforming method may retrieve the sensing object from the sensing image as the sensing object skeleton in a skeleton form. Alternatively, the video object deforming method may retrieve the video object as the video object shape representing the shape of the video object, from the interactive video  303 . 
         [0130]    The video object detecting method may retrieve the video object from the interactive video. That is, the video object detecting method may retrieve an object such as a video image of a person or a thing from the interactive video. 
         [0131]    Depending on embodiments, the video object detecting method may retrieve the video object from the interactive video by the model-based tracking method. 
         [0132]    Depending on embodiments, the video object detecting method may retrieve the video object as the video object skeleton in the skeleton form, from the interactive video. Alternatively, the video object detecting method may retrieve the video object as the video object shape representing a shape of the video object, from the interactive video. 
         [0133]    In operation  820 , the video object deforming method may calculate a difference between an object shape related to the sensing object and a depth image related to the video object. 
         [0134]    In operation  830 , the video object deforming method may generate a difference map based on the calculated difference. That is, the difference map may express a difference between the video object and the sensing object. 
         [0135]    The video object deforming method may further include segmenting a color video object from a color image and deforming the video object. 
         [0136]    Based on the depth image related to the video object, the video object deforming method may segment the color video object from the color image related to the video object in operation  840 . 
         [0137]    In addition, in operation  850 , the video object deforming method may deform the video object based on the generated difference map, the segmented color video object, and a 3D morphable object model stored in the DB. 
         [0138]    The video object deforming method may texture the color video object with respect to the 3D morphable object model stored in the DB. In addition, the video object deforming method may perform 3D model deformation with respect to the textured 3D morphable object model, based on the generated difference map. Furthermore, the video object deforming method may deform the video object by performing a 2D projection with respect to the 3D morphable object model passed through the 3D model deformation. 
         [0139]    As shown in operation  860 , the video object deforming method may further include synthesizing the deformed video object with the interactive video. 
         [0140]    Accordingly, the video object deforming method may be provided with not only an original video object before the deformation but also a deformed video object to which the difference between an image of the user and the original video object is reflected. 
         [0141]    The methods according to the above-described example embodiments may be recorded in non-transitory computer-readable media including program instructions to implement various operations embodied by a computer. The media may also include, alone or in combination with the program instructions, data files, data structures, and the like. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the example embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of non-transitory computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD ROM discs and DVDs; magneto-optical media such as optical discs; and hardware devices that are specially configured to store and perform program instructions, such as read-only memory (ROM), random access memory (RAM), flash memory, and the like. The computer-readable media may also be a distributed network, so that the program instructions are stored and executed in a distributed fashion. The program instructions may be executed by one or more processors. The computer-readable media may also be embodied in at least one application specific integrated circuit (ASIC) or Field Programmable Gate Array (FPGA), which executes (processes like a processor) program instructions. The media may be transfer media such as optical lines, metal lines, or waveguides including a carrier wave for transmitting a signal designating the program command and the data construction. Examples of program instructions include both machine code, such as produced by a compiler, and files containing higher level code that may be executed by the computer using an interpreter. The described hardware devices may be configured to act as one or more software modules in order to perform the operations of the above-described example embodiments, or vice versa. 
         [0142]    Although example embodiments have been shown and described, it would be appreciated by those skilled in the art that changes may be made in these example embodiments without departing from the principles and spirit of the disclosure, the scope of which is defined in the claims and their equivalents.