Abstract:
An automatic calibration method suitable for an electronic apparatus comprising a proximity sensor is disclosed. The automatic calibration method includes steps of: resetting data comprising a minimal index and a maximal index when a specific function is activated; updating the minimal index and the maximal index according to readings from the proximity sensor; resetting the minimal index when the proximity sensor detects a far event according to a threshold value of the proximity sensor; resetting the maximal index when the proximity sensor detects a near event according to the threshold value; and, re-calculating the threshold value according to the minimal index or the maximal index every time the minimal index or the maximal index is updated.

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
       [0001]    This application claims the priority benefit of U.S. Provisional Application Ser. No. 61/915,530, filed Dec. 13, 2013, and the full disclosures of which are incorporated herein by reference. 
     
    
     BACKGROUND 
       [0002]    1. Field of Invention 
         [0003]    The present application relates to an image processing method. More particularly, the present application relates to how to apply a parallax effect onto a still image. 
         [0004]    2. Description of Related Art 
         [0005]    Recently, people get used to record their daily life by shooting photographs, and they can review the photographs in a digital album on their devices. After the photos are captured, some users may perform some post processes to the photographs, such as filtering, warping, and applying various parametric transformations. However, aforesaid effects are limited by contents of the original image. User can only observes objects and background scene in the photographs in the original way as the photograph was taken. 
         [0006]    In general, animations or videos are more attractive to users in comparison with still images, because the users can observes how the object moves in the scene. The animations or the videos occupy much more storage space, such that some users prefer to shoot photographs then filming a video. In general, it is easy to generate a still image from a video by taking a screenshot, but it is hard to establish an animation or a video from a still image. 
       SUMMARY 
       [0007]    This disclosure provides a method and a system for creating a parallax video from a still image. The parallax video provides user a chance to observe the still image from different simulated visual angle, such that a 2.5D or 3D-alike effect is applied on the still image. 
         [0008]    An aspect of the present disclosure is to provide a method for creating a parallax video from a still image. The method includes steps of: performing an image segmentation for separating at least a foreground object within the still image from an image background of the still image; filling a blank region of the image background, the blank region corresponding to where the separated foreground object locates within the still image; creating a virtual three-dimensional (3D) scene consisting of the foreground object and the image background; analyzing the virtual 3D scene for determining a camera path; and, creating the parallax video by observing the virtual 3D scene with a virtual camera moving along the camera path. 
         [0009]    Another aspect of the present disclosure is to provide an electronic apparatus, which includes a data storage module and a control, module. The data storage module is configured for storing a still image. The control module is coupled with the data storage module. The control module includes a segmentation unit, an in-painting unit, a scene simulation unit, a scene analyzer unit and a video generator unit. The segmentation unit is configured for separating at least a foreground object within the still image from an image background of the still image. The in-painting unit is configured for filling a blank region of the image background, the blank region corresponding to where the separated foreground object locates within the still image. The scene simulation unit is configured for creating a virtual three-dimensional (3D) scene consisting of the foreground object and the image background. The scene analyzer unit is configured for analyzing the virtual 3D scene for determining a camera path. The video generator unit is configured for generating a parallax video by observing the virtual 3D scene with a virtual camera moving along the camera path. 
         [0010]    Another aspect of the present disclosure is to provide a non-transitory computer readable storage medium, containing instructions for performing a method includes steps of: performing an image segmentation for separating at least a foreground object within the still image from an image background of the still image; filling a blank region of the image background, the blank region corresponding to where the separated foreground object locates within the still image; creating a virtual three-dimensional (3D) scene consisting of the foreground object and the image background; analyzing the virtual 3D scene for determining a camera path; and, creating a parallax video by observing the virtual 3D scene with a virtual camera moving along the camera path. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0011]    The disclosure can be more fully understood by reading the following detailed description of the embodiments, with reference made to the accompanying drawings as follows: 
           [0012]      FIG. 1  is a schematic diagram illustrating an electronic apparatus according to an embodiment of the disclosure; 
           [0013]      FIG. 2  is a flow-chart diagram illustrating a method for creating a parallax video from a still image according to an embodiment of the disclosure; 
           [0014]      FIG. 3A  is a flow-chart diagram illustrating further steps according to an embodiment of the disclosure; 
           [0015]      FIG. 3B  is a flow-chart diagram illustrating further steps according to an embodiment of the disclosure; 
           [0016]      FIG. 3C  is a flow-chart diagram illustrating further detail steps shown in  FIG. 3B  for demonstrating how the camera path is determined according to an embodiment of the disclosure; 
           [0017]      FIG. 4A  illustrates an example of the still image in a demonstrational case according to an embodiment of the disclosure; 
           [0018]      FIG. 4B  illustrates an image segmentation result of the still image shown in  FIG. 4A : 
           [0019]      FIG. 4C  illustrates the image background in the demonstrational case; 
           [0020]      FIG. 4D  illustrates the virtual 3D scene in the demonstrational case; 
           [0021]      FIG. 4E  and  FIG. 4F  illustrate the virtual camera moving relative to the virtual 3D scene in the demonstrational case: 
           [0022]      FIG. 4G  and  FIG. 4H  illustrate examples of different leakage ratios of the in-painting region IPR in the demonstrational case; and 
           [0023]      FIG. 5A  to  FIG. 5D  is a schematic diagram illustrating the camera path. 
       
    
    
     DETAILED DESCRIPTION 
       [0024]    Reference is made to  FIG. 1 , which is a schematic diagram illustrating an electronic apparatus  100  according to an embodiment of the disclosure. As shown in  FIG. 1 , the electronic apparatus  100  includes a control module  120  and a data storage module  140 . In some embodiments, there are some still images stored in the data storage module  140 . The data storage module  140  can be a memory card, a hard-drive or a communication unit connected to an on-line storage server. The control module  120  of the electronic apparatus  100  is able to create a parallax video (or a parallax animation) from the still image. The control module  120  is configured to establish a virtual three-dimensional (3D) scene simulated from the still image. During the parallax video, the user can observe the virtual 3D scene (i.e., the still image) from different visual angles, such as to provide users different views and 3D-alike experiences toward the still image. Details of how to create the parallax video from the still image is explained in following paragraphs. 
         [0025]    In some embodiments, the electronic apparatus  100  further includes a camera module  160  and a display panel  180 . Aforesaid still image can be captured by the camera module  160  of the electronic apparatus  100 , but the disclosure is not limited to this. The still image can also be a photograph inputted from other devices (e.g., a digital camera or a smartphone other than the electronic apparatus  100 ), an image downloaded over internet, etc. The parallax video created by the still image can be shown on the display panel  180  such as 2D display panel. 
         [0026]    The control module  120  of the electronic apparatus  100  includes a segmentation unit  121 , an in-painting unit  122 , a scene simulation unit  123 , a scene analyzer unit  124  and a video generator unit  125 . The control module  120  can be a central processor unit (CPU), a graphic processing unit (GPU), an image processing circuit, or an equivalent computing circuit. The segmentation unit  121 , the in-painting unit  122 , the scene simulation unit  123  the scene analyzer unit  124  and the video generator unit  125  can be realized by software instructions/programs executed by the control module  120 , firmware instructions/programs or circuits for executing specific applications. 
         [0027]    Reference is also made to  FIG. 2 .  FIG. 2  is a flow-chart diagram illustrating a method  200  for creating a parallax video from a still image according to an embodiment of the disclosure. In some embodiments, the method  200  can be executed by the electronic apparatus  100  shown in  FIG. 1 . As shown in  FIG. 2 , after the still image is provided (the still image is existed in the data storage module  140  or the still image is captured by the camera module  160 ), the method  200  executes step S 210  for performing an image segmentation for separating at least a foreground object within the still image from an image background of the still image by the segmentation unit  121 . Reference is also made to  FIG. 4A  and  FIG. 4B .  FIG. 4A  illustrates an example of the still image IMG in a demonstrational case according to an embodiment of the disclosure.  FIG. 4B  illustrates an image segmentation result of the still image IMG shown in  FIG. 4A . 
         [0028]    As shown in  FIG. 4A  and  FIG. 4B , the original still image IMG includes a foreground object OBJ (e.g., a person in this demonstrational case) and an image background BG. In some applications, the segmentation unit  121  can analyze depth distributions within the still image IMG (e.g. generating a depth histogram of the still image IMG), and identify objects within the still image IMG by the depth distributions, so as to identify the foreground object OBJ from the image background BG. After the foreground object OBJ is identified, the segmentation unit  121  separates pixels of the foreground object. OBJ apart from the image background BG (as shown in  FIG. 4B ). The original data of aforesaid pixels are removed from the still image IMG to form the image background BG, such that a blank region BR (as shown in  FIG. 4B ) is left on the image background BG. In other words, the blank region BR corresponds to where the separated foreground object OBJ locates within the still image IMG. 
         [0029]    In other cases, the original still image IMG may include more than one foreground object. The foreground object is not limited to a person, and it can be a car, a pet, a store, an item, a sign or any object which is located at a specific depth and able be separated from the image background BG. 
         [0030]    The method  200  executes step S 220  for filling the blank region BR of the image background BG by the in-painting unit  122 . Reference is also made to  FIG. 3A  and  FIG. 4C .  FIG. 3A  is a flow-chart diagram illustrating further steps S 221 -S 222  of step S 220  shown in  FIG. 2  according to an embodiment of the disclosure.  FIG. 4C  illustrates the image background BG in the demonstrational case. In this embodiment, step S 221  is executed for analyzing neighborhood pixels NB adjacent to the blank region of the image background. Step S 222  is executed for filling the blank region BR (referring to  FIG. 4B ) up into an in-painting region IPR (referring to  FIG. 4C ) with a suitable pattern analyzed from the neighborhood pixels NB. 
         [0031]    In an embodiment, the in-painting unit  122  analyzes colors of each of the neighborhood pixels NB, finds out average values (or mode values, or medium values) of RIG/B data of the neighborhood pixels NB, and paint the blank region BR into the in-painting region IPR with an average color (or the most common color, or the medium color) of the neighborhood pixels NB. In another embodiment, the in-painting unit  122  further analyzes strips, shapes, edges, regular patterns within the neighborhood pixels NB, and fill the blank region BR up according to the pattern of the neighborhood pixels NB. The in-painting unit  122  is configured to simulate or predict the contents within the blank region BR (i.e., the in-painting region IPR) even though the contents of the blank region BR is not recorded in the original still image IMG, so as to make the in-painting region IPR unnoticeable by users as possible. 
         [0032]    The method  200  executes step S 230  for creating a virtual three-dimensional (3D) scene consisting of the foreground object OBJ and the image background BG by the scene simulation unit  123 . Reference is also made to  FIG. 4D , which illustrates the virtual 3D scene TDS in the demonstrational case. As shown in  FIG. 40 , the virtual 3D scene TDS consists of the foreground object OBJ and the image background BG, and the foreground object OBJ is configured to be located in front of the image background BG by a depth difference DP. The depth difference DP can be determined or calculated according to the depth histogram of the still image IMG. As shown in  FIG. 4D , the foreground object OBJ and the image background BG are now located at different depths in the virtual 3D scene TDS. When an observing spots is moving (e.g., moving vertically or horizontally), the foreground object OBJ in the virtual 3D scene TDS will be viewed to be located at different positions relative to the background BG. In some embodiments, the virtual three-dimensional scene is established by a 3D simulator, a virtual reality scene generator, a 3D modeling process or any equivalent simulator in accordance with the foreground object. OBJ, the image background BG and the depth distributions thereof. Further details of how to establish a virtual 3D scene are known by a person in the art and not to be discussed here. 
         [0033]    Afterward, the method  200  executes step S 240  for analyzing the virtual 3D scene TDS for determining a camera path by the scene analyzer unit  124 . Then, the method  200  executes step S 250  for creating the parallax video by observing the virtual 3D scene TDS with a virtual camera moving along the camera path by the video generator unit  125 . Reference is also made to  FIG. 4E  and  FIG. 4F .  FIG. 4E  and  FIG. 4F  illustrate the virtual camera VC moving relative to the virtual 3D scene TDS in the demonstrational case. As shown in  FIG. 4E , the video generator unit  125  simulates the virtual camera VC moving from the node ND1 via the node ND2 to the node ND3 along the camera path PATH. The virtual camera VC observed at the nodes ND1, ND2 and ND3 will observe the virtual 3D scene TDS from different visual angles and different distances, such that visual windows observed by the virtual camera VC at the nodes ND1 ND2 and ND3 will be quite different from each others. The relationship between the foreground object OBJ and the in-painting region IPR observed by the virtual camera VC at the nodes ND1, ND2 and ND3 are also different. Accordingly, the parallax video is created to display the still image in a more vivid way. 
         [0034]    However, the camera path PATH of the virtual camera VC can not be a random path. For example, when the virtual camera VC is moved to a node (referring to the node ND4 as shown in  FIG. 4F ) too far from the foreground object OBJ and the image background BG, a visual window of the virtual camera VC from the node ND4 can not be completely covered by the virtual 3D scene, and the visual window go beyond the boundary of the image background BG. The original still image IMG has no data about pixels outside the boundary of the image background BG. Therefore, if the visual window goes beyond the boundary of the image background BG, a part of the visual window observed by the virtual camera VC will be void without any valid pixel data. On the other hands, the in-painting region IPR filled up by the in-painting unit  122  is not really existed in the original still image IMG. If a large portion of the in-painting region IPR is revealed to the user, the user may notice some flaws or unnatural bugs within the in-painting region IPR. Therefore, the camera path PATH must be planned carefully to optimize the parallax video. 
         [0035]    In this embodiment, the camera path PATH is determined in step S 240  according to analysis of the virtual 3D scene. Reference is also made to  FIG. 3B , which is a flow-chart diagram illustrating further steps S 241 ˜S 243  of step S 240  shown in  FIG. 2  according to an embodiment of the disclosure. As shown in  FIG. 3B , step S 241  is executed for determining whether the blank region BR (referring to  FIG. 4B ) of the image background BG is filled as the in-painting region (referring to  FIG. 4C ). In some embodiments, the blank region BR is completely filled up with the suitable pattern as the in-painting region IPR. However, the disclosure is not limited to this. In some other embodiment, at least part of the blank region BR (e.g. a part adjacent to edges of the blank region BR) is filled with the suitable pattern as the in-painting region IPR, and another part of the blank region BR (e.g., a part adjacent to a center of the blank region BR) remains blank without display data. In this case, step S 241  is executed for determining whether the blank region BR (referring to  FIG. 4B ) of the image background BG is at least partially filled as the in-painting region. 
         [0036]    Step S 241  in this embodiment is an optional step for double checking whether the blank region BR is filled as the in-painting region IPR. If the blank region BR is not filled, the method  200  returns to step S 220 . 
         [0037]    If the blank region BR is filled as the in-painting region IPR, step S 242  is executed for estimating different leakage ratios of the in-painting region IPR. Step S 243  is executed for determining the camera path PATH based on aforesaid estimation. The leakage ratios correspond to how much the in-painting region IPR is observable from different candidate nodes by the virtual camera VC in the virtual 3D scene TDS. Reference is also made to  FIG. 4G  and  FIG. 4H , which illustrate examples of different leakage ratios of the in-painting region IPR in the demonstrational case. 
         [0038]    As shown in  FIG. 4G  the virtual camera is substantially located in front of a center of the image background BG. A projection area of the foreground object OBJ (observed from the virtual camera) is formed on the image background BG. The projection area of the foreground object OBJ on the image background BG overlaps about 75% of the in-painting region IPR in  FIG. 4G . In this case, a leakage region LK of the in-painting region IPR will be observable by the virtual camera VC. The leakage ratio is calculated in this case shown in  FIG. 4G  is about 25% (i.e., a percentage between the observable area and the total area of the in-painting region IPR). 
         [0039]    As shown in  FIG. 4H , the virtual camera is moved to the right side in front of the image background BG. Another projection area of the foreground object OBJ (observed from the virtual camera) is formed on the image background BG. The projection area of the foreground object OBJ on the image background BG overlaps about 55% of the in-painting region IPR in  FIG. 4H . In this case, another leakage region LK of the in-painting region IPR will be observable by the virtual camera VC. The leakage ratio in this case shown in  FIG. 4H  is about 45%. According to practical experiences, flaws or bugs of the in-painting region IPR will be easy to notice when the leakage ratio is higher than 50%. In some embodiments, the scene analyzer unit  124  determines the camera path according to the leakage ratios of nodes on the camera path PATH, so as to ensure that each of nodes along the camera path PATH has a leakage ratio less than 50%. Furthermore, in the embodiments that the blank region BR is partially filled as the in-painting region IPR, the remaining part of the blank region BR without display data is not allowed to be observable by the virtual camera VC, and the determination of the camera path PATH is required to avoid the leakage of the remaining part of the blank region BR without display data. 
         [0040]    Reference is made to  FIG. 3C  and  FIG. 5A  to  FIG. 5D .  FIG. 3C  is a flow-chart diagram illustrating further detail steps S 243   a ˜S 243   i  of step S 243  shown in  FIG. 3B  for demonstrating how the camera path PATH is determined according to an embodiment of the disclosure.  FIG. 5A  to  FIG. 5D  is a schematic diagram illustrating the camera path PATH. 
         [0041]    As shown in  FIG. 5C , the camera path PATH includes an initial node NDi, a terminal node NDt and at least one intermediate node (in the embodiment shown in  FIG. 5C , there are two intermediate nodes NDm1, NDm2) between the initial node NDi and the terminal node NDt. 
         [0042]    As shown in  FIG. 3C , step S 243   a  is executed for setting the initial node NDi. In some embodiments, the initial node NDi can be set at an original viewing point (i.e., where the user takes the still image IMG in the first place) of the still image IMG. For example, a focal distance of the still image IMG is known and recorded along the still image IMG. Therefore, the original viewing point can be located along a vertical axis extending from the center of the still image IMG according to the focal distance. Step S 243   b  is executed for setting the terminal node NDt according to a position and a size of the foreground object OBJ within the still image IMG. 
         [0043]    In some embodiments, the terminal node NDt is set to a right side from the initial node NDi when the position of the foreground object OBJ is located at a right half of the still image IMG; on the other hand, the terminal node NDt is set to a left side from the initial node NDi when the position of the foreground object OBJ is located at a left half of the still image IMG; such that the visual camera can move from one side relative to the foreground object OBJ to the other side relative to the foreground object OBJ, and the parallax video created in this way will has opposite views relative to the foreground object OBJ. 
         [0044]    As shown in  FIG. 3C  and  FIG. 5A , step S 243   c  is executed for finding a target vector Vt from the initial node NDi toward the terminal node NDt. Step S 243   d  is executed for creating refraction vectors Vr around the target vector Vt. Step S 243   e  is executed for assigning ends of the refraction vectors Vr and target vector Vt as the candidate nodes NDc1˜NDc5. The refraction vectors Vr are created along an orientation of the target vector Vt with different angles deviated from the target vector Vt. 
         [0045]    As shown in  FIG. 3C  and  FIG. 5A , step S 243   f  is executed for verifying whether each visual window of the virtual camera VC from each of the candidate nodes NDc1˜NDc5 is completely covered by the virtual 3D scene TDS (referring to  FIG. 4E  and  FIG. 4F ). If any visual window exceeds the virtual 3D scene TDS, the corresponding candidate node is disqualified (e.g., the node ND4 in  FIG. 4F  will be disqualified). When the visual window is completely covered by the virtual 3D scene TDS, the corresponding candidate node is qualified (e.g., the nodes ND1, ND2, ND3 in  FIG. 4E  will be qualified). In this case, it assumes that the candidate nodes NDc4 and NDc5 are disqualified in step S 243   f.    
         [0046]    As shown in  FIG. 3C  and  FIG. 5A , step S 243   g  is executed for finding the optimal vector by selecting an optimal node from the qualified candidate nodes (NDc1, NDc2, NDc3) according to the leakage ratios. The leakage ratio of the optimal node of the camera path is less than a criteria upper bound. In some embodiment, the criteria upper bound is that 50% of the in-painting region IPR is observable by the virtual camera VC from one of the candidate nodes NDc1, NDc2, NDc3. The optimal node is selected from the qualified candidate nodes with their leakage ratio less than 50%. In some embodiment, the optimal node is further elected from the qualified candidate nodes with their leakage ratio more than 20% and less than 50%, such that the parallax effect between the foreground object and the image background will be more noticeable to users. In this case, it assumes that the qualified candidate nodes NDc3 is selected as the optimal node, and the refraction vector Vr corresponding to the candidate nodes NDc3 is selected as the optimal vector. 
         [0047]    As shown in  FIG. 3C  and  FIG. 5A , step S 243   h  is executed for assigning an end (i.e., the candidate nodes NDc3) of the optimal vector as a next intermediate node NDm1. 
         [0048]    Step S 243   i  is executed for determining whether the camera path PATH is completed. When the camera path PATH is not completed, it goes to repeat steps S 243   c ˜S 243   h.    
         [0049]    As shown in  FIG. 5B , the intermediate node NDm1 is the current intermediate node NDm1 Step S 243   c  is executed for finding a target vector Vt from the current intermediate node NDm1 toward the terminal node NDt. Steps S 243   d  to S 243   h  is further executed for assign the next intermediate node NDm2. Then, the Steps S 243   c  to S 243   h  are repeated until the camera path PATH is completed. 
         [0050]    In the embodiment shown in  FIG. 5A  to  FIG. 5C , there are two intermediate nodes NDm1 and NDm2, but the disclosure is not limited to this. As shown in  FIG. 5D  according to some other embodiments, the camera path PATH can includes more intermediate nodes, such as the intermediate nodes NDm1˜NDm5 shown in  FIG. 5D . 
         [0051]    Another embodiment of the disclosure is a non-transitory computer readable storage medium. The non-transitory computer readable storage medium contains instructions for performing the method  200  in aforesaid embodiment shown in  FIG. 2 ,  FIG. 3A  to  FIG. 3C . 
         [0052]    Based on aforesaid embodiment, the disclosure provides a method and an electronic apparatus capable of creating a parallax video from a still image, such that user can experience a 2.5D or 3D-alike animation effect generated from any still image. While the parallax video is created by a virtual camera, the method will automatically determine an optimal camera path for the virtual camera, such that the parallax video can show objects within the still image from different visual angles with diversity, and also prevent the user from seeing too much leakage of the in-painting region. 
         [0053]    It will be apparent to those skilled in the art that various modifications and variations can be made to the structure of the present application without departing from the scope or spirit of the application. In view of the foregoing, it is intended that the present application cover modifications and variations of this application provided they fall within the scope of the following claims.