Patent Publication Number: US-11398046-B2

Title: Object position determination circuit

Description:
BACKGROUND OF THE INVENTION 
     1. Field of the Invention 
     The present invention is related to image processing, and more particularly, to a circuit for determining a position of a specific object in an image. 
     2. Description of the Prior Art 
     In current face identification systems, deep learning or neural networks are utilized to analyze and process an image in order to identify a position of a human face therein. An artificial intelligence module for deep learning requires a high computation amount. When there is a large content of image data, the artificial intelligence module may be overloaded, therefore requiring an artificial intelligence module having greater ability. This increases design and manufacturing costs. 
     SUMMARY OF THE INVENTION 
     Thus, an objective of the present invention is to provide an object position determination circuit, which can perform object position detection on a partial region of subsequent frame(s) according to a detection result of previous frame(s) only, to reduce loading of the artificial intelligence module. 
     In an embodiment of the present invention, an object position determination circuit is disclosed. The object position determination circuit comprises a receiving circuit and a detecting circuit. In operations of the object position determination circuit, the receiving circuit receives an image signal; and the detecting circuit detects a position of an object in an N th  frame of the image signal, determines a partial region within an (N+M) th  frame of the image signal according to the position of the object in the N th  frame, and only detects the partial region within the (N+M) th  frame to determine a position of the object in the (N+M) th  frame, wherein N and M are positive integers. 
     In another embodiment of the present invention, a circuit architecture comprising an object position determination circuit and an image processing circuit is disclosed. The object position determination circuit comprises a receiving circuit, a detecting circuit, and an output circuit, wherein the receiving circuit receives an image signal; the detecting circuit detects a position of an object in an N th  frame of the image signal, determines a partial region within an (N+M) th  frame of the image signal according to the position of the object in the N th  frame, and only detects the partial region within the (N+M) th  frame to determine a position of the object in the (N+M) th  frame, wherein N and M are positive integers; and the output circuit outputs coordinate ranges within the N th  frame and the (N+M) th  frame to respectively serve as the positions of the object. The image processing circuit is configured to receive the image signal and process the image signal according to the coordinate ranges within the N th  frame and the (N+M) th  frame, in order to generate multiple output images to a display panel for being displayed thereon. 
     These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a diagram illustrating an electronic device according to an embodiment of the present invention. 
         FIG. 2  is a timing diagram illustrating operations of an object position determination circuit according to an embodiment of the present invention. 
         FIG. 3  is a diagram illustrating object detection operations according to a first embodiment of the present invention. 
         FIG. 4  is a diagram illustrating object detection operations according to a second embodiment of the present invention. 
         FIG. 5  is a diagram illustrating object detection operations according to a third embodiment of the present invention. 
         FIG. 6  is a diagram illustrating an image processing circuit processing images according to object position information. 
         FIG. 7  is a flowchart illustrating an image processing method according to an embodiment of the present invention. 
     
    
    
     DETAILED DESCRIPTION 
       FIG. 1  is a diagram illustrating an electronic device according to an embodiment of the present invention. As shown in  FIG. 1 , the electronic device comprises an image processing circuit  110 , an object position determination circuit  120  and a display panel  130 , where the object position determination circuit  120  comprises a receiving circuit  122 , a detecting circuit  124  and an output circuit  128 . In this embodiment, the image processing circuit  110  and the object position determination circuit  120  may be integrated into a single chip, and the electronic device may be any electronic device comprising a display panel, such as a desktop computer, laptop computer, or mobile phone, etc. 
     In operations of the electronic device, the image processing circuit  110  is configured to receive an image signal Din, and generate multiple output images (frames) to the display panel  130  for being displayed thereon. Simultaneously, the object position determination circuit  120  determines positions of an object (e.g. a human face) in multiple frames according to contents of the image signal Din, and generates object position information (e.g. region coordinates of the object in each frame) to the image processing circuit, to allow the image processing circuit to process the image signal Din. Determination (or detection) of object position in an image is performed by utilizing deep learning or neural networks which require a high computation amount, and processing capability of the object position determination circuit  120  is not enough to perform the determination (or detection) of object position on each frame immediately; for example, the time for the object position determination circuit  120  to process a whole frame might be equal to the time for the display panel  130  to display four to five frames. Thus, the object position determination circuit  120  in this embodiment performs object position detection according to a partial region of a frame, in order to provide the image processing circuit with enough object position information under a condition where the processing capability of the object position determination circuit  120  meets the requirement. 
     Refer to  FIG. 1  and  FIG. 2  together, where  FIG. 2  is a timing diagram illustrating operations of the object position determination circuit  120  according to an embodiment of the present invention. As shown in  FIG. 2 , this embodiment assumes that the time for the object position determination circuit  120  to process a whole frame is roughly equal to the time for the display panel to display four to five frames. Thus, during a process of the image processing circuit  110  receiving frames I 0  to I 3  and performing further processing on them, the detecting circuit  124  skips the frames I 1  to I 3  and directly performs object position detection on whole contents of a frame I 4  in order to determine a position of the object in the frame I 4  and output a coordinate range which serves as position information F 4  of the object in the frame I 4 . Then, a partial region of at least one of the frames I 5  to I 8  is determined according to the position information F 4  (i.e. the coordinate range mentioned above) of the object in the frame I 4 , and the detecting circuit  124  only detects the partial region to determine a position of the object in at least one of the frames I 5  to I 8 . In this embodiment, the detecting circuit  124  at least generates position information F 8  of the object in the frame I 8 , and the detecting circuit  124  then determines a partial region I 9 ′ of a frame I 9  according to the position information F 8  of the object in the frame I 8 , where the detecting circuit  124  only detects the partial region I 9 ′ to determine a position of the object in the frame I 9 . 
     In this embodiment, the detecting circuit  124  may determine whether to perform object detection on at least one of the frames I 5  to I 7  or directly skip (or omit) the frames I 5  to I 7  according to a size or an occupied ratio of the object in the frame I 4 . For example, referring to a first embodiment shown in  FIG. 3  and assuming that the size or occupied ratio of the object in the frame I 4  is less than a threshold value, the detecting circuit  124  performs object detection on each of the frames I 5  to I 8  to generate object position information. In detail, the detecting circuit  124  may select a partial region I 5 ′ of the frame I 5  according to an object detection result of the frame I 4  (the position information F 4 ) for performing object detection thereon (other regions do not undergo object detection), select a partial region I 6 ′ of the frame I 6  according to an object detection result of the frame I 5  (the position information F 5 ) for performing object detection thereon (other regions do not undergo object detection), select a partial region I 7 ′ of the frame I 7  according to an object detection result of the frame I 6  (the position information F 6 ) for performing object detection thereon (other regions do not undergo object detection), select a partial region I 8 ′ of the frame I 8  according to an object detection result of the frame I 7  (the position information F 7 ) for performing object detection thereon (other regions do not undergo object detection), and select a partial region I 9 ′ of the frame I 9  according to an object detection result of the frame I 8  (the position information F 8 ) for performing object detection thereon (other regions do not undergo object detection), where the rest may be deduced by analogy. In this embodiment, considering movement of the object, a size of the partial region of each of the frames I 6 , I 7 , I 8  and I 9  that is selected and undergoes object detection is not less than a size of a previous frame that is selected and undergoes object detection; for example, each of the frames I 5  to I 8  may select a region having 10% size of a frame to perform object detection thereon, and the frame I 9  may select a region having 50% size of a frame to perform object detection thereon. 
     Referring to a second embodiment shown in  FIG. 4  and assuming that the size or occupied ratio of the object in the frame I 4  is greater than a threshold value, the detecting circuit  124  may only perform object detection on the frames I 7  and I 8  to generate the position information F 7  and F 8 , and does not perform object detection on the frames I 5  and I 6 . In detail, the detecting circuit  124  may select a partial region I 7 ′ of the frame I 7  for performing object detection thereon (other regions do not undergo object detection) according to the object detection result of the frame I 4  (the position information F 4 ), select a partial region I 8 ′ of the frame I 8  for performing object detection thereon (other regions do not undergo object detection) according to the object detection result of the frame I 7  (the position information F 7 ), and select a partial region I 9 ′ of the frame I 9  for performing object detection thereon (other regions do not undergo object detection) according to the object detection result of the frame I 8  (the position information F 8 ), where the rest may be deduced by analogy. In this embodiment, considering the movement of the object, a size of the partial region of each of the frames I 8  and I 9  that is selected and undergoes object detection is not less than a size of a previous frame that is selected and undergoes object detection; for example, each of the frames I 7  and I 8  may select a region having 20% size of a frame to perform object detection thereon, and the frame I 9  may select a region having 50% size of a frame to perform object detection thereon. 
     Referring to a third embodiment shown in  FIG. 5  and assuming that the size or occupied ratio of the object in the frame I 4  is too big or too high, the detecting circuit  124  may perform object detection only on the frame I 8  to generate the position information F 8 , and object detection is not performed on the frames I 5  to I 7 . In detail, the detecting circuit  124  may select a partial region I 8 ′ of the frame I 8  for performing object detection thereon (other regions do not undergo object detection) according to the object detection result of the frame I 4  (the position information F 4 ), and select a partial region of the frame I 9  for performing object detection thereon (other regions do not undergo object detection) according to the object detection result of the frame I 8  (the position information F 8 ), where the rest may be deduced by analogy. In this embodiment, considering the movement of the object, the partial region which is selected from the frame I 9  and undergoes object detection is not less than the region selected from the previous frame which undergoes object detection; for example, a region having 40% size of a frame can be selected from the frame I 8  for undergoing object detection, and a region having 50% size of a frame can be selected from the frame I 9  for undergoing object detection. 
     Operations of the detecting circuit  124  performing object detection on frames I 10  to I 14  are similar to those performed on frames I 5  to I 9 ; that is, a partial region of at least one of the frames I 10  to I 13  is determined according to the position information F 9  of the object in the frame I 9 , and the detecting circuit  124  only detects the partial region to determine a position of the object in at least one of the frames I 10  to I 13 . In this embodiment, the detecting circuit  124  at least generates position information of the object in the frame I 13 , and the detecting circuit  124  then determines a partial region of the frame I 14  according to the position information of the object in the frame I 13 , and the detecting circuit  124  only detects the partial region to determine a position of the object in the frame I 14 , where the rest may be deduced by analogy. 
     Returning to  FIG. 2 , the frames I 0  to I 10  are sequentially processed by the image processing circuit  110  and the transmitted to the display panel  130  for being displayed. Thus, the object position determination circuit  120  may perform object detection on the frame I 4  while the image processing circuit  110  processes the frame I 0 , and temporarily store the position information F 4  of the object in the frame I 4 . The object position determination circuit  120  sequentially performs object detection on the frames I 5  to I 8  while the image processing circuit  110  processes the frame I 4 , to transmit the position information F 5  to F 8  of the object in the frames I 5  to I 8  to the image processing circuit  110 , so that the image processing circuit  110  processes images according to the above position information (such as the position information F 5  to F 8 ). Additionally, considering the movement of the object, a greater region may be selected from the frame I 9  for undergoing object detection, so that the object position determination circuit  120  may perform object position detection on a partial region I 9 ′ of the frame I 9  while the image processing circuit  110  processes the frame I 7 , and transmit the position information F 9  of the object in the frame I 9  to the image processing circuit  110  for being processed. 
     As mentioned in the above embodiments, since only a partial region needs to undergo object detection, the detecting circuit  124  may perform object detection or determination on each frame or most frames under a condition where calculation capability of deep learning or neural network can meet requirements, to finish the object detection operations more efficiently. 
     In an embodiment, the image processing circuit  110  may add a pattern on a frame according to object position information from the object position determination circuit  120  in order to label the object. Taking  FIG. 6  as an example and assuming that the detected object is a human face, the object position determination circuit  120  transmits an object detection result (i.e. a coordinate range  610  shown in  FIG. 6 ) of the frame I 4  to the image processing circuit  110  to serve as the position information of the object, and the image processing circuit  110  adds a rectangle on the frame I 4  to label a position of the object. Then, the object position determination circuit  120  selects a region  622  from the frame I 5  for object detection according to the coordinate range  610 , where the region  622  includes the coordinate range  610 , and the region  622  and the coordinate range  610  have a same center position (center coordinate) in an embodiment. The object position determination circuit  120  then transmits an object detection result (i.e. a coordinate range  620  shown in  FIG. 6 ) of the frame I 5  to the image processing circuit  110  to serve as the position information of the object, and the image processing circuit  110  adds a rectangle on the frame I 5  to label a position of the object. Then, the object position determination circuit  120  selects a region  632  from the frame I 6  for object detection according to the coordinate range  620 , where the region  632  includes the coordinate range  620 , and the region  632  and the coordinate range  620  have a same center position (center coordinate) in an embodiment. The object position determination circuit  120  then transmits an object detection result (i.e. a coordinate range  630  shown in  FIG. 6 ) of the frame I 6  to the image processing circuit  110  to serve as the position information of the object, and the image processing circuit  110  adds a rectangle on the frame I 6  to label a position of the object. 
     In another embodiment, the image processing circuit  110  may perform image processing on different regions within a frame in different manners according to the object position information from the object position determination circuit  120 . Taking  FIG. 6  as an example, regarding the frame I 4 , the image processing circuit  120  may process a human face portion within the region  610  in a first image processing manner (e.g. a contrast ratio, color modification, etc.), and process other portions outside the region  610  in a second image processing manner. Regarding the frames I 5  and I 6 , the image processing circuit  110  may process a human face portion within the regions  620  and  630  in a first image processing manner (e.g. a contrast ratio, color modification, etc.), and process the portions outside the regions  620  and  630  in a second image processing manner. 
       FIG. 7  is a flowchart illustrating an image processing method according to an embodiment of the present invention. Referring to all descriptions disclosed above, the image processing method is shown as follows. 
     Step  700 : the flow starts. 
     Step  702 : receive an image signal. 
     Step  704 : detect a position of an object in an N th  frame of the image signal, and determine a partial region within an (N+M) th  frame of the image signal according to the position of the object in the N th  frame, and only detect the partial region within the (N+M) th  frame to determine a position of the object in the (N+M) th  frame, wherein N and M are positive integers. 
     Step  706 : output coordinate ranges within the N th  frame and the (N+M) th  frame to respectively serve as the positions of the object. 
     Step  708 : process the image signal according to the coordinate ranges within the N th  frame and the (N+M) th  frame, to generate multiple output images to a display panel for being displayed thereon. 
     Briefly summarized, in the circuit and image processing method of the present invention, by performing object detection on a partial region of a frame only, a detection circuit can perform object detection or determination on each frame or most frames under a condition where calculation capability of deep learning or neural network can meet requirements, so that object detection operations can be finished more efficiently and loading of circuits for image identification can be reduced. 
     Those skilled in the art will readily observe that numerous modifications and alterations of the device and method may be made while retaining the teachings of the invention. Accordingly, the above disclosure should be construed as limited only by the metes and bounds of the appended claims.