Method and system for object detection in an image plane

Disclosed is an object detection method and system in an image plane. A Hidden Markov Model (HMM) is employed and its associated parameters are initialized for an image plane. Updating HMM parameters is accomplished by referring to the previous estimated object mask in a spatial domain. With the updated HMM parameters and a decoding algorithm, a refined state sequence is obtained and a better object mask is restored from the refined state sequence. Consequently, estimation of the HMM parameters can be rapidly achieved and robust object detection can be effected. This allows the resultant object mask to be closer to the real object area, and the false detection in the background area can be decreased.

FIELD OF THE INVENTION

The present invention generally relates to a method and system for object detection in an image plane.

BACKGROUND OF THE INVENTION

Object detection plays an important role in many video applications, such as computer vision, and video surveillance systems. In general, object detection is one of the major factors for the success of video systems.

Japan Patent No. 61003591 disclosed a technique for storing background picture in the first picture memory, and store image containing objects in the second picture memory. By subtracting the data in these two picture memories, the result is the scene change, where the objects are.

U.S. patent and publication documents also disclosed several techniques for object detection. For example, U.S. Pat. No. 5,099,322 uses an object detector to detect abrupt changes between two consecutive images, and uses a decision processor to determine whether scene changes occur by means of feature computing. U.S. Pat. No. 6,999,604 uses a color normalizer to normalize the colors in an image, and uses a color transformer for color transformation so that the image can be enhanced and the area suspected of an object is enhanced to facilitate object detection. Finally, a comparison against the default color histogram is performed, and a fuzzy adaptive algorithm is used to find the moving object in the image.

U.S. Patent Publication No. 2004/0017938 disclosed a technique with a default color feature of objects. During detection, anything that matches the default color feature is determined to be an object. U.S. Patent Publication No. 2005/0111696 disclosed a technique with long exposure to capture the current image at a low illumination, and comparing the current image against the previous reference image to detect the changes. U.S. Patent Publication No. 2004/0086152 divides the image into blocks, and compares the current image block against the previous corresponding image block for the difference of frequency domain transformation parameter. When the difference exceeds a certain threshold, the image block is determined to have changed.

Gaussian Mixture Model (GMM) is usually used for modeling each pixel or region to make the background model adaptive to the changing illumination. Those pixels that do not fit the model are considered as foreground.

Dedeoglu Y. disclosed an article in 2005, “Human Action Recognition Using Gaussian Mixture Model Based Background Segmentation,” using Gaussian Mixture Model to perform real-time moving object detection.

Hidden Markov Model (HMM) is used for modeling a non-stationary process, and uses the time-axis continuity constraint in the continuous pixel intensity. In other words, if a pixel is detected as foreground, the pixel is expected to stay as foreground for a period of time. The advantages of HMM are as follows. (1) Selection of training data is not required, and (2) Using different hidden states to learn the statistical characteristics of foreground and background from a mixed sequence of foreground symbols and background symbols.

An HMM can be expressed as H:=(N,M,A,π,P1,P2), where N is the number of states, M is the number of symbols, A is the state transition probability matrix, A={aij,i,j=1, . . . N}, aijis the transiting probability from state i to state j, π={π1, . . . , πN}, πiis the initial probability of state i, and P=(pi, . . . , pn), piis the probability of state i.

J. Kato presented a technique in the article, “An HMM-Based Segmentation Method for Traffic Monitoring Movies,” IEEE Trans. PAMI, Vol. 24, No. 9, pp. 1291-1296, 2002, using a grey scale to construct an HMM on the time axis for each pixel. There are three states for each pixel, i.e. background state, foreground state, and shadow state, for detecting objects.

FIG. 1shows a schematic view of a flowchart of a conventional HMM. As shown inFIG. 1, a conventional HMM procedure includes three steps: (1) initializing HMM parameters, as shown in step101; (2) training stage, that is, estimating and updating the HMM parameters through Baum-Welch algorithm, as shown in step103; and (3) using Viterbi algorithm and the HMM parameters from the previous step to estimate the state for input data (foreground state and background state), as shown in step105. Baum-Welch algorithm is used for training HMM parameters.

Using Baum-Welch algorithm, the state transition probability matrix A, the initial probability πiof each state i, and the probability piof each state i can be trained from the previous sample and updated. The Baum-Welch algorithm is an iterative likelihood maximization method. Therefore, it is time-consuming for estimating and updating the HMM parameters.

SUMMARY OF THE INVENTION

Examples of the present invention may provide a method and system for object detection in an image plane. The present invention uses HMM to improve the robustness of the object mask in image spatial domain. The object mask obtained at the previous time is used to assist in estimating the HMM parameters at the current time. HMM is then used to estimate the background and foreground (object) at the current time with stable and robust object detection effect. The object mask at the current time is closer to the actual object range, and the false detection in foreground and background can be decreased.

The present invention constructs an HMM model for each image, unlike the conventional techniques having an HMM model for each pixel. The present invention uses two states, the foreground state and the background state. The shadow problem is solved by the fusion of the result of GMM on luma and the result of GMM on chroma.

Accordingly, the method for object detection in an image plane of the present invention includes the following steps. First, an HMM model is constructed for an image, and the HMM parameters are initialized. Then, an object mask Ωh(t−1) at the previous time is used to assist in updating the HMM parameters at the current time. Based on the HMM parameters at the current time, the object mask at the current time can be restored from states which are obtained by a decoding algorithm.

In the present invention, the HMM model can be expressed as H:=(N,M, A,π, P1,P2), where N=2 (two states), i.e., S1is the foreground state and S2is the background state, M=2 (two symbols), i.e., background symbol β and foreground symbol α, P1and P2are the probability density function (PDF) for S1and S2, respectively. P1(x=α) is the probability that foreground symbol occurs during the background situation, and P1(x=β) is the probability that background symbol occurs during the background situation. On the other hand, P2(x=α) is the probability that foreground symbol occurs during the foreground situation, and P2(x=β) is the probability that background symbol occurs during the foreground situation.

Therefore, the examples of the system for object detection in an image plane of the present invention may be realized by an HMM, a parameter estimation unit, a state estimation unit, a unit for restoring states to an object mask, and a delay buffer.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 2shows a two-dimensional representation of an object mask corresponding to an image being expressed by a one-dimensional signal, where Ωf1 1is the two-dimensional representation of an object mask corresponding to an image. The one-dimensional signal representation ωf1, called ID sequence, for the object mask of the image, can be considered as a non-stationary random process including a plurality of states and each state having its own subprocess. In the example of the one-dimensional signal representation ωf1, symbols ‘0’ and ‘1’ respectively represent foreground and background for the image.

The ID signal representation has two states. As shown inFIG. 3, S1is the background state and S2is the foreground state. Each state is a Markov chain with stationary statistics. Therefore, the signal characteristics of an object mask, i.e., a one-dimensional random process ωƒ1represented by an ID sequence, can be represented by an HMM model.

The HMM is expressed as H:=(N,M,A,π,P1P2), where N=2, i.e., S1is the background state and S2is the foreground state, M=2, i.e., background symbol β and foreground symbol α, A is the state transition probability matrix, A={aij,i,j=1, . . . N}, aijis the transition probability from state i to state j, π={π1, . . . , πN}, πiis the initial state probability of state i, and P1and P2are the probability density function (PDF) for S1and S2, respectively. P1(x=α) is the probability that foreground symbol occurs during the background situation, and P1(x=β) is the probability that background symbol occurs during the background situation. On the other hand, P2(x=α) is the probability that foreground symbol occurs during the foreground situation, and P2(x=β) is the probability that background symbol occurs during the foreground situation.

Therefore, inFIG. 3, a12is the transition probability from background state S1to foreground state S2, a21is the transition probability from foreground state S2to background state S1, a11is the transition probability from background state S1to background state S1, and a22is the transition probability from foreground S2to foreground state S2.

To rapidly estimate the HMM parameters, the present invention transforms a re-estimating background mask problem into an HMM training problem by using a new method in the existent HMM training stage to obtain HMM parameters.FIG. 4shows a flowchart illustrating the operating steps for object detection in an image plane of the present invention.

As shown inFIG. 4, the present invention first constructs an HMM for the current image, and initializes the HMM parameters, as shown in step401. Then, step403is to obtain a new mask Ω(t) on the spatial domain at the current time through the object mask Ωh(t−1 ) at previous time, and update the HMM parameters λ(t). Step405is to re-estimate the object mask at the current time based on the parameter λ(t) and a decoding algorithm.

FIG. 5shows a schematic view of a block diagram further describing the steps inFIG. 4. As shown inFIG. 5, after performing the object segmentation procedure on the current input image, the initialization of HMM parameters in step401includes the setting for the state transition probability matrix, the probabilities of P1(x=α) and P1(x=β), and the initial state probabilities of background state S1and foreground state S2. It is worth noting that for the state transition probability matrix {aij,ij=1,2}, when i≠j, aii>aij.

In step403, the mask Ω(t) to be updated represents the binary mask of subtracting foreground mask Ωh(t−1) at previous time t−1 from a foreground mask Ωƒ1(t); that is, Ω(t)=Ωh(t−1)AND Ωf1(t). Let ξ denote the occupy-ratio of foreground symbol in Ω(t), the probability of foreground symbol can be approximated as P1(x=α)=ξ. Therefore, the probability of background symbol in background state is P1(x=β)=1−P1(x=α). The HMM parameters can be updated using the above approximation.

After having the updated HMM parameters, the object mask Ωh(t−1 ) at the previous time is read in a one-dimensional way, either vertically or horizontally, as shown in step405. A decoding technique, such as Viterbi decoding algorithm, is used to re-estimate the state of Ωƒ1(x,y,t), where Ωƒ1(x,y,t)=1 if at time t, the pixel (x,y) of the input image (x,y) belongs to the foreground, and Ωƒ1(x,y,t)=0 or if at t, the pixel (x,y) of the input image belongs to the background.

In other words, the statistic model of the background is estimated. If some part (fusion of the foreground and background symbols) of Ωf1(t) matches the background statistic model, the part will be recognized as background. The estimated Ωƒ1(x,y,t) with one-dimensional states will be restored to two-dimensional object mask of the same size as the original image. Therefore, the object mask Ωf1(t) is refined, and results in a better object mask.

According to the present invention, in step405, the reading of the previous object mask Ωh(t−1) and the updating of the new mask Ω(t) can be performed in different scale options. The two common scales are scale=1 and scale=2. If the original resolution of the input signal is used in execution, the scale is set to be 1. If the original input signal is down-sampled to Ω′(t) for replacing the Ω(t) in estimating the HMM parameters λ(t), the scale is said to be 2. When scale=2, the refined state sequence is denoted as Ω′h(t) which must be up-sampled to the object mask Ω″h(t) (with original size) during the HMM procedure. According to the experimental results, the object mask obtained when scale=2 will lead to more robust object mask, and be closer to the actual object.

The present invention uses only two states, the foreground state and the background state. The shadow can be removed from the object mask by means of fusion of the results of GMM on luma and the results of GMM on chroma.

FIG. 6shows a schematic block diagram of the system of the present invention. As shown inFIG. 6, a system for object detection in an image plane includes an HMM601, a parameter estimation unit603, a state estimation unit605, a mask restoration unit607for restoring states to an object mask, and a delay buffer609.

The HMM601is initialized to H:=(N,M,A,π,P1,P2), and is coupled with an object segmentation unit611. The parameter estimation unit603uses the object mask Ωh(t−1) at previous time t−1 to update the HMM parameters λ(t) at current time t. Based on λ(t), state estimation unit605uses a decoder to estimate a corresponding state sequence. The mask restoration unit607for restoring states to an object mask transforms the state sequence into an object mask Ωh(t), and stores the object mask. The delay buffer609propagates the object mask Ωh(t−1) at previous time t−1 to the parameter estimation unit603.

Unlike the conventional methods to construct an HMM for each pixel, the present invention only constructs an HMM for an image and results in a binary object mask.

It is worth noting that in an actual object detection environment, the background area is larger than the foreground area. Therefore, in initializing the state probability, the initial state probability of the background is larger than the initial state probability of the foreground. In a simulation experiment of the present invention, 23 images are captured, and an HMM is constructed for an image100. The initial state probability π1of background is 0.9, and the initial state probability π2of foreground is 0.1. In comparison with the conventional object detection techniques, the results show that the foreground is more stable and the background is clearer when using the present invention. The complete object mask can almost be extracted. Therefore, the present invention not only improves the robustness of the object mask, but also improves the clear background to further decrease the false detection. The detection rate of the present invention is also higher.

In addition, the simulation experiments for HMM procedure of the present invention is performed under scale=1 and scale=2. The results show that when scale=2, the method of the present invention will result in a more distinguishable object mask in comparison with scale=1.

Although the present invention has been described with reference to the preferred embodiments, it will be understood that the invention is not limited to the details described thereof. Various substitutions and modifications have been suggested in the foregoing descriptions, and others will occur to those of ordinary skill in the art. Therefore, all such substitutions and modifications are intended to be embraced within the scope of the invention as defined in the appended claims.