Method and system for tracking objects within a video

This disclosure relates generally to computer vision, and more particularly to method and system for tracking objects within a video. In one embodiment, a method for tracking objects within a video is disclosed. The method includes receiving one or more regions of interest (ROIs) corresponding to one or more objects in an initial frame of the video, extracting a set of scale and rotation invariant interest data points in each of the ROIs, clustering the set of scale and rotation invariant interest data points in a ROI into a set of clusters based on corresponding locations in the ROI, determining an optimal set of interest data points from each of the set of clusters based on corresponding feature response values and spread values, and initiating tracking of the optimal set of interest data points in subsequent frames of the video to track the one or more objects in the video.

This application claims the benefit of Indian Patent Application Serial No. 201841003127 filed Jan. 26, 2018 which is hereby incorporated by reference in its entirety.

FIELD

This disclosure relates generally to computer vision, and more particularly to method and system for tracking objects within a video.

BACKGROUND

Computer vision in general and real-time object tracking in particular has numerous applications such as surveillance systems, augmented reality (AR), human computer interaction (HCI), medical imaging, and so forth. As will be appreciated, there are a number of techniques for tracking objects in real-time. Such techniques may be broadly categorized into point based tracking, kernel based tracking, and contour based tracking.

However, tracking objects in real-time using existing techniques on low-end electronic devices (e.g., embedded devices, cameras, mobile phones with low computational capability, etc.) may be quite challenging due to hardware constraints (e.g., low computational capability) of such devices. In real world applications, once the objects are tracked in a frame sequence, per frame processing such as augmenting on the tracked objects, estimating pose of the object, and so forth may further bring down the real-time performance of such devices. The impact on performance of the devices may further result in missing tracks in the frame sequences, thereby impacting performance of the tracking technique itself. In other words, existing techniques are inefficient, slow, and not robust particular on the low-end electronic devices. Moreover, existing techniques are limited because of a tradeoff between the speed of tracking and the robustness of tracking.

SUMMARY

In one embodiment, a method for tracking objects within a video is disclosed. In one example, the method may include receiving one or more regions of interest (ROIs) corresponding to one or more objects in an initial frame of the video. The method may further include extracting a set of scale and rotation invariant interest data points in each of the ROIs. The method may further include clustering the set of scale and rotation invariant interest data points in a ROI into a set of clusters based on corresponding locations in the ROI. The method may further include determining an optimal set of interest data points from each of the set of clusters based on corresponding feature response values and spread values. The method may further include initiating tracking of the optimal set of interest data points in subsequent frames of the video to track the one or more objects in the video.

In one embodiment, a system for tracking objects within a video is disclosed. In one example, the system may include at least one processor and a memory communicatively coupled to the at least one processor. The memory may store processor-executable instructions, which, on execution, may cause the processor to receive one or more regions of interest (ROIs) corresponding to one or more objects in an initial frame of the video. The processor-executable instructions, on execution, may further cause the processor to extract a set of scale and rotation invariant interest data points in each of the ROIs. The processor-executable instructions, on execution, may further cause the processor to cluster the set of scale and rotation invariant interest data points in a ROI into a set of clusters based on corresponding locations in the ROI. The processor-executable instructions, on execution, may further cause the processor to determine an optimal set of interest data points from each of the set of clusters based on corresponding feature response values and spread values. The processor-executable instructions, on execution, may further cause the processor to initiate tracking of the optimal set of interest data points in subsequent frames of the video to track the one or more objects in the video.

In one embodiment, a non-transitory computer-readable medium storing computer-executable instructions for tracking objects within a video is disclosed. In one example, the stored instructions, when executed by a processor, may cause the processor to perform operations including receiving one or more regions of interest (ROIs) corresponding to one or more objects in an initial frame of the video. The operations may further include extracting a set of scale and rotation invariant interest data points in each of the ROIs. The operations may further include clustering the set of scale and rotation invariant interest data points in a ROI into a set of clusters based on corresponding locations in the ROI. The operations may further include determining an optimal set of interest data points from each of the set of clusters based on corresponding feature response values and spread values. The operations may further include initiating tracking of the optimal set of interest data points in subsequent frames of the video to track the one or more objects in the video.

DETAILED DESCRIPTION

Referring now toFIG. 1, an exemplary system100for tracking objects within a video is illustrated in accordance with some embodiments of the present disclosure. In particular, the system100may include a tracking device (e.g., server, desktop, laptop, notebook, netbook, tablet, smartphone, mobile phone, or any other computing device) that implements a tracking engine so as to track objects within the video. It should be noted that, in some embodiments, the tracking device may be a low computational capability computing device such as embedded devices, cameras, mobile phones with low computational capability, and so forth. As will be described in greater detail in conjunction withFIGS. 2-5, the tracking engine may receive one or more regions of interest (ROIs) corresponding to one or more objects in an initial frame of the video, extract a set of scale and rotation invariant interest data points in each of the ROIs, cluster the set of scale and rotation invariant interest data points in a ROI into a set of clusters based on corresponding locations in the ROI, determine an optimal set of interest data points from each of the set of clusters based on corresponding feature response values and spread values, and initiate tracking of the optimal set of interest data points in subsequent frames of the video to track the one or more objects in the video.

The system100may include one or more processors101, a computer-readable medium (e.g., a memory)102, and a display103. The computer-readable storage medium102may store instructions that, when executed by the one or more processors101, cause the one or more processors101to track objects within the video in accordance with aspects of the present disclosure. The computer-readable storage medium102may also store various data (e.g., frames of the video, ROIs, scale and rotation invariant interest data points, clusters, optimal set of interest data points, feature response values for data points, spread values for data points, grid indices, active data points, orientations between active data points, distance between active data points, symmetry between active data points, outlier data points, threshold for number of active data points, etc.) that may be captured, processed, and/or required by the system100. The system100may interact with a user via a user interface104accessible via the display103. The system100may also interact with one or more external devices105over a communication network106for sending or receiving various data. The external devices105may include, but are not limited to, a remote server, a digital device, or another computing system.

Referring now toFIG. 2, a functional block diagram of the tracking engine200, implemented by the system100ofFIG. 1, is illustrated in accordance with some embodiments of the present disclosure. The tracking engine200may include various modules that perform various functions so as to perform tracking of objects within the video. In some embodiments, the tracking engine200may include an object (ROI) detection module201, an initialization module202, a data points database203, and an object or a ROI tracking module204. Additionally, in some embodiments, the initialization module202may further include an extraction module205, a clustering module206, and a data points selection module207. Further, in some embodiments, the object (ROI) tracking module204may further include a tracking module208and a validation module209. As will be appreciated by those skilled in the art, all such aforementioned modules and the database201-209may be represented as a single module or a combination of different modules. Moreover, as will be appreciated by those skilled in the art, each of the modules and the database201-209may reside, in whole or in parts, on one device or multiple devices in communication with each other.

The object (ROI) detection module201may detect one or more objects in an initial or a real-time frame of the video. As will be appreciated, the detected objects may be tracked and analyzed in subsequent frame sequences of the video. In some embodiments, the object (ROI) detection module201may return an array of detected objects. The location of each object in the initial frame may be defined by a bounding box, also referred to as the ROI. Thus, the bounding box containing the object of interest may act as the ROI for the tracking engine.

The initialization module202may select spread out and high feature response value data points as an optimal set of interest data points ‘M’ from a given ROI. It should be noted that the optimal set of interest data points ‘M’ may be tracked be tracked in real-time by the tracking engine. As stated above, in some embodiments, the initialization module202may include the extraction module205, the clustering module206, and the data points selection module207.

The extraction module205may receive the detected bounding box (i.e., the ROI) for each of the objects to be tracked from the object (ROI) detection module201via a connection C1. The extraction module205may then extract the scale and rotation invariant interest data points (e.g., pixels) in each of the ROIs. As will be appreciated, the scale and rotation invariant interest data points are extracted for a given ROI in the initial frame of the video. In some embodiments, the set of scale and rotation invariant interest data points may be extracted based on the corresponding feature response values and spread values within the ROI. It should be noted that, in some embodiments, the set of scale and rotation invariant interest data points cover at least a substantial portion of boundaries of the ROI. Further, it should be noted that, a number of data points (N) within the set of scale and rotation invariant interest data points may be determined based on at least one of a required speed of tracking and a required robustness of tracking.

The clustering module206may receive the extracted scale and rotation invariant interest data points for each of the ROIs from the extraction module205via a connection C2. The clustering module206may then cluster the set of scale and rotation invariant interest data points in a ROI, based on corresponding locations in the ROI, into a set of clusters. In some embodiments, the clustering module206may cluster the extracted scale and rotation invariant interest data points into K-clusters based on their co-ordinates (i.e., locations in the ROI) proximity to the cluster centroids. It should be noted that the hyper-parameter ‘K’ may be selected based on the size of the ROI in the initial frame of the video.

The data points selection module207may receive the clustered scale and rotation invariant interest data points for each of the ROIs from the clustering module206via a connection C3. The data points selection module207may then determine an optimal set of interest data points210from each of the set of clusters based on corresponding feature response values and spread values. As will be appreciated, feature response value for a given data point within a ROI may be based on a significance of the given data point in accurately representing the ROI. For example, data points with high feature response values may include, but are not limited to, corner data points, edge data points, gradient data points, minimum data points, and maximum data points. Similarly, spread value for a given data point within a ROI may be based on a significance of the given data point in adequately covering the ROI.

In some embodiments, the data points selection module207may determine the optimal set of interest data points210from each of the clusters by splitting the ROI into a number of grid cells, assigning the clustered scale and rotation invariant interest data points corresponding grid indices based on corresponding locations in the grid cells, and selecting the optimal set of interest data points210based on the corresponding feature response values and the corresponding grid indices. As will be appreciated, the grid indices correspond to the spread values of the data points within a given grid cell. Thus, the ROI may be split into grid cells, and the clustered N data points may be assigned grid indices along with their cluster indices. The selection of optimal data points from a cluster may then be performed based on the feature response values and grid indices.

In some embodiments, the data points selection module207may select the ‘M/K’ interest data points from each of the ‘K’ clusters as the optimal set of interest data points210. It should be noted that the number of interest data points (M/K) to be selected for the optimal set of interest data points210may be defined by a hyper-parameter ‘M’. Further, it should be noted that the hyper-parameter ‘M’ may be minimal data points (M) pre-defined for the tracking device based on a computational capability of the tracking device. In some embodiments, the hyper-parameter ‘M’ may be automatically adjusted based on a computational capability of the tracking device. As will be appreciated, the number of interest data points (M/K) in the optimal set of interest data points210may be an optimal number of data points to perform speedy as well as robust tracking for a given tracking device.

The data points database203may receive the optimal set of interest data points210(i.e., selected data points) from the data points selection module207via a connection C4. The data points database203may then store the selected optimal set of interest data points210. It should be noted that the process of determining the optimal set of interest data points210may be performed only once during initialization of the tracking engine200. Further, it should be noted that the optimal set of interest data points210may be a reduced number of data points but may be spread out prominent data points, thereby facilitating fast and robust tracking of the objects.

As will be described in greater detail below, the optimal set of interest data points210may be employed by the object (ROI) tracking module204for tracking detected objects in real time. For example, the optimal set of interest data points210may be initial set of data points to initialize the object tracking. Additionally, the optimal set of interest data points210may act as reference set for the validation of the tracked data points. For example, the symmetry of the optimal set of interest data points210may be used to remove outlier data points during tracking in subsequent frame sequences of the video.

The object (ROI) tracking module204may track the detected objects bounding boxes (i.e., ROIs) through subsequent frame sequences. As stated above, in some embodiments, the object (ROI) tracking module204may include a tracking module208and a validation module209. The tracking module208may receive the optimal set of interest data points210, in the reference frame (i.e., the initial frame) to be tracked through frame sequences, from the data points database203via a connection C5. The tracking module208may then track the optimal set of interest data points210, and find their matching locations in the instant frame.

As will be appreciated, the optimal set of interest data points210selected in the ROI of the initial frame during initialization of the tracking engine200may be used to start the ROI tracking. The data points tracked in the second frame may be considered as active data points (validated tracked data points) for the third frame. Further, the active data points in the second frame may be employed for tracking ROI in the third frame, and so on. Thus, the optimal set of interest data points are tracked by iteratively determining locations of active data points from among the optimal set of interest data points in each of the subsequent video frames. Further, the tracked data points in a frame may be validated based on the reference optimal set of interest data points, and the valid data points may be considered as the active data points to be tracked in the next frame. It should be noted that when the number of active data points in any frame is reduced to less than a predefined threshold (T), the tracking module208may abandon the tracking, and invoke fresh ROI detection instead.

The validation module209may receive the tracked interest data points in an instant frame from the tracking module208via a connection C6. The validation module209may then identify outlier data points with respect to the reference interest data points (i.e., optimal set of interest data points210computed during the initialization of the tracking engine200). Thus, the validation module209may validate the tracked data points in a frame based on the reference optimal set of interest data points210computed during the initialization of the tracking engine200. Further, the validation module209may filter out the identified outlier data points, thereby leaving the filtered data points211for tracking the object in the subsequent frames.

In some embodiments, the validation module209validates the active data points based on at least one of an orientation, a distance, or a symmetry between two or more of the active data points in each of the subsequent video frames, and remove outlier active data points based on the validation. Thus, the validation module209may use the symmetry and positioning of the reference interest data points and the matched data points in the current frame to filter the outlier active data points. A connection C7may define residual active data points211left after filtering outliers.

As will be appreciated, the residual active data points211are the data points to be tracked for the next frame. In other words, the residual active data points211are the data points in the current frame of the video that may be persisted, and that may be used for tracking the objects in the next frame. The tracking module208may receive the residual active data points211for tracking the objects in the next frame via a connection C8.

As the tracking in frame sequences progresses, the number of residual active data points211may keep reducing. Thus, when the residual active data points become less than a predefined threshold (T), the tracking may be terminated. In such cases, the object (ROI) tracking module204may invoke the object (ROI) detection module201to re-detect the objects in the current frame via a connection C9. The tracking engine200may then restart the tracking after initialization by the initialization module202for the re-detected ROIs. It should be noted that, in some embodiments, the predefined threshold (T) may be based on the end application. For example, if the purpose of the tracking is pose estimation, the pre-defined threshold (T) may be 4. However, if the purpose of the tracking is tracking a moving object (e.g., a cricket ball during a live match), the pre-defined threshold (T) may be 8.

As will be appreciated, each of the connections, C1-C9, may acquire or transfer data from one module or the database201-209to the other module or the database201-209using standard wired or wireless data connections means. For example, each of the connections, C1-C9, may employ one or more connection protocols including, but not limited to, serial advanced technology attachment (SATA), integrated drive electronics (IDE), IEEE-1394, universal serial bus (USB), fiber channel, small computer systems interface (SCSI), STD Bus, RS-232, RS-422, RS-485, I2C, SPI, Microwire, 1-Wire, IEEE 1284, Intel® QuickPathInterconnect, InfiniBand, and PCIe.

By way of an example, the tracking engine200described above may perform fast and robust tracking of objects in real-time on a computing device (e.g., low computational capability devices) using clustering and grid based mechanism. The tracking engine200may be fed with regions of interest (ROIs) in an initial frame (e.g., real-time frame) of the video which needs to be tracked in successive frames of the video. The ROIs may be output from the object (ROI) detection module201representing objects' bounding boxes in the initial frame which are to be tracked and augmented in frame sequences. The speed of tracking may be directly proportional to the number of ROIs to be tracked in the frame sequences.

The tracking engine200may then determine an optimal set of interest data points in each of the input ROIs which are to be tracked so as to locate the ROIs in successive frames. As will be appreciated, the speed of tracking may be dependent on the number of data points in the optimal set. Thus, in some embodiments, the optimal set includes a minimum number of data points that are good enough for tracking. However, the tracking robustness may decrease with decrease in the number of data points in the optimal set. So the optimal set of interest data points may be selected in such a way that the data points cover most parts of the ROIs boundaries. Such selection of data points may add to the robustness during tracking even in case of partial occlusions. Further, orientations, distances, symmetry between the interest data points may be used to re-validate the tracked data points in the successive frames to add further to the robustness of the tracking.

As will be appreciated, unlike the existing tracking techniques, the tracking engine200described above employ clustering and grid based mechanism for selecting an optimal set of interest data points thus enhancing the tracker speed. Further, the selection of the optimal set of interest data points may be prioritized based on both location (spread out data points) and feature response values so as to increase the speed of tracking while adding robustness to the tracking. Moreover, such selection of good trackable data points may enhance the occlusion handling capability of the tracking engine200.

It should be noted that the tracking engine200may be implemented in programmable hardware devices such as programmable gate arrays, programmable array logic, programmable logic devices, and so forth. Alternatively, the tracking engine200may be implemented in software for execution by various types of processors. An identified engine of executable code may, for instance, include one or more physical or logical blocks of computer instructions which may, for instance, be organized as an object, procedure, function, module, or other construct. Nevertheless, the executables of an identified engine need not be physically located together, but may include disparate instructions stored in different locations which, when joined logically together, include the engine and achieve the stated purpose of the engine. Indeed, an engine of executable code could be a single instruction, or many instructions, and may even be distributed over several different code segments, among different applications, and across several memory devices.

As will be appreciated by one skilled in the art, a variety of processes may be employed for tracking objects within a video. For example, the exemplary system100and the associated tracking engine200may perform tracking of objects within the video by the processes discussed herein. In particular, as will be appreciated by those of ordinary skill in the art, control logic and/or automated routines for performing the techniques and steps described herein may be implemented by the system100and the associated tracking engine200, either by hardware, software, or combinations of hardware and software. For example, suitable code may be accessed and executed by the one or more processors on the system100to perform some or all of the techniques described herein. Similarly application specific integrated circuits (ASICs) configured to perform some or all of the processes described herein may be included in the one or more processors on the system100.

For example, referring now toFIG. 3, exemplary control logic300for tracking objects within a video via a system, such as system100, is depicted via a flowchart in accordance with some embodiments of the present disclosure. As illustrated in the flowchart, the control logic300may include the steps of receiving one or more regions of interest (ROIs) corresponding to one or more objects in an initial frame of the video at step301, extracting a set of scale and rotation invariant interest data points in each of the ROIs at step302, clustering the set of scale and rotation invariant interest data points in a ROI into a set of clusters based on corresponding locations in the ROI at step303, determining an optimal set of interest data points from each of the set of clusters based on corresponding feature response values and spread values at step304, and initiating tracking of the optimal set of interest data points in subsequent frames of the video to track the one or more objects in the video at step305. In some embodiments, the control logic300may further include the step of detecting the one or more ROIs in the initial video frame. It should be noted that, in some embodiments, each of the one or more ROIs may include a bounding box within which the corresponding object may be located.

In some embodiments, the set of scale and rotation invariant interest data points may be extracted at step302based on the corresponding feature response values and spread values within the ROI. Additionally, it should be noted that, in some embodiments, the set of scale and rotation invariant interest data points may cover at least a substantial portion of boundaries of the ROI. Further, in some embodiments, a number of data points (N) within the set of scale and rotation invariant interest data points may be determined based on at least one of a required speed of tracking and a required robustness of tracking.

In some embodiments, the set of scale and rotation invariant interest data points may be clustered at step303based on corresponding coordinates proximity to cluster centroids. It should be noted that, in some embodiments, a number of clusters (K) within the set of clusters may be selected based on a size of the ROI.

In some embodiments, the optimal set of interest data points may be determined at step304by splitting the ROI into a number of grid cells, assigning the set of scale and rotation invariant interest data points corresponding grid indices based on corresponding locations in the grid cells, and selecting the optimal set of interest data points based on the corresponding feature response values and the corresponding grid indices, wherein the grid indices correspond to the spread values. It should be noted that, in some embodiments, a number of data points (M/K) within the set of optimal interest data points may be determined based on a number of clusters (K) and a number of minimal data points (M) pre-defined for the tracking device based on a computational capability of the tracking device.

In some embodiments, the optimal set of interest data points may be tracked at step305by iteratively determining locations of active data points from among the optimal set of interest data points in each of the subsequent video frames. Additionally, in some embodiments, the control logic300may further include the steps of validating the active data points based on at least one of an orientation, a distance, or a symmetry between two or more of the active data points in each of the subsequent video frames, and removing outlier active data points based on the validation. Further, in some embodiments, the control logic300may include the steps of terminating the tracking if the number of active data points in an instant video frame falls below a pre-defined threshold, and detecting one or more new ROIs in the instant video frame to re-initiate the tracking.

Referring now toFIG. 4, exemplary control logic400for tracking objects in real-time using clustering and grid based mechanism is depicted in greater detail via a flowchart in accordance with some embodiments of the present disclosure. As illustrated in the flowchart, at step401, the control logic400may extract the scale and rotation invariant interest data points in a region of interest (ROI) for a provided input frame and ROI bounding box. Initially, for a given input frame and a ROI bounding box, the scale and rotation invariant interest data points in the ROI may be extracted. As will be appreciated, the extraction of scale and rotation invariant interest data points needs to be fast and accurate. Thus, only the top ‘N’ data points may be identified and extracted. In some embodiments, the top ‘N’ data points may be sorted and selected based on the feature response values and spread values in the ROI. It should be noted that the hyper-parameter ‘N’ should not be so small that the spread out data points are missed. Further, it should be noted that the hyper-parameter ‘N’ should not be so high that the time required for clustering is high. The spread out interest data points with good response feature values brings the robustness during tracking. The extracted scale and rotation invariant interest data points and their feature response values may be subsequently used during the determination of the optimal set of interest data points.

At step402, the control logic400may cluster the extracted scale and rotation invariant interest data points into ‘K’ clusters based on their co-ordinates (i.e., locations in the ROI) proximity to cluster centroids. It should be noted that the hyper-parameter ‘K’ may be selected based on the size of the ROI in the initial image fame of the video. In some embodiments, ‘K’ should have a minimum value of ‘4’ for ensuring selection of spread out data points within the ROI, thereby ensuring robust and efficient tracking.

At step403, the control logic400may select the interest data points from each of the ‘K’ clusters based on the feature response values and grid indices. As will be appreciated, the number of data points needs to be reduced so as to improve performance of the control logic400in terms of speed. However, the reduction should be performed in such a way that robustness of the control logic400is retained and the control logic400does not miss the tracked object frequently. The total number of data points to be selected may be defined by a hyper-parameter ‘M’ which forms the minimal interest data points set. The value of ‘M’ may depend on the hardware computation capability of the tracking device. As will be appreciated, a higher value of ‘M’ may slow down the control logic400while a lower value of ‘M’ may affect the robustness of the control logic400. Once the interest data points are clustered, ‘M/K’ interest data points may be selected from each of the ‘K’ clusters.

The ROI may be split into grid cells and the clustered ‘N’ data points may be assigned grid indices along with their cluster indices. The selection of ‘M/K’ interest data points from each of the ‘K’ clusters may be performed based on the feature response values and the grid indices. By way of an example, if a cluster spans ‘3’ grid cells then equal number of interest data points may be selected from each of these grids with priority given to data points with higher feature response values. However, if the number of interest data points to be selected in a grid cell for a cluster are less, then the data points available in that grid may be selected and rest of the data points may be selected from the other grid cells with preference given to the data points with higher features response values.

Further, for example, if a grid cell may fall under two or more clusters, then the number of data points to be selected from the grid cell for a given cluster remains same given that the total data points selected from the grid cell in all the clusters should not exceed the limit of ‘M/K’ data points. If it exceeds the limit, then the number of data points to be selected for each cluster in that grid cell may be reduced to ‘M/K/O’ data points where ‘O’ is the number of clusters whose data points lie in that grid cell and rest of the data points for that cluster may be selected from other grid cells with preference given to data points with higher feature response values.

At step404, the control logic400may form minimal interest data points set from the ‘M’ spread out and high feature response value data points selected at step403. The minimal interest data points set may be used as initial set of data points to start ROI tracking in successive frames of the video. The spatial alignment and symmetry of the minimal interest data points set may act as a reference for validating the tracked interest data points in the successive frames of the video. The steps401-404may be performed once during initialization of the ROI tracking, and the reduced spread out and prominent minimal interest data points set so determined may facilitate fast and robust tracking of the object.

At step405, the control logic400may store the formed minimal interest data points set in a database. The minimal interest data points set stored in the database may be further used for the real-time tracking of the objects in the successive frames of the video.

At step406, the control logic400may initialize the ROI tracking using the minimal interest data points set stored in the database. The minimal interest data points set selected in the ROI of the initial frame during initialization of the control logic400may be used to start the ROI tracking. The data points tracked in the second frame may be considered as active data points (validated tracked data points) for the third frame (i.e., the active data points in the second frame may be used for tracking ROI in the third frame), and so on.

At step407, the control logic400may validate the tracked data points in a frame based on the reference minimal interest data points set. The validated data points may be considered as active data points set to be tracked in a next successive frame. As stated above, when the number of active data points in any frame is reduced to less than a predefined threshold (T), the control logic may abandon tracking and may invoke re-detection of ROI instead.

At step408, the control logic400may remove outliers by filtering out wrongly tracked data points that affect the performance of the control logic400in consecutive frames leading to wrong ROI tracking. The tracked interest data points in a current frame may be validated based on the reference minimal interest data points set. As will be appreciated, the validation may be performed to remove outliers by filtering out wrongly tracked data points, which will affect the tracking performance in the consecutive frames leading to wrong ROI tracking.

Referring now toFIGS. 5A-5E, an exemplary scenario for tracking objects within a video is illustrated in accordance with some embodiments of the present disclosure. In the exemplary scenario, the objects may be tracked in real-time and analytics may be performed on the tracked objects. As illustrated inFIG. 5A, the object (ROI) detection module201may detect the objects in an image frame500of the video, and may return ROIs501A,501B bounding the objects in the frame500. Each of the ROIs501A,50B may then be passed onto the initialization module202, which may extract the scale and rotation invariant interest data points in the ROIs501A,501B via the extraction module205. For example,FIG. 5Billustrates the scale and rotation invariant interest data points extracted from the ROI501A.

Further, the initialization module202may cluster the extracted scale and rotation invariant interest data points into ‘K’ clusters (e.g., ‘4’ or ‘5’ clusters) via the clustering module206. For example,FIG. 5Cillustrates the scale and rotation invariant interest data points clustered into 4 clusters502A-502D. Further, the initialization module202may select some of the data points from each of the clusters502A-502D via the data points selection module207. The number of data points to be selected may be dependent on the hardware capability of the tracking device. Further, the number of data points to be selected should be optimal for a given tracking device. In other words, the number of data points should not be so less that it causes the tracking device to loose the track of object frequently. At the same time, the number of data points should not be so high that it makes the tracking device slow. In some embodiments, a minimum of 50 data points need to be selected for a fast as well as robust tracking.

Further, the ROI region may be divided into grid cells (e.g., 3×3 grid having 9 grid cells). For example,FIG. 5Dillustrates the ROI501A with the clustered scale and rotation invariant interest data points divided into 9 grid cells in a 3×3 grid. In some embodiments, the number of grid cells may increase if the ROI region is bigger in size so that the spread out data points may be selected. Further, the grid indices may be assigned to the interest data points and then equal number of interest data points may be selected from each grid cell for a given cluster with priority given to the data points with higher feature response values. The selected interest data points represent the minimal interest data points set. For example,FIG. 5Eillustrates the ROI501A with the minimal interest data points set.

By way of an example, 50 data points need to be selected for a given ROI from among 5 clusters. If there is a cluster with data points lying in three grid cells then, in such case, 3, 3, and 3 data points from each grid cell may be selected and the last data point with highest feature response value from among the remaining data points may be selected. In this manner, 10 data points from each cluster may be selected, therefore providing 50 data points from five clusters. Further, if the grid cell lies in two clusters then half of the data points in that grid cell may be selected from the first cluster and the remaining half from the second cluster need to be selected, with overall number of data points selected from each cluster to be same as 10 data points. The selected spread out high feature response values interest data points (i.e., minimal interest data points set) may act as reference and may be used for triggering the tracking device. In other words, these data points may act as the active data points to be tracked for the next frame. The spatial symmetry and orientations of the minimal interest data points set may be computed, which may act as reference for filtering outlier data points during the tracking process. This is performed for each object ROI and may be stored in the data points database203for tracking and processing in real-time.

Further, during tracking of the objects (ROIs), the precomputed minimal interest data points set stored in the data points database203may be used as the active data points by the object (ROI) tracking module204. The object (ROI) tracking module204may determine the matching data points in the current frame (i.e., tracked data points) for the active data points from the previous frame via the tracking module208. The object (ROI) tracking module204may then validate the tracked data points for removing the outliners via the validation module209. The filtered tracked data points (or the residual active data points) may now become the active data points for the next frame. Additionally, in some embodiments, augment analytics may be performed on each of the tracked objects in real-time.

The tracking process may iteratively continue until the completion of frames of the video, or until the completion of the tracking, or when the number of active data points to be tracked in the next frame falls below the predefined threshold (T). For example, if the number of tracked data points are less than the predefined threshold (T), then the tracking is abandoned. The tracking device may then invoke the object (ROI) detection module201so as to re-detect the objects to be tracked. The process may then be repeated by passing the re-detected ROIs corresponding to the objects to the initialization module202.

The disclosed methods and systems may be implemented on a conventional or a general-purpose computer system, such as a personal computer (PC) or server computer. Referring now toFIG. 6, a block diagram of an exemplary computer system601for implementing embodiments consistent with the present disclosure is illustrated. Variations of computer system601may be used for implementing system100for tracking objects within a video. Computer system601may include a central processing unit (“CPU” or “processor”)602. Processor602may include at least one data processor for executing program components for executing user-generated or system-generated requests. A user may include a person, a person using a device such as such as those included in this disclosure, or such a device itself. The processor may include specialized processing units such as integrated system (bus) controllers, memory management control units, floating point units, graphics processing units, digital signal processing units, etc. The processor may include a microprocessor, such as AMD Athlon, Duron or Opteron, ARM's application, embedded or secure processors, IBM PowerPC, Intel's Core, Itanium, Xeon, Celeron or other line of processors, etc. The processor602may be implemented using mainframe, distributed processor, multi-core, parallel, grid, or other architectures. Some embodiments may utilize embedded technologies like application-specific integrated circuits (ASICs), digital signal processors (DSPs), Field Programmable Gate Arrays (FPGAs), etc.

Using the I/O interface603, the computer system601may communicate with one or more I/O devices. For example, the input device604may be an antenna, keyboard, mouse, joystick, (infrared) remote control, camera, card reader, fax machine, dongle, biometric reader, microphone, touch screen, touchpad, trackball, sensor (e.g., accelerometer, light sensor, GPS, altimeter, gyroscope, proximity sensor, or the like), stylus, scanner, storage device, transceiver, video device/source, visors, etc. Output device605may be a printer, fax machine, video display (e.g., cathode ray tube (CRT), liquid crystal display (LCD), light-emitting diode (LED), plasma, or the like), audio speaker, etc. In some embodiments, a transceiver606may be disposed in connection with the processor602. The transceiver may facilitate various types of wireless transmission or reception. For example, the transceiver may include an antenna operatively connected to a transceiver chip (e.g., Texas Instruments WiLink WL1283, Broadcom BCM4750IUB8, Infineon Technologies X-Gold 618-PMB9600, or the like), providing IEEE 602.11a/b/g/n, Bluetooth, FM, global positioning system (GPS), 2G/3G HSDPA/HSUPA communications, etc.

In some embodiments, the processor602may be disposed in communication with a communication network608via a network interface607. The network interface607may communicate with the communication network608. The network interface may employ connection protocols including, without limitation, direct connect, Ethernet (e.g., twisted pair 10/100/1000 Base T), transmission control protocol/internet protocol (TCP/IP), token ring, IEEE 602.11a/b/g/n/x, etc. The communication network608may include, without limitation, a direct interconnection, local area network (LAN), wide area network (WAN), wireless network (e.g., using Wireless Application Protocol), the Internet, etc. Using the network interface607and the communication network608, the computer system601may communicate with devices609,610, and611. These devices may include, without limitation, personal computer(s), server(s), fax machines, printers, scanners, various mobile devices such as cellular telephones, smartphones (e.g., Apple iPhone, Blackberry, Android-based phones, etc.), tablet computers, eBook readers (Amazon Kindle, Nook, etc.), laptop computers, notebooks, gaming consoles (Microsoft Xbox, Nintendo DS, Sony PlayStation, etc.), or the like. In some embodiments, the computer system601may itself embody one or more of these devices.

As will be appreciated by those skilled in the art, the techniques described in the various embodiments discussed above provide for efficient, speedy, and robust tracking of objects in real-time. The techniques employ a combination of clustering and grid based approach to track objects in frame sequences in a speedy manner while maintaining required robustness. The combination of clustering and grid based approach facilitate selection of optimal set of interest data points in the ROI to be tracked, thereby making it feasible to perform real-time tracking on low-end computing devices (i.e., devices with low computational capability). Additionally, the techniques described above prioritize the selection based on the interest data point's location and feature response values, which enhances the performance of the tracking. Unlike existing techniques, the techniques described in the embodiments discussed above select a reduced set of data points to enhance the tracking speed without compromising much on robustness of the tracking device by selecting spread out data points covering most parts of the ROI to be tracked.