A video system for maximizing tracking speed of an object, the system comprising: a camera configured to generate an image frame; and an object pattern tracker having a pattern detector configured to detect pattern features of an object in the image frame of a video, wherein the object pattern tracker is configured to determine a number of feature correspondences between the pattern features of the object in the image frame and object features in a reference frame; wherein the object pattern tracker has a detected pattern feature processor configured to determine whether the number of feature correspondences of the object in the image frame exceeds a predetermined object pattern threshold level, wherein the object pattern tracker is configured to apply a template based tracking to track the object in the image frame when the number of feature correspondences of the object does not exceed the predetermined object pattern threshold level to achieve a first tracking object speed of the object, and wherein the object pattern tracker is configured to apply a corner based tracking to track the object in the image frame when the number of feature correspondences of the object exceeds the predetermined object pattern threshold level to achieve a second tracking object speed of the object, and wherein the object pattern tracker applies one of the template based tracking and the corner based tracking to track the object in the image frame independently of a selected type of tracking for a previous image frame in the video, such that the type of tracking between template based and corner based tracking is applied independently for each frame in the sequence of images of the video. a camera configured to generate an image frame; and an object pattern tracker having a pattern detector configured to detect pattern features of an object in the image frame of a video, wherein the object pattern tracker is configured to determine a number of feature correspondences between the pattern features of the object in the image frame and object features in a reference frame; wherein the object pattern tracker has a detected pattern feature processor configured to determine whether the number of feature correspondences of the object in the image frame exceeds a predetermined object pattern threshold level, wherein the object pattern tracker is configured to apply a template based tracking to track the object in the image frame when the number of feature correspondences of the object does not exceed the predetermined object pattern threshold level to achieve a first tracking object speed of the object, and wherein the object pattern tracker is configured to apply a corner based tracking to track the object in the image frame when the number of feature correspondences of the object exceeds the predetermined object pattern threshold level to achieve a second tracking object speed of the object, and wherein the object pattern tracker applies one of the template based tracking and the corner based tracking to track the object in the image frame independently of a selected type of tracking for a previous image frame in the video, such that the type of tracking between template based and corner based tracking is applied independently for each frame in the sequence of images of the video.