A method comprising: producing a reference image of a scene; capturing an image of the scene with a foreground object that is not attached to the scene, the foreground object including a point of interest that is a distinct physical aspect; warping either the image or the reference image of the scene that does not include the foreground object so the image and the reference image have a same view; comparing the image to the reference image after warping to detect pixels that belong to the point of interest on the foreground object, wherein comparing the image to the reference image after warping to detect pixels that belong to the point of interest on the foreground object comprises: generating a mask for the foreground object; segmenting the foreground object from the image using the mask; and detecting the pixels using the foreground object segmented from the image; detecting the point of interest on the foreground object using the detected pixels, wherein detecting the point of interest on the foreground object is performed using no predetermined geometric constraints on the foreground object by use of a classifier; displaying the image on a display; rendering an augmentation on the display over the image based on the point of interest; and tracking the foreground object in subsequently captured images, wherein the tracking comprises detecting a scene change by detecting consistent and stationary regions that appear in the mask over a number of frames and updating the reference image if there is a scene change. producing a reference image of a scene; capturing an image of the scene with a foreground object that is not attached to the scene, the foreground object including a point of interest that is a distinct physical aspect; warping either the image or the reference image of the scene that does not include the foreground object so the image and the reference image have a same view; comparing the image to the reference image after warping to detect pixels that belong to the point of interest on the foreground object, wherein comparing the image to the reference image after warping to detect pixels that belong to the point of interest on the foreground object comprises: generating a mask for the foreground object; segmenting the foreground object from the image using the mask; and detecting the pixels using the foreground object segmented from the image; generating a mask for the foreground object; segmenting the foreground object from the image using the mask; and detecting the pixels using the foreground object segmented from the image; detecting the point of interest on the foreground object using the detected pixels, wherein detecting the point of interest on the foreground object is performed using no predetermined geometric constraints on the foreground object by use of a classifier; displaying the image on a display; rendering an augmentation on the display over the image based on the point of interest; and tracking the foreground object in subsequently captured images, wherein the tracking comprises detecting a scene change by detecting consistent and stationary regions that appear in the mask over a number of frames and updating the reference image if there is a scene change.