A method of tracking scored candidate objects in a sequence of image frames acquired by a camera, the scored candidate objects comprising a set of existing scored candidate objects associated with a prior image frame and a set of new scored candidate objects associated with a next image frame, wherein each of the scored candidate objects has a respective composite object score that represents a likelihood that the scored candidate object corresponds to an actual pallet structure in its associated image frame, the method comprising: determining, by a computer, a respective location for each new scored candidate object in the next image frame; and analyzing the new scored candidate objects, wherein said analyzing comprises: for each particular existing scored candidate object: identifying, by the computer, a respective first location of this particular existing scored candidate object in the prior image frame, estimating, by the computer, a respective second location in the next image frame, wherein the respective second location is estimated based on movement of the camera between acquisition of the prior image frame and the next image frame, calculating, by the computer, a respective prediction window, wherein the respective prediction window defines a set of pixel locations in the next image frame nearby the respective second location, and for any new scored candidate object having a respective location within the respective prediction window, tentatively matching that new scored candidate object with this particular existing scored candidate object. determining, by a computer, a respective location for each new scored candidate object in the next image frame; and analyzing the new scored candidate objects, wherein said analyzing comprises: for each particular existing scored candidate object: identifying, by the computer, a respective first location of this particular existing scored candidate object in the prior image frame, estimating, by the computer, a respective second location in the next image frame, wherein the respective second location is estimated based on movement of the camera between acquisition of the prior image frame and the next image frame, calculating, by the computer, a respective prediction window, wherein the respective prediction window defines a set of pixel locations in the next image frame nearby the respective second location, and for any new scored candidate object having a respective location within the respective prediction window, tentatively matching that new scored candidate object with this particular existing scored candidate object. identifying, by the computer, a respective first location of this particular existing scored candidate object in the prior image frame, estimating, by the computer, a respective second location in the next image frame, wherein the respective second location is estimated based on movement of the camera between acquisition of the prior image frame and the next image frame, calculating, by the computer, a respective prediction window, wherein the respective prediction window defines a set of pixel locations in the next image frame nearby the respective second location, and for any new scored candidate object having a respective location within the respective prediction window, tentatively matching that new scored candidate object with this particular existing scored candidate object.