An imaging system, configured as a rolling shutter camera, comprising: an image capture device comprising a lens and an image sensor, the lens configured to direct light from an environment surrounding the image capture device to the image sensor and the image sensor configured to sequentially capture a plurality of image segments of an image based on the light from the environment, the image representing a field of view, FOV, of the outward facing image capture device, the FOV comprising a portion of the environment and including a plurality of sparse points, wherein each sparse point is identifiable in part based on a corresponding subset of the plurality of image segments; non-transitory data storage configured to sequentially receive the plurality of image segments from the image sensor and store instructions for estimating at least one of a position and orientation of the image capture device within the environment; and at least one hardware processor operably coupled to the non-transitory data storage and configured by the instructions to: sequentially identify one or more sparse points of the plurality of sparse points when each subset of image segments corresponding to the one or more sparse points is received at the non-transitory data storage, and characterized in that the at least one hardware processor is further configured by the instructions to: periodically estimate the at least one of the position and orientation of the image capture device within the environment based on a rolling set of sparse points comprising a predetermined number of most recently identified sparse points selected from the identified sparse points, and excluding sparse points that are not within the predetermined number of most recently identified sparse points, wherein each individual periodic estimation is based on a different set of sparse points relative to a set of sparse points used for a preceding estimation, and periodically update the at least one of the position and orientation of the image capture device within the environment based at least in part on the rolling set of sparse points. an image capture device comprising a lens and an image sensor, the lens configured to direct light from an environment surrounding the image capture device to the image sensor and the image sensor configured to sequentially capture a plurality of image segments of an image based on the light from the environment, the image representing a field of view, FOV, of the outward facing image capture device, the FOV comprising a portion of the environment and including a plurality of sparse points, wherein each sparse point is identifiable in part based on a corresponding subset of the plurality of image segments; non-transitory data storage configured to sequentially receive the plurality of image segments from the image sensor and store instructions for estimating at least one of a position and orientation of the image capture device within the environment; and at least one hardware processor operably coupled to the non-transitory data storage and configured by the instructions to: sequentially identify one or more sparse points of the plurality of sparse points when each subset of image segments corresponding to the one or more sparse points is received at the non-transitory data storage, and characterized in that the at least one hardware processor is further configured by the instructions to: periodically estimate the at least one of the position and orientation of the image capture device within the environment based on a rolling set of sparse points comprising a predetermined number of most recently identified sparse points selected from the identified sparse points, and excluding sparse points that are not within the predetermined number of most recently identified sparse points, wherein each individual periodic estimation is based on a different set of sparse points relative to a set of sparse points used for a preceding estimation, and periodically update the at least one of the position and orientation of the image capture device within the environment based at least in part on the rolling set of sparse points. periodically estimate the at least one of the position and orientation of the image capture device within the environment based on a rolling set of sparse points comprising a predetermined number of most recently identified sparse points selected from the identified sparse points, and excluding sparse points that are not within the predetermined number of most recently identified sparse points, wherein each individual periodic estimation is based on a different set of sparse points relative to a set of sparse points used for a preceding estimation, and periodically update the at least one of the position and orientation of the image capture device within the environment based at least in part on the rolling set of sparse points.