An augmented reality display system (60, 250) configured to be worn by a user, the system comprising: one or more imaging devices; one or more processors (140, 150), the processors configured to: obtain a current image of a real-world environment; perform frame-to-frame tracking on the current image, such that salient points included in a previous image are projected onto the current image, each salient point representing a unique portion of the real-world environment which can be tracked; perform map-to-frame tracking on the current image when one or more areas of the current image contain less than either a threshold number of salient points or a threshold density of salient points, wherein map-to-frame tracking comprises: obtaining descriptors of the salient points in the current image, and matching the obtained descriptors of the salient points with descriptors stored in a map database, the stored descriptors corresponding to descriptors of distinguishable visual features of the real-world environment, and the map database including real-world locations which are three-dimensional coordinates of the distinguishable visual features, such that real-world locations of the salient points are identified via the matching; and determine, based on the identification of the real-world locations of the salient points, at least an orientation of the one or more imaging devices in the real-world environment. one or more imaging devices; one or more processors (140, 150), the processors configured to: obtain a current image of a real-world environment; perform frame-to-frame tracking on the current image, such that salient points included in a previous image are projected onto the current image, each salient point representing a unique portion of the real-world environment which can be tracked; perform map-to-frame tracking on the current image when one or more areas of the current image contain less than either a threshold number of salient points or a threshold density of salient points, wherein map-to-frame tracking comprises: obtaining descriptors of the salient points in the current image, and matching the obtained descriptors of the salient points with descriptors stored in a map database, the stored descriptors corresponding to descriptors of distinguishable visual features of the real-world environment, and the map database including real-world locations which are three-dimensional coordinates of the distinguishable visual features, such that real-world locations of the salient points are identified via the matching; and determine, based on the identification of the real-world locations of the salient points, at least an orientation of the one or more imaging devices in the real-world environment. obtain a current image of a real-world environment; perform frame-to-frame tracking on the current image, such that salient points included in a previous image are projected onto the current image, each salient point representing a unique portion of the real-world environment which can be tracked; perform map-to-frame tracking on the current image when one or more areas of the current image contain less than either a threshold number of salient points or a threshold density of salient points, wherein map-to-frame tracking comprises: obtaining descriptors of the salient points in the current image, and matching the obtained descriptors of the salient points with descriptors stored in a map database, the stored descriptors corresponding to descriptors of distinguishable visual features of the real-world environment, and the map database including real-world locations which are three-dimensional coordinates of the distinguishable visual features, such that real-world locations of the salient points are identified via the matching; and obtaining descriptors of the salient points in the current image, and matching the obtained descriptors of the salient points with descriptors stored in a map database, the stored descriptors corresponding to descriptors of distinguishable visual features of the real-world environment, and the map database including real-world locations which are three-dimensional coordinates of the distinguishable visual features, such that real-world locations of the salient points are identified via the matching; and determine, based on the identification of the real-world locations of the salient points, at least an orientation of the one or more imaging devices in the real-world environment.