A computer-implemented method of generating computer mediated reality data on a client device, the method comprising: capturing image data with a camera integrated in the client device, the image data representing a near real-time view of an environment around the client device; capturing location data with a location sensor integrated in the client device, the location data describing a spatial position of the client device in the environment; generating local map data based on the image data and the location data, the local map data including one or more three-dimensional point clouds spatially describing one or more objects in the environment around the client device; accessing a set of viewpoints stored locally on the client device; identifying a first image in the image data as a novel viewpoint based at least in part on the first image having less than a minimum overlap in field of view with each viewpoint in the set of viewpoints; transmitting the local map data and the first image as the novel viewpoint to an external server; receiving a local portion of world map data at the client device from the external server, wherein the world map data is generated by fusing a plurality of local maps generated by one or more client devices and viewpoints captured by one or more client devices, wherein the local portion is selected based on the local map data; determining a distance between a mapping point in the local portion of world map data and the spatial position of the client device based on location data and the local portion of world map data; generating a computer mediated reality image at the mapping point in the local portion of world map data based on the image data and the location data; and displaying the computer mediated reality image at the mapping point. capturing image data with a camera integrated in the client device, the image data representing a near real-time view of an environment around the client device; capturing location data with a location sensor integrated in the client device, the location data describing a spatial position of the client device in the environment; generating local map data based on the image data and the location data, the local map data including one or more three-dimensional point clouds spatially describing one or more objects in the environment around the client device; accessing a set of viewpoints stored locally on the client device; identifying a first image in the image data as a novel viewpoint based at least in part on the first image having less than a minimum overlap in field of view with each viewpoint in the set of viewpoints; transmitting the local map data and the first image as the novel viewpoint to an external server; receiving a local portion of world map data at the client device from the external server, wherein the world map data is generated by fusing a plurality of local maps generated by one or more client devices and viewpoints captured by one or more client devices, wherein the local portion is selected based on the local map data; determining a distance between a mapping point in the local portion of world map data and the spatial position of the client device based on location data and the local portion of world map data; generating a computer mediated reality image at the mapping point in the local portion of world map data based on the image data and the location data; and displaying the computer mediated reality image at the mapping point.