A method of operating an augmented reality system, the method comprising: allowing an end user to visualize direct light from a three-dimensional scene in an ambient environment; generating virtual image data from the point of the view of the end user; determining a region of spatial overlap (200a, 200b) between a real object (10, 12) in the three-dimensional scene and a virtual object in the virtual image data; determining a color of the real object (10, 12) in the overlap region (200a, 200b); displaying the virtual image data as a virtual image to the end user that, along with the visualized direct light, creates a three-dimensional augmented scene; and obscuring the real object (10, 12) in the overlap region (200a, 200b) from the perspective of the end user based on the determined color of the real object (10, 12) in the overlap region (200a, 200b), characterised in that obscuring the real object (10, 12) in the overlap region (200a, 200b) comprises: generating interference data based on the determined color of the real object (10, 12) in the overlap region (200a, 200b) by adding an intensity to the real object (10, 12) in the overlap region (200a, 200b) to match an intensity of the virtual image in the overlap regions (200a, 200b); and displaying the interference data as an interference image over the overlap region (200a, 200b) to the end user, such that the interference image combines with the direct light from the real object (10, 12) to create a background for the virtual object in the overlap region (200a, 200b). allowing an end user to visualize direct light from a three-dimensional scene in an ambient environment; generating virtual image data from the point of the view of the end user; determining a region of spatial overlap (200a, 200b) between a real object (10, 12) in the three-dimensional scene and a virtual object in the virtual image data; determining a color of the real object (10, 12) in the overlap region (200a, 200b); displaying the virtual image data as a virtual image to the end user that, along with the visualized direct light, creates a three-dimensional augmented scene; and obscuring the real object (10, 12) in the overlap region (200a, 200b) from the perspective of the end user based on the determined color of the real object (10, 12) in the overlap region (200a, 200b), characterised in that obscuring the real object (10, 12) in the overlap region (200a, 200b) comprises: generating interference data based on the determined color of the real object (10, 12) in the overlap region (200a, 200b) by adding an intensity to the real object (10, 12) in the overlap region (200a, 200b) to match an intensity of the virtual image in the overlap regions (200a, 200b); and displaying the interference data as an interference image over the overlap region (200a, 200b) to the end user, such that the interference image combines with the direct light from the real object (10, 12) to create a background for the virtual object in the overlap region (200a, 200b). generating interference data based on the determined color of the real object (10, 12) in the overlap region (200a, 200b) by adding an intensity to the real object (10, 12) in the overlap region (200a, 200b) to match an intensity of the virtual image in the overlap regions (200a, 200b); and displaying the interference data as an interference image over the overlap region (200a, 200b) to the end user, such that the interference image combines with the direct light from the real object (10, 12) to create a background for the virtual object in the overlap region (200a, 200b).