A method for generating multi-resolution lightfield image pyramids, wherein the multi-resolution lightfield image pyramids represent the lightfield at different resolutions, the method comprising: Receiving, by an image processor, a plurality of multi-dimensional image data sets of a volume captured at different resolutions for each of a plurality of different camera positions, wherein the volume represents an object or a patient; Rendering, by the image processor, an image of the volume for each of the plurality of image data sets, wherein each of the different resolutions corresponds to a level of a plurality of image pyramids, wherein a top level of each of the plurality of image pyramids corresponds to the lowest resolution, a bottom level of each of the plurality of image pyramids corresponds to the highest resolution, and a variety of intermediate levels of each of the plurality of image pyramids each corresponds to a resolution greater than the top level and less than the bottom level; storing, by the server, the rendered images for each of the different camera positions together as one of the plurality of lightfield image pyramids; dividing, by the image processor, the rendered images into a plurality of image tiles wherein rendered images with higher resolutions are divided into more image tiles than rendered images with lower resolutions; assigning, by the image processor, a unique identifier to each of the image tiles; receiving, by a graphical user interface, a request for a lightfield volume rendering of a viewpoint of a volume; identifying, by the image processor, a position, an orientation and a zoom level for the lightfield volume rendering of the volume based on the requested viewpoint; generating, by the image processor, a list of the subset of images of the volume for rendering the lightfield volume rendering, wherein the images of the volume comprise image tiles from the plurality of lightfield image pyramids; requesting, by the image processor from the server, the subset of images of the volume based on a plurality of unique identifiers associated with the image tiles; receiving, by the image processor, a subset of images of the volume from the plurality of multi-resolution lightfield image pyramids corresponding to the identified position, the identified orientation and the identified zoom level wherein each image of the subset of images is received by accessing only the necessary image data of one of the different levels of an image pyramid of the plurality of image pyramids; and generating, by the image processor on a display, the lightfield volume rendering using the subset of images of the volume; wherein receiving the multi-dimensional image data sets comprises receiving image data sets parameterized with enclosing spheres (505, 507) and wherein parameterizing the image data sets comprises: defining a first sphere and a second sphere inside the first sphere, wherein the volume lies inside the second sphere; and path tracing from points on the first sphere from each point on the second sphere. Receiving, by an image processor, a plurality of multi-dimensional image data sets of a volume captured at different resolutions for each of a plurality of different camera positions, wherein the volume represents an object or a patient; Rendering, by the image processor, an image of the volume for each of the plurality of image data sets, wherein each of the different resolutions corresponds to a level of a plurality of image pyramids, wherein a top level of each of the plurality of image pyramids corresponds to the lowest resolution, a bottom level of each of the plurality of image pyramids corresponds to the highest resolution, and a variety of intermediate levels of each of the plurality of image pyramids each corresponds to a resolution greater than the top level and less than the bottom level; storing, by the server, the rendered images for each of the different camera positions together as one of the plurality of lightfield image pyramids; dividing, by the image processor, the rendered images into a plurality of image tiles wherein rendered images with higher resolutions are divided into more image tiles than rendered images with lower resolutions; assigning, by the image processor, a unique identifier to each of the image tiles; receiving, by a graphical user interface, a request for a lightfield volume rendering of a viewpoint of a volume; identifying, by the image processor, a position, an orientation and a zoom level for the lightfield volume rendering of the volume based on the requested viewpoint; generating, by the image processor, a list of the subset of images of the volume for rendering the lightfield volume rendering, wherein the images of the volume comprise image tiles from the plurality of lightfield image pyramids; requesting, by the image processor from the server, the subset of images of the volume based on a plurality of unique identifiers associated with the image tiles; receiving, by the image processor, a subset of images of the volume from the plurality of multi-resolution lightfield image pyramids corresponding to the identified position, the identified orientation and the identified zoom level wherein each image of the subset of images is received by accessing only the necessary image data of one of the different levels of an image pyramid of the plurality of image pyramids; and generating, by the image processor on a display, the lightfield volume rendering using the subset of images of the volume;