A mediated-reality system for surgical visualization, comprising: a head-mounted display assembly comprising: a frame configured to be worn on a user's head; an image capture device (113a, 113b) coupled to the frame, the image capture device (113a, 113b) is a plenoptic camera including an image sensor, and an array of sensor elements (405a-d) corresponding to different portions of the image sensor that receive light from different lenslets, each portion of the sensor elements (405a-d) having a different field of view; a display device coupled to the frame, the display device configured to display an image towards an eye of the user; a computing device in communication with the display device and the image capture device (113a, 113b), the computing device configured to: receive image data from the image capture device (113a, 113b); render a virtual camera having a position and a field of view, and wherein only data from a portion of sensor elements (405a-d) for which the position of the virtual camera falls within the field of view of the portion of sensor element (405a-d), or for which the field of view of the portion of sensor element (405a-d) is within the field of view of the virtual camera, is used to render the image from the virtual camera, wherein the virtual camera may be positioned in front of the sensor elements (405a-d), or the virtual camera may be positioned behind the sensor elements (405a-d); and present an image corresponding to the virtual camera from the image data via the display device. a head-mounted display assembly comprising: a frame configured to be worn on a user's head; an image capture device (113a, 113b) coupled to the frame, the image capture device (113a, 113b) is a plenoptic camera including an image sensor, and an array of sensor elements (405a-d) corresponding to different portions of the image sensor that receive light from different lenslets, each portion of the sensor elements (405a-d) having a different field of view; a display device coupled to the frame, the display device configured to display an image towards an eye of the user; a frame configured to be worn on a user's head; an image capture device (113a, 113b) coupled to the frame, the image capture device (113a, 113b) is a plenoptic camera including an image sensor, and an array of sensor elements (405a-d) corresponding to different portions of the image sensor that receive light from different lenslets, each portion of the sensor elements (405a-d) having a different field of view; a display device coupled to the frame, the display device configured to display an image towards an eye of the user; a computing device in communication with the display device and the image capture device (113a, 113b), the computing device configured to: receive image data from the image capture device (113a, 113b); render a virtual camera having a position and a field of view, and wherein only data from a portion of sensor elements (405a-d) for which the position of the virtual camera falls within the field of view of the portion of sensor element (405a-d), or for which the field of view of the portion of sensor element (405a-d) is within the field of view of the virtual camera, is used to render the image from the virtual camera, wherein the virtual camera may be positioned in front of the sensor elements (405a-d), or the virtual camera may be positioned behind the sensor elements (405a-d); and present an image corresponding to the virtual camera from the image data via the display device. receive image data from the image capture device (113a, 113b); render a virtual camera having a position and a field of view, and wherein only data from a portion of sensor elements (405a-d) for which the position of the virtual camera falls within the field of view of the portion of sensor element (405a-d), or for which the field of view of the portion of sensor element (405a-d) is within the field of view of the virtual camera, is used to render the image from the virtual camera, wherein the virtual camera may be positioned in front of the sensor elements (405a-d), or the virtual camera may be positioned behind the sensor elements (405a-d); and present an image corresponding to the virtual camera from the image data via the display device.