An apparatus for generating an output video stream, the apparatus comprising: a processor for providing a plurality of reference video streams for a plurality of reference viewpoints for a scene; a receiver for receiving a viewpoint request from a remote client, the viewpoint request being indicative of a requested viewpoint of the scene for the output video stream; a generator for generating an output video stream comprising a first video stream comprising first output frames from a first reference video stream of the plurality of reference video streams for a first viewpoint and a second video stream comprising second output frames from a second reference video stream of the plurality of reference video streams for a second reference viewpoint, the second output frames being differentially encoded relative to the first output frames and the first output frames not being differentially encoded relative to the second output frames; and a controller for determining the first viewpoint and the second viewpoint as viewpoints of the plurality of reference viewpoints in response to the viewpoint request; wherein the controller is arranged to, in response to a detection during the generation of the output video stream of a change in the viewpoint request meeting a criterion, change the first viewpoint from being a first reference viewpoint of the plurality of reference viewpoints and the second viewpoint from being a second reference viewpoint of the plurality of reference viewpoints to the first viewpoint being the second reference viewpoint and the second viewpoint being the first reference viewpoint; characterized by the apparatus being arranged to include a number of additional video streams in the output video stream, each additional video stream comprising output frames from an additional reference video stream of the plurality of reference video streams differentially encoded relative to the first output frames; and wherein the controller is arranged to determine the number of additional video streams in response to at least one of a variation measure for the viewpoint requests, and a latency measure for a communication channel for transmitting the output video stream. a processor for providing a plurality of reference video streams for a plurality of reference viewpoints for a scene; a receiver for receiving a viewpoint request from a remote client, the viewpoint request being indicative of a requested viewpoint of the scene for the output video stream; a generator for generating an output video stream comprising a first video stream comprising first output frames from a first reference video stream of the plurality of reference video streams for a first viewpoint and a second video stream comprising second output frames from a second reference video stream of the plurality of reference video streams for a second reference viewpoint, the second output frames being differentially encoded relative to the first output frames and the first output frames not being differentially encoded relative to the second output frames; and a controller for determining the first viewpoint and the second viewpoint as viewpoints of the plurality of reference viewpoints in response to the viewpoint request; wherein the controller is arranged to, in response to a detection during the generation of the output video stream of a change in the viewpoint request meeting a criterion, change the first viewpoint from being a first reference viewpoint of the plurality of reference viewpoints and the second viewpoint from being a second reference viewpoint of the plurality of reference viewpoints to the first viewpoint being the second reference viewpoint and the second viewpoint being the first reference viewpoint; characterized by the apparatus being arranged to include a number of additional video streams in the output video stream, each additional video stream comprising output frames from an additional reference video stream of the plurality of reference video streams differentially encoded relative to the first output frames; and wherein the controller is arranged to determine the number of additional video streams in response to at least one of a variation measure for the viewpoint requests, and a latency measure for a communication channel for transmitting the output video stream.