An encoding method for multi-layered frame-compatible video delivery, comprising: a) base layer processing images or video frames of multiple views through a base layer, comprising: multiplexing the multiple views in the base layer to create a base layer frame-compatible representation of the images or video frames of the multiple views; storing past decoded frame-compatible reference pictures for the base layer; and encoding the base layer frame-compatible representation by performing disparity compensation at frame-compatible level using the past decoded frame-compatible reference pictures for the base layer; b) enhancement layer processing the images or video frames of multiple views through one or more enhancement layers, the enhancement layer processing for each of the one or more enhancement layers comprising: multiplexing the multiple views in the respective enhancement layer to create an enhancement layer frame-compatible representation of the images or video frames of the multiple views for the respective enhancement layer, the enhancement layer frame-compatible representation of the respective enhancement layer having a higher spatial resolution than the base layer frame-compatible representation; de-multiplexing decoded frame-compatible reference pictures for the respective enhancement layer; de-multiplexing past decoded frame-compatible reference pictures for the base layer; storing the de-multiplexed reference pictures in multiple enhancement layer reference picture buffers, one for each view of the multiple views; generating a frame-compatible reference picture by multiplexing the reference pictures from the multiple enhancement layer reference picture buffers; and encoding the enhancement layer by performing motion or disparity compensation using the frame-compatible reference picture. a) base layer processing images or video frames of multiple views through a base layer, comprising: multiplexing the multiple views in the base layer to create a base layer frame-compatible representation of the images or video frames of the multiple views; storing past decoded frame-compatible reference pictures for the base layer; and encoding the base layer frame-compatible representation by performing disparity compensation at frame-compatible level using the past decoded frame-compatible reference pictures for the base layer; multiplexing the multiple views in the base layer to create a base layer frame-compatible representation of the images or video frames of the multiple views; storing past decoded frame-compatible reference pictures for the base layer; and encoding the base layer frame-compatible representation by performing disparity compensation at frame-compatible level using the past decoded frame-compatible reference pictures for the base layer; b) enhancement layer processing the images or video frames of multiple views through one or more enhancement layers, the enhancement layer processing for each of the one or more enhancement layers comprising: multiplexing the multiple views in the respective enhancement layer to create an enhancement layer frame-compatible representation of the images or video frames of the multiple views for the respective enhancement layer, the enhancement layer frame-compatible representation of the respective enhancement layer having a higher spatial resolution than the base layer frame-compatible representation; de-multiplexing decoded frame-compatible reference pictures for the respective enhancement layer; de-multiplexing past decoded frame-compatible reference pictures for the base layer; storing the de-multiplexed reference pictures in multiple enhancement layer reference picture buffers, one for each view of the multiple views; generating a frame-compatible reference picture by multiplexing the reference pictures from the multiple enhancement layer reference picture buffers; and encoding the enhancement layer by performing motion or disparity compensation using the frame-compatible reference picture. multiplexing the multiple views in the respective enhancement layer to create an enhancement layer frame-compatible representation of the images or video frames of the multiple views for the respective enhancement layer, the enhancement layer frame-compatible representation of the respective enhancement layer having a higher spatial resolution than the base layer frame-compatible representation; de-multiplexing decoded frame-compatible reference pictures for the respective enhancement layer; de-multiplexing past decoded frame-compatible reference pictures for the base layer; storing the de-multiplexed reference pictures in multiple enhancement layer reference picture buffers, one for each view of the multiple views; generating a frame-compatible reference picture by multiplexing the reference pictures from the multiple enhancement layer reference picture buffers; and encoding the enhancement layer by performing motion or disparity compensation using the frame-compatible reference picture.