A processor based method for reconstructing a depth image encoded as a depth bitstream including a base layer bitstream and a set of enhancement layer bitstreams, wherein the set of enhancement layer bitstreams are arranged in a low to high order, and wherein a value of a pixel in the depth image represents a distance to a three-dimensional point in a scene, comprising the steps of: decoding the base layer bitstream to produce pixels of a reconstructed base layer image corresponding to the depth image; decoding, in the low to high order, each enhancement layer bitstream, wherein the decoding of each enhancement layer bitstream produces a reconstructed residual image, further comprising, maintaining a context model using an edge map corresponding to depth discontinuities of the depth image by utilizing statistics of the data of the enhancement layer bitstream being coded; entropy decoding each enhancement layer bitstream using the context model to determine a significance value for each pixel and a sign bit for each significant pixel, wherein if the significance value of said each pixel is set to 1 then a residual value of said each pixel of the reconstructed residual image falls outside an uncertainty interval, which indicates a maximum allowable error for a pixel to be decoded, and if the significance value of said each pixel is set to 0 then the residual value of said each pixel is within the uncertainty interval; and reconstructing a pixel value of the reconstructed residual image according to the significance value, sign bit and the uncertainty interval; adding the reconstructed residual images to the reconstructed base layer image to produce a reconstructed depth image, wherein the reconstructed depth image has a maximum error relative to the depth image corresponding to the uncertainty interval associated with the highest enhancement layer; and using the reconstructed depth image for virtual view synthesis. decoding the base layer bitstream to produce pixels of a reconstructed base layer image corresponding to the depth image; decoding, in the low to high order, each enhancement layer bitstream, wherein the decoding of each enhancement layer bitstream produces a reconstructed residual image, further comprising, maintaining a context model using an edge map corresponding to depth discontinuities of the depth image by utilizing statistics of the data of the enhancement layer bitstream being coded; entropy decoding each enhancement layer bitstream using the context model to determine a significance value for each pixel and a sign bit for each significant pixel, wherein if the significance value of said each pixel is set to 1 then a residual value of said each pixel of the reconstructed residual image falls outside an uncertainty interval, which indicates a maximum allowable error for a pixel to be decoded, and if the significance value of said each pixel is set to 0 then the residual value of said each pixel is within the uncertainty interval; and reconstructing a pixel value of the reconstructed residual image according to the significance value, sign bit and the uncertainty interval; adding the reconstructed residual images to the reconstructed base layer image to produce a reconstructed depth image, wherein the reconstructed depth image has a maximum error relative to the depth image corresponding to the uncertainty interval associated with the highest enhancement layer; and using the reconstructed depth image for virtual view synthesis.