A method of decoding video data to determine a predicted enhancement layer sample based on a sample of a base layer, wherein the base layer sample represents a luminance value and/or a chrominance value, wherein samples of the base layer have a first bit depth, and samples of the enhancement layer have a second bit depth that is greater than the first bit depth, the method comprising: receiving the base layer sample; applying a preliminary mapping function to the base layer sample to produce a preliminary predicted sample having the second bit depth, wherein the preliminary mapping function comprises: a left bit shift, multiplication by a number greater than or equal to 2, or a look-up table that maps base layer sample values onto corresponding enhancement layer sample values; determining an intensity category and a pattern category associated with the base layer sample, wherein determining the intensity category comprises comparing the luminance value of the base layer sample, the chrominance value of the base layer sample, or a combination of the luminance and chrominance values with a range of possible intensity values divided into a plurality of bands, to determine which band the base layer sample is located in, wherein each band corresponds to an intensity category, wherein determining the pattern category comprises comparing an intensity value associated with the base layer sample with intensity values of spatially adjacent samples, and assigning a pattern category to the base layer sample based on the intensity value of the base layer sample relative to the intensity value of each adjacent sample, and wherein each intensity category and each pattern category has a coefficient associated therewith for adjusting the preliminary predicted samples to produce refined predicted samples; multiplying the preliminary predicted sample by the coefficient associated with the determined intensity category and by the coefficient associated with the determined pattern category to determine a refined predicted sample from the preliminary predicted sample. receiving the base layer sample; applying a preliminary mapping function to the base layer sample to produce a preliminary predicted sample having the second bit depth, wherein the preliminary mapping function comprises: a left bit shift, multiplication by a number greater than or equal to 2, or a look-up table that maps base layer sample values onto corresponding enhancement layer sample values; a left bit shift, multiplication by a number greater than or equal to 2, or a look-up table that maps base layer sample values onto corresponding enhancement layer sample values; determining an intensity category and a pattern category associated with the base layer sample, wherein determining the intensity category comprises comparing the luminance value of the base layer sample, the chrominance value of the base layer sample, or a combination of the luminance and chrominance values with a range of possible intensity values divided into a plurality of bands, to determine which band the base layer sample is located in, wherein each band corresponds to an intensity category, wherein determining the pattern category comprises comparing an intensity value associated with the base layer sample with intensity values of spatially adjacent samples, and assigning a pattern category to the base layer sample based on the intensity value of the base layer sample relative to the intensity value of each adjacent sample, and wherein each intensity category and each pattern category has a coefficient associated therewith for adjusting the preliminary predicted samples to produce refined predicted samples; multiplying the preliminary predicted sample by the coefficient associated with the determined intensity category and by the coefficient associated with the determined pattern category to determine a refined predicted sample from the preliminary predicted sample.