A method of decoding multi-layer video data, the method comprising: determining, for a first coding unit of video data at a first temporal location in a first view, whether one or more reference picture lists for coding the first coding unit contain at least one temporal reference picture associated with the same view as the first coding unit, at a second, different temporal location; based on determining that the one or more reference picture lists for coding the first coding unit do not contain at least one temporal reference picture, disabling an inter-view residual prediction process, wherein disabling the inter-view residual prediction process comprises skipping decoding of a weighting factor for the inter-view residual prediction process for the first coding unit and each other coding unit of a picture containing the first coding unit, and wherein the inter-view residual prediction process includes: determining, for the first coding unit, a temporal motion vector which indicates a temporal reference picture; determining a disparity reference block in a disparity reference picture at the first temporal location in a second view that is different than the first view, wherein the disparity reference block is indicated by a disparity vector associated with the first coding unit; determining a temporal-disparity reference block at the second temporal location in the second view, wherein the temporal-disparity reference block is indicated by the temporal motion vector and the disparity vector in combination; and determining a residual predictor for predicting residual data associated with the first coding unit, wherein the residual predictor is determined based on differences between samples of the disparity reference block and corresponding samples of the temporal-disparity reference block, and wherein the residual data associated with the first coding unit represents differences between samples of the first coding unit and corresponding samples of a reference block in the temporal reference picture; and decoding the first coding unit of video data relative to at least one reference block of video data of a reference picture in the one or more reference picture lists. determining, for a first coding unit of video data at a first temporal location in a first view, whether one or more reference picture lists for coding the first coding unit contain at least one temporal reference picture associated with the same view as the first coding unit, at a second, different temporal location; based on determining that the one or more reference picture lists for coding the first coding unit do not contain at least one temporal reference picture, disabling an inter-view residual prediction process, wherein disabling the inter-view residual prediction process comprises skipping decoding of a weighting factor for the inter-view residual prediction process for the first coding unit and each other coding unit of a picture containing the first coding unit, and wherein the inter-view residual prediction process includes: determining, for the first coding unit, a temporal motion vector which indicates a temporal reference picture; determining a disparity reference block in a disparity reference picture at the first temporal location in a second view that is different than the first view, wherein the disparity reference block is indicated by a disparity vector associated with the first coding unit; determining a temporal-disparity reference block at the second temporal location in the second view, wherein the temporal-disparity reference block is indicated by the temporal motion vector and the disparity vector in combination; and determining a residual predictor for predicting residual data associated with the first coding unit, wherein the residual predictor is determined based on differences between samples of the disparity reference block and corresponding samples of the temporal-disparity reference block, and wherein the residual data associated with the first coding unit represents differences between samples of the first coding unit and corresponding samples of a reference block in the temporal reference picture; and determining, for the first coding unit, a temporal motion vector which indicates a temporal reference picture; determining a disparity reference block in a disparity reference picture at the first temporal location in a second view that is different than the first view, wherein the disparity reference block is indicated by a disparity vector associated with the first coding unit; determining a temporal-disparity reference block at the second temporal location in the second view, wherein the temporal-disparity reference block is indicated by the temporal motion vector and the disparity vector in combination; and determining a residual predictor for predicting residual data associated with the first coding unit, wherein the residual predictor is determined based on differences between samples of the disparity reference block and corresponding samples of the temporal-disparity reference block, and wherein the residual data associated with the first coding unit represents differences between samples of the first coding unit and corresponding samples of a reference block in the temporal reference picture; and decoding the first coding unit of video data relative to at least one reference block of video data of a reference picture in the one or more reference picture lists.