A method of performing enhancement layer temporal motion vector prediction (EL TMVP) in scalable video coding, the method comprising: - a) obtaining a reference picture list associated with a current prediction unit of a current picture in an enhancement layer; - b) obtaining a candidate reference picture list from two candidate reference picture lists associated with a co-located prediction unit that is co-located with the current prediction unit, the co-located prediction unit being part of a co-located picture of the current picture for TMVP, and the co-located picture being an inter-layer reference picture generated from a base layer; - c) determining at least one of the following conditions for avoiding temporal difference-based motion vector scaling: (i) that reference pictures in the reference picture list associated with the current prediction unit have the same picture order count values as reference pictures in the candidate reference picture list associated with the co-located prediction unit, (ii) that the candidate reference picture list has a same reference list index as the reference picture list associated with the current prediction unit, or (iii) that the candidate reference picture list comprises a reference picture having a same POC distance from a picture associated with the co-located prediction unit as the POC distance between a picture associated with the current prediction unit and the reference picture in the reference picture list associated with the current prediction unit; and - d) if at least one of the determined conditions applies, selecting the candidate reference picture list as the motion vector prediction reference picture list. - a) obtaining a reference picture list associated with a current prediction unit of a current picture in an enhancement layer; - b) obtaining a candidate reference picture list from two candidate reference picture lists associated with a co-located prediction unit that is co-located with the current prediction unit, the co-located prediction unit being part of a co-located picture of the current picture for TMVP, and the co-located picture being an inter-layer reference picture generated from a base layer; - c) determining at least one of the following conditions for avoiding temporal difference-based motion vector scaling: (i) that reference pictures in the reference picture list associated with the current prediction unit have the same picture order count values as reference pictures in the candidate reference picture list associated with the co-located prediction unit, (ii) that the candidate reference picture list has a same reference list index as the reference picture list associated with the current prediction unit, or (iii) that the candidate reference picture list comprises a reference picture having a same POC distance from a picture associated with the co-located prediction unit as the POC distance between a picture associated with the current prediction unit and the reference picture in the reference picture list associated with the current prediction unit; and (i) that reference pictures in the reference picture list associated with the current prediction unit have the same picture order count values as reference pictures in the candidate reference picture list associated with the co-located prediction unit, (ii) that the candidate reference picture list has a same reference list index as the reference picture list associated with the current prediction unit, or (iii) that the candidate reference picture list comprises a reference picture having a same POC distance from a picture associated with the co-located prediction unit as the POC distance between a picture associated with the current prediction unit and the reference picture in the reference picture list associated with the current prediction unit; and - d) if at least one of the determined conditions applies, selecting the candidate reference picture list as the motion vector prediction reference picture list.