A video decoding method performed by a decoding apparatus, the method comprising: obtaining picture order count information of reference pictures for constructing a reference picture set including information on the reference pictures, which are used for inter prediction of a current picture and are prior to the current picture in a decoding order, from a bitstream; deriving picture order count values for specifying the reference pictures based on the picture order count information, and constructing the reference picture set including information on the reference pictures; constructing a reference picture list based on information on the reference pictures in the reference picture set; and performing an inter prediction on a current block based on the reference picture list to derive a predicted sample of the current block; characterized in that the step of constructing the reference picture set includes deriving a picture order count value for specifying an i-th reference picture based on a picture order count difference derived from the picture order count information, the picture order count information specifies the picture order count difference between the current picture and the i-th reference picture when the i-th reference picture is a first reference picture specified by the picture order count information, and the picture order count information specifies the picture order count difference between the i-th reference picture and an (i-1)-th reference picture when the i-th reference picture is a second reference picture or a subsequent reference picture thereof specified by the picture order count information. obtaining picture order count information of reference pictures for constructing a reference picture set including information on the reference pictures, which are used for inter prediction of a current picture and are prior to the current picture in a decoding order, from a bitstream; deriving picture order count values for specifying the reference pictures based on the picture order count information, and constructing the reference picture set including information on the reference pictures; constructing a reference picture list based on information on the reference pictures in the reference picture set; and performing an inter prediction on a current block based on the reference picture list to derive a predicted sample of the current block; characterized in that the step of constructing the reference picture set includes deriving a picture order count value for specifying an i-th reference picture based on a picture order count difference derived from the picture order count information, the picture order count information specifies the picture order count difference between the current picture and the i-th reference picture when the i-th reference picture is a first reference picture specified by the picture order count information, and the picture order count information specifies the picture order count difference between the i-th reference picture and an (i-1)-th reference picture when the i-th reference picture is a second reference picture or a subsequent reference picture thereof specified by the picture order count information.