A video encoding method for encoding a plurality of pictures into a temporally scalable bitstream by using a hierarchical coding structure that assigns one of a plurality of temporal levels to each of the plurality of pictures, the video encoding method comprising: writing a parameter specifying a temporal level of each of the plurality of pictures into a bitstream; judging, for each of the plurality of pictures, whether the temporal level of the picture is the lowest temporal level of the plurality of temporal levels specified in the parameters, wherein the picture is judged to be a key picture if and only if the temporal level of the picture is the lowest temporal level of the plurality of temporal levels specified in the parameters; judging whether a current picture is a temporal switching point, wherein the current picture is judged to be a temporal switching point if and only if there is a key picture such that the current picture is the first picture in display time of the pictures having the same temporal level as the current picture and which follow said key picture in display time; selecting only key pictures as valid reference pictures for the current picture, out of all reference pictures in a reference picture memory, when the current picture is a key picture or a temporal switching point; and selecting the key pictures and non-key pictures, from among all reference pictures included in the reference picture memory, as the valid reference pictures when the current picture is neither a temporal switching point nor a key picture, wherein the non-key picture is selected only if the non-key picture follows the last key picture of the key pictures preceding the current picture in display time, precedes the first key picture of the key pictures following the current picture in display time, and has a temporal level lower than or equal to a temporal level of the current picture; constructing a reference picture list including one or more of the valid reference pictures, the reference picture list being for the current picture; and encoding the current picture into the bitstream by performing motion prediction on the current picture using the one or more valid reference pictures in the reference picture list. writing a parameter specifying a temporal level of each of the plurality of pictures into a bitstream; judging, for each of the plurality of pictures, whether the temporal level of the picture is the lowest temporal level of the plurality of temporal levels specified in the parameters, wherein the picture is judged to be a key picture if and only if the temporal level of the picture is the lowest temporal level of the plurality of temporal levels specified in the parameters; judging whether a current picture is a temporal switching point, wherein the current picture is judged to be a temporal switching point if and only if there is a key picture such that the current picture is the first picture in display time of the pictures having the same temporal level as the current picture and which follow said key picture in display time; selecting only key pictures as valid reference pictures for the current picture, out of all reference pictures in a reference picture memory, when the current picture is a key picture or a temporal switching point; and selecting the key pictures and non-key pictures, from among all reference pictures included in the reference picture memory, as the valid reference pictures when the current picture is neither a temporal switching point nor a key picture, wherein the non-key picture is selected only if the non-key picture follows the last key picture of the key pictures preceding the current picture in display time, precedes the first key picture of the key pictures following the current picture in display time, and has a temporal level lower than or equal to a temporal level of the current picture; constructing a reference picture list including one or more of the valid reference pictures, the reference picture list being for the current picture; and encoding the current picture into the bitstream by performing motion prediction on the current picture using the one or more valid reference pictures in the reference picture list.