A coding apparatus, comprising an image coding unit configured to classify image data of pictures constituting moving image data into a plurality of temporal hierarchies to provide a temporal scalability, to code the image data of the pictures in each of the classified hierarchies, to divide the plurality of hierarchies into a predetermined number of hierarchy sets, wherein the lowest hierarchy set includes two or more of the plurality of hierarchies, and a hierarchy set higher than the lowest hierarchy set includes one hierarchy, to set a picture encoding interval (cpb_removal_delay) and a picture encoding timing (dpb_output_delay) in each hierarchy set, wherein the picture encoding intervals in at least the lowest hierarchy set are equal intervals; to generate a predetermined number of video streams having the coded image data of the pictures in each of the divided hierarchy sets, and to determine a decoding time stamp for each picture in each hierarchy set based on the picture encoding interval (cpb_removal_delay) and picture encoding timing in each hierarchy set; wherein the video stream having the image data of the pictures in the lowest hierarchy set is a base stream and the video streams having the image data of the pictures in the other hierarchy sets are enhanced streams; and characterized in that the image coding unit performs coding so that a encoding timing for coded image data of pictures in the hierarchy set higher than the lowest hierarchy set is an intermediate timing of encoding timings for coded image data of pictures in all hierarchy sets lower than the hierarchy set. an image coding unit configured to classify image data of pictures constituting moving image data into a plurality of temporal hierarchies to provide a temporal scalability, to code the image data of the pictures in each of the classified hierarchies, to divide the plurality of hierarchies into a predetermined number of hierarchy sets, wherein the lowest hierarchy set includes two or more of the plurality of hierarchies, and a hierarchy set higher than the lowest hierarchy set includes one hierarchy, to set a picture encoding interval (cpb_removal_delay) and a picture encoding timing (dpb_output_delay) in each hierarchy set, wherein the picture encoding intervals in at least the lowest hierarchy set are equal intervals; to generate a predetermined number of video streams having the coded image data of the pictures in each of the divided hierarchy sets, and to determine a decoding time stamp for each picture in each hierarchy set based on the picture encoding interval (cpb_removal_delay) and picture encoding timing in each hierarchy set; to classify image data of pictures constituting moving image data into a plurality of temporal hierarchies to provide a temporal scalability, to code the image data of the pictures in each of the classified hierarchies, to divide the plurality of hierarchies into a predetermined number of hierarchy sets, wherein the lowest hierarchy set includes two or more of the plurality of hierarchies, and a hierarchy set higher than the lowest hierarchy set includes one hierarchy, to set a picture encoding interval (cpb_removal_delay) and a picture encoding timing (dpb_output_delay) in each hierarchy set, wherein the picture encoding intervals in at least the lowest hierarchy set are equal intervals; to generate a predetermined number of video streams having the coded image data of the pictures in each of the divided hierarchy sets, and to determine a decoding time stamp for each picture in each hierarchy set based on the picture encoding interval (cpb_removal_delay) and picture encoding timing in each hierarchy set; wherein the video stream having the image data of the pictures in the lowest hierarchy set is a base stream and the video streams having the image data of the pictures in the other hierarchy sets are enhanced streams; and characterized in that the image coding unit performs coding so that a encoding timing for coded image data of pictures in the hierarchy set higher than the lowest hierarchy set is an intermediate timing of encoding timings for coded image data of pictures in all hierarchy sets lower than the hierarchy set.