An image decoding method for decoding image data formed by a plurality of frames, the image decoding method characterized by having a current category decoding step for decoding a category number j of a current frame, the current frame being classified into one of N categories, with N ≥ 2, a reference image specifying data decoding step for decoding a reference image specifying data which specifies a reference frame, for said decoded category number, a predicted image producing step for producing a predicted image from the reference frame specified by said reference image specifying data, wherein the reference frame is selected from image data of a plurality of reference frames of an i-th category which have been decoded previously, for the current frame classified as j-th category, wherein for the number of the categories 1 ≤ i ≤ j is applied, a difference decoding step for decoding a difference between a decoded image of the current frame and the predicted image, a decoded image producing step for producing the decoded image of the current frame from said decoded difference data and said predicted image, and a decoded image storing step for storing said produced decoded image data of the current frame into a reference image memory, the reference image memory having memories allocated to specific categories, wherein said reference image specifying data decoding step has a tentative frame number setting step for setting tentative frame numbers with respect to the image data of the reference frames each belonging to the i-th category which is the j-th category of the current frame and below, among a plurality of image data stored in said reference image memory and not setting tentative frame numbers with respect to the image data of reference frames belonging to categories other than the i-th category, the tentative frame numbers being serial numbers and being set based on a decoding order of the reference frames, and a tentative frame number decoding step for obtaining the tentative frame number which specifies an image data to be selected at said predicted image producing step, by decoding the reference image specifying data. a current category decoding step for decoding a category number j of a current frame, the current frame being classified into one of N categories, with N ≥ 2, a reference image specifying data decoding step for decoding a reference image specifying data which specifies a reference frame, for said decoded category number, a predicted image producing step for producing a predicted image from the reference frame specified by said reference image specifying data, wherein the reference frame is selected from image data of a plurality of reference frames of an i-th category which have been decoded previously, for the current frame classified as j-th category, wherein for the number of the categories 1 ≤ i ≤ j is applied, a difference decoding step for decoding a difference between a decoded image of the current frame and the predicted image, a decoded image producing step for producing the decoded image of the current frame from said decoded difference data and said predicted image, and a decoded image storing step for storing said produced decoded image data of the current frame into a reference image memory, the reference image memory having memories allocated to specific categories, wherein said reference image specifying data decoding step has a tentative frame number setting step for setting tentative frame numbers with respect to the image data of the reference frames each belonging to the i-th category which is the j-th category of the current frame and below, among a plurality of image data stored in said reference image memory and not setting tentative frame numbers with respect to the image data of reference frames belonging to categories other than the i-th category, the tentative frame numbers being serial numbers and being set based on a decoding order of the reference frames, and a tentative frame number decoding step for obtaining the tentative frame number which specifies an image data to be selected at said predicted image producing step, by decoding the reference image specifying data.