A method for operating an encoder for encoding in a direct mode when coding moving images, comprising the steps of: receiving by the encoder input images to be encoded; storing said images in a memory; and outputting from said memory a current block to be encoded, characterised in that the method further comprises the steps of: (a) obtaining a first motion vector from the same position block of a backward reference picture for the current block to be encoded in a temporal direct mode; (b) deriving a first bidirectional motion vector of the current block by scaling the obtained first motion vector; (c) calculating a first cost for encoding a backward-based temporal direct mode using the first bidirectional motion vector and two prediction blocks to which the first bidirectional motion vector points; (d) obtaining a second motion vector from the same position block of a forward reference picture for the current block to be encoded in a temporal direct mode; (e) deriving a second bidirectional motion vector of the current block by scaling the obtained second motion vector; (f) calculating a second cost for encoding in a forward-based temporal direct mode using the second bidirectional motion vector and two prediction blocks to which the second bidirectional motion vector points; (g) obtaining a third motion vector from neighboring blocks for the current block to be encoded in a spatial direct mode; (h) deriving a third bidirectional motion vector of the current block by scaling the obtained third motion vector; (i) calculating a third cost for encoding in a spatial direct mode using the third bidirectional motion vector and two prediction blocks to which the third bidirectional motion vector points; and determining one direct mode having the lowest cost as a result of comparing the first, second and third costs as an optimal encoding mode, performing encoding in the determined direct mode, and inserting information into the encoded bit stream indicating which of the direct modes has been selected as the optimal encoding mode. receiving by the encoder input images to be encoded; storing said images in a memory; and outputting from said memory a current block to be encoded, characterised in that the method further comprises the steps of: (a) obtaining a first motion vector from the same position block of a backward reference picture for the current block to be encoded in a temporal direct mode; (b) deriving a first bidirectional motion vector of the current block by scaling the obtained first motion vector; (c) calculating a first cost for encoding a backward-based temporal direct mode using the first bidirectional motion vector and two prediction blocks to which the first bidirectional motion vector points; (d) obtaining a second motion vector from the same position block of a forward reference picture for the current block to be encoded in a temporal direct mode; (e) deriving a second bidirectional motion vector of the current block by scaling the obtained second motion vector; (f) calculating a second cost for encoding in a forward-based temporal direct mode using the second bidirectional motion vector and two prediction blocks to which the second bidirectional motion vector points; (g) obtaining a third motion vector from neighboring blocks for the current block to be encoded in a spatial direct mode; (h) deriving a third bidirectional motion vector of the current block by scaling the obtained third motion vector; (i) calculating a third cost for encoding in a spatial direct mode using the third bidirectional motion vector and two prediction blocks to which the third bidirectional motion vector points; and (a) obtaining a first motion vector from the same position block of a backward reference picture for the current block to be encoded in a temporal direct mode; (b) deriving a first bidirectional motion vector of the current block by scaling the obtained first motion vector; (c) calculating a first cost for encoding a backward-based temporal direct mode using the first bidirectional motion vector and two prediction blocks to which the first bidirectional motion vector points; (d) obtaining a second motion vector from the same position block of a forward reference picture for the current block to be encoded in a temporal direct mode; (e) deriving a second bidirectional motion vector of the current block by scaling the obtained second motion vector; (f) calculating a second cost for encoding in a forward-based temporal direct mode using the second bidirectional motion vector and two prediction blocks to which the second bidirectional motion vector points; (g) obtaining a third motion vector from neighboring blocks for the current block to be encoded in a spatial direct mode; (h) deriving a third bidirectional motion vector of the current block by scaling the obtained third motion vector; (i) calculating a third cost for encoding in a spatial direct mode using the third bidirectional motion vector and two prediction blocks to which the third bidirectional motion vector points; and determining one direct mode having the lowest cost as a result of comparing the first, second and third costs as an optimal encoding mode, performing encoding in the determined direct mode, and inserting information into the encoded bit stream indicating which of the direct modes has been selected as the optimal encoding mode.