An image processing device, comprising: a decoding unit configured to decode encoded data including an initial difference coefficient that is a difference between a DC coefficient and a predetermined initial value, a replacement difference coefficient that is a difference between the DC coefficient and a (0, 0) coefficient located at the beginning of a quantization matrix whose size is limited to be not greater than a transmission size that is the maximum size allowed in transmission, and further difference coefficients each being a difference between two adjacent coefficients in a sequence of coefficients of the quantization matrix arranged in scan order; an inverse DPCM unit configured to set the DC coefficient by adding the initial value to the initial difference coefficient and to set the quantization matrix by setting the (0, 0) coefficient located at the beginning of the quantization matrix by adding the DC coefficient to the replacement difference coefficient and by setting the remaining coefficients of the quantization matrix by stepwise adding the further difference coefficients to the (0, 0) coefficient, wherein a remaining coefficient at a particular position in scan order is the sum of the (0, 0) coefficient and the difference coefficient up to said particular position; a matrix size transformation unit configured to up-convert the quantization matrix into an up-converted quantization matrix; a replacement unit configured to replace a (0, 0) coefficient located at the beginning of the up-converted quantization matrix set by the matrix size transformation unit with the DC coefficient set by the inverse DPCM unit; and a dequantization unit configured to dequantize quantized data obtained by decoding the encoded data, using the up-converted quantization matrix in which the (0, 0) coefficient located at the beginning of the up-converted quantization matrix has been replaced with the DC coefficient by the replacement unit. a decoding unit configured to decode encoded data including an initial difference coefficient that is a difference between a DC coefficient and a predetermined initial value, a replacement difference coefficient that is a difference between the DC coefficient and a (0, 0) coefficient located at the beginning of a quantization matrix whose size is limited to be not greater than a transmission size that is the maximum size allowed in transmission, and further difference coefficients each being a difference between two adjacent coefficients in a sequence of coefficients of the quantization matrix arranged in scan order; an inverse DPCM unit configured to set the DC coefficient by adding the initial value to the initial difference coefficient and to set the quantization matrix by setting the (0, 0) coefficient located at the beginning of the quantization matrix by adding the DC coefficient to the replacement difference coefficient and by setting the remaining coefficients of the quantization matrix by stepwise adding the further difference coefficients to the (0, 0) coefficient, wherein a remaining coefficient at a particular position in scan order is the sum of the (0, 0) coefficient and the difference coefficient up to said particular position; a matrix size transformation unit configured to up-convert the quantization matrix into an up-converted quantization matrix; a replacement unit configured to replace a (0, 0) coefficient located at the beginning of the up-converted quantization matrix set by the matrix size transformation unit with the DC coefficient set by the inverse DPCM unit; and a dequantization unit configured to dequantize quantized data obtained by decoding the encoded data, using the up-converted quantization matrix in which the (0, 0) coefficient located at the beginning of the up-converted quantization matrix has been replaced with the DC coefficient by the replacement unit.